bpf: LLVM BTF inner map struct type def missing

[Mahe Tardy <mahe.tardy@gmail.com>]

Hello,

While writing a BPF prog using map of maps I bumped into this compiler
bug that GitHub user thediveo and Isovalent colleague Timo Beckers
already discussed in the cilium/ebpf discussions [^1].

The issue is that a struct only used in a inner map is not included in
the program BTF, so it needs a dummy declaration elsewhere to work.

For example such program:

	#include "vmlinux.h"
	#include <bpf/bpf_helpers.h>

	struct missing_type {
		uint64_t foo;
	};

	// struct missing_type bar; // commented on purpose

	struct {
		__uint(type, BPF_MAP_TYPE_ARRAY_OF_MAPS);
		__type(key, uint32_t);
		__type(value, uint32_t);
		__uint(max_entries, 16);
		__array(
			values, struct {
				__uint(type, BPF_MAP_TYPE_HASH);
				__type(key, uint64_t);
				__type(value, struct missing_type);
				__uint(max_entries, 32);
			});
	} outer_map SEC(".maps");

Then do:

	bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h
	clang -target bpf -g -O2 -c prog.c -o prog.o
	bpftool btf dump file prog.o

Will result in:

	[1] PTR '(anon)' type_id=3
	[2] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
	[3] ARRAY '(anon)' type_id=2 index_type_id=4 nr_elems=12
	[4] INT '__ARRAY_SIZE_TYPE__' size=4 bits_offset=0 nr_bits=32 encoding=(none)
	[5] PTR '(anon)' type_id=6
	[6] TYPEDEF 'uint32_t' type_id=7
	[7] TYPEDEF 'u32' type_id=8
	[8] TYPEDEF '__u32' type_id=9
	[9] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32 encoding=(none)
	[10] PTR '(anon)' type_id=11
	[11] ARRAY '(anon)' type_id=2 index_type_id=4 nr_elems=16
	[12] PTR '(anon)' type_id=13
	[13] STRUCT '(anon)' size=32 vlen=4
		'type' type_id=14 bits_offset=0
		'key' type_id=16 bits_offset=64
		'value' type_id=21 bits_offset=128
		'max_entries' type_id=22 bits_offset=192
	[14] PTR '(anon)' type_id=15
	[15] ARRAY '(anon)' type_id=2 index_type_id=4 nr_elems=1
	[16] PTR '(anon)' type_id=17
	[17] TYPEDEF 'uint64_t' type_id=18
	[18] TYPEDEF 'u64' type_id=19
	[19] TYPEDEF '__u64' type_id=20
	[20] INT 'unsigned long long' size=8 bits_offset=0 nr_bits=64 encoding=(none)
	[21] PTR '(anon)' type_id=28
	[22] PTR '(anon)' type_id=23
	[23] ARRAY '(anon)' type_id=2 index_type_id=4 nr_elems=32
	[24] ARRAY '(anon)' type_id=12 index_type_id=4 nr_elems=0
	[25] STRUCT '(anon)' size=32 vlen=5
		'type' type_id=1 bits_offset=0
		'key' type_id=5 bits_offset=64
		'value' type_id=5 bits_offset=128
		'max_entries' type_id=10 bits_offset=192
		'values' type_id=24 bits_offset=256
	[26] VAR 'outer_map' type_id=25, linkage=global
	[27] DATASEC '.maps' size=0 vlen=1
		type_id=26 offset=0 size=32 (VAR 'outer_map')
	[28] FWD 'missing_type' fwd_kind=struct

You can see that the outer map is [25], with values [24] with type to
[12] thus [13] and then the value of [13] is [21] which points to type
[28]. And [28] is a forward declaration. Thus if we try to load this
program (there's no program but the libbpf error msg is explicit):

	bpftool prog load prog.o /sys/fs/bpf/prog

Output is

	libbpf: map 'outer_map.inner': can't determine value size for type [28]: -22.

Now if you uncomment the commented line in the example (or use this type
in a function as suggested by Timo), the BTF looks like this:

	[1] PTR '(anon)' type_id=3
	[2] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
	[3] ARRAY '(anon)' type_id=2 index_type_id=4 nr_elems=12
	[4] INT '__ARRAY_SIZE_TYPE__' size=4 bits_offset=0 nr_bits=32 encoding=(none)
	[5] PTR '(anon)' type_id=6
	[6] TYPEDEF 'uint32_t' type_id=7
	[7] TYPEDEF 'u32' type_id=8
	[8] TYPEDEF '__u32' type_id=9
	[9] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32 encoding=(none)
	[10] PTR '(anon)' type_id=11
	[11] ARRAY '(anon)' type_id=2 index_type_id=4 nr_elems=16
	[12] PTR '(anon)' type_id=13
	[13] STRUCT '(anon)' size=32 vlen=4
		'type' type_id=14 bits_offset=0
		'key' type_id=16 bits_offset=64
		'value' type_id=21 bits_offset=128
		'max_entries' type_id=22 bits_offset=192
	[14] PTR '(anon)' type_id=15
	[15] ARRAY '(anon)' type_id=2 index_type_id=4 nr_elems=1
	[16] PTR '(anon)' type_id=17
	[17] TYPEDEF 'uint64_t' type_id=18
	[18] TYPEDEF 'u64' type_id=19
	[19] TYPEDEF '__u64' type_id=20
	[20] INT 'unsigned long long' size=8 bits_offset=0 nr_bits=64 encoding=(none)
	[21] PTR '(anon)' type_id=27
	[22] PTR '(anon)' type_id=23
	[23] ARRAY '(anon)' type_id=2 index_type_id=4 nr_elems=32
	[24] ARRAY '(anon)' type_id=12 index_type_id=4 nr_elems=0
	[25] STRUCT '(anon)' size=32 vlen=5
		'type' type_id=1 bits_offset=0
		'key' type_id=5 bits_offset=64
		'value' type_id=5 bits_offset=128
		'max_entries' type_id=10 bits_offset=192
		'values' type_id=24 bits_offset=256
	[26] VAR 'outer_map' type_id=25, linkage=global
	[27] STRUCT 'missing_type' size=8 vlen=1
		'foo' type_id=17 bits_offset=0
	[28] VAR 'bar' type_id=27, linkage=global
	[29] DATASEC '.bss' size=0 vlen=1
		type_id=28 offset=0 size=8 (VAR 'bar')
	[30] DATASEC '.maps' size=0 vlen=1
		type_id=26 offset=0 size=32 (VAR 'outer_map')

And then the type [27] exists, loading can now proceed.

I tested it with latest LLVM-project head when writing this e789f8bdf369
("[libc][math] Add Generic Comparison Operations for floating point
types (#144983)").

If you think it's reasonable to fix, I would be interested looking into
this.

[^1]: https://github.com/cilium/ebpf/discussions/1658#discussioncomment-12491339