Re: [PATCH dwarves v2 2/2] btf_encoder: Normalize array index type for parallel dwarf loading case

[Arnaldo Carvalho de Melo <acme@kernel.org>]

Em Thu, May 12, 2022 at 03:55:14PM -0700, Andrii Nakryiko escreveu:
> On Wed, May 11, 2022 at 10:18 PM Yonghong Song <yhs@fb.com> wrote:
> >
> > With latest llvm15 built kernel (make -j LLVM=1), I hit the following
> > error when build selftests (make -C tools/testing/selftests/bpf -j LLVM=1):
> >   In file included from skeleton/pid_iter.bpf.c:3:
> >   .../selftests/bpf/tools/build/bpftool/vmlinux.h:84050:9: error: unknown type name
> >        '__builtin_va_list___2'; did you mean '__builtin_va_list'?
> >   typedef __builtin_va_list___2 va_list___2;
> >           ^~~~~~~~~~~~~~~~~~~~~
> >           __builtin_va_list
> >   note: '__builtin_va_list' declared here
> >   In file included from skeleton/profiler.bpf.c:3:
> >   .../selftests/bpf/tools/build/bpftool/vmlinux.h:84050:9: error: unknown type name
> >        '__builtin_va_list__ _2'; did you mean '__builtin_va_list'?
> >   typedef __builtin_va_list___2 va_list___2;
> >           ^~~~~~~~~~~~~~~~~~~~~
> >           __builtin_va_list
> >   note: '__builtin_va_list' declared here
> >
> > The error can be easily explained with after-dedup vmlinux btf:
> >   [21] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
> >   [2300] STRUCT '__va_list_tag' size=24 vlen=4
> >         'gp_offset' type_id=2 bits_offset=0
> >         'fp_offset' type_id=2 bits_offset=32
> >         'overflow_arg_area' type_id=32 bits_offset=64
> >         'reg_save_area' type_id=32 bits_offset=128
> >   [2308] TYPEDEF 'va_list' type_id=2309
> >   [2309] TYPEDEF '__builtin_va_list' type_id=2310
> >   [2310] ARRAY '(anon)' type_id=2300 index_type_id=21 nr_elems=1
> >
> >   [5289] PTR '(anon)' type_id=2308
> >   [158520] STRUCT 'warn_args' size=32 vlen=2
> >         'fmt' type_id=14 bits_offset=0
> >         'args' type_id=2308 bits_offset=64
> >   [27299] INT '__ARRAY_SIZE_TYPE__' size=4 bits_offset=0 nr_bits=32 encoding=(none)
> >   [34590] TYPEDEF '__builtin_va_list' type_id=34591
> >   [34591] ARRAY '(anon)' type_id=2300 index_type_id=27299 nr_elems=1
> >
> > Note that two array index_type_id's are different so the va_list and __builtin_va_list
> > will have two versions in the BTF. With this, vmlinux.h contains the following code,
> >   typedef __builtin_va_list va_list;
> >   typedef __builtin_va_list___2 va_list___2;
> > Since __builtin_va_list is a builtin type for the compiler,
> > libbpf does not generate
> >   typedef <...> __builtin_va_list
> > and this caused __builtin_va_list___2 is not defined and hence compilation error.
> > This happened when pahole is running with more than one jobs when parsing dwarf
> > and generating btfs.
> >
> > Function btf_encoder__encode_cu() is used to do btf encoding for
> > each cu. The function will try to find an "int" type for the cu
> > if it is available, otherwise, it will create a special type
> > with name __ARRAY_SIZE_TYPE__. For example,
> >   file1: yes 'int' type
> >   file2: no 'int' type
> >
> > In serial mode, file1 is processed first, followed by file2.
> > both will have 'int' type as the array index type since file2
> > will inherit the index type from file1.
> >
> > In parallel mode though, arrays in file1 will have index type 'int',
> > and arrays in file2 wil have index type '__ARRAY_SIZE_TYPE__'.
> > This will prevent some legitimate dedup and may have generated
> > vmlinux.h having compilation error.
> >
> > This patch fixed the issue by creating an 'int' type as the
> > array index type, so all array index type should be the same
> > for all cu's even in parallel mode.
> >
> > Signed-off-by: Yonghong Song <yhs@fb.com>
> > ---
> >  btf_encoder.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> >
> 
> LGTM, it should work reliably.
> 
> Acked-by: Andrii Nakryiko <andrii@kernel.org>

Applied and testing.

- Arnaldo
 
> > Changelog:
> >   v1 -> v2:
> >    - change creation of array index type to be 'int' type,
> >      the same as the type encoder tries to search in the current
> >      types.
> >
> > diff --git a/btf_encoder.c b/btf_encoder.c
> > index 1a42094..9e708e4 100644
> > --- a/btf_encoder.c
> > +++ b/btf_encoder.c
> > @@ -1460,7 +1460,8 @@ int btf_encoder__encode_cu(struct btf_encoder *encoder, struct cu *cu)
> >
> >                 bt.name = 0;
> >                 bt.bit_size = 32;
> > -               btf_encoder__add_base_type(encoder, &bt, "__ARRAY_SIZE_TYPE__");
> > +               bt.is_signed = true;
> > +               btf_encoder__add_base_type(encoder, &bt, "int");
> >                 encoder->has_index_type = true;
> >         }
> >
> > --
> > 2.30.2
> >

-- 

- Arnaldo