Re: [PATCH dwarves v4 0/4] Emit global variables in BTF

[Stephen Brennan <stephen.s.brennan@oracle.com>]

Andrii Nakryiko <andrii.nakryiko@gmail.com> writes:
> On Fri, Oct 4, 2024 at 2:21 PM Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
>>
>> On Fri, Oct 04, 2024 at 10:26:24AM -0700, Stephen Brennan wrote:
>> > Hi all,
>> >
>> > This is v4 of the series which adds global variables to pahole's generated BTF.
>> >
>> > Since v3:
>> >
>> > 1. Gathered Alan's Reviewed-by + Tested-by, and Jiri's Acked-by.
>> > 2. Consistently start shndx loops at 1, and use size_t.
>> > 3. Since patch 1 of v3 was already applied, I dropped it out of this series.
>> >
>> > v3: https://lore.kernel.org/dwarves/20241002235253.487251-1-stephen.s.brennan@oracle.com/
>> > v2: https://lore.kernel.org/dwarves/20240920081903.13473-1-stephen.s.brennan@oracle.com/
>> > v1: https://lore.kernel.org/dwarves/20240912190827.230176-1-stephen.s.brennan@oracle.com/
>> >
>> > Thanks everyone for your review, tests, and consideration!
>>
>> Looks ok, I run the existing regression tests:
>>
>> acme@x1:~/git/pahole$ tests/tests
>>   1: Validation of BTF encoding of functions; this may take some time: Ok
>>   2: Pretty printing of files using DWARF type information: Ok
>>   3: Parallel reproducible DWARF Loading/Serial BTF encoding: Ok
>> /home/acme/git/pahole
>> acme@x1:~/git/pahole$
>>
>> And now I'm building a kernel with clang + Thin LTO + Rust enabled in
>> the kernel to test other fixes I have merged and doing that with your
>> patch series.
>>
>> Its all in the next branch and will move to master later today or
>> tomorrow when I finish the clang+LTO+Rust tests.
>
> pahole-staging testing in libbpf CI started failing recently, can you
> please double-check and see if this was caused by these changes? They
> seem to be related to encoding BTF for per-CPU global variables, so
> might be relevant ([0] for full run logs)
>
>   #33      btf_dump:FAIL
>   libbpf: extern (var ksym) 'bpf_prog_active': not found in kernel BTF
>   libbpf: failed to load object 'kfunc_call_test_subprog'
>   libbpf: failed to load BPF skeleton 'kfunc_call_test_subprog': -22
>   test_subprog:FAIL:skel unexpected error: -22
>   #126/17  kfunc_call/subprog:FAIL
>   test_subprog_lskel:FAIL:skel unexpected error: -2
>   #126/18  kfunc_call/subprog_lskel:FAIL
>   #126     kfunc_call:FAIL
>   test_ksyms_module_lskel:FAIL:test_ksyms_module_lskel__open_and_load
> unexpected error: -2
>   #135/1   ksyms_module/lskel:FAIL
>   libbpf: extern (var ksym) 'bpf_testmod_ksym_percpu': not found in kernel BTF
>   libbpf: failed to load object 'test_ksyms_module'
>   libbpf: failed to load BPF skeleton 'test_ksyms_module': -22
>   test_ksyms_module_libbpf:FAIL:test_ksyms_module__open unexpected error: -22
>   #135/2   ksyms_module/libbpf:FAIL
>
>
>   [0] https://github.com/libbpf/libbpf/actions/runs/11204199648/job/31142297399#step:4:12480

Hi Andrii,

Thanks for the report.

The error: "'bpf_prog_active' not found in kernel BTF" sounds like it's
related to a bug that was present in v4 of this patch series:

https://lore.kernel.org/dwarves/ZwPob57HKYbfNpOH@x1/T/#t

Basically due to poor testing of a small refactor on my part, pahole
failed to emit almost all of the variables for BTF, so it would very
likely cause this error. And I think this broken commit may have been
hanging around in the git repository for the weekend, maybe Arnaldo can
confirm whether or not it was fixed up.

I cannot see the git SHA for the pahole branch which was used in this CI
run, so I can't say for sure. But I do see that the "tmp.master" branch
is now fixed up, so a re-run would verify whether this is the root
cause.

Thanks,
Stephen