Re: [RESEND] BTF is not generated for gcc-built kernel with the latest pahole

[Arnaldo Carvalho de Melo <acme@kernel.org>]

Em Tue, Aug 01, 2023 at 06:36:49PM +0100, Alan Maguire escreveu:
> On 01/08/2023 02:01, Masami Hiramatsu (Google) wrote:
> > On Mon, 31 Jul 2023 16:45:24 +0100
> > Alan Maguire <alan.maguire@oracle.com> wrote:
> > 
> >> On 27/07/2023 02:51, Masami Hiramatsu (Google) wrote:
> >>> On Thu, 27 Jul 2023 09:38:14 +0900
> >>> Masami Hiramatsu (Google) <mhiramat@kernel.org> wrote:
> >>>
> >>>>> Yep, BPF generation is more selective about what it emits in 1.25 to
> >>>>> avoid cases where a kernel function signature is ambiguous (multiple
> >>>>> functions of the same name with different signatures) or where it has
> >>>>> unexpected register use. You can observe this via pahole's --verbose
> >>>>> option (a lot of outut is emitted):
> >>>>>
> >>>>> In a built kernel directory (where unstripped vmlinux is present):
> >>>>> $ PAHOLE_FLAGS=$(./scripts/pahole_flags)
> >>>>> $ PAHOLE=/usr/local/bin/pahole
> >>>>> $ pahole --verbose -J $PAHOLE_FLAGS vmlinux > /tmp/pahole.out
> >>>>
> >>>> So this will generate BTF from vmlinux DWARF info.
> >>>>
> >>>>> If you want to investigate why a function has been left out, look for
> >>>>> "skipping" verbose output like this:
> >>>>>
> >>>>> skipping addition of 'access_error'(access_error) due to multiple
> >>>>> inconsistent function prototypes
> >>>>> skipping addition of
> >>>>> 'acpi_ex_convert_to_object_type_string'(acpi_ex_convert_to_object_type_string.isra.0)
> >>>>> due to unexpected register used for parameter
> >>>>
> >>>> Ah, that's nice. Let me try.
> >>>
> >>> $ pahole --version 
> >>> v1.23
> >>>
> >>
> >> shouldn't this be v1.25? Is it possible pahole is picking up the wrong
> >> libdwarves? what does "ldd pahole" say?
> > 
> > Here it is.
> > $ ldd pahole 
> > 	linux-vdso.so.1 (0x00007ffd6b1e2000)
> > 	libdwarves_reorganize.so.1 => /opt/local/pahole/libdwarves_reorganize.so.1 (0x00007f1ddaad9000)
> > 	libdwarves.so.1 => /opt/local/pahole/libdwarves.so.1 (0x00007f1ddaa72000)
> > 	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1dda82a000)
> > 	libdw.so.1 => /usr/local/lib/x86_64-linux-gnu/libdw.so.1 (0x00007f1dda78c000)
> > 	libelf.so.1 => /usr/local/lib/x86_64-linux-gnu/libelf.so.1 (0x00007f1dda771000)
> > 	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f1dda753000)
> > 	/lib64/ld-linux-x86-64.so.2 (0x00007f1ddaaef000)
> > 	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f1dda74e000)
> > 	liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f1dda723000)
> > 	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1dda71e000)
> > 
> > Maybe libdwarvers is not related, I could build pahole v1.23 and v1.22 without
> > sync the submodule.
> > 
> > I also confirmed the same issue on Ubuntu 22.04's combination, which
> > update pahole from v1.22 to v1.25 recently. (but gcc is still v11.3)
> > 
> >             gcc-11.3 | clang-16.0
> > ---------------------------+---------------- 
> > v1.22          OK         OK
> > (ubuntu)
> > v1.22          OK         OK
> > (local)
> > v1.23          NG         OK
> > (local)
> > v1.24          NG         -
> > (local)
> > v1.25          NG         -
> > (ubuntu)
> > v1.25          NG         OK
> > (local)
> > 
> > So, as far as I checked, there is something wrong between v1.22 and v1.23
> > which is also related to gcc-11.3.
> > 
> 
> One thing that is notable about gcc 11 is that I believe it's the first
> gcc release to emit DWARF5 by default. I wonder if it's possible that
> the kernel emitted DWARF5, but pahole was built with libraries that
> didn't support it yet? Not sure how that fits with the fact that

To check that please tweak:

⬢[acme@toolbox perf-tools-next]$ grep DWARF ../build/v6.2-rc5+/.config
CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y
# CONFIG_DEBUG_INFO_DWARF4 is not set
# CONFIG_DEBUG_INFO_DWARF5 is not set
⬢[acme@toolbox perf-tools-next]$

i.e. disable CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT and enable
CONFIG_DEBUG_INFO_DWARF4.

- Arnaldo

> pahole v1.22 works though. If you have a chance, it might be worth
> experimenting with your kernel .config to specify DWARF4 to see if
> that makes a difference.
> 
> You can check DWARF version associated with CUs with
> 
> readelf --debug-dump=info vmlinux |grep -A 2 'Compilation Unit'