Re: [PATCH dwarves 0/2] dwarf_loader: improve cus__merging_cu()

[Arnaldo Carvalho de Melo <acme@kernel.org>]

Em Fri, Apr 02, 2021 at 12:44:50PM -0700, Bill Wendling escreveu:
> I tried porting the .config we're using to the official branch and
> couldn't replicate the problem. It's probably something local.
> 
> On Thu, Apr 1, 2021 at 3:00 PM Yonghong Song <yhs@fb.com> wrote:
> > On 4/1/21 1:56 PM, Bill Wendling wrote:
> > > On Thu, Apr 1, 2021 at 12:35 PM Bill Wendling <morbo@google.com> wrote:
> > >>
> > >> On Wed, Mar 31, 2021 at 7:58 PM Yonghong Song <yhs@fb.com> wrote:
> > >>>
> > >>> Function cus__merging_cu() is introduced in Commit 39227909db3c
> > >>> ("dwarf_loader: Permit merging all DWARF CU's for clang LTO built
> > >>> binary") to test whether cross-cu references may happen.
> > >>> The original implementation anticipates compilation flags
> > >>> in dwarf, but later some concerns about binary size surfaced
> > >>> and the decision is to scan .debug_abbrev as a faster way
> > >>> to check cross-cu references. Also putting a note in vmlinux
> > >>> to indicate whether lto is enabled for built or not can
> > >>> provide a much faster way.
> > >>>
> > >>> This patch set implemented this two approaches, first
> > >>> checking the note (in Patch #2), if not found, then
> > >>> check .debug_abbrev (in Patch #1).
> > >>>
> > >>> Yonghong Song (2):
> > >>>    dwarf_loader: check .debug_abbrev for cross-cu references
> > >>>    dwarf_loader: check .notes section for lto build info
> > >>>
> > >>>   dwarf_loader.c | 76 ++++++++++++++++++++++++++++++++++++--------------
> > >>>   1 file changed, 55 insertions(+), 21 deletions(-)
> > >>>
> > >> With this series of patches, the compilation passes for me with
> > >> ThinLTO. You may add this if you like:
> > >>
> > >> Tested-by: Bill Wendling <morbo@google.com>
> > >
> > > I did notice these warnings following the "pahole -J .tmp_vmlinux.btf"
> > > command. I don't know the severity of them, but it might be good to
> > > investigate.
> > >
> > > $ ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > >    BTFIDS  vmlinux
> > > WARN: multiple IDs found for 'inode': 355, 8746 - using 355
> > > WARN: multiple IDs found for 'file': 588, 8779 - using 588
> > > WARN: multiple IDs found for 'path': 411, 8780 - using 411
> > > WARN: multiple IDs found for 'seq_file': 1414, 8836 - using 1414
> > > WARN: multiple IDs found for 'vm_area_struct': 538, 8873 - using 538
> > > WARN: multiple IDs found for 'task_struct': 28, 8880 - using 28
> > > WARN: multiple IDs found for 'inode': 355, 9484 - using 355
> > > WARN: multiple IDs found for 'file': 588, 9517 - using 588
> > > WARN: multiple IDs found for 'path': 411, 9518 - using 411
> > > WARN: multiple IDs found for 'seq_file': 1414, 9578 - using 1414
> > > WARN: multiple IDs found for 'vm_area_struct': 538, 9615 - using 538
> > > WARN: multiple IDs found for 'task_struct': 28, 9622 - using 28
> > > WARN: multiple IDs found for 'seq_file': 1414, 12223 - using 1414
> > > WARN: multiple IDs found for 'file': 588, 12237 - using 588
> > > WARN: multiple IDs found for 'path': 411, 12238 - using 411
> > > ...
> >
> > I didn't see it with my config. Maybe you can share your config file?

I'm seeing these here:

[acme@five bpf]$ rm -f ../build/bpf_clang_thin_lto/*vmlinu*
[acme@five bpf]$ time make -j28 LLVM=1 LLVM_IAS=1 O=../build/bpf_clang_thin_lto/ vmlinux
make[1]: Entering directory '/home/acme/git/build/bpf_clang_thin_lto'
  GEN     Makefile
  DESCEND  objtool
  DESCEND  bpf/resolve_btfids
  CALL    /home/acme/git/bpf/scripts/atomic/check-atomics.sh
  CALL    /home/acme/git/bpf/scripts/checksyscalls.sh
  CHK     include/generated/compile.h
  GEN     .version
  CHK     include/generated/compile.h
  UPD     include/generated/compile.h
  CC      init/version.o
  AR      init/built-in.a
  GEN     .tmp_initcalls.lds
  LTO     vmlinux.o
  OBJTOOL vmlinux.o
vmlinux.o: warning: objtool: aesni_gcm_init_avx_gen2()+0x12: unsupported stack pointer realignment
vmlinux.o: warning: objtool: aesni_gcm_enc_update_avx_gen2()+0x12: unsupported stack pointer realignment
vmlinux.o: warning: objtool: aesni_gcm_dec_update_avx_gen2()+0x12: unsupported stack pointer realignment
vmlinux.o: warning: objtool: aesni_gcm_finalize_avx_gen2()+0x12: unsupported stack pointer realignment
vmlinux.o: warning: objtool: aesni_gcm_init_avx_gen4()+0x12: unsupported stack pointer realignment
vmlinux.o: warning: objtool: aesni_gcm_enc_update_avx_gen4()+0x12: unsupported stack pointer realignment
vmlinux.o: warning: objtool: aesni_gcm_dec_update_avx_gen4()+0x12: unsupported stack pointer realignment
vmlinux.o: warning: objtool: aesni_gcm_finalize_avx_gen4()+0x12: unsupported stack pointer realignment
  MODPOST vmlinux.symvers
  MODINFO modules.builtin.modinfo
  GEN     modules.builtin
  LD      .tmp_vmlinux.btf
  BTF     .btf.vmlinux.bin.o
  LD      .tmp_vmlinux.kallsyms1
  KSYMS   .tmp_vmlinux.kallsyms1.S
  AS      .tmp_vmlinux.kallsyms1.S
  LD      .tmp_vmlinux.kallsyms2
  KSYMS   .tmp_vmlinux.kallsyms2.S
  AS      .tmp_vmlinux.kallsyms2.S
  LD      vmlinux
  BTFIDS  vmlinux
WARN: multiple IDs found for 'inode': 232, 28822 - using 232
WARN: multiple IDs found for 'file': 374, 28855 - using 374
WARN: multiple IDs found for 'path': 379, 28856 - using 379
WARN: multiple IDs found for 'vm_area_struct': 177, 28929 - using 177
WARN: multiple IDs found for 'task_struct': 97, 28966 - using 97
WARN: multiple IDs found for 'seq_file': 510, 29059 - using 510
WARN: multiple IDs found for 'inode': 232, 29345 - using 232
WARN: multiple IDs found for 'file': 374, 29429 - using 374
WARN: multiple IDs found for 'path': 379, 29430 - using 379
WARN: multiple IDs found for 'vm_area_struct': 177, 29471 - using 177
WARN: multiple IDs found for 'task_struct': 97, 29481 - using 97
WARN: multiple IDs found for 'seq_file': 510, 29512 - using 510
  SORTTAB vmlinux
  SYSMAP  System.map
make[1]: Leaving directory '/home/acme/git/build/bpf_clang_thin_lto'

[acme@five pahole]$ clang -v
clang version 11.0.0 (Fedora 11.0.0-2.fc33)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/10
Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/10
Selected GCC installation: /usr/lib/gcc/x86_64-redhat-linux/10
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
[acme@five pahole]$

[acme@five bpf]$ git log --oneline -10
49b9da70941c3c8a (HEAD -> bpf_perf_enable) kbuild: add an elfnote with type BUILD_COMPILER_LTO_INFO
5c4f082a143c786e kbuild: move LINUX_ELFNOTE_BUILD_SALT to elfnote.h
42c8b565decb3662 bpf: Introduce helpers to enable/disable perf event fds in a map
f73ea1eb4cce6637 (bpf-next/master, bpf-next/for-next) bpf: selftests: Specify CONFIG_DYNAMIC_FTRACE in the testing config
f07669df4c8df0b7 libbpf: Remove redundant semi-colon
6ac4c6f887f5a8ef bpf: Remove repeated struct btf_type declaration
2daae89666ad2532 bpf, cgroup: Delete repeated struct bpf_prog declaration
2ec9898e9c70b93a bpf: Remove unused parameter from ___bpf_prog_run
007bdc12d4b46656 bpf, selftests: test_maps generating unrecognized data section
82506665179209e4 tcp: reorder tcp_congestion_ops for better cache locality
[acme@five bpf]$

I'll try after a 'make mrproper'

- Arnaldo