Re: objtool "no non-local symbols" error with tip of tree LLVM

Nathan Chancellor
2022-05-16T22:49:41Z

On Mon, May 16, 2022 at 11:40:05PM +0200, Peter Zijlstra wrote:
> On Mon, May 16, 2022 at 01:47:15PM -0700, Nathan Chancellor wrote:
> > Hi Josh and Peter,
> >
> > After a recent change in LLVM [1], I see warnings (errors?) from objtool
> > when building x86_64 allmodconfig on 5.15 and 5.17:
> >
> > $ make -skj"$(nproc)" KCONFIG_ALLCONFIG=<(echo CONFIG_WERROR) LLVM=1 allmodconfig all
> > ...
> > mm/highmem.o: warning: objtool: no non-local symbols !?
> > mm/highmem.o: warning: objtool: gelf_update_symshndx: invalid section index
> > make[2]: *** [scripts/Makefile.build:288: mm/highmem.o] Error 255
> > ...
> > security/tomoyo/load_policy.o: warning: objtool: no non-local symbols !?
> > security/tomoyo/load_policy.o: warning: objtool: gelf_update_symshndx: invalid section index
> > make[3]: *** [scripts/Makefile.build:288: security/tomoyo/load_policy.o] Error 255
> > ...
> >
> > I don't see the same errors on x86_64 allmodconfig on mainline so I
> > bisected the 5.17 branch and came upon commit 4abff6d48dbc ("objtool:
> > Fix code relocs vs weak symbols"). I wanted to see what 5.17 might be
> > missing and came to commit ed53a0d97192 ("x86/alternative: Use
> > .ibt_endbr_seal to seal indirect calls") in mainline, which I think just
> > hides the issue for allmodconfig. I can reproduce this problem with a
> > more selective set of config values on mainline:
> >
> > $ make -skj"$(nproc)" LLVM=1 defconfig
> >
> > $ scripts/config -e KASAN -e SECURITY_TOMOYO -e SECURITY_TOMOYO_OMIT_USERSPACE_LOADER
> >
> > $ make -skj"$(nproc)" LLVM=1 olddefconfig security/tomoyo/load_policy.o
> > security/tomoyo/load_policy.o: warning: objtool: no non-local symbols !?
> > security/tomoyo/load_policy.o: warning: objtool: gelf_update_symshndx: invalid section index
> > make[3]: *** [scripts/Makefile.build:288: security/tomoyo/load_policy.o] Error 255
> > ...
> >
> > Looking at the object file, the '.text.asan.module_ctor' section has
> > disappeared.
> >
> > Before:
> >
> > $ llvm-nm -S security/tomoyo/load_policy.o
> > 0000000000000000 0000000000000001 t asan.module_ctor
> >
> > $ llvm-readelf -s security/tomoyo/load_policy.o
> >
> > Symbol table '.symtab' contains 4 entries:
> > Num: Value Size Type Bind Vis Ndx Name
> > 0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
> > 1: 0000000000000000 0 FILE LOCAL DEFAULT ABS load_policy.c
> > 2: 0000000000000000 0 SECTION LOCAL DEFAULT 3 .text.asan.module_ctor
> > 3: 0000000000000000 1 FUNC LOCAL DEFAULT 3 asan.module_ctor
> >
> > After:
> >
> > $ llvm-nm -S security/tomoyo/load_policy.o
> > 0000000000000000 0000000000000001 t asan.module_ctor
> >
> > $ llvm-readelf -s security/tomoyo/load_policy.o
> >
> > Symbol table '.symtab' contains 3 entries:
> > Num: Value Size Type Bind Vis Ndx Name
> > 0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
> > 1: 0000000000000000 0 FILE LOCAL DEFAULT ABS load_policy.c
> > 2: 0000000000000000 1 FUNC LOCAL DEFAULT 3 asan.module_ctor
> >
>
> The problem seems to be that we need to add a local symbols because LLVM
> helpfully stripped all unused section symbols.
>
> The way we do that, is by moving a the first non-local symbol to the
> end, thereby creating a hole where we can insert a new local symbol.
> Because ELF very helpfully mandates that local symbols must come before
> non-local symbols and keeps the symbols index of the first non-local in
> sh_info.
>
> Thing is, the above object files don't appear to have a non-local symbol
> so the swizzle thing isn't needed, and apparently the value in sh_info
> isn't valid either.
>
> Does something simple like this work? If not, I'll try and reproduce
> tomorrow, it shouldn't be too hard to fix.

That diff obviously gets rid of the "no non-local symbols" message but I
still see the "invalid section index" message. I'll be offline tomorrow
but if you have issues reproducing it, I'll be happy to help on
Wednesday. At the time I am writing this, apt.llvm.org packages have not
been updated to include that LLVM change I mentioned; hopefully they
will be soon.

Thanks for the quick response!
Nathan

> diff --git a/tools/objtool/elf.c b/tools/objtool/elf.c
> index 583a3ec987b5..baabf38a2a11 100644
> --- a/tools/objtool/elf.c
> +++ b/tools/objtool/elf.c
> @@ -618,8 +618,7 @@ static int elf_move_global_symbol(struct elf *elf, struct section *symtab,
>
> sym = find_symbol_by_index(elf, first_non_local);
> if (!sym) {
> - WARN("no non-local symbols !?");
> - return first_non_local;
> + return symtab->sh.sh_size / sizeof(sym->sym);
> }
>
> s = elf_getscn(elf->elf, symtab->idx);
>