objtool "no non-local symbols" error with tip of tree LLVM

Nathan Chancellor
2022-05-16T21:07:41Z

Hi Josh and Peter,

After a recent change in LLVM [1], I see warnings (errors?) from objtool
when building x86_64 allmodconfig on 5.15 and 5.17:

$ make -skj"$(nproc)" KCONFIG_ALLCONFIG=<(echo CONFIG_WERROR) LLVM=1 allmodconfig all
...
mm/highmem.o: warning: objtool: no non-local symbols !?
mm/highmem.o: warning: objtool: gelf_update_symshndx: invalid section index
make[2]: *** [scripts/Makefile.build:288: mm/highmem.o] Error 255
...
security/tomoyo/load_policy.o: warning: objtool: no non-local symbols !?
security/tomoyo/load_policy.o: warning: objtool: gelf_update_symshndx: invalid section index
make[3]: *** [scripts/Makefile.build:288: security/tomoyo/load_policy.o] Error 255
...

I don't see the same errors on x86_64 allmodconfig on mainline so I
bisected the 5.17 branch and came upon commit 4abff6d48dbc ("objtool:
Fix code relocs vs weak symbols"). I wanted to see what 5.17 might be
missing and came to commit ed53a0d97192 ("x86/alternative: Use
.ibt_endbr_seal to seal indirect calls") in mainline, which I think just
hides the issue for allmodconfig. I can reproduce this problem with a
more selective set of config values on mainline:

$ make -skj"$(nproc)" LLVM=1 defconfig

$ scripts/config -e KASAN -e SECURITY_TOMOYO -e SECURITY_TOMOYO_OMIT_USERSPACE_LOADER

$ make -skj"$(nproc)" LLVM=1 olddefconfig security/tomoyo/load_policy.o
security/tomoyo/load_policy.o: warning: objtool: no non-local symbols !?
security/tomoyo/load_policy.o: warning: objtool: gelf_update_symshndx: invalid section index
make[3]: *** [scripts/Makefile.build:288: security/tomoyo/load_policy.o] Error 255
...

Looking at the object file, the '.text.asan.module_ctor' section has
disappeared.

Before:

$ llvm-nm -S security/tomoyo/load_policy.o
0000000000000000 0000000000000001 t asan.module_ctor

$ llvm-readelf -s security/tomoyo/load_policy.o

Symbol table '.symtab' contains 4 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
1: 0000000000000000 0 FILE LOCAL DEFAULT ABS load_policy.c
2: 0000000000000000 0 SECTION LOCAL DEFAULT 3 .text.asan.module_ctor
3: 0000000000000000 1 FUNC LOCAL DEFAULT 3 asan.module_ctor

After:

$ llvm-nm -S security/tomoyo/load_policy.o
0000000000000000 0000000000000001 t asan.module_ctor

$ llvm-readelf -s security/tomoyo/load_policy.o

Symbol table '.symtab' contains 3 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
1: 0000000000000000 0 FILE LOCAL DEFAULT ABS load_policy.c
2: 0000000000000000 1 FUNC LOCAL DEFAULT 3 asan.module_ctor

As far as I understand it, the kernel uses constructors for at least
KASAN and KCSAN, hence why that change impacts the kernel. Beyond that,
I am not really sure whether the LLVM change is problematic or objtool
just is not accounting for something that it should. I am happy to
provide any additional information that might help understand what is
going wrong here.

[1]: https://github.com/llvm/llvm-project/commit/badd088c57d7d18acd337b7868fe8c7974c88c5b

Cheers,
Nathan