Re: [PATCH v5 16/21] kbuild: generate KSYMTAB entries by modpost

[Greg Ungerer <gregungerer@westnet.com.au>]

Hi Masahiro,

On 15/5/23 01:27, Masahiro Yamada wrote:
> Commit 7b4537199a4a ("kbuild: link symbol CRCs at final link, removing
> CONFIG_MODULE_REL_CRCS") made modpost output CRCs in the same way
> whether the EXPORT_SYMBOL() is placed in *.c or *.S.
> 
> This commit applies a similar approach to the entire data structure of
> EXPORT_SYMBOL() for further cleanups. The EXPORT_SYMBOL() compilation
> is split into two stages.
> 
> When a source file is compiled, EXPORT_SYMBOL() is converted into a
> dummy symbol in the .export_symbol section.
> 
> For example,
> 
>      EXPORT_SYMBOL(foo);
>      EXPORT_SYMBOL_NS_GPL(bar, BAR_NAMESPACE);
> 
> will be encoded into the following assembly code:
> 
>      .section ".export_symbol","a"
>      __export_symbol__foo:
>              .asciz ""
>              .balign 8
>              .quad foo
>      .previous
> 
>      .section ".export_symbol","a"
>      __export_symbol_gpl_bar:
>              .asciz "BAR_NAMESPACE"
>              .balign 8
>              .quad bar
>      .previous
> 
> They are just markers to tell modpost the name, license, and namespace
> of the symbols. They will be dropped from the final vmlinux and modules
> because the *(.export_symbol) will go into /DISCARD/ in the linker script.
> 
> Then, modpost extracts all the information about EXPORT_SYMBOL() from the
> .export_symbol section, and generates C code:
> 
>      KSYMTAB_FUNC(foo, "", "");
>      KSYMTAB_FUNC(bar, "_gpl", "BAR_NAMESPACE");
> 
> KSYMTAB_FUNC() (or KSYMTAB_DATA() if it is data) is expanded to struct
> kernel_symbol that will be linked to the vmlinux or a module.
> 
> With this change, EXPORT_SYMBOL() works in the same way for *.c and *.S
> files, providing the following benefits.
> 
> [1] Deprecate EXPORT_DATA_SYMBOL()
> 
> In the old days, EXPORT_SYMBOL() was only available in C files. To export
> a symbol in *.S, EXPORT_SYMBOL() was placed in a separate *.c file.
> arch/arm/kernel/armksyms.c is one example written in the classic manner.
> 
> Commit 22823ab419d8 ("EXPORT_SYMBOL() for asm") removed this limitation.
> Since then, EXPORT_SYMBOL() can be placed close to the symbol definition
> in *.S files. It was a nice improvement.
> 
> However, as that commit mentioned, you need to use EXPORT_DATA_SYMBOL()
> for data objects on some architectures.
> 
> In the new approach, modpost checks symbol's type (STT_FUNC or not),
> and outputs KSYMTAB_FUNC() or KSYMTAB_DATA() accordingly.
> 
> There are only two users of EXPORT_DATA_SYMBOL:
> 
>    EXPORT_DATA_SYMBOL_GPL(empty_zero_page)    (arch/ia64/kernel/head.S)
>    EXPORT_DATA_SYMBOL(ia64_ivt)               (arch/ia64/kernel/ivt.S)
> 
> They are transformed as follows and output into .vmlinux.export.c
> 
>    KSYMTAB_DATA(empty_zero_page, "_gpl", "");
>    KSYMTAB_DATA(ia64_ivt, "", "");
> 
> The other EXPORT_SYMBOL users in ia64 assembly are output as
> KSYMTAB_FUNC().
> 
> EXPORT_DATA_SYMBOL() is now deprecated.
> 
> [2] merge <linux/export.h> and <asm-generic/export.h>
> 
> There are two similar header implementations:
> 
>    include/linux/export.h        for .c files
>    include/asm-generic/export.h  for .S files
> 
> Ideally, the functionality should be consistent between them, but they
> tend to diverge.
> 
> Commit 8651ec01daed ("module: add support for symbol namespaces.") did
> not support the namespace for *.S files.
> 
> This commit shifts the essential implementation part to C, which supports
> EXPORT_SYMBOL_NS() for *.S files.
> 
> <asm/export.h> and <asm-generic/export.h> will remain as a wrapper of
> <linux/export.h> for a while.
> 
> They will be removed after #include <asm/export.h> directives are all
> replaced with #include <linux/export.h>.
> 
> [3] Implement CONFIG_TRIM_UNUSED_KSYMS in one-pass algorithm (by a later commit)
> 
> When CONFIG_TRIM_UNUSED_KSYMS is enabled, Kbuild recursively traverses
> the directory tree to determine which EXPORT_SYMBOL to trim. If an
> EXPORT_SYMBOL turns out to be unused by anyone, Kbuild begins the
> second traverse, where some source files are recompiled with their
> EXPORT_SYMBOL() tuned into a no-op.
> 
> We can do this better now; modpost can selectively emit KSYMTAB entries
> that are really used by modules.
> 
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

This breaks building kernels with an m68k-uclinux-gcc toolchain that have
modules configured. Before this change they built and ran fine.
They build and run fine if CONFIG_MODULES is not set.

A few hundred errors like this spew out:

     scripts/mod/modpost -o Module.symvers -T modules.order vmlinux.o
     ERROR: modpost: vmlinux: .export_symbol section references '', but it does not seem to be an export symbol
     ERROR: modpost: vmlinux: .export_symbol section references '', but it does not seem to be an export symbol
     ERROR: modpost: vmlinux: .export_symbol section references '', but it does not seem to be an export symbol
     ...

This is still broken all the way through to the current 6.6-rc7, though the error
messages are slightly better:

     ERROR: modpost: vmlinux: local symbol 'system_state' was exported
     ERROR: modpost: vmlinux: local symbol 'static_key_initialized' was exported
     ERROR: modpost: vmlinux: local symbol 'reset_devices' was exported
     ...

I tried a bunch of different binutils and gcc versions (binutils-2.251 through
2.40 and gcc versions 8.3.0 through 12.3.0). If I compile with an m68k-linux
targeted toolchain then it works - no modpost processing problems.

nm reports the same information for symbols in both cases, eg:

$ m68k-uclinux-nm vmlinux.o | grep system_state
00000000 r __export_symbol_system_state
00000008 B system_state
0000000c d __UNIQUE_ID___addressable_system_state320

$ m68k-linux-nm vmlinux.o | grep system_state
00000000 r __export_symbol_system_state
00000008 B system_state
0000000c d __UNIQUE_ID___addressable_system_state320

Tracing in scripts/mod/modpost.c I see that for this symbol example
("system_state") that ELF_ST_BIND(sym->st_info) is 0x0 for the
m68k-uclinux toolchain case, so STB_LOCAL, whereas for the m68k-linux
case it is 0x1, so STB_GLOBAL.

Any idea what is going on here?

Regards
Greg