[PATCH 0/4] kbuild: fix and clean-up after avoiding kallsyms weak reference

[PATCH 0/4] kbuild: fix and clean-up after avoiding kallsyms weak reference

[Masahiro Yamada]

I noticed a compile-time regression after commit 951bcae6c5a0
("kallsyms: Avoid weak references for kallsyms symbols").

1/4 fixes the unneeded kallsyms step 3.

2/4, 3/4, and 4/4 are follow-up cleanups.



Masahiro Yamada (4):
  kbuild: avoid unneeded kallsyms step 3
  kbuild: change scripts/mksysmap into sed script
  kbuild: fix shortlog for AS in link-vmlinux.sh
  kbuild: remove PROVIDE() for kallsyms symbols

 include/asm-generic/vmlinux.lds.h | 19 ------------
 kernel/kallsyms_internal.h        |  5 ----
 scripts/kallsyms.c                |  6 ----
 scripts/link-vmlinux.sh           | 49 ++++++++++++++++---------------
 scripts/mksysmap                  | 28 ++++--------------
 5 files changed, 32 insertions(+), 75 deletions(-)

-- 
2.40.1

[PATCH 1/4] kbuild: avoid unneeded kallsyms step 3

[Masahiro Yamada]

Since commit 951bcae6c5a0 ("kallsyms: Avoid weak references for kallsyms
symbols"), the kallsyms step 3 always occurs.

You can compare the build logs.

[Before 951bcae6c5a0]

  $ git checkout 951bcae6c5a0^
  $ make defconfig all
       [ snip ]
    LD      .tmp_vmlinux.kallsyms1
    NM      .tmp_vmlinux.kallsyms1.syms
    KSYMS   .tmp_vmlinux.kallsyms1.S
    AS      .tmp_vmlinux.kallsyms1.S
    LD      .tmp_vmlinux.kallsyms2
    NM      .tmp_vmlinux.kallsyms2.syms
    KSYMS   .tmp_vmlinux.kallsyms2.S
    AS      .tmp_vmlinux.kallsyms2.S
    LD      vmlinux

[After 951bcae6c5a0]

  $ git checkout 951bcae6c5a0
  $ make defconfig all
       [ snip ]
    LD      .tmp_vmlinux.kallsyms1
    NM      .tmp_vmlinux.kallsyms1.syms
    KSYMS   .tmp_vmlinux.kallsyms1.S
    AS      .tmp_vmlinux.kallsyms1.S
    LD      .tmp_vmlinux.kallsyms2
    NM      .tmp_vmlinux.kallsyms2.syms
    KSYMS   .tmp_vmlinux.kallsyms2.S
    AS      .tmp_vmlinux.kallsyms2.S
    LD      .tmp_vmlinux.kallsyms3             # should not happen
    NM      .tmp_vmlinux.kallsyms3.syms        # should not happen
    KSYMS   .tmp_vmlinux.kallsyms3.S           # should not happen
    AS      .tmp_vmlinux.kallsyms3.S           # should not happen
    LD      vmlinux

The resulting vmlinux is correct, but it always requires an additional
linking step.

The symbols produced by kallsyms are excluded from kallsyms itself
because they were previously missing in step 1. With those symbols
excluded, the numbers of symbols matched between step 1 and step 2,
eliminating the need for step 3. Now, this has a negative effect.

Since 951bcae6c5a0, the PROVIDE() directives provide the fallback
definitions, which are not trimmed from the sysmap in step 1 because
${kallsymso_prev} is empty at this point.

In step2, ${kallsymso_prev} is set, and the kallsyms_* symbols are
trimmed again from the sysmap.

Due to the table size difference between step 1 and step 2 (the former
is larger due to the presence of kallsyms_*), step 3 is triggered.

Now the kallsyms_* symbols are always linked, let's stop omitting them
from kallsyms. This avoids unnecessary step 3.

Fixes: 951bcae6c5a0 ("kallsyms: Avoid weak references for kallsyms symbols")
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

 scripts/link-vmlinux.sh |  6 +++---
 scripts/mksysmap        | 11 +----------
 2 files changed, 4 insertions(+), 13 deletions(-)

diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index 7862a8101747..b0d39a927fbc 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -179,7 +179,7 @@ kallsyms_step()
 	kallsyms_S=${kallsyms_vmlinux}.S
 
 	vmlinux_link ${kallsyms_vmlinux} "${kallsymso_prev}" ${btf_vmlinux_bin_o}
-	mksysmap ${kallsyms_vmlinux} ${kallsyms_vmlinux}.syms ${kallsymso_prev}
+	mksysmap ${kallsyms_vmlinux} ${kallsyms_vmlinux}.syms
 	kallsyms ${kallsyms_vmlinux}.syms ${kallsyms_S}
 
 	info AS ${kallsyms_S}
@@ -193,7 +193,7 @@ kallsyms_step()
 mksysmap()
 {
 	info NM ${2}
-	${CONFIG_SHELL} "${srctree}/scripts/mksysmap" ${1} ${2} ${3}
+	${CONFIG_SHELL} "${srctree}/scripts/mksysmap" ${1} ${2}
 }
 
 sorttable()
@@ -282,7 +282,7 @@ if is_enabled CONFIG_DEBUG_INFO_BTF && is_enabled CONFIG_BPF; then
 	${RESOLVE_BTFIDS} vmlinux
 fi
 
-mksysmap vmlinux System.map ${kallsymso}
+mksysmap vmlinux System.map
 
 if is_enabled CONFIG_BUILDTIME_TABLE_SORT; then
 	info SORTTAB vmlinux
diff --git a/scripts/mksysmap b/scripts/mksysmap
index 57ff5656d566..e46bafe333bd 100755
--- a/scripts/mksysmap
+++ b/scripts/mksysmap
@@ -4,7 +4,7 @@
 # tools to retrieve the actual addresses of symbols in the kernel.
 #
 # Usage
-# mksysmap vmlinux System.map [exclude]
+# mksysmap vmlinux System.map
 
 
 #####
@@ -92,13 +92,4 @@ ${NM} -n ${1} | sed >${2} -e "
 # ppc stub
 /\.long_branch\./d
 /\.plt_branch\./d
-
-# ---------------------------------------------------------------------------
-# Ignored kallsyms symbols
-#
-# If the 3rd parameter exists, symbols from it will be omitted from the output.
-# This makes kallsyms have the identical symbol lists in the step 1 and 2.
-# Without this, the step2 would get new symbols generated by scripts/kallsyms.c
-# when CONFIG_KALLSYMS_ALL is enabled. That might require one more pass.
-$(if [ $# -ge 3 ]; then ${NM} ${3} | sed -n '/ U /!s:.* \([^ ]*\)$:/ \1$/d:p'; fi)
 "
-- 
2.40.1

[PATCH 2/4] kbuild: change scripts/mksysmap into sed script

[Masahiro Yamada]

The previous commit removed the subshell execution from scripts/mksysmap,
which is now simple enough to become a sed script.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

 scripts/link-vmlinux.sh |  2 +-
 scripts/mksysmap        | 19 ++++++-------------
 2 files changed, 7 insertions(+), 14 deletions(-)

diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index b0d39a927fbc..c22a213ea6a9 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -193,7 +193,7 @@ kallsyms_step()
 mksysmap()
 {
 	info NM ${2}
-	${CONFIG_SHELL} "${srctree}/scripts/mksysmap" ${1} ${2}
+	${NM} -n "${1}" | "${srctree}/scripts/mksysmap" > "${2}"
 }
 
 sorttable()
diff --git a/scripts/mksysmap b/scripts/mksysmap
index e46bafe333bd..c12723a04655 100755
--- a/scripts/mksysmap
+++ b/scripts/mksysmap
@@ -1,22 +1,16 @@
-#!/bin/sh -x
-# Based on the vmlinux file create the System.map file
+#!/bin/sed -f
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# sed script to filter out symbols that are not needed for System.map,
+# or not suitable for kallsyms. The input should be 'nm -n <file>'.
+#
 # System.map is used by module-init tools and some debugging
 # tools to retrieve the actual addresses of symbols in the kernel.
 #
-# Usage
-# mksysmap vmlinux System.map
-
-
-#####
-# Generate System.map (actual filename passed as second argument)
-# The following refers to the symbol type as per nm(1).
-
 # readprofile starts reading symbols when _stext is found, and
 # continue until it finds a symbol which is not either of 'T', 't',
 # 'W' or 'w'.
 #
-
-${NM} -n ${1} | sed >${2} -e "
 # ---------------------------------------------------------------------------
 # Ignored symbol types
 #
@@ -92,4 +86,3 @@ ${NM} -n ${1} | sed >${2} -e "
 # ppc stub
 /\.long_branch\./d
 /\.plt_branch\./d
-"
-- 
2.40.1

[PATCH 3/4] kbuild: fix shortlog for AS in link-vmlinux.sh

[Masahiro Yamada]

In convention, the short log prints the output file, not the input file.

Change the suffix for 'AS' since it assembles *.S into *.o.

[Before]

  LD      .tmp_vmlinux.kallsyms1
  NM      .tmp_vmlinux.kallsyms1.syms
  KSYMS   .tmp_vmlinux.kallsyms1.S
  AS      .tmp_vmlinux.kallsyms1.S
  LD      .tmp_vmlinux.kallsyms2
  NM      .tmp_vmlinux.kallsyms2.syms
  KSYMS   .tmp_vmlinux.kallsyms2.S
  AS      .tmp_vmlinux.kallsyms2.S
  LD      vmlinux

[After]

  LD      .tmp_vmlinux.kallsyms1
  NM      .tmp_vmlinux.kallsyms1.syms
  KSYMS   .tmp_vmlinux.kallsyms1.S
  AS      .tmp_vmlinux.kallsyms1.o
  LD      .tmp_vmlinux.kallsyms2
  NM      .tmp_vmlinux.kallsyms2.syms
  KSYMS   .tmp_vmlinux.kallsyms2.S
  AS      .tmp_vmlinux.kallsyms2.o
  LD      vmlinux

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

 scripts/link-vmlinux.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index c22a213ea6a9..7aca51b24e9f 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -182,7 +182,7 @@ kallsyms_step()
 	mksysmap ${kallsyms_vmlinux} ${kallsyms_vmlinux}.syms
 	kallsyms ${kallsyms_vmlinux}.syms ${kallsyms_S}
 
-	info AS ${kallsyms_S}
+	info AS ${kallsymso}
 	${CC} ${NOSTDINC_FLAGS} ${LINUXINCLUDE} ${KBUILD_CPPFLAGS} \
 	      ${KBUILD_AFLAGS} ${KBUILD_AFLAGS_KERNEL} \
 	      -c -o ${kallsymso} ${kallsyms_S}
-- 
2.40.1

[PATCH 4/4] kbuild: remove PROVIDE() for kallsyms symbols

[Masahiro Yamada]

This reimplements commit 951bcae6c5a0 ("kallsyms: Avoid weak references
for kallsyms symbols").

I am not a big fan of PROVIDE() because it always satisfies the linker
even in situations that should result in a link error. In other words,
it can potentially shift a compile-time error into a run-time error.

Duplicating kallsyms_* in vmlinux.lds.h also reduces maintainability.

I shuffled scripts/link-vmlinux.sh to prepend one more kallsyms step.
The following logs illustrates how it works.

[Before]

    LD      .tmp_vmlinux.kallsyms1
    NM      .tmp_vmlinux.kallsyms1.syms
    KSYMS   .tmp_vmlinux.kallsyms1.S
    AS      .tmp_vmlinux.kallsyms1.o
    LD      .tmp_vmlinux.kallsyms2
    NM      .tmp_vmlinux.kallsyms2.syms
    KSYMS   .tmp_vmlinux.kallsyms2.S
    AS      .tmp_vmlinux.kallsyms2.o
    LD      vmlinux

[After]

    KSYMS   .tmp_vmlinux.kallsyms0.S          # added
    AS      .tmp_vmlinux.kallsyms0.o          # added
    LD      .tmp_vmlinux.kallsyms1
    NM      .tmp_vmlinux.kallsyms1.syms
    KSYMS   .tmp_vmlinux.kallsyms1.S
    AS      .tmp_vmlinux.kallsyms1.o
    LD      .tmp_vmlinux.kallsyms2
    NM      .tmp_vmlinux.kallsyms2.syms
    KSYMS   .tmp_vmlinux.kallsyms2.S
    AS      .tmp_vmlinux.kallsyms2.o
    LD      vmlinux

Step 0 takes /dev/null as input, and generates .tmp_vmlinux.kallsyms0.o,
which has a valid kallsyms format with zero symbols, and can be linked
to vmlinux. Since it is really small, the added compile-time cost is
negligible.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

 include/asm-generic/vmlinux.lds.h | 19 -------------
 kernel/kallsyms_internal.h        |  5 ----
 scripts/kallsyms.c                |  6 -----
 scripts/link-vmlinux.sh           | 45 ++++++++++++++++---------------
 4 files changed, 24 insertions(+), 51 deletions(-)

diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index 870753fbb123..9752eb420ffa 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -448,30 +448,11 @@
 #endif
 #endif
 
-/*
- * Some symbol definitions will not exist yet during the first pass of the
- * link, but are guaranteed to exist in the final link. Provide preliminary
- * definitions that will be superseded in the final link to avoid having to
- * rely on weak external linkage, which requires a GOT when used in position
- * independent code.
- */
-#define PRELIMINARY_SYMBOL_DEFINITIONS					\
-	PROVIDE(kallsyms_addresses = .);				\
-	PROVIDE(kallsyms_offsets = .);					\
-	PROVIDE(kallsyms_names = .);					\
-	PROVIDE(kallsyms_num_syms = .);					\
-	PROVIDE(kallsyms_relative_base = .);				\
-	PROVIDE(kallsyms_token_table = .);				\
-	PROVIDE(kallsyms_token_index = .);				\
-	PROVIDE(kallsyms_markers = .);					\
-	PROVIDE(kallsyms_seqs_of_names = .);
-
 /*
  * Read only Data
  */
 #define RO_DATA(align)							\
 	. = ALIGN((align));						\
-	PRELIMINARY_SYMBOL_DEFINITIONS					\
 	.rodata           : AT(ADDR(.rodata) - LOAD_OFFSET) {		\
 		__start_rodata = .;					\
 		*(.rodata) *(.rodata.*)					\
diff --git a/kernel/kallsyms_internal.h b/kernel/kallsyms_internal.h
index 85480274fc8f..925f2ab22639 100644
--- a/kernel/kallsyms_internal.h
+++ b/kernel/kallsyms_internal.h
@@ -4,11 +4,6 @@
 
 #include <linux/types.h>
 
-/*
- * These will be re-linked against their real values during the second link
- * stage. Preliminary values must be provided in the linker script using the
- * PROVIDE() directive so that the first link stage can complete successfully.
- */
 extern const unsigned long kallsyms_addresses[];
 extern const int kallsyms_offsets[];
 extern const u8 kallsyms_names[];
diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c
index 653b92f6d4c8..6b90f52fd707 100644
--- a/scripts/kallsyms.c
+++ b/scripts/kallsyms.c
@@ -246,12 +246,6 @@ static void shrink_table(void)
 		}
 	}
 	table_cnt = pos;
-
-	/* When valid symbol is not registered, exit to error */
-	if (!table_cnt) {
-		fprintf(stderr, "No valid symbol.\n");
-		exit(1);
-	}
 }
 
 static void read_map(const char *in)
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index 7aca51b24e9f..242a92e24f20 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -165,27 +165,25 @@ kallsyms()
 		kallsymopt="${kallsymopt} --lto-clang"
 	fi
 
-	info KSYMS ${2}
-	scripts/kallsyms ${kallsymopt} ${1} > ${2}
+	info KSYMS "${2}.S"
+	scripts/kallsyms ${kallsymopt} "${1}" > "${2}.S"
+
+	info AS "${2}.o"
+	${CC} ${NOSTDINC_FLAGS} ${LINUXINCLUDE} ${KBUILD_CPPFLAGS} \
+	      ${KBUILD_AFLAGS} ${KBUILD_AFLAGS_KERNEL} \
+	      -c -o "${2}.o" "${2}.S"
+	kallsymso=${2}.o
 }
 
 # Perform one step in kallsyms generation, including temporary linking of
 # vmlinux.
 kallsyms_step()
 {
-	kallsymso_prev=${kallsymso}
 	kallsyms_vmlinux=.tmp_vmlinux.kallsyms${1}
-	kallsymso=${kallsyms_vmlinux}.o
-	kallsyms_S=${kallsyms_vmlinux}.S
 
-	vmlinux_link ${kallsyms_vmlinux} "${kallsymso_prev}" ${btf_vmlinux_bin_o}
-	mksysmap ${kallsyms_vmlinux} ${kallsyms_vmlinux}.syms
-	kallsyms ${kallsyms_vmlinux}.syms ${kallsyms_S}
-
-	info AS ${kallsymso}
-	${CC} ${NOSTDINC_FLAGS} ${LINUXINCLUDE} ${KBUILD_CPPFLAGS} \
-	      ${KBUILD_AFLAGS} ${KBUILD_AFLAGS_KERNEL} \
-	      -c -o ${kallsymso} ${kallsyms_S}
+	vmlinux_link "${kallsyms_vmlinux}" "${kallsymso}" "${btf_vmlinux_bin_o}"
+	mksysmap "${kallsyms_vmlinux}" "${kallsyms_vmlinux}.syms"
+	kallsyms "${kallsyms_vmlinux}.syms" "${kallsyms_vmlinux}"
 }
 
 # Create map file with all symbols from ${1}
@@ -235,15 +233,15 @@ if is_enabled CONFIG_DEBUG_INFO_BTF; then
 fi
 
 kallsymso=""
-kallsymso_prev=""
-kallsyms_vmlinux=""
 if is_enabled CONFIG_KALLSYMS; then
 
 	# kallsyms support
 	# Generate section listing all symbols and add it into vmlinux
-	# It's a three step process:
+	# It's a four step process:
+	# 0)  Generate a dummy __kallsyms, which has zero symbols, but a valid
+	#     format.
 	# 1)  Link .tmp_vmlinux.kallsyms1 so it has all symbols and sections,
-	#     but __kallsyms is empty.
+	#     with a dummy __kallsyms.
 	#     Running kallsyms on that gives us .tmp_kallsyms1.o with
 	#     the right size
 	# 2)  Link .tmp_vmlinux.kallsyms2 so it now has a __kallsyms section of
@@ -262,13 +260,18 @@ if is_enabled CONFIG_KALLSYMS; then
 	# a)  Verify that the System.map from vmlinux matches the map from
 	#     ${kallsymso}.
 
-	kallsyms_step 1
-	kallsyms_step 2
+	# step 0
+	kallsyms /dev/null .tmp_vmlinux.kallsyms0
 
-	# step 3
-	size1=$(${CONFIG_SHELL} "${srctree}/scripts/file-size.sh" ${kallsymso_prev})
+	# step 1
+	kallsyms_step 1
+	size1=$(${CONFIG_SHELL} "${srctree}/scripts/file-size.sh" ${kallsymso})
+
+	# step 2
+	kallsyms_step 2
 	size2=$(${CONFIG_SHELL} "${srctree}/scripts/file-size.sh" ${kallsymso})
 
+	# step 3
 	if [ $size1 -ne $size2 ] || [ -n "${KALLSYMS_EXTRA_PASS}" ]; then
 		kallsyms_step 3
 	fi
-- 
2.40.1

Re: [PATCH 4/4] kbuild: remove PROVIDE() for kallsyms symbols

[kernel test robot]

Hi Masahiro,

kernel test robot noticed the following build errors:

[auto build test ERROR on masahiroy-kbuild/for-next]
[also build test ERROR on linus/master masahiroy-kbuild/fixes next-20240521]
[cannot apply to v6.9]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Masahiro-Yamada/kbuild-avoid-unneeded-kallsyms-step-3/20240520-204508
base:   https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git for-next
patch link:    https://lore.kernel.org/r/20240520124212.2351033-5-masahiroy%40kernel.org
patch subject: [PATCH 4/4] kbuild: remove PROVIDE() for kallsyms symbols
config: x86_64-rhel-8.3-bpf (https://download.01.org/0day-ci/archive/20240521/202405211448.fglQOQ9W-lkp@intel.com/config)
compiler: gcc-13 (Ubuntu 13.2.0-4ubuntu3) 13.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240521/202405211448.fglQOQ9W-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202405211448.fglQOQ9W-lkp@intel.com/

All errors (new ones prefixed by >>):

   ld: vmlinux.o: in function `get_symbol_offset':
>> kernel/kallsyms.c:125:(.text+0x2111af): undefined reference to `kallsyms_markers'
>> ld: kernel/kallsyms.c:125:(.text+0x2111b5): undefined reference to `kallsyms_names'
   ld: kernel/kallsyms.c:146:(.text+0x2111eb): undefined reference to `kallsyms_names'
   ld: vmlinux.o: in function `get_symbol_pos':
>> kernel/kallsyms.c:330:(.text+0x211218): undefined reference to `kallsyms_relative_base'
>> ld: kernel/kallsyms.c:330:(.text+0x211226): undefined reference to `kallsyms_num_syms'
   ld: vmlinux.o: in function `kallsyms_sym_address':
>> kernel/kallsyms.c:159:(.text+0x211245): undefined reference to `kallsyms_offsets'
>> ld: kernel/kallsyms.c:159:(.text+0x211285): undefined reference to `kallsyms_offsets'
   ld: kernel/kallsyms.c:159:(.text+0x211296): undefined reference to `kallsyms_offsets'
   ld: kernel/kallsyms.c:159:(.text+0x2112b7): undefined reference to `kallsyms_offsets'
   ld: kernel/kallsyms.c:159:(.text+0x2112ec): undefined reference to `kallsyms_offsets'
   ld: vmlinux.o: in function `get_ksymbol_mod':
>> kernel/kallsyms.c:623:(.text+0x211434): undefined reference to `kallsyms_num_syms'
   ld: vmlinux.o: in function `kallsyms_expand_symbol':
>> kernel/kallsyms.c:50:(.text+0x21167d): undefined reference to `kallsyms_names'
   ld: kernel/kallsyms.c:51:(.text+0x211684): undefined reference to `kallsyms_names'
>> ld: kernel/kallsyms.c:73:(.text+0x2116b6): undefined reference to `kallsyms_token_index'
>> ld: kernel/kallsyms.c:73:(.text+0x2116bf): undefined reference to `kallsyms_token_table'
   ld: kernel/kallsyms.c:77:(.text+0x2116c6): undefined reference to `kallsyms_token_table'
   ld: vmlinux.o: in function `update_iter':
   kernel/kallsyms.c:740:(.text+0x211757): undefined reference to `kallsyms_num_syms'
   ld: vmlinux.o: in function `kallsyms_sym_address':
   kernel/kallsyms.c:159:(.text+0x211782): undefined reference to `kallsyms_offsets'
>> ld: kernel/kallsyms.c:163:(.text+0x211790): undefined reference to `kallsyms_relative_base'
   ld: vmlinux.o: in function `kallsyms_get_symbol_type':
   kernel/kallsyms.c:108:(.text+0x21179e): undefined reference to `kallsyms_names'
   ld: kernel/kallsyms.c:108:(.text+0x2117aa): undefined reference to `kallsyms_token_index'
   ld: vmlinux.o: in function `get_ksymbol_core':
>> kernel/kallsyms.c:693:(.text+0x2117b1): undefined reference to `kallsyms_token_table'
   ld: vmlinux.o: in function `kallsyms_lookup_names':
   kernel/kallsyms.c:218:(.text+0x211a48): undefined reference to `kallsyms_num_syms'
   ld: vmlinux.o: in function `get_symbol_seq':
>> kernel/kallsyms.c:203:(.text+0x211a83): undefined reference to `kallsyms_seqs_of_names'
>> ld: kernel/kallsyms.c:203:(.text+0x211a90): undefined reference to `kallsyms_seqs_of_names'
   ld: kernel/kallsyms.c:203:(.text+0x211a9c): undefined reference to `kallsyms_seqs_of_names'
   ld: kernel/kallsyms.c:203:(.text+0x211af2): undefined reference to `kallsyms_seqs_of_names'
   ld: kernel/kallsyms.c:203:(.text+0x211aff): undefined reference to `kallsyms_seqs_of_names'
   ld: vmlinux.o:kernel/kallsyms.c:203: more undefined references to `kallsyms_seqs_of_names' follow
   ld: vmlinux.o: in function `kallsyms_sym_address':
   kernel/kallsyms.c:159:(.text+0x211fb0): undefined reference to `kallsyms_offsets'
   ld: kernel/kallsyms.c:163:(.text+0x211fbe): undefined reference to `kallsyms_relative_base'
   ld: vmlinux.o: in function `get_symbol_seq':
   kernel/kallsyms.c:203:(.text+0x212049): undefined reference to `kallsyms_seqs_of_names'
   ld: kernel/kallsyms.c:203:(.text+0x212056): undefined reference to `kallsyms_seqs_of_names'
   ld: kernel/kallsyms.c:203:(.text+0x21205d): undefined reference to `kallsyms_seqs_of_names'
   ld: vmlinux.o: in function `kallsyms_sym_address':
   kernel/kallsyms.c:159:(.text+0x212071): undefined reference to `kallsyms_offsets'
   ld: kernel/kallsyms.c:163:(.text+0x21207f): undefined reference to `kallsyms_relative_base'
   ld: vmlinux.o: in function `kallsyms_on_each_symbol':
   kernel/kallsyms.c:293:(.text+0x2120e5): undefined reference to `kallsyms_num_syms'
   ld: vmlinux.o: in function `kallsyms_sym_address':
   kernel/kallsyms.c:159:(.text+0x21214b): undefined reference to `kallsyms_offsets'
   ld: kernel/kallsyms.c:163:(.text+0x212159): undefined reference to `kallsyms_relative_base'
   ld: kernel/kallsyms.c:163:(.text+0x212224): undefined reference to `kallsyms_relative_base'
   ld: vmlinux.o: in function `get_symbol_seq':
   kernel/kallsyms.c:203:(.text+0x212239): undefined reference to `kallsyms_seqs_of_names'
   ld: kernel/kallsyms.c:203:(.text+0x212240): undefined reference to `kallsyms_seqs_of_names'
   ld: kernel/kallsyms.c:203:(.text+0x212253): undefined reference to `kallsyms_seqs_of_names'
   ld: vmlinux.o: in function `kallsyms_sym_address':
   kernel/kallsyms.c:159:(.text+0x21225f): undefined reference to `kallsyms_offsets'
   ld: vmlinux.o: in function `crash_save_vmcoreinfo_init':
>> kernel/vmcore_info.c:214:(.init.text+0x429d6): undefined reference to `kallsyms_names'
>> ld: kernel/vmcore_info.c:215:(.init.text+0x429f0): undefined reference to `kallsyms_num_syms'
>> ld: kernel/vmcore_info.c:216:(.init.text+0x42a0a): undefined reference to `kallsyms_token_table'
>> ld: kernel/vmcore_info.c:217:(.init.text+0x42a24): undefined reference to `kallsyms_token_index'
>> ld: kernel/vmcore_info.c:219:(.init.text+0x42a3e): undefined reference to `kallsyms_offsets'
>> ld: kernel/vmcore_info.c:220:(.init.text+0x42a58): undefined reference to `kallsyms_relative_base'
   pahole: .tmp_vmlinux.btf: Invalid argument
   .btf.vmlinux.bin.o: file not recognized: file format not recognized


vim +125 kernel/kallsyms.c

^1da177e4c3f41 Linus Torvalds   2005-04-16   36  
ad6ccfad6f759a Manish Katiyar   2009-05-12   37  /*
ad6ccfad6f759a Manish Katiyar   2009-05-12   38   * Expand a compressed symbol data into the resulting uncompressed string,
e3f26752f0f8a6 Chen Gang        2013-04-15   39   * if uncompressed string is too long (>= maxlen), it will be truncated,
ad6ccfad6f759a Manish Katiyar   2009-05-12   40   * given the offset to where the symbol is in the compressed stream.
ad6ccfad6f759a Manish Katiyar   2009-05-12   41   */
e3f26752f0f8a6 Chen Gang        2013-04-15   42  static unsigned int kallsyms_expand_symbol(unsigned int off,
e3f26752f0f8a6 Chen Gang        2013-04-15   43  					   char *result, size_t maxlen)
^1da177e4c3f41 Linus Torvalds   2005-04-16   44  {
^1da177e4c3f41 Linus Torvalds   2005-04-16   45  	int len, skipped_first = 0;
cde26a6e17ec36 Masahiro Yamada  2020-02-02   46  	const char *tptr;
cde26a6e17ec36 Masahiro Yamada  2020-02-02   47  	const u8 *data;
^1da177e4c3f41 Linus Torvalds   2005-04-16   48  
ad6ccfad6f759a Manish Katiyar   2009-05-12   49  	/* Get the compressed symbol length from the first symbol byte. */
^1da177e4c3f41 Linus Torvalds   2005-04-16  @50  	data = &kallsyms_names[off];
^1da177e4c3f41 Linus Torvalds   2005-04-16   51  	len = *data;
^1da177e4c3f41 Linus Torvalds   2005-04-16   52  	data++;
73bbb94466fd3f Miguel Ojeda     2021-04-05   53  	off++;
73bbb94466fd3f Miguel Ojeda     2021-04-05   54  
73bbb94466fd3f Miguel Ojeda     2021-04-05   55  	/* If MSB is 1, it is a "big" symbol, so needs an additional byte. */
73bbb94466fd3f Miguel Ojeda     2021-04-05   56  	if ((len & 0x80) != 0) {
73bbb94466fd3f Miguel Ojeda     2021-04-05   57  		len = (len & 0x7F) | (*data << 7);
73bbb94466fd3f Miguel Ojeda     2021-04-05   58  		data++;
73bbb94466fd3f Miguel Ojeda     2021-04-05   59  		off++;
73bbb94466fd3f Miguel Ojeda     2021-04-05   60  	}
^1da177e4c3f41 Linus Torvalds   2005-04-16   61  
ad6ccfad6f759a Manish Katiyar   2009-05-12   62  	/*
ad6ccfad6f759a Manish Katiyar   2009-05-12   63  	 * Update the offset to return the offset for the next symbol on
ad6ccfad6f759a Manish Katiyar   2009-05-12   64  	 * the compressed stream.
ad6ccfad6f759a Manish Katiyar   2009-05-12   65  	 */
73bbb94466fd3f Miguel Ojeda     2021-04-05   66  	off += len;
^1da177e4c3f41 Linus Torvalds   2005-04-16   67  
ad6ccfad6f759a Manish Katiyar   2009-05-12   68  	/*
ad6ccfad6f759a Manish Katiyar   2009-05-12   69  	 * For every byte on the compressed symbol data, copy the table
ad6ccfad6f759a Manish Katiyar   2009-05-12   70  	 * entry for that byte.
ad6ccfad6f759a Manish Katiyar   2009-05-12   71  	 */
^1da177e4c3f41 Linus Torvalds   2005-04-16   72  	while (len) {
^1da177e4c3f41 Linus Torvalds   2005-04-16  @73  		tptr = &kallsyms_token_table[kallsyms_token_index[*data]];
^1da177e4c3f41 Linus Torvalds   2005-04-16   74  		data++;
^1da177e4c3f41 Linus Torvalds   2005-04-16   75  		len--;
^1da177e4c3f41 Linus Torvalds   2005-04-16   76  
^1da177e4c3f41 Linus Torvalds   2005-04-16   77  		while (*tptr) {
^1da177e4c3f41 Linus Torvalds   2005-04-16   78  			if (skipped_first) {
e3f26752f0f8a6 Chen Gang        2013-04-15   79  				if (maxlen <= 1)
e3f26752f0f8a6 Chen Gang        2013-04-15   80  					goto tail;
^1da177e4c3f41 Linus Torvalds   2005-04-16   81  				*result = *tptr;
^1da177e4c3f41 Linus Torvalds   2005-04-16   82  				result++;
e3f26752f0f8a6 Chen Gang        2013-04-15   83  				maxlen--;
^1da177e4c3f41 Linus Torvalds   2005-04-16   84  			} else
^1da177e4c3f41 Linus Torvalds   2005-04-16   85  				skipped_first = 1;
^1da177e4c3f41 Linus Torvalds   2005-04-16   86  			tptr++;
^1da177e4c3f41 Linus Torvalds   2005-04-16   87  		}
^1da177e4c3f41 Linus Torvalds   2005-04-16   88  	}
^1da177e4c3f41 Linus Torvalds   2005-04-16   89  
e3f26752f0f8a6 Chen Gang        2013-04-15   90  tail:
e3f26752f0f8a6 Chen Gang        2013-04-15   91  	if (maxlen)
^1da177e4c3f41 Linus Torvalds   2005-04-16   92  		*result = '\0';
^1da177e4c3f41 Linus Torvalds   2005-04-16   93  
ad6ccfad6f759a Manish Katiyar   2009-05-12   94  	/* Return to offset to the next symbol. */
^1da177e4c3f41 Linus Torvalds   2005-04-16   95  	return off;
^1da177e4c3f41 Linus Torvalds   2005-04-16   96  }
^1da177e4c3f41 Linus Torvalds   2005-04-16   97  
ad6ccfad6f759a Manish Katiyar   2009-05-12   98  /*
ad6ccfad6f759a Manish Katiyar   2009-05-12   99   * Get symbol type information. This is encoded as a single char at the
ad6ccfad6f759a Manish Katiyar   2009-05-12  100   * beginning of the symbol name.
ad6ccfad6f759a Manish Katiyar   2009-05-12  101   */
^1da177e4c3f41 Linus Torvalds   2005-04-16  102  static char kallsyms_get_symbol_type(unsigned int off)
^1da177e4c3f41 Linus Torvalds   2005-04-16  103  {
ad6ccfad6f759a Manish Katiyar   2009-05-12  104  	/*
ad6ccfad6f759a Manish Katiyar   2009-05-12  105  	 * Get just the first code, look it up in the token table,
ad6ccfad6f759a Manish Katiyar   2009-05-12  106  	 * and return the first char from this token.
ad6ccfad6f759a Manish Katiyar   2009-05-12  107  	 */
^1da177e4c3f41 Linus Torvalds   2005-04-16  108  	return kallsyms_token_table[kallsyms_token_index[kallsyms_names[off + 1]]];
^1da177e4c3f41 Linus Torvalds   2005-04-16  109  }
^1da177e4c3f41 Linus Torvalds   2005-04-16  110  
^1da177e4c3f41 Linus Torvalds   2005-04-16  111  
ad6ccfad6f759a Manish Katiyar   2009-05-12  112  /*
ad6ccfad6f759a Manish Katiyar   2009-05-12  113   * Find the offset on the compressed stream given and index in the
ad6ccfad6f759a Manish Katiyar   2009-05-12  114   * kallsyms array.
ad6ccfad6f759a Manish Katiyar   2009-05-12  115   */
^1da177e4c3f41 Linus Torvalds   2005-04-16  116  static unsigned int get_symbol_offset(unsigned long pos)
^1da177e4c3f41 Linus Torvalds   2005-04-16  117  {
aad094701c6355 Jan Beulich      2006-12-08  118  	const u8 *name;
73bbb94466fd3f Miguel Ojeda     2021-04-05  119  	int i, len;
^1da177e4c3f41 Linus Torvalds   2005-04-16  120  
ad6ccfad6f759a Manish Katiyar   2009-05-12  121  	/*
ad6ccfad6f759a Manish Katiyar   2009-05-12  122  	 * Use the closest marker we have. We have markers every 256 positions,
ad6ccfad6f759a Manish Katiyar   2009-05-12  123  	 * so that should be close enough.
ad6ccfad6f759a Manish Katiyar   2009-05-12  124  	 */
^1da177e4c3f41 Linus Torvalds   2005-04-16 @125  	name = &kallsyms_names[kallsyms_markers[pos >> 8]];
^1da177e4c3f41 Linus Torvalds   2005-04-16  126  
ad6ccfad6f759a Manish Katiyar   2009-05-12  127  	/*
ad6ccfad6f759a Manish Katiyar   2009-05-12  128  	 * Sequentially scan all the symbols up to the point we're searching
ad6ccfad6f759a Manish Katiyar   2009-05-12  129  	 * for. Every symbol is stored in a [<len>][<len> bytes of data] format,
ad6ccfad6f759a Manish Katiyar   2009-05-12  130  	 * so we just need to add the len to the current pointer for every
ad6ccfad6f759a Manish Katiyar   2009-05-12  131  	 * symbol we wish to skip.
ad6ccfad6f759a Manish Katiyar   2009-05-12  132  	 */
73bbb94466fd3f Miguel Ojeda     2021-04-05  133  	for (i = 0; i < (pos & 0xFF); i++) {
73bbb94466fd3f Miguel Ojeda     2021-04-05  134  		len = *name;
73bbb94466fd3f Miguel Ojeda     2021-04-05  135  
73bbb94466fd3f Miguel Ojeda     2021-04-05  136  		/*
73bbb94466fd3f Miguel Ojeda     2021-04-05  137  		 * If MSB is 1, it is a "big" symbol, so we need to look into
73bbb94466fd3f Miguel Ojeda     2021-04-05  138  		 * the next byte (and skip it, too).
73bbb94466fd3f Miguel Ojeda     2021-04-05  139  		 */
73bbb94466fd3f Miguel Ojeda     2021-04-05  140  		if ((len & 0x80) != 0)
73bbb94466fd3f Miguel Ojeda     2021-04-05  141  			len = ((len & 0x7F) | (name[1] << 7)) + 1;
73bbb94466fd3f Miguel Ojeda     2021-04-05  142  
73bbb94466fd3f Miguel Ojeda     2021-04-05  143  		name = name + len + 1;
73bbb94466fd3f Miguel Ojeda     2021-04-05  144  	}
^1da177e4c3f41 Linus Torvalds   2005-04-16  145  
^1da177e4c3f41 Linus Torvalds   2005-04-16  146  	return name - kallsyms_names;
^1da177e4c3f41 Linus Torvalds   2005-04-16  147  }
^1da177e4c3f41 Linus Torvalds   2005-04-16  148  
30f3bb09778de6 Zhen Lei         2022-11-15  149  unsigned long kallsyms_sym_address(int idx)
2213e9a66bb87d Ard Biesheuvel   2016-03-15  150  {
2213e9a66bb87d Ard Biesheuvel   2016-03-15  151  	if (!IS_ENABLED(CONFIG_KALLSYMS_BASE_RELATIVE))
2213e9a66bb87d Ard Biesheuvel   2016-03-15  152  		return kallsyms_addresses[idx];
2213e9a66bb87d Ard Biesheuvel   2016-03-15  153  
2213e9a66bb87d Ard Biesheuvel   2016-03-15  154  	/* values are unsigned offsets if --absolute-percpu is not in effect */
2213e9a66bb87d Ard Biesheuvel   2016-03-15  155  	if (!IS_ENABLED(CONFIG_KALLSYMS_ABSOLUTE_PERCPU))
2213e9a66bb87d Ard Biesheuvel   2016-03-15  156  		return kallsyms_relative_base + (u32)kallsyms_offsets[idx];
2213e9a66bb87d Ard Biesheuvel   2016-03-15  157  
2213e9a66bb87d Ard Biesheuvel   2016-03-15  158  	/* ...otherwise, positive offsets are absolute values */
2213e9a66bb87d Ard Biesheuvel   2016-03-15 @159  	if (kallsyms_offsets[idx] >= 0)
2213e9a66bb87d Ard Biesheuvel   2016-03-15  160  		return kallsyms_offsets[idx];
2213e9a66bb87d Ard Biesheuvel   2016-03-15  161  
2213e9a66bb87d Ard Biesheuvel   2016-03-15  162  	/* ...and negative offsets are relative to kallsyms_relative_base - 1 */
2213e9a66bb87d Ard Biesheuvel   2016-03-15 @163  	return kallsyms_relative_base - 1 - kallsyms_offsets[idx];
2213e9a66bb87d Ard Biesheuvel   2016-03-15  164  }
2213e9a66bb87d Ard Biesheuvel   2016-03-15  165  
76903a9648744c Yonghong Song    2023-08-25  166  static void cleanup_symbol_name(char *s)
8b8e6b5d3b013b Sami Tolvanen    2021-04-08  167  {
8b8e6b5d3b013b Sami Tolvanen    2021-04-08  168  	char *res;
8b8e6b5d3b013b Sami Tolvanen    2021-04-08  169  
6eb4bd92c1cedc Nick Desaulniers 2021-10-04  170  	if (!IS_ENABLED(CONFIG_LTO_CLANG))
76903a9648744c Yonghong Song    2023-08-25  171  		return;
6eb4bd92c1cedc Nick Desaulniers 2021-10-04  172  
6eb4bd92c1cedc Nick Desaulniers 2021-10-04  173  	/*
6eb4bd92c1cedc Nick Desaulniers 2021-10-04  174  	 * LLVM appends various suffixes for local functions and variables that
6eb4bd92c1cedc Nick Desaulniers 2021-10-04  175  	 * must be promoted to global scope as part of LTO.  This can break
6eb4bd92c1cedc Nick Desaulniers 2021-10-04  176  	 * hooking of static functions with kprobes. '.' is not a valid
8cc32a9bbf2934 Yonghong Song    2023-06-28  177  	 * character in an identifier in C. Suffixes only in LLVM LTO observed:
6eb4bd92c1cedc Nick Desaulniers 2021-10-04  178  	 * - foo.llvm.[0-9a-f]+
6eb4bd92c1cedc Nick Desaulniers 2021-10-04  179  	 */
8cc32a9bbf2934 Yonghong Song    2023-06-28  180  	res = strstr(s, ".llvm.");
76903a9648744c Yonghong Song    2023-08-25  181  	if (res)
6eb4bd92c1cedc Nick Desaulniers 2021-10-04  182  		*res = '\0';
6eb4bd92c1cedc Nick Desaulniers 2021-10-04  183  
76903a9648744c Yonghong Song    2023-08-25  184  	return;
8b8e6b5d3b013b Sami Tolvanen    2021-04-08  185  }
8b8e6b5d3b013b Sami Tolvanen    2021-04-08  186  
60443c88f3a89f Zhen Lei         2022-11-02  187  static int compare_symbol_name(const char *name, char *namebuf)
60443c88f3a89f Zhen Lei         2022-11-02  188  {
33f0467fe06934 Yonghong Song    2023-08-24  189  	/* The kallsyms_seqs_of_names is sorted based on names after
33f0467fe06934 Yonghong Song    2023-08-24  190  	 * cleanup_symbol_name() (see scripts/kallsyms.c) if clang lto is enabled.
33f0467fe06934 Yonghong Song    2023-08-24  191  	 * To ensure correct bisection in kallsyms_lookup_names(), do
33f0467fe06934 Yonghong Song    2023-08-24  192  	 * cleanup_symbol_name(namebuf) before comparing name and namebuf.
33f0467fe06934 Yonghong Song    2023-08-24  193  	 */
33f0467fe06934 Yonghong Song    2023-08-24  194  	cleanup_symbol_name(namebuf);
33f0467fe06934 Yonghong Song    2023-08-24  195  	return strcmp(name, namebuf);
60443c88f3a89f Zhen Lei         2022-11-02  196  }
60443c88f3a89f Zhen Lei         2022-11-02  197  
19bd8981dc2ee3 Zhen Lei         2022-11-02  198  static unsigned int get_symbol_seq(int index)
19bd8981dc2ee3 Zhen Lei         2022-11-02  199  {
19bd8981dc2ee3 Zhen Lei         2022-11-02  200  	unsigned int i, seq = 0;
19bd8981dc2ee3 Zhen Lei         2022-11-02  201  
19bd8981dc2ee3 Zhen Lei         2022-11-02  202  	for (i = 0; i < 3; i++)
19bd8981dc2ee3 Zhen Lei         2022-11-02 @203  		seq = (seq << 8) | kallsyms_seqs_of_names[3 * index + i];
19bd8981dc2ee3 Zhen Lei         2022-11-02  204  
19bd8981dc2ee3 Zhen Lei         2022-11-02  205  	return seq;
19bd8981dc2ee3 Zhen Lei         2022-11-02  206  }
19bd8981dc2ee3 Zhen Lei         2022-11-02  207  

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

Re: [PATCH 4/4] kbuild: remove PROVIDE() for kallsyms symbols

[Masahiro Yamada]

On Tue, May 21, 2024 at 4:13 PM kernel test robot <lkp@intel.com> wrote:
>
> Hi Masahiro,
>
> kernel test robot noticed the following build errors:
>
> [auto build test ERROR on masahiroy-kbuild/for-next]
> [also build test ERROR on linus/master masahiroy-kbuild/fixes next-20240521]
> [cannot apply to v6.9]
> [If your patch is applied to the wrong git tree, kindly drop us a note.
> And when submitting patch, we suggest to use '--base' as documented in
> https://git-scm.com/docs/git-format-patch#_base_tree_information]
>
> url:    https://github.com/intel-lab-lkp/linux/commits/Masahiro-Yamada/kbuild-avoid-unneeded-kallsyms-step-3/20240520-204508
> base:   https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git for-next
> patch link:    https://lore.kernel.org/r/20240520124212.2351033-5-masahiroy%40kernel.org
> patch subject: [PATCH 4/4] kbuild: remove PROVIDE() for kallsyms symbols
> config: x86_64-rhel-8.3-bpf (https://download.01.org/0day-ci/archive/20240521/202405211448.fglQOQ9W-lkp@intel.com/config)
> compiler: gcc-13 (Ubuntu 13.2.0-4ubuntu3) 13.2.0
> reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240521/202405211448.fglQOQ9W-lkp@intel.com/reproduce)
>
> If you fix the issue in a separate patch/commit (i.e. not just a new version of
> the same patch/commit), kindly add following tags
> | Reported-by: kernel test robot <lkp@intel.com>
> | Closes: https://lore.kernel.org/oe-kbuild-all/202405211448.fglQOQ9W-lkp@intel.com/
>
> All errors (new ones prefixed by >>):


Thanks, I will move kallsyms step 0 before btf creation.




-- 
Best Regards
Masahiro Yamada