[PATCH v2 1/3] modpost: check section mismatch in reference to .dtb.init.rodata

[PATCH v2 1/3] modpost: check section mismatch in reference to .dtb.init.rodata

[Masahiro Yamada]

Built-in DTB files are discarded because KERNEL_DTB() is a part of
INIT_DATA, as defined in include/asm-generic/vmlinux.lds.h.

Currently, modpost warns about mismatched section references to init
data only when the destination section is prefixed with ".init.".
However, ".dtb.init.rodata" is also discarded.

This commit has revealed some missing annotations.

overlays[] references builtin DTBs, which become dangling pointers
after early boot.

testdrv_probe() is not an __init function, yet it holds a reference to
overlays[]. The assumption is that this function is executed only at
the boot time even though it remains in memory. I annotated it as __ref
because otherwise I do not know how to suppress the modpost warning.

I marked the dtb_start as __initdata in the Xtensa boot code, although
modpost does not warn about it because __dtb_start is not yet defined
at the time of modpost.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

Changes in v2:
  - New patch

 arch/xtensa/kernel/setup.c | 2 +-
 drivers/of/unittest.c      | 5 +++--
 scripts/mod/modpost.c      | 2 +-
 3 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/arch/xtensa/kernel/setup.c b/arch/xtensa/kernel/setup.c
index bdec4a773af0..d9f290cf726e 100644
--- a/arch/xtensa/kernel/setup.c
+++ b/arch/xtensa/kernel/setup.c
@@ -55,7 +55,7 @@ extern int initrd_below_start_ok;
 #endif
 
 #ifdef CONFIG_USE_OF
-void *dtb_start = __dtb_start;
+static __initdata void *dtb_start = __dtb_start;
 #endif
 
 extern unsigned long loops_per_jiffy;
diff --git a/drivers/of/unittest.c b/drivers/of/unittest.c
index fd8cb931b1cc..56516bf4171d 100644
--- a/drivers/of/unittest.c
+++ b/drivers/of/unittest.c
@@ -3585,7 +3585,7 @@ OVERLAY_INFO_EXTERN(overlay_bad_symbol);
 OVERLAY_INFO_EXTERN(overlay_bad_unresolved);
 
 /* entries found by name */
-static struct overlay_info overlays[] = {
+static __initdata struct overlay_info overlays[] = {
 	OVERLAY_INFO(overlay_base, -9999, 0),
 	OVERLAY_INFO(overlay, 0, 0),
 	OVERLAY_INFO(overlay_0, 0, 0),
@@ -4054,7 +4054,8 @@ static const struct pci_device_id testdrv_pci_ids[] = {
 	{ 0, }
 };
 
-static int testdrv_probe(struct pci_dev *pdev, const struct pci_device_id *id)
+/* testdrv_probe() references overlays[], which is discarded data  */
+static int __ref testdrv_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 {
 	struct overlay_info *info;
 	struct device_node *dn;
diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
index 107393a8c48a..2e4b0816fdc1 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -776,7 +776,7 @@ static void check_section(const char *modname, struct elf_info *elf,
 	".pci_fixup_enable", ".pci_fixup_resume", \
 	".pci_fixup_resume_early", ".pci_fixup_suspend"
 
-#define ALL_INIT_SECTIONS ".init.*"
+#define ALL_INIT_SECTIONS ".init.*", ".dtb.init.rodata"
 #define ALL_EXIT_SECTIONS ".exit.*"
 
 #define DATA_SECTIONS ".data", ".data.rel"
-- 
2.43.0

[PATCH v2 2/3] kbuild: move non-boot builtin DTBs to .init.rodata section

[Masahiro Yamada]

Some architectures support embedding boot DTB(s) in vmlinux. These
architectures, except MIPS and MicroBlaze, expect a single DTB in
the .dtb.init.rodata section. MIPS embeds multiple DTBs in vmlinux.
MicroBlaze embeds a DTB in its own __fdt_blob section instead of the
.dtb.init.rodata section.

For example, RISC-V previously allowed embedding multiple DTBs, but
only the first DTB in the .dtb.init.rodata section was used. Commit
2672031b20f6 ("riscv: dts: Move BUILTIN_DTB_SOURCE to common Kconfig")
ensured only one boot DTB is embedded.

Meanwhile, commit 7b937cc243e5 ("of: Create of_root if no dtb provided
by firmware") introduced another DTB into the .dtb.init.rodata section.

The symbol dump (sorted by address) for ARCH=riscv nommu_k210_defconfig
is now as follows:

    00000000801290e0 D __dtb_start
    00000000801290e0 D __dtb_k210_generic_begin
    000000008012b571 D __dtb_k210_generic_end
    000000008012b580 D __dtb_empty_root_begin
    000000008012b5c8 D __dtb_empty_root_end
    000000008012b5e0 D __dtb_end

The .dtb.init.rodata section contains the following two DTB files:

    arch/riscv/boot/dts/canaan/k210_generic.dtb
    drivers/of/empty_root.dtb

This is not an immediate problem because the boot code chooses the
first DTB, k210_generic.dtb. The second one, empty_root.dtb is ignored.
However, relying on the link order (i.e., the order in Makefiles) is
fragile.

Only the boot DTB should be placed in the .dtb.init.rodata because the
arch boot code generally does not know the DT name, thus it uses the
__dtb_start symbol to find it.

empty_root.dtb is looked up by name, so it can be moved to the generic
.init.rodata section.

When CONFIG_OF_UNITTEST is enabled, more unittest DTBOs are embedded in
the .dtb.init.rodata section. These are also looked up by name, so can
be moved to the .init.rodata section.

The implementation is kind of cheesy; the section is .dtb.init.rodata
under the arch/ directory, and .init.rodata section otherwise. This will
be refactored later.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Rob Herring (Arm) <robh@kernel.org>
---

Changes in v2:
  - Split __initdata annotation to 1/3

 scripts/Makefile.dtbs | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/scripts/Makefile.dtbs b/scripts/Makefile.dtbs
index 46009d5f1486..55998b878e54 100644
--- a/scripts/Makefile.dtbs
+++ b/scripts/Makefile.dtbs
@@ -34,12 +34,14 @@ $(obj)/dtbs-list: $(dtb-y) FORCE
 # Assembly file to wrap dtb(o)
 # ---------------------------------------------------------------------------
 
+builtin-dtb-section = $(if $(filter arch/%, $(obj)),.dtb.init.rodata,.init.rodata)
+
 # Generate an assembly file to wrap the output of the device tree compiler
 quiet_cmd_wrap_S_dtb = WRAP    $@
       cmd_wrap_S_dtb = {								\
 		symbase=__$(patsubst .%,%,$(suffix $<))_$(subst -,_,$(notdir $*));	\
 		echo '\#include <asm-generic/vmlinux.lds.h>';				\
-		echo '.section .dtb.init.rodata,"a"';					\
+		echo '.section $(builtin-dtb-section),"a"';				\
 		echo '.balign STRUCT_ALIGNMENT';					\
 		echo ".global $${symbase}_begin";					\
 		echo "$${symbase}_begin:";						\
-- 
2.43.0

[PATCH v2 3/3] kbuild: add generic support for built-in boot DTBs

[Masahiro Yamada]

Some architectures embed boot DTBs in vmlinux. A potential issue for
these architectures is a race condition during parallel builds because
Kbuild descends into arch/*/boot/dts/ twice.

One build thread is initiated by the 'dtbs' target, which is a
prerequisite of the 'all' target in the top-level Makefile:

  ifdef CONFIG_OF_EARLY_FLATTREE
  all: dtbs
  endif

For architectures that support the built-in boot dtb, arch/*/boot/dts/
is visited also during the ordinary directory traversal in order to
build obj-y objects that wrap DTBs.

Since these build threads are unaware of each other, they can run
simultaneously during parallel builds.

This commit introduces a generic build rule to scripts/Makefile.vmlinux
to support embedded boot DTBs in a race-free way. Architectures that
want to use this rule need to select CONFIG_GENERIC_BUILTIN_DTB.

After the migration, Makefiles under arch/*/boot/dts/ will be visited
only once to build only *.dtb files.

This change also aims to unify the CONFIG options used for built-in DTBs
support. Currently, different architectures use different CONFIG options
for the same purposes.

The CONFIG options are unified as follows:

 - CONFIG_GENERIC_BUILTIN_DTB

   This enables the generic rule for embedded boot DTBs. This will be
   renamed to CONFIG_BUILTIN_DTB after all architectures migrate to the
   generic rule.

 - CONFIG_BUILTIN_DTB_NAME

   This specifies the path to the embedded DTB.
   (relative to arch/*/boot/dts/)

 - CONFIG_BUILTIN_DTB_ALL

   If this is enabled, all DTB files compiled under arch/*/boot/dts/ are
   embedded into vmlinux. Only used by MIPS.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

(no changes since v1)

 Makefile                 |  7 ++++++-
 drivers/of/Kconfig       |  6 ++++++
 scripts/Makefile.vmlinux | 44 ++++++++++++++++++++++++++++++++++++++++
 scripts/link-vmlinux.sh  |  4 ++++
 4 files changed, 60 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index b4a941a30c73..efe8ac7d356c 100644
--- a/Makefile
+++ b/Makefile
@@ -1417,6 +1417,10 @@ ifdef CONFIG_OF_EARLY_FLATTREE
 all: dtbs
 endif
 
+ifdef CONFIG_GENERIC_BUILTIN_DTB
+vmlinux: dtbs
+endif
+
 endif
 
 PHONY += scripts_dtc
@@ -1484,7 +1488,8 @@ CLEAN_FILES += vmlinux.symvers modules-only.symvers \
 	       modules.builtin modules.builtin.modinfo modules.nsdeps \
 	       modules.builtin.ranges vmlinux.o.map \
 	       compile_commands.json rust/test \
-	       rust-project.json .vmlinux.objs .vmlinux.export.c
+	       rust-project.json .vmlinux.objs .vmlinux.export.c \
+               .builtin-dtbs-list .builtin-dtb.S
 
 # Directories & files removed with 'make mrproper'
 MRPROPER_FILES += include/config include/generated          \
diff --git a/drivers/of/Kconfig b/drivers/of/Kconfig
index dd726c7056bf..5142e7d7fef8 100644
--- a/drivers/of/Kconfig
+++ b/drivers/of/Kconfig
@@ -2,6 +2,12 @@
 config DTC
 	bool
 
+config GENERIC_BUILTIN_DTB
+	bool
+
+config BUILTIN_DTB_ALL
+	bool
+
 menuconfig OF
 	bool "Device Tree and Open Firmware support"
 	help
diff --git a/scripts/Makefile.vmlinux b/scripts/Makefile.vmlinux
index dfb408aa19c6..50b341c3b8cf 100644
--- a/scripts/Makefile.vmlinux
+++ b/scripts/Makefile.vmlinux
@@ -17,6 +17,50 @@ quiet_cmd_cc_o_c = CC      $@
 %.o: %.c FORCE
 	$(call if_changed_dep,cc_o_c)
 
+quiet_cmd_as_o_S = AS      $@
+      cmd_as_o_S = $(CC) $(a_flags) -c -o $@ $<
+
+%.o: %.S FORCE
+	$(call if_changed_dep,as_o_S)
+
+# Built-in dtb
+# ---------------------------------------------------------------------------
+
+quiet_cmd_wrap_dtbs = WRAP    $@
+      cmd_wrap_dtbs = {							\
+	echo '\#include <asm-generic/vmlinux.lds.h>';			\
+	echo '.section .dtb.init.rodata,"a"';				\
+	while read dtb; do						\
+		symbase=__dtb_$$(basename -s .dtb "$${dtb}" | tr - _);	\
+		echo '.balign STRUCT_ALIGNMENT';			\
+		echo ".global $${symbase}_begin";			\
+		echo "$${symbase}_begin:";				\
+		echo '.incbin "'$$dtb'" ';				\
+		echo ".global $${symbase}_end";				\
+		echo "$${symbase}_end:";				\
+	done < $<;							\
+	} > $@
+
+.builtin-dtbs.S: .builtin-dtbs-list FORCE
+	$(call if_changed,wrap_dtbs)
+
+quiet_cmd_gen_dtbs_list = GEN     $@
+      cmd_gen_dtbs_list = \
+	$(if $(CONFIG_BUILTIN_DTB_NAME), echo "arch/$(SRCARCH)/boot/dts/$(CONFIG_BUILTIN_DTB_NAME).dtb",:) > $@
+
+.builtin-dtbs-list: arch/$(SRCARCH)/boot/dts/dtbs-list FORCE
+	$(call if_changed,$(if $(CONFIG_BUILTIN_DTB_ALL),copy,gen_dtbs_list))
+
+targets += .builtin-dtbs-list
+
+ifdef CONFIG_GENERIC_BUILTIN_DTB
+targets += .builtin-dtbs.S .builtin-dtbs.o
+vmlinux: .builtin-dtbs.o
+endif
+
+# vmlinux
+# ---------------------------------------------------------------------------
+
 ifdef CONFIG_MODULES
 targets += .vmlinux.export.o
 vmlinux: .vmlinux.export.o
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index c27b4e969f20..bd196944e350 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -68,6 +68,10 @@ vmlinux_link()
 		libs="${KBUILD_VMLINUX_LIBS}"
 	fi
 
+	if is_enabled CONFIG_GENERIC_BUILTIN_DTB; then
+		objs="${objs} .builtin-dtbs.o"
+	fi
+
 	if is_enabled CONFIG_MODULES; then
 		objs="${objs} .vmlinux.export.o"
 	fi
-- 
2.43.0

Re: [PATCH v2 1/3] modpost: check section mismatch in reference to .dtb.init.rodata

[Rob Herring (Arm)]

On Tue, 10 Sep 2024 18:44:52 +0900, Masahiro Yamada wrote:
> Built-in DTB files are discarded because KERNEL_DTB() is a part of
> INIT_DATA, as defined in include/asm-generic/vmlinux.lds.h.
> 
> Currently, modpost warns about mismatched section references to init
> data only when the destination section is prefixed with ".init.".
> However, ".dtb.init.rodata" is also discarded.
> 
> This commit has revealed some missing annotations.
> 
> overlays[] references builtin DTBs, which become dangling pointers
> after early boot.
> 
> testdrv_probe() is not an __init function, yet it holds a reference to
> overlays[]. The assumption is that this function is executed only at
> the boot time even though it remains in memory. I annotated it as __ref
> because otherwise I do not know how to suppress the modpost warning.
> 
> I marked the dtb_start as __initdata in the Xtensa boot code, although
> modpost does not warn about it because __dtb_start is not yet defined
> at the time of modpost.
> 
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---
> 
> Changes in v2:
>   - New patch
> 
>  arch/xtensa/kernel/setup.c | 2 +-
>  drivers/of/unittest.c      | 5 +++--
>  scripts/mod/modpost.c      | 2 +-
>  3 files changed, 5 insertions(+), 4 deletions(-)
> 

Acked-by: Rob Herring (Arm) <robh@kernel.org>