[PATCH v4 0/2] kbuild: Support directory targets for DTBs

[PATCH v4 0/2] kbuild: Support directory targets for DTBs

[Rob Herring (Arm)]

This series adds support for using directory targets to build DTBs. This 
is primarily useful for architectures with lots of vendor DTS 
subdirectories.

v2 had some issues reported after I applied. In fixing David's issue,
that introduced random parallel build issues Arnd reported. I didn't get
it figured out quickly, so I ended up dropping it. I finally got
around to revisiting it with some LLM help this time. Having multiple 
paths in the build system to building DTBs complicates things, so I've 
reworked things to always use the single target and directory targets 
for building.

Arnd, can you give this a spin on your randconfig builds?
    
Please ack and I'll take this in the DT tree.

Signed-off-by: Rob Herring (Arm) <robh@kernel.org>
---
Changes in v4:
- Don't create dtbs-list files in every directory
- Link to v3: https://patch.msgid.link/20261007-dtb-dirs-v3-v3-0-b3ab32b64fa6@kernel.org

Changes in v3:
 - Convert %.dtbo? rules to single target
 - Convert dtbs rule to use directory target
 - Link to v2: https://lore.kernel.org/all/20251120204812.1983452-4-robh@kernel.org/

---
Rob Herring (Arm) (2):
      kbuild: Convert %.dtb(o) targets to single target rules
      kbuild: Support directory targets for building DTBs

 Makefile                          | 38 ++++++++++++++++++++++++++------------
 drivers/of/unittest-data/Makefile |  5 +++++
 scripts/Makefile.build            |  7 +++++++
 scripts/Makefile.dtbs             |  2 ++
 4 files changed, 40 insertions(+), 12 deletions(-)
---
base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
change-id: 20261007-dtb-dirs-v3-008fa5ffbc74

Best regards,
--  
Rob Herring (Arm) <robh@kernel.org>

[PATCH v4 1/2] kbuild: Convert %.dtb(o) targets to single target rules

[Rob Herring (Arm)]

Single .dtb and .dtbo targets can be built using the existing kbuild
single target infrastructure. The only difference is the custom rules
work on a relative path under the arch dts directory, so we have to
fixup the path for the single targets.

With this both relative and full paths work:

make arch/arm64/boot/dts/arm/juno.dtb
make arm/juno.dtb

Complicating matters is building DTBs works without $ARCH compiler as
only the host preprocessor is used. This is useful for building all the
DTBs in the tree without installing a bunch of cross-compilers. The
single target rule needs to depend on 'dtbs_prepare' rather the
'prepare' to maintain support for that.

Reviewed-by: Nicolas Schier <nsc@kernel.org>
Tested-by: Nicolas Schier <nsc@kernel.org>
Signed-off-by: Rob Herring (Arm) <robh@kernel.org>
---
v3:
 - New patch
---
 Makefile | 26 +++++++++++++++++---------
 1 file changed, 17 insertions(+), 9 deletions(-)

diff --git a/Makefile b/Makefile
index 4ad67b737af7..c307ee9293bc 100644
--- a/Makefile
+++ b/Makefile
@@ -301,7 +301,7 @@ no-dot-config-targets := $(clean-targets) \
 			 run-command
 no-sync-config-targets := $(no-dot-config-targets) %install modules_sign kernelrelease \
 			  image_name
-single-targets := %.a %.i %.ko %.lds %.ll %.lst %.mod %.o %.rsi %.s %/
+single-targets := %.a %.i %.ko %.lds %.ll %.lst %.mod %.o %.rsi %.s %.dtb %.dtbo %/
 
 config-build	:=
 mixed-build	:=
@@ -1663,12 +1663,6 @@ dtbindingtree := Documentation/devicetree/bindings
 
 ifneq ($(dtstree),)
 
-%.dtb: dtbs_prepare
-	$(Q)$(MAKE) $(build)=$(dtstree) $(dtstree)/$@
-
-%.dtbo: dtbs_prepare
-	$(Q)$(MAKE) $(build)=$(dtstree) $(dtstree)/$@
-
 PHONY += dtbs dtbs_prepare dtbs_install dtbs_check
 dtbs: dtbs_prepare
 	$(Q)$(MAKE) $(build)=$(dtstree) need-dtbslist=1
@@ -1683,7 +1677,7 @@ export CHECK_DTBS=y
 endif
 
 ifneq ($(CHECK_DTBS),)
-dtbs_prepare: dt_binding_schemas
+scripts_dtc: dt_binding_schemas
 endif
 
 dtbs_check: dtbs
@@ -2190,6 +2184,8 @@ modpost: $(if $(single-build),, $(if $(KBUILD_BUILTIN), vmlinux.o)) \
 #
 #   make foo/bar/baz/
 
+single-build-prep := prepare
+
 ifdef single-build
 
 # .ko is special because modpost is needed
@@ -2212,8 +2208,20 @@ ifneq ($(KBUILD_MODPOST_NOFINAL),1)
 endif
 	$(Q)rm -f modules.order
 
+# For compatibility with existing %.dtb targets without full path
+single-dtb := $(filter-out $(dtstree)/%, $(filter %.dtb %.dtbo, $(single-no-ko)))
+single-no-ko := $(filter-out $(single-dtb), $(single-no-ko)) $(addprefix $(dtstree)/, $(single-dtb))
+
 single-goals := $(addprefix $(build-dir)/, $(single-no-ko))
 
+# DT-only targets need host tools, but no architecture-specific preparation.
+ifneq ($(dtstree),)
+single-dt-patterns := $(addprefix $(dtstree)/,%.dtb %.dtbo)
+ifeq ($(filter-out $(single-dt-patterns),$(single-no-ko)),)
+single-build-prep := dtbs_prepare
+endif
+endif
+
 KBUILD_MODULES := y
 
 endif
@@ -2225,7 +2233,7 @@ prepare: outputmakefile
 # make menuconfig etc.
 # Error messages still appears in the original language
 PHONY += $(build-dir)
-$(build-dir): prepare
+$(build-dir): $(single-build-prep)
 	$(Q)$(MAKE) $(build)=$@ need-builtin=1 need-modorder=1 $(single-goals)
 
 clean-dirs := $(addprefix _clean_, $(clean-dirs))

-- 
2.53.0

[PATCH v4 2/2] kbuild: Support directory targets for building DTBs

[Rob Herring (Arm)]

It is useful to be able to build all the DTBs for a vendor. One can list
all the .dts files in a directory and convert those to %.dtb targets,
but that doesn't work for base+overlay DTB targets.

The 'dtbs' and 'dtbs_check' targets can use the directory rule targets
as well which avoids different make paths for building DTBs.

Signed-off-by: Rob Herring (Arm) <robh@kernel.org>
---
v4:
 - Don't create a dtbs-list file in every directory

v3:
 - Reworked on top of single target rule for %.dtb
 - Make dtbs target use directory rule.
 - Move adding boot/dts subdir out of the arch code. Now supported on
   every arch (not that useful without vendor dirs).
 - Removed the need to look at MAKECMDGOALS
 - Fixed unnecessary rebuilding DT base+overlay targets
 - Link to v2: https://lore.kernel.org/all/20251120204812.1983452-4-robh@kernel.org/

v2:
 - Convert arm, mips and riscv. The other DT enabled arches don't have
   vendor directories.
 - Link to v1: https://lore.kernel.org/all/20251113225952.867138-1-robh@kernel.org/
---
 Makefile                          | 14 ++++++++++----
 drivers/of/unittest-data/Makefile |  5 +++++
 scripts/Makefile.build            |  7 +++++++
 scripts/Makefile.dtbs             |  2 ++
 4 files changed, 24 insertions(+), 4 deletions(-)

diff --git a/Makefile b/Makefile
index c307ee9293bc..6d1d1c803ee8 100644
--- a/Makefile
+++ b/Makefile
@@ -1664,8 +1664,8 @@ dtbindingtree := Documentation/devicetree/bindings
 ifneq ($(dtstree),)
 
 PHONY += dtbs dtbs_prepare dtbs_install dtbs_check
-dtbs: dtbs_prepare
-	$(Q)$(MAKE) $(build)=$(dtstree) need-dtbslist=1
+dtbs:
+	$(Q)$(MAKE) $(dtstree)/
 
 # include/config/kernel.release is actually needed when installing DTBs because
 # INSTALL_DTBS_PATH contains $(KERNELRELEASE). However, we do not want to make
@@ -2185,6 +2185,7 @@ modpost: $(if $(single-build),, $(if $(KBUILD_BUILTIN), vmlinux.o)) \
 #   make foo/bar/baz/
 
 single-build-prep := prepare
+single-dtbslist-arg :=
 
 ifdef single-build
 
@@ -2216,9 +2217,13 @@ single-goals := $(addprefix $(build-dir)/, $(single-no-ko))
 
 # DT-only targets need host tools, but no architecture-specific preparation.
 ifneq ($(dtstree),)
-single-dt-patterns := $(addprefix $(dtstree)/,%.dtb %.dtbo)
+single-dt-patterns := $(dtstree)/ $(addprefix $(dtstree)/,%.dtb %.dtbo %/)
+ifneq ($(filter $(single-dt-patterns),$(single-no-ko)),)
+single-dtbslist-arg := need-dtbslist=1
+endif
 ifeq ($(filter-out $(single-dt-patterns),$(single-no-ko)),)
 single-build-prep := dtbs_prepare
+single-dtbslist-arg := need-dtbslist=1
 endif
 endif
 
@@ -2234,7 +2239,8 @@ prepare: outputmakefile
 # Error messages still appears in the original language
 PHONY += $(build-dir)
 $(build-dir): $(single-build-prep)
-	$(Q)$(MAKE) $(build)=$@ need-builtin=1 need-modorder=1 $(single-goals)
+	$(Q)$(MAKE) $(build)=$@ need-builtin=1 need-modorder=1 \
+		$(single-dtbslist-arg) $(single-goals)
 
 clean-dirs := $(addprefix _clean_, $(clean-dirs))
 PHONY += $(clean-dirs) clean
diff --git a/drivers/of/unittest-data/Makefile b/drivers/of/unittest-data/Makefile
index 01a966e39f23..bac6ee9327e2 100644
--- a/drivers/of/unittest-data/Makefile
+++ b/drivers/of/unittest-data/Makefile
@@ -104,3 +104,8 @@ static_test_1-dtbs := static_base_1.dtb $(apply_static_overlay_1)
 static_test_2-dtbs := static_base_2.dtb $(apply_static_overlay_2)
 
 dtb-$(CONFIG_OF_OVERLAY) += static_test_1.dtb static_test_2.dtb
+
+# Include the static overlay tests in normal kernel builds.
+ifdef CONFIG_OF_OVERLAY
+need-dtbslist := 1
+endif
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 4349108e75e1..d22b3b2d43b3 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -38,6 +38,13 @@ include $(srctree)/scripts/Makefile.compiler
 include $(kbuild-file)
 include $(srctree)/scripts/Makefile.lib
 
+# Make DT directory targets reachable for architectures with a DTS tree.
+ifeq ($(obj),arch/$(SRCARCH))
+ifneq ($(wildcard $(src)/boot/dts/),)
+subdir-y += boot/dts
+endif
+endif
+
 # flags that take effect in current and sub directories
 KBUILD_AFLAGS += $(subdir-asflags-y)
 KBUILD_CFLAGS += $(subdir-ccflags-y)
diff --git a/scripts/Makefile.dtbs b/scripts/Makefile.dtbs
index c4e466390284..c7ec2a2a308e 100644
--- a/scripts/Makefile.dtbs
+++ b/scripts/Makefile.dtbs
@@ -23,7 +23,9 @@ dtb-y           := $(addprefix $(obj)/, $(dtb-y))
 multi-dtb-y     := $(addprefix $(obj)/, $(multi-dtb-y))
 real-dtb-y      := $(addprefix $(obj)/, $(real-dtb-y))
 
+ifdef need-dtbslist
 always-y        += $(dtb-y)
+endif
 targets         += $(real-dtb-y)
 
 # dtbs-list

-- 
2.53.0

Re: [PATCH v4 2/2] kbuild: Support directory targets for building DTBs

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] kbuild: parallel build race in dtbs target and extra boot/dts traversal
--

commit 3f00c18c0d96a2ac1d59c7335959fec4fa041ff7
Author: Rob Herring (Arm) <robh@kernel.org>

kbuild: Support directory targets for building DTBs

This commit enables building DTBs using directory targets so that all DTBs
for a vendor directory can be built, including base+overlay DTB targets,
and updates the dtbs target to use the directory target rule.

> diff --git a/Makefile b/Makefile
> index c307ee9293bcd..6d1d1c803ee86 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1664,8 +1664,8 @@ dtbindingtree := Documentation/devicetree/bindings
>  ifneq ($(dtstree),)
>  
>  PHONY += dtbs dtbs_prepare dtbs_install dtbs_check
> -dtbs: dtbs_prepare
> -	$(Q)$(MAKE) $(build)=$(dtstree) need-dtbslist=1
> +dtbs:
> +	$(Q)$(MAKE) $(dtstree)/

[Severity: High]
Does dropping dtbs_prepare from the dtbs target prerequisites and spawning a
separate top-level make for $(dtstree)/ introduce a parallel build race when
vmlinux and dtbs are built together (for example, during make -j all when
CONFIG_OF_EARLY_FLATTREE=y, make -j vmlinux when CONFIG_GENERIC_BUILTIN_DTB=y,
or make -j image.fit on arm64)?

Previously, vmlinux (via . -> prepare -> prepare0 -> archprepare) and dtbs
(via dtbs_prepare) shared the same top-level make dependency graph, which
serialized include/config/kernel.release, scripts_basic, and scripts_dtc:

Makefile:
    scripts: scripts_basic scripts_dtc
    	$(Q)$(MAKE) $(build)=$(@)
    ...
    archprepare: outputmakefile archheaders archscripts scripts include/config/kernel.release \
    ...
    dtbs_prepare: include/config/kernel.release scripts_dtc

With dtbs running $(MAKE) $(dtstree)/, the child make enables single-build
and evaluates $(build-dir): $(single-build-prep) where single-build-prep is
dtbs_prepare. In a parallel build, the parent make (building archprepare) and
the child make (building dtbs_prepare) have independent dependency graphs and
can run concurrently:

1. Both make instances can run filechk for include/config/kernel.release at
   the same time, racing on the shared include/config/.tmp_kernel.release
   temporary file in scripts/Kbuild.include:

   define filechk
   	$(check-FORCE)
   	$(Q)set -e;						\
   	mkdir -p $(dir $@);					\
   	trap "rm -f $(tmp-target)" EXIT;			\
   	{ $(filechk_$(1)); } > $(tmp-target);			\
   	if [ ! -r $@ ] || ! cmp -s $@ $(tmp-target); then	\
   		$(kecho) '  UPD     $@';			\
   		mv -f $(tmp-target) $@;				\
   	fi
   endef

2. Both make instances can concurrently invoke $(MAKE) $(build)=scripts/basic
   and $(MAKE) $(build)=scripts/dtc in the same output directory, racing on
   building the host binaries (fixdep, dtc, fdtoverlay) and their .*.cmd
   files.

[ ... ]

> diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> index 4349108e75e1f..d22b3b2d43b32 100644
> --- a/scripts/Makefile.build
> +++ b/scripts/Makefile.build
> @@ -38,6 +38,13 @@ include $(srctree)/scripts/Makefile.compiler
>  include $(kbuild-file)
>  include $(srctree)/scripts/Makefile.lib
>  
> +# Make DT directory targets reachable for architectures with a DTS tree.
> +ifeq ($(obj),arch/$(SRCARCH))
> +ifneq ($(wildcard $(src)/boot/dts/),)
> +subdir-y += boot/dts
> +endif
> +endif
> +

[Severity: High]
Because subdir-y += boot/dts is added unconditionally whenever $(obj) is
arch/$(SRCARCH), won't the main "." build pass (as well as non-DT builds like
make vmlinux or make modules) now recurse into arch/$(SRCARCH)/boot/dts and
all vendor subdirectories even when need-dtbslist is not set?

During make -j, the parent make's "." traversal will descend into
arch/$(SRCARCH)/boot/dts in parallel with the child $(MAKE) $(dtstree)/
spawned by the dtbs target. In every vendor directory where dtb-y or dtb- is
non-empty, scripts/Makefile.build includes scripts/Makefile.dtbs (which adds
$(real-dtb-y) to targets) and then includes the .cmd files for existing
targets:

scripts/Makefile.build:
    ifneq ($(need-dtbslist)$(dtb-y)$(dtb-)$(filter %.dtb %.dtb.o %.dtbo.o,$(targets)),)
    include $(srctree)/scripts/Makefile.dtbs
    endif
    ...
    existing-targets := $(wildcard $(sort $(targets)))

    -include $(foreach f,$(existing-targets),$(dir $(f)).$(notdir $(f)).cmd)

If the parallel dtbs sub-make has just created a .dtb file and cmd_and_fixdep
is still writing the corresponding .*.dtb.cmd file while the "." build walk
is evaluating that directory, can the "." sub-make read a partially written
.*.dtb.cmd file and fail with a syntax error?

[ ... ]

$>

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261016-dtb-dirs-v3-v4-0-0ec42fd8ef9b@kernel.org?part=2