* [PATCH v5 0/8] scripts/make_fit: Support ramdisks and faster operations @ 2025-11-14 14:27 Simon Glass 2025-11-14 14:27 ` [PATCH v5 5/8] kbuild: Split out module targets into a variable Simon Glass 2025-11-14 14:27 ` [PATCH v5 6/8] kbuild: Allow adding modules into the FIT ramdisk Simon Glass 0 siblings, 2 replies; 6+ messages in thread From: Simon Glass @ 2025-11-14 14:27 UTC (permalink / raw) To: linux-arm-kernel Cc: Nicolas Schier, Masahiro Yamada, Chen-Yu Tsai, Tom Rini, Ahmad Fatoum, J . Neuschäfer, Simon Glass, Ard Biesheuvel, Bill Wendling, Catalin Marinas, David Sterba, Josh Poimboeuf, Justin Stitt, Kees Cook, Miguel Ojeda, Nathan Chancellor, Nick Desaulniers, Nick Terrell, Nicolas Schier, Parth Pancholi, Rong Xu, Tamir Duberstein, Thomas Weißschuh, Will Deacon, linux-kbuild, linux-kernel, llvm This series updates 'make image.fit' to support adding a ramdisk to the FIT, either one provided as a parameter or one created from all the kernel modules. It also includes a few performance improvement, so that building a FIT from ~450MB of kernel/module/devicetree files only takes a few seconds on a modern machine. Changes in v5: - Fix 'use' typo - Add a new patch to split out module targets into a variable - Build modules automatically if needed (fix from Nicolas Schier) Changes in v4: - Update the commit message - Provide the list of modules from the Makefile - Reduce verbosity (don't print every module filename) - Rename the Makefile variable from 'EXTRA' to 'MAKE_FIT_FLAGS' - Use an empty FIT_MODULES to disable the feature, instead of '0' - Make use of the 'modules' dependency to ensure modules are built - Pass the list of modules to the script Changes in v3: - Move the ramdisk chunk into the correct patch - Add a comment at the top of the file about the -r option - Count the ramdisk in the total files - Update the commit message - Add a way to add built modules into the FIT Changes in v2: - Don't compress the ramdisk as it is already compressed Simon Glass (8): scripts/make_fit: Speed up operation scripts/make_fit: Support an initial ramdisk scripts/make_fit: Move dtb processing into a function scripts/make_fit: Provide a way to add built modules kbuild: Split out module targets into a variable kbuild: Allow adding modules into the FIT ramdisk scripts/make_fit: Support a few more parallel compressors scripts/make_fit: Compress dtbs in parallel Makefile | 8 +- arch/arm64/Makefile | 1 + scripts/Makefile.lib | 10 +- scripts/make_fit.py | 264 +++++++++++++++++++++++++++++++++++++------ 4 files changed, 243 insertions(+), 40 deletions(-) -- 2.43.0 base-commit: 4a71531471926e3c391665ee9c42f4e0295a4585 branch: fita5 ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v5 5/8] kbuild: Split out module targets into a variable 2025-11-14 14:27 [PATCH v5 0/8] scripts/make_fit: Support ramdisks and faster operations Simon Glass @ 2025-11-14 14:27 ` Simon Glass 2025-11-14 14:27 ` [PATCH v5 6/8] kbuild: Allow adding modules into the FIT ramdisk Simon Glass 1 sibling, 0 replies; 6+ messages in thread From: Simon Glass @ 2025-11-14 14:27 UTC (permalink / raw) To: linux-arm-kernel Cc: Nicolas Schier, Masahiro Yamada, Chen-Yu Tsai, Tom Rini, Ahmad Fatoum, J . Neuschäfer, Simon Glass, Bill Wendling, Justin Stitt, Miguel Ojeda, Nathan Chancellor, Nick Desaulniers, Nicolas Schier, Tamir Duberstein, Thomas Weißschuh, linux-kbuild, linux-kernel, llvm Add a modules-targets variable to list the targets which cause modules to be built, since we want to add a conditional target. Signed-off-by: Simon Glass <sjg@chromium.org> --- Changes in v5: - Add a new patch to split out module targets into a variable Makefile | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 638bc09a546a..8cd46222fc48 100644 --- a/Makefile +++ b/Makefile @@ -772,7 +772,12 @@ endif # in addition to whatever we do anyway. # Just "make" or "make all" shall build modules as well -ifneq ($(filter all modules nsdeps compile_commands.json clang-%,$(MAKECMDGOALS)),) +modules-targets := all +modules-targets += modules +modules-targets += nsdeps +modules-targets += compile_commands.json +modules-targets += clang-% +ifneq ($(filter $(modules-targets),$(MAKECMDGOALS)),) KBUILD_MODULES := y endif -- 2.43.0 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v5 6/8] kbuild: Allow adding modules into the FIT ramdisk 2025-11-14 14:27 [PATCH v5 0/8] scripts/make_fit: Support ramdisks and faster operations Simon Glass 2025-11-14 14:27 ` [PATCH v5 5/8] kbuild: Split out module targets into a variable Simon Glass @ 2025-11-14 14:27 ` Simon Glass 2025-11-14 15:29 ` Thomas Weißschuh 1 sibling, 1 reply; 6+ messages in thread From: Simon Glass @ 2025-11-14 14:27 UTC (permalink / raw) To: linux-arm-kernel Cc: Nicolas Schier, Masahiro Yamada, Chen-Yu Tsai, Tom Rini, Ahmad Fatoum, J . Neuschäfer, Simon Glass, Reviewed-by : Nicolas Schier, Nathan Chancellor, Ard Biesheuvel, Catalin Marinas, Josh Poimboeuf, Kees Cook, Miguel Ojeda, Nicolas Schier, Parth Pancholi, Rong Xu, Tamir Duberstein, Thomas Weißschuh, Will Deacon, linux-kbuild, linux-kernel Support 'make image.fit FIT_MODULES=1' to put all the modules into a ramdisk image within the FIT. Add image.fit as a target which requires modules, so that modules will built automatically when using FIT_MODULES=1 Signed-off-by: Simon Glass <sjg@chromium.org> Suggested-by: Ahmad Fatoum <a.fatoum@pengutronix.de> Suggested-by: Reviewed-by: Nicolas Schier <nsc@kernel.org> Acked-by: Nathan Chancellor <nathan@kernel.org> --- Changes in v5: - Build modules automatically if needed (fix from Nicolas Schier) Changes in v4: - Rename the Makefile variable from 'EXTRA' to 'MAKE_FIT_FLAGS' - Use an empty FIT_MODULES to disable the feature, instead of '0' - Make use of the 'modules' dependency to ensure modules are built - Pass the list of modules to the script Makefile | 1 + arch/arm64/Makefile | 1 + scripts/Makefile.lib | 10 ++++++++-- 3 files changed, 10 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index 8cd46222fc48..4eccaef95826 100644 --- a/Makefile +++ b/Makefile @@ -773,6 +773,7 @@ endif # Just "make" or "make all" shall build modules as well modules-targets := all +modules-targets += $(if $(FIT_MODULES),image.fit) modules-targets += modules modules-targets += nsdeps modules-targets += compile_commands.json diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile index 73a10f65ce8b..7036f251ab40 100644 --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -174,6 +174,7 @@ endif all: $(notdir $(KBUILD_IMAGE)) image.fit: dtbs +image.fit: $(if $(FIT_MODULES),modules) vmlinuz.efi image.fit: Image $(BOOT_TARGETS): vmlinux diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 1d581ba5df66..c6a3aa653035 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -398,11 +398,17 @@ MAKE_FIT := $(srctree)/scripts/make_fit.py # Use this to override the compression algorithm FIT_COMPRESSION ?= gzip +# Set this to 1 to include an initrd with all the kernel modules +FIT_MODULES ?= + quiet_cmd_fit = FIT $@ - cmd_fit = $(MAKE_FIT) -o $@ --arch $(UIMAGE_ARCH) --os linux \ - --name '$(UIMAGE_NAME)' \ + cmd_fit = $(if $(FIT_MODULES), \ + find $(objtree) -name '*.ko' > $(objtree)/.modules-list 2>/dev/null &&) \ + $(MAKE_FIT) -o $@ --arch $(UIMAGE_ARCH) --os linux \ + --name '$(UIMAGE_NAME)' $(MAKE_FIT_FLAGS) \ $(if $(findstring 1,$(KBUILD_VERBOSE)),-v) \ $(if $(FIT_DECOMPOSE_DTBS),--decompose-dtbs) \ + $(if $(FIT_MODULES),--modules @$(objtree)/.modules-list) \ --compress $(FIT_COMPRESSION) -k $< @$(word 2,$^) # XZ -- 2.43.0 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH v5 6/8] kbuild: Allow adding modules into the FIT ramdisk 2025-11-14 14:27 ` [PATCH v5 6/8] kbuild: Allow adding modules into the FIT ramdisk Simon Glass @ 2025-11-14 15:29 ` Thomas Weißschuh 2025-11-14 19:50 ` Nicolas Schier 0 siblings, 1 reply; 6+ messages in thread From: Thomas Weißschuh @ 2025-11-14 15:29 UTC (permalink / raw) To: Simon Glass Cc: linux-arm-kernel, Nicolas Schier, Masahiro Yamada, Chen-Yu Tsai, Tom Rini, Ahmad Fatoum, J . Neuschäfer, Reviewed-by : Nicolas Schier, Nathan Chancellor, Ard Biesheuvel, Catalin Marinas, Josh Poimboeuf, Kees Cook, Miguel Ojeda, Nicolas Schier, Parth Pancholi, Rong Xu, Tamir Duberstein, Will Deacon, linux-kbuild, linux-kernel On Fri, Nov 14, 2025 at 07:27:32AM -0700, Simon Glass wrote: > Support 'make image.fit FIT_MODULES=1' to put all the modules into a > ramdisk image within the FIT. > > Add image.fit as a target which requires modules, so that modules will > built automatically when using FIT_MODULES=1 > > Signed-off-by: Simon Glass <sjg@chromium.org> > Suggested-by: Ahmad Fatoum <a.fatoum@pengutronix.de> > Suggested-by: Reviewed-by: Nicolas Schier <nsc@kernel.org> > Acked-by: Nathan Chancellor <nathan@kernel.org> > --- > > Changes in v5: > - Build modules automatically if needed (fix from Nicolas Schier) > > Changes in v4: > - Rename the Makefile variable from 'EXTRA' to 'MAKE_FIT_FLAGS' > - Use an empty FIT_MODULES to disable the feature, instead of '0' > - Make use of the 'modules' dependency to ensure modules are built > - Pass the list of modules to the script > > Makefile | 1 + > arch/arm64/Makefile | 1 + > scripts/Makefile.lib | 10 ++++++++-- > 3 files changed, 10 insertions(+), 2 deletions(-) > > diff --git a/Makefile b/Makefile > index 8cd46222fc48..4eccaef95826 100644 > --- a/Makefile > +++ b/Makefile > @@ -773,6 +773,7 @@ endif > # Just "make" or "make all" shall build modules as well > > modules-targets := all > +modules-targets += $(if $(FIT_MODULES),image.fit) > modules-targets += modules > modules-targets += nsdeps > modules-targets += compile_commands.json > diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile > index 73a10f65ce8b..7036f251ab40 100644 > --- a/arch/arm64/Makefile > +++ b/arch/arm64/Makefile > @@ -174,6 +174,7 @@ endif > all: $(notdir $(KBUILD_IMAGE)) > > image.fit: dtbs > +image.fit: $(if $(FIT_MODULES),modules) > > vmlinuz.efi image.fit: Image > $(BOOT_TARGETS): vmlinux > diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib > index 1d581ba5df66..c6a3aa653035 100644 > --- a/scripts/Makefile.lib > +++ b/scripts/Makefile.lib > @@ -398,11 +398,17 @@ MAKE_FIT := $(srctree)/scripts/make_fit.py > # Use this to override the compression algorithm > FIT_COMPRESSION ?= gzip > > +# Set this to 1 to include an initrd with all the kernel modules > +FIT_MODULES ?= '0' will also trigger that behavior. > + > quiet_cmd_fit = FIT $@ > - cmd_fit = $(MAKE_FIT) -o $@ --arch $(UIMAGE_ARCH) --os linux \ > - --name '$(UIMAGE_NAME)' \ > + cmd_fit = $(if $(FIT_MODULES), \ > + find $(objtree) -name '*.ko' > $(objtree)/.modules-list 2>/dev/null &&) \ This will include stale module files. You can get an up-to-date list from $(objtree)/modules.order with a bit post-processing. Maybe kbuild can be extended to also create a list of the .ko files. (I would be interested in that for my own usecases, too) > + $(MAKE_FIT) -o $@ --arch $(UIMAGE_ARCH) --os linux \ > + --name '$(UIMAGE_NAME)' $(MAKE_FIT_FLAGS) \ > $(if $(findstring 1,$(KBUILD_VERBOSE)),-v) \ > $(if $(FIT_DECOMPOSE_DTBS),--decompose-dtbs) \ > + $(if $(FIT_MODULES),--modules @$(objtree)/.modules-list) \ > --compress $(FIT_COMPRESSION) -k $< @$(word 2,$^) > > # XZ > -- > 2.43.0 > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v5 6/8] kbuild: Allow adding modules into the FIT ramdisk 2025-11-14 15:29 ` Thomas Weißschuh @ 2025-11-14 19:50 ` Nicolas Schier 2025-11-17 8:30 ` Thomas Weißschuh 0 siblings, 1 reply; 6+ messages in thread From: Nicolas Schier @ 2025-11-14 19:50 UTC (permalink / raw) To: Thomas Weißschuh Cc: Simon Glass, linux-arm-kernel, Masahiro Yamada, Chen-Yu Tsai, Tom Rini, Ahmad Fatoum, J . Neuschäfer, Nathan Chancellor, Ard Biesheuvel, Catalin Marinas, Josh Poimboeuf, Kees Cook, Miguel Ojeda, Parth Pancholi, Rong Xu, Tamir Duberstein, Will Deacon, linux-kbuild, linux-kernel On Fri, Nov 14, 2025 at 04:29:33PM +0100, Thomas Weißschuh wrote: > On Fri, Nov 14, 2025 at 07:27:32AM -0700, Simon Glass wrote: > > Support 'make image.fit FIT_MODULES=1' to put all the modules into a > > ramdisk image within the FIT. > > > > Add image.fit as a target which requires modules, so that modules will > > built automatically when using FIT_MODULES=1 > > > > Signed-off-by: Simon Glass <sjg@chromium.org> > > Suggested-by: Ahmad Fatoum <a.fatoum@pengutronix.de> > > Suggested-by: Reviewed-by: Nicolas Schier <nsc@kernel.org> > > Acked-by: Nathan Chancellor <nathan@kernel.org> > > --- > > > > Changes in v5: > > - Build modules automatically if needed (fix from Nicolas Schier) > > > > Changes in v4: > > - Rename the Makefile variable from 'EXTRA' to 'MAKE_FIT_FLAGS' > > - Use an empty FIT_MODULES to disable the feature, instead of '0' > > - Make use of the 'modules' dependency to ensure modules are built > > - Pass the list of modules to the script > > > > Makefile | 1 + > > arch/arm64/Makefile | 1 + > > scripts/Makefile.lib | 10 ++++++++-- > > 3 files changed, 10 insertions(+), 2 deletions(-) > > > > diff --git a/Makefile b/Makefile > > index 8cd46222fc48..4eccaef95826 100644 > > --- a/Makefile > > +++ b/Makefile > > @@ -773,6 +773,7 @@ endif > > # Just "make" or "make all" shall build modules as well > > > > modules-targets := all > > +modules-targets += $(if $(FIT_MODULES),image.fit) > > modules-targets += modules > > modules-targets += nsdeps > > modules-targets += compile_commands.json > > diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile > > index 73a10f65ce8b..7036f251ab40 100644 > > --- a/arch/arm64/Makefile > > +++ b/arch/arm64/Makefile > > @@ -174,6 +174,7 @@ endif > > all: $(notdir $(KBUILD_IMAGE)) > > > > image.fit: dtbs > > +image.fit: $(if $(FIT_MODULES),modules) > > > > vmlinuz.efi image.fit: Image > > $(BOOT_TARGETS): vmlinux > > diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib > > index 1d581ba5df66..c6a3aa653035 100644 > > --- a/scripts/Makefile.lib > > +++ b/scripts/Makefile.lib > > @@ -398,11 +398,17 @@ MAKE_FIT := $(srctree)/scripts/make_fit.py > > # Use this to override the compression algorithm > > FIT_COMPRESSION ?= gzip > > > > +# Set this to 1 to include an initrd with all the kernel modules > > +FIT_MODULES ?= > > '0' will also trigger that behavior. > > > + > > quiet_cmd_fit = FIT $@ > > - cmd_fit = $(MAKE_FIT) -o $@ --arch $(UIMAGE_ARCH) --os linux \ > > - --name '$(UIMAGE_NAME)' \ > > + cmd_fit = $(if $(FIT_MODULES), \ > > + find $(objtree) -name '*.ko' > $(objtree)/.modules-list 2>/dev/null &&) \ > > This will include stale module files. You can get an up-to-date list from > $(objtree)/modules.order with a bit post-processing. > Maybe kbuild can be extended to also create a list of the .ko files. > (I would be interested in that for my own usecases, too) oh yes, thanks for the pointer. This is indeed quite simple and much better than calling find. For in-tree kmods: compiled-modules = $(patsubst %.o,%.ko,$(call read-file, $(objtree)/modules.order)) But as we need the list of modules in a file, we can also add a $(call write-file,FILE,TEXT) macro (cp. read-file in scripts/Kbuild.include). Thomas, is this sufficient for your use case? Or do you also need a make target outputting the list of kmods? Kind regards, Nicolas ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v5 6/8] kbuild: Allow adding modules into the FIT ramdisk 2025-11-14 19:50 ` Nicolas Schier @ 2025-11-17 8:30 ` Thomas Weißschuh 0 siblings, 0 replies; 6+ messages in thread From: Thomas Weißschuh @ 2025-11-17 8:30 UTC (permalink / raw) To: Nicolas Schier Cc: Simon Glass, linux-arm-kernel, Masahiro Yamada, Chen-Yu Tsai, Tom Rini, Ahmad Fatoum, J . Neuschäfer, Nathan Chancellor, Ard Biesheuvel, Catalin Marinas, Josh Poimboeuf, Kees Cook, Miguel Ojeda, Parth Pancholi, Rong Xu, Tamir Duberstein, Will Deacon, linux-kbuild, linux-kernel On Fri, Nov 14, 2025 at 08:50:27PM +0100, Nicolas Schier wrote: > On Fri, Nov 14, 2025 at 04:29:33PM +0100, Thomas Weißschuh wrote: > > On Fri, Nov 14, 2025 at 07:27:32AM -0700, Simon Glass wrote: (...) > > > quiet_cmd_fit = FIT $@ > > > - cmd_fit = $(MAKE_FIT) -o $@ --arch $(UIMAGE_ARCH) --os linux \ > > > - --name '$(UIMAGE_NAME)' \ > > > + cmd_fit = $(if $(FIT_MODULES), \ > > > + find $(objtree) -name '*.ko' > $(objtree)/.modules-list 2>/dev/null &&) \ > > > > This will include stale module files. You can get an up-to-date list from > > $(objtree)/modules.order with a bit post-processing. > > Maybe kbuild can be extended to also create a list of the .ko files. > > (I would be interested in that for my own usecases, too) > > oh yes, thanks for the pointer. This is indeed quite simple and much > better than calling find. For in-tree kmods: > > compiled-modules = $(patsubst %.o,%.ko,$(call read-file, $(objtree)/modules.order)) Tiny nitpick: IMO 'built' modules would be more accurate than 'compiled' modules. > But as we need the list of modules in a file, we can also add a > $(call write-file,FILE,TEXT) macro (cp. read-file in > scripts/Kbuild.include). > Thomas, is this sufficient for your use case? Or do you also need a > make target outputting the list of kmods? For my usecase I need it in a file [0], passing them on the command line will likely run into argument list length limitations. A new file 'modules.built' generated from modules.order would be useful both for this patch, mine and other tools like scripts/package/builddeb. (Or 'modules.loadable', matching the scheme from 'modules.builtin') On the other hand the modules.order seems to have quite some special logic in the Makefile, so it may not be straigtforward to add such a file. [0] https://lore.kernel.org/lkml/20250429-module-hashes-v3-9-00e9258def9e@weissschuh.net/#Z31scripts:module-hashes.sh Thomas ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2025-11-17 8:30 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2025-11-14 14:27 [PATCH v5 0/8] scripts/make_fit: Support ramdisks and faster operations Simon Glass 2025-11-14 14:27 ` [PATCH v5 5/8] kbuild: Split out module targets into a variable Simon Glass 2025-11-14 14:27 ` [PATCH v5 6/8] kbuild: Allow adding modules into the FIT ramdisk Simon Glass 2025-11-14 15:29 ` Thomas Weißschuh 2025-11-14 19:50 ` Nicolas Schier 2025-11-17 8:30 ` Thomas Weißschuh
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox