public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
* [PATCH v5 0/2] kbuild: distributed build support for Clang ThinLTO
@ 2025-10-28 18:28 xur
  2025-10-28 18:28 ` [PATCH v5 1/2] kbuild: move vmlinux.a build rule to scripts/Makefile.vmlinux_a xur
                   ` (5 more replies)
  0 siblings, 6 replies; 19+ messages in thread
From: xur @ 2025-10-28 18:28 UTC (permalink / raw)
  To: Masahiro Yamada, Nathan Chancellor, Nicolas Schier,
	Nick Desaulniers, Bill Wendling, Justin Stitt, Miguel Ojeda,
	Thomas Gleixner, Alice Ryhl, Sami Tolvanen,
	Mike Rapoport (Microsoft), Rafael Aquini, Michael Ellerman,
	Stafford Horne, Christophe Leroy, Piotr Gorski, Rong Xu,
	Teresa Johnson
  Cc: linux-kernel, linux-kbuild, llvm

From: Rong Xu <xur@google.com>

ChangeLog:

V5: Handle single quote escaping in echo _c_flags to fix
    Piotr Gorski's reported error.

V4: This is based on the reimplementation provided by Masahiro Yamada
on May 26. The difference is that this version now saves the compiler
flags (_c_flags) from the Front-End (FE) compilation and re-uses them
for the subsequent Back-End (BE) compilation.
 
Rong Xu (2):
  kbuild: move vmlinux.a build rule to scripts/Makefile.vmlinux_a
  kbuild: distributed build support for Clang ThinLTO

 .gitignore                 |  2 +
 Makefile                   | 25 +++++-------
 arch/Kconfig               | 19 +++++++++
 scripts/Makefile.lib       |  7 ++++
 scripts/Makefile.thinlto   | 40 ++++++++++++++++++
 scripts/Makefile.vmlinux_a | 83 ++++++++++++++++++++++++++++++++++++++
 scripts/mod/modpost.c      | 15 +++++--
 7 files changed, 174 insertions(+), 17 deletions(-)
 create mode 100644 scripts/Makefile.thinlto
 create mode 100644 scripts/Makefile.vmlinux_a


base-commit: dcb6fa37fd7bc9c3d2b066329b0d27dedf8becaa
-- 
2.51.1.851.g4ebd6896fd-goog


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [PATCH v5 1/2] kbuild: move vmlinux.a build rule to scripts/Makefile.vmlinux_a
  2025-10-28 18:28 [PATCH v5 0/2] kbuild: distributed build support for Clang ThinLTO xur
@ 2025-10-28 18:28 ` xur
  2026-03-16 13:33   ` Nicolas Schier
  2025-10-28 18:28 ` [PATCH v5 2/2] kbuild: distributed build support for Clang ThinLTO xur
                   ` (4 subsequent siblings)
  5 siblings, 1 reply; 19+ messages in thread
From: xur @ 2025-10-28 18:28 UTC (permalink / raw)
  To: Masahiro Yamada, Nathan Chancellor, Nicolas Schier,
	Nick Desaulniers, Bill Wendling, Justin Stitt, Miguel Ojeda,
	Thomas Gleixner, Alice Ryhl, Sami Tolvanen,
	Mike Rapoport (Microsoft), Rafael Aquini, Michael Ellerman,
	Stafford Horne, Christophe Leroy, Piotr Gorski, Rong Xu,
	Teresa Johnson
  Cc: linux-kernel, linux-kbuild, llvm

From: Rong Xu <xur@google.com>

From: Masahiro Yamada <masahiroy@kernel.org>

Move the build rule for vmlinux.a to a separate file in preparation
for supporting distributed builds with Clang ThinLTO.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Tested-by: Rong Xu <xur@google.com>
---
 Makefile                   | 16 +++++--------
 scripts/Makefile.vmlinux_a | 46 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 52 insertions(+), 10 deletions(-)
 create mode 100644 scripts/Makefile.vmlinux_a

diff --git a/Makefile b/Makefile
index b34a1f4c03967..89a25bac2bbab 100644
--- a/Makefile
+++ b/Makefile
@@ -1198,7 +1198,7 @@ export ARCH_DRIVERS	:= $(drivers-y) $(drivers-m)
 KBUILD_VMLINUX_OBJS := built-in.a $(patsubst %/, %/lib.a, $(filter %/, $(libs-y)))
 KBUILD_VMLINUX_LIBS := $(filter-out %/, $(libs-y))
 
-export KBUILD_VMLINUX_LIBS
+export KBUILD_VMLINUX_OBJS KBUILD_VMLINUX_LIBS
 export KBUILD_LDS          := arch/$(SRCARCH)/kernel/vmlinux.lds
 
 ifdef CONFIG_TRIM_UNUSED_KSYMS
@@ -1207,16 +1207,12 @@ ifdef CONFIG_TRIM_UNUSED_KSYMS
 KBUILD_MODULES := y
 endif
 
-# '$(AR) mPi' needs 'T' to workaround the bug of llvm-ar <= 14
-quiet_cmd_ar_vmlinux.a = AR      $@
-      cmd_ar_vmlinux.a = \
-	rm -f $@; \
-	$(AR) cDPrST $@ $(KBUILD_VMLINUX_OBJS); \
-	$(AR) mPiT $$($(AR) t $@ | sed -n 1p) $@ $$($(AR) t $@ | grep -F -f $(srctree)/scripts/head-object-list.txt)
+PHONY += vmlinux_a
+vmlinux_a: $(KBUILD_VMLINUX_OBJS) scripts/head-object-list.txt FORCE
+	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.vmlinux_a
 
-targets += vmlinux.a
-vmlinux.a: $(KBUILD_VMLINUX_OBJS) scripts/head-object-list.txt FORCE
-	$(call if_changed,ar_vmlinux.a)
+vmlinux.a: vmlinux_a
+	@:
 
 PHONY += vmlinux_o
 vmlinux_o: vmlinux.a $(KBUILD_VMLINUX_LIBS)
diff --git a/scripts/Makefile.vmlinux_a b/scripts/Makefile.vmlinux_a
new file mode 100644
index 0000000000000..9774f02b43b2f
--- /dev/null
+++ b/scripts/Makefile.vmlinux_a
@@ -0,0 +1,46 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+PHONY := __default
+__default: vmlinux.a
+
+include include/config/auto.conf
+include $(srctree)/scripts/Kbuild.include
+include $(srctree)/scripts/Makefile.lib
+
+# Link of built-in-fixup.a
+# ---------------------------------------------------------------------------
+
+# '$(AR) mPi' needs --thin to workaround the bug of llvm-ar <= 14
+quiet_cmd_ar_builtin_fixup = AR      $@
+      cmd_ar_builtin_fixup = \
+	rm -f $@; \
+	$(AR) cDPrS --thin $@ $(KBUILD_VMLINUX_OBJS); \
+	$(AR) mPi --thin $$($(AR) t $@ | sed -n 1p) $@ $$($(AR) t $@ | grep -F -f $(srctree)/scripts/head-object-list.txt)
+
+targets += built-in-fixup.a
+built-in-fixup.a: $(KBUILD_VMLINUX_OBJS) scripts/head-object-list.txt FORCE
+	$(call if_changed,ar_builtin_fixup)
+
+# vmlinux.a
+# ---------------------------------------------------------------------------
+
+targets += vmlinux.a
+vmlinux.a: built-in-fixup.a FORCE
+	$(call if_changed,copy)
+
+# Add FORCE to the prerequisites of a target to force it to be always rebuilt.
+# ---------------------------------------------------------------------------
+
+PHONY += FORCE
+FORCE:
+
+# Read all saved command lines and dependencies for the $(targets) we
+# may be building above, using $(if_changed{,_dep}). As an
+# optimization, we don't need to read them if the target does not
+# exist, we will rebuild anyway in that case.
+
+existing-targets := $(wildcard $(sort $(targets)))
+
+-include $(foreach f,$(existing-targets),$(dir $(f)).$(notdir $(f)).cmd)
+
+.PHONY: $(PHONY)
-- 
2.51.1.851.g4ebd6896fd-goog


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH v5 2/2] kbuild: distributed build support for Clang ThinLTO
  2025-10-28 18:28 [PATCH v5 0/2] kbuild: distributed build support for Clang ThinLTO xur
  2025-10-28 18:28 ` [PATCH v5 1/2] kbuild: move vmlinux.a build rule to scripts/Makefile.vmlinux_a xur
@ 2025-10-28 18:28 ` xur
  2025-10-28 19:22 ` [PATCH v5 0/2] " Piotr Gorski
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 19+ messages in thread
From: xur @ 2025-10-28 18:28 UTC (permalink / raw)
  To: Masahiro Yamada, Nathan Chancellor, Nicolas Schier,
	Nick Desaulniers, Bill Wendling, Justin Stitt, Miguel Ojeda,
	Thomas Gleixner, Alice Ryhl, Sami Tolvanen,
	Mike Rapoport (Microsoft), Rafael Aquini, Michael Ellerman,
	Stafford Horne, Christophe Leroy, Piotr Gorski, Rong Xu,
	Teresa Johnson
  Cc: linux-kernel, linux-kbuild, llvm

From: Rong Xu <xur@google.com>

Add distributed ThinLTO build support for the Linux kernel.
This new mode offers several advantages: (1) Increased
flexibility in handling user-specified build options.
(2) Improved user-friendliness for developers. (3) Greater
convenience for integrating with objtool and livepatch.

Note that "distributed" in this context refers to a term
that differentiates in-process ThinLTO builds by invoking
backend compilation through the linker, not necessarily
building in distributed environments.

Distributed ThinLTO is enabled via the
`CONFIG_LTO_CLANG_THIN_DIST` Kconfig option. For example:
 > make LLVM=1 defconfig
 > scripts/config -e LTO_CLANG_THIN_DIST
 > make LLVM=1 oldconfig
 > make LLVM=1 vmlinux -j <..>

The build flow proceeds in four stages:
  1. Perform FE compilation, mirroring the in-process ThinLTO mode.
  2. Thin-link the generated IR files and object files.
  3. Find all IR files and perform BE compilation, using the flags
    stored in the .*.o.cmd files.
  4. Link the BE results to generate the final vmlinux.o.

NOTE: This patch currently implements the build for the main kernel
image (vmlinux) only. Kernel module support is planned for a
subsequent patch.

Tested on the following arch: x86, arm64, loongarch, and
riscv.

The earlier implementation details can be found here:
https://discourse.llvm.org/t/rfc-distributed-thinlto-build-for-kernel/85934

Signed-off-by: Rong Xu <xur@google.com>
Co-developed-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---
 .gitignore                 |  2 ++
 Makefile                   |  9 +++++----
 arch/Kconfig               | 19 ++++++++++++++++++
 scripts/Makefile.lib       |  7 +++++++
 scripts/Makefile.thinlto   | 40 ++++++++++++++++++++++++++++++++++++++
 scripts/Makefile.vmlinux_a | 37 +++++++++++++++++++++++++++++++++++
 scripts/mod/modpost.c      | 15 +++++++++++---
 7 files changed, 122 insertions(+), 7 deletions(-)
 create mode 100644 scripts/Makefile.thinlto

diff --git a/.gitignore b/.gitignore
index 86a1ba0d90353..d34f28ca55400 100644
--- a/.gitignore
+++ b/.gitignore
@@ -55,6 +55,7 @@
 *.zst
 Module.symvers
 dtbs-list
+builtin.order
 modules.order
 
 #
@@ -66,6 +67,7 @@ modules.order
 /vmlinux.32
 /vmlinux.map
 /vmlinux.symvers
+/vmlinux.thinlto-index
 /vmlinux.unstripped
 /vmlinux-gdb.py
 /vmlinuz
diff --git a/Makefile b/Makefile
index 89a25bac2bbab..a608a082599d9 100644
--- a/Makefile
+++ b/Makefile
@@ -1000,10 +1000,10 @@ export CC_FLAGS_SCS
 endif
 
 ifdef CONFIG_LTO_CLANG
-ifdef CONFIG_LTO_CLANG_THIN
-CC_FLAGS_LTO	:= -flto=thin -fsplit-lto-unit
-else
+ifdef CONFIG_LTO_CLANG_FULL
 CC_FLAGS_LTO	:= -flto
+else
+CC_FLAGS_LTO	:= -flto=thin -fsplit-lto-unit
 endif
 CC_FLAGS_LTO	+= -fvisibility=hidden
 
@@ -1572,6 +1572,7 @@ endif # CONFIG_MODULES
 CLEAN_FILES += vmlinux.symvers modules-only.symvers \
 	       modules.builtin modules.builtin.modinfo modules.nsdeps \
 	       modules.builtin.ranges vmlinux.o.map vmlinux.unstripped \
+	       vmlinux.thinlto-index builtin.order \
 	       compile_commands.json rust/test \
 	       rust-project.json .vmlinux.objs .vmlinux.export.c \
                .builtin-dtbs-list .builtin-dtb.S
@@ -2014,7 +2015,7 @@ clean: $(clean-dirs)
 	$(call cmd,rmfiles)
 	@find . $(RCS_FIND_IGNORE) \
 		\( -name '*.[aios]' -o -name '*.rsi' -o -name '*.ko' -o -name '.*.cmd' \
-		-o -name '*.ko.*' \
+		-o -name '*.ko.*' -o -name '*.o.thinlto.bc' \
 		-o -name '*.dtb' -o -name '*.dtbo' \
 		-o -name '*.dtb.S' -o -name '*.dtbo.S' \
 		-o -name '*.dt.yaml' -o -name 'dtbs-list' \
diff --git a/arch/Kconfig b/arch/Kconfig
index 74ff011335322..f323ff2f12f87 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -864,6 +864,25 @@ config LTO_CLANG_THIN
 	    https://clang.llvm.org/docs/ThinLTO.html
 
 	  If unsure, say Y.
+
+config LTO_CLANG_THIN_DIST
+	bool "Clang ThinLTO in distributed mode (EXPERIMENTAL)"
+	depends on HAS_LTO_CLANG && ARCH_SUPPORTS_LTO_CLANG_THIN
+	select LTO_CLANG
+	help
+	  This option enables Clang's ThinLTO in distributed build mode.
+	  In this mode, the linker performs the thin-link, generating
+	  ThinLTO index files. Subsequently, the build system explicitly
+	  invokes ThinLTO backend compilation using these index files
+	  and pre-linked IR objects. The resulting native object files
+	  are with the .thinlto-native.o suffix.
+
+	  This build mode offers improved visibility into the ThinLTO
+	  process through explicit subcommand exposure. It also makes
+	  final native object files directly available, benefiting
+	  tools like objtool and kpatch. Additionally, it provides
+	  crucial granular control over back-end options, enabling
+	  module-specific compiler options, and simplifies debugging.
 endchoice
 
 config ARCH_SUPPORTS_AUTOFDO_CLANG
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 1d581ba5df66f..65bf1e4d31c0a 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -239,6 +239,12 @@ ifdef CONFIG_LTO_CLANG
 cmd_ld_single = $(if $(objtool-enabled)$(is-single-obj-m), ; $(LD) $(ld_flags) -r -o $(tmp-target) $@; mv $(tmp-target) $@)
 endif
 
+ifdef CONFIG_LTO_CLANG_THIN_DIST
+# Save the _c_flags, sliently.
+quiet_cmd_save_c_flags =
+      cmd_save_c_flags = printf '\n%s\n' 'saved_c_flags_$@ := $(call escsq,$(_c_flags))' >> $(dot-target).cmd
+endif
+
 quiet_cmd_cc_o_c = CC $(quiet_modtag)  $@
       cmd_cc_o_c = $(CC) $(c_flags) -c -o $@ $< \
 		$(cmd_ld_single) \
@@ -246,6 +252,7 @@ quiet_cmd_cc_o_c = CC $(quiet_modtag)  $@
 
 define rule_cc_o_c
 	$(call cmd_and_fixdep,cc_o_c)
+	$(call cmd,save_c_flags)
 	$(call cmd,checksrc)
 	$(call cmd,checkdoc)
 	$(call cmd,gen_objtooldep)
diff --git a/scripts/Makefile.thinlto b/scripts/Makefile.thinlto
new file mode 100644
index 0000000000000..03349ac69de5f
--- /dev/null
+++ b/scripts/Makefile.thinlto
@@ -0,0 +1,40 @@
+PHONY := __default
+__default:
+
+include include/config/auto.conf
+include $(srctree)/scripts/Kbuild.include
+include $(srctree)/scripts/Makefile.lib
+
+native-objs := $(patsubst %.o,%.thinlto-native.o,$(call read-file, vmlinux.thinlto-index))
+
+__default: $(native-objs)
+
+# Generate .thinlto-native.o (obj) from .o (bitcode) and .thinlto.bc (summary) files
+# ---------------------------------------------------------------------------
+quiet_cmd_cc_o_bc = CC $(quiet_modtag)  $@
+      be_flags = $(shell sed -n '/saved_c_flags_/s/.*:= //p' \
+		 $(dir $(<)).$(notdir $(<)).cmd)
+      cmd_cc_o_bc = \
+      $(CC) $(be_flags) -x ir -fno-lto -Wno-unused-command-line-argument \
+      -fthinlto-index=$(word 2, $^) -c -o $@ $<
+
+targets += $(native-objs)
+$(native-objs): %.thinlto-native.o: %.o %.o.thinlto.bc   FORCE
+	$(call if_changed,cc_o_bc)
+
+# Add FORCE to the prerequisites of a target to force it to be always rebuilt.
+# ---------------------------------------------------------------------------
+
+PHONY += FORCE
+FORCE:
+
+# Read all saved command lines and dependencies for the $(targets) we
+# may be building above, using $(if_changed{,_dep}). As an
+# optimization, we don't need to read them if the target does not
+# exist, we will rebuild anyway in that case.
+
+existing-targets := $(wildcard $(sort $(targets)))
+
+-include $(foreach f,$(existing-targets),$(dir $(f)).$(notdir $(f)).cmd)
+
+.PHONY: $(PHONY)
diff --git a/scripts/Makefile.vmlinux_a b/scripts/Makefile.vmlinux_a
index 9774f02b43b2f..382cebe1ee571 100644
--- a/scripts/Makefile.vmlinux_a
+++ b/scripts/Makefile.vmlinux_a
@@ -21,6 +21,41 @@ targets += built-in-fixup.a
 built-in-fixup.a: $(KBUILD_VMLINUX_OBJS) scripts/head-object-list.txt FORCE
 	$(call if_changed,ar_builtin_fixup)
 
+ifdef CONFIG_LTO_CLANG_THIN_DIST
+
+quiet_cmd_builtin.order = GEN     $@
+      cmd_builtin.order = $(AR) t $< > $@
+
+targets += builtin.order
+builtin.order: built-in-fixup.a FORCE
+	$(call if_changed,builtin.order)
+
+quiet_cmd_ld_thinlto_index = LD      $@
+      cmd_ld_thinlto_index = \
+	$(LD) $(KBUILD_LDFLAGS) -r --thinlto-index-only=$@ @$<
+
+targets += vmlinux.thinlto-index
+vmlinux.thinlto-index: builtin.order FORCE
+	$(call if_changed,ld_thinlto_index)
+
+quiet_cmd_ar_vmlinux.a = GEN     $@
+      cmd_ar_vmlinux.a =					\
+	rm -f $@;						\
+	while read -r obj; do					\
+		if grep -q $${obj} $(word 2, $^); then		\
+			echo $${obj%.o}.thinlto-native.o;	\
+		else						\
+			echo $${obj};				\
+		fi;						\
+	done < $< | xargs $(AR) cDPrS --thin $@
+
+targets += vmlinux.a
+vmlinux.a: builtin.order vmlinux.thinlto-index FORCE
+	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.thinlto
+	$(call if_changed,ar_vmlinux.a)
+
+else
+
 # vmlinux.a
 # ---------------------------------------------------------------------------
 
@@ -28,6 +63,8 @@ targets += vmlinux.a
 vmlinux.a: built-in-fixup.a FORCE
 	$(call if_changed,copy)
 
+endif
+
 # Add FORCE to the prerequisites of a target to force it to be always rebuilt.
 # ---------------------------------------------------------------------------
 
diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
index 47c8aa2a69392..38f62eb4bd8ba 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -1473,13 +1473,22 @@ static void extract_crcs_for_object(const char *object, struct module *mod)
 	char cmd_file[PATH_MAX];
 	char *buf, *p;
 	const char *base;
-	int dirlen, ret;
+	int dirlen, baselen_without_suffix, ret;
 
 	base = get_basename(object);
 	dirlen = base - object;
 
-	ret = snprintf(cmd_file, sizeof(cmd_file), "%.*s.%s.cmd",
-		       dirlen, object, base);
+	baselen_without_suffix = strlen(object) - dirlen - strlen(".o");
+
+	/*
+	 * When CONFIG_LTO_CLANG_THIN_DIST=y, the ELF is *.thinlto-native.o
+	 * but the symbol CRCs are recorded in *.o.cmd file.
+	 */
+	if (strends(object, ".thinlto-native.o"))
+		baselen_without_suffix -= strlen(".thinlto-native");
+
+	ret = snprintf(cmd_file, sizeof(cmd_file), "%.*s.%.*s.o.cmd",
+		       dirlen, object, baselen_without_suffix, base);
 	if (ret >= sizeof(cmd_file)) {
 		error("%s: too long path was truncated\n", cmd_file);
 		return;
-- 
2.51.1.851.g4ebd6896fd-goog


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* Re: [PATCH v5 0/2] kbuild: distributed build support for Clang ThinLTO
  2025-10-28 18:28 [PATCH v5 0/2] kbuild: distributed build support for Clang ThinLTO xur
  2025-10-28 18:28 ` [PATCH v5 1/2] kbuild: move vmlinux.a build rule to scripts/Makefile.vmlinux_a xur
  2025-10-28 18:28 ` [PATCH v5 2/2] kbuild: distributed build support for Clang ThinLTO xur
@ 2025-10-28 19:22 ` Piotr Gorski
  2025-12-04 17:49 ` Nathan Chancellor
                   ` (2 subsequent siblings)
  5 siblings, 0 replies; 19+ messages in thread
From: Piotr Gorski @ 2025-10-28 19:22 UTC (permalink / raw)
  To: xur, Masahiro Yamada, Nathan Chancellor, Nicolas Schier,
	Nick Desaulniers, Bill Wendling, Justin Stitt, Miguel Ojeda,
	Thomas Gleixner, Alice Ryhl, Sami Tolvanen,
	Mike Rapoport (Microsoft), Rafael Aquini, Michael Ellerman,
	Stafford Horne, Christophe Leroy, Teresa Johnson
  Cc: linux-kernel, linux-kbuild, llvm

Built and booted without any problems.

lucjan at cachyos ~ 20:19:27
❯ zgrep -i THIN_DIST /proc/config.gz
CONFIG_LTO_CLANG_THIN_DIST=y
lucjan at cachyos ~ 20:19:45
❯ cat /proc/version
Linux version 6.17.5-1.3-cachyos (linux-cachyos@cachyos) (clang version 
21.1.3, LLD 21.1.4) #1 SMP PREEMPT_DYNAMIC Tue, 28 Oct 2025 18:39:23 +0000

Thanks!

W dniu 28.10.2025 o 19:28, xur@google.com pisze:
> From: Rong Xu <xur@google.com>
>
> ChangeLog:
>
> V5: Handle single quote escaping in echo _c_flags to fix
>      Piotr Gorski's reported error.
>
> V4: This is based on the reimplementation provided by Masahiro Yamada
> on May 26. The difference is that this version now saves the compiler
> flags (_c_flags) from the Front-End (FE) compilation and re-uses them
> for the subsequent Back-End (BE) compilation.
>   
> Rong Xu (2):
>    kbuild: move vmlinux.a build rule to scripts/Makefile.vmlinux_a
>    kbuild: distributed build support for Clang ThinLTO
>
>   .gitignore                 |  2 +
>   Makefile                   | 25 +++++-------
>   arch/Kconfig               | 19 +++++++++
>   scripts/Makefile.lib       |  7 ++++
>   scripts/Makefile.thinlto   | 40 ++++++++++++++++++
>   scripts/Makefile.vmlinux_a | 83 ++++++++++++++++++++++++++++++++++++++
>   scripts/mod/modpost.c      | 15 +++++--
>   7 files changed, 174 insertions(+), 17 deletions(-)
>   create mode 100644 scripts/Makefile.thinlto
>   create mode 100644 scripts/Makefile.vmlinux_a
>
>
> base-commit: dcb6fa37fd7bc9c3d2b066329b0d27dedf8becaa

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v5 0/2] kbuild: distributed build support for Clang ThinLTO
  2025-10-28 18:28 [PATCH v5 0/2] kbuild: distributed build support for Clang ThinLTO xur
                   ` (2 preceding siblings ...)
  2025-10-28 19:22 ` [PATCH v5 0/2] " Piotr Gorski
@ 2025-12-04 17:49 ` Nathan Chancellor
  2025-12-04 18:36   ` Rong Xu
  2026-03-12  8:25 ` Nathan Chancellor
  2026-03-13 14:50 ` Nicolas Schier
  5 siblings, 1 reply; 19+ messages in thread
From: Nathan Chancellor @ 2025-12-04 17:49 UTC (permalink / raw)
  To: xur
  Cc: Masahiro Yamada, Nicolas Schier, Nick Desaulniers, Bill Wendling,
	Justin Stitt, Miguel Ojeda, Thomas Gleixner, Alice Ryhl,
	Sami Tolvanen, Mike Rapoport (Microsoft), Rafael Aquini,
	Michael Ellerman, Stafford Horne, Christophe Leroy, Piotr Gorski,
	Teresa Johnson, linux-kernel, linux-kbuild, llvm

Hi Rong,

On Tue, Oct 28, 2025 at 06:28:20PM +0000, xur@google.com wrote:

First of all, my apologies for taking so long to get to testing and
reviewing this patchset.

> Rong Xu (2):
>   kbuild: move vmlinux.a build rule to scripts/Makefile.vmlinux_a
>   kbuild: distributed build support for Clang ThinLTO
> 
>  .gitignore                 |  2 +
>  Makefile                   | 25 +++++-------
>  arch/Kconfig               | 19 +++++++++
>  scripts/Makefile.lib       |  7 ++++
>  scripts/Makefile.thinlto   | 40 ++++++++++++++++++
>  scripts/Makefile.vmlinux_a | 83 ++++++++++++++++++++++++++++++++++++++
>  scripts/mod/modpost.c      | 15 +++++--
>  7 files changed, 174 insertions(+), 17 deletions(-)
>  create mode 100644 scripts/Makefile.thinlto
>  create mode 100644 scripts/Makefile.vmlinux_a

Overall, this seems reasonable from a high level perspective. I have
been testing it with my arm64 and x86_64 distribution configurations for
the past couple of days and I have not noticed any issues.

Did you take a look at the robot report from patch 2?

  https://lore.kernel.org/202511052257.Bb85ptQG-lkp@intel.com/

It seems like it could be caused by different optimizations?

I plan to take a more in-depth look at the implementation after I am
back home from Plumbers in a couple of weeks (just to make sure I
understand it from a maintainer's perspective). Based on that, I will
either apply it to kbuild-next for 6.20/7.0 or ask for further
interations, while still aiming to get it into that release.

Cheers,
Natha

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v5 0/2] kbuild: distributed build support for Clang ThinLTO
  2025-12-04 17:49 ` Nathan Chancellor
@ 2025-12-04 18:36   ` Rong Xu
  0 siblings, 0 replies; 19+ messages in thread
From: Rong Xu @ 2025-12-04 18:36 UTC (permalink / raw)
  To: Nathan Chancellor
  Cc: Masahiro Yamada, Nicolas Schier, Nick Desaulniers, Bill Wendling,
	Justin Stitt, Miguel Ojeda, Thomas Gleixner, Alice Ryhl,
	Sami Tolvanen, Mike Rapoport (Microsoft), Rafael Aquini,
	Michael Ellerman, Stafford Horne, Christophe Leroy, Piotr Gorski,
	Teresa Johnson, linux-kernel, linux-kbuild, llvm

Thanks Nathan for testing and reviewing the patch!

On Thu, Dec 4, 2025 at 9:50 AM Nathan Chancellor <nathan@kernel.org> wrote:
>
> Hi Rong,
>
> On Tue, Oct 28, 2025 at 06:28:20PM +0000, xur@google.com wrote:
>
> First of all, my apologies for taking so long to get to testing and
> reviewing this patchset.
>
> > Rong Xu (2):
> >   kbuild: move vmlinux.a build rule to scripts/Makefile.vmlinux_a
> >   kbuild: distributed build support for Clang ThinLTO
> >
> >  .gitignore                 |  2 +
> >  Makefile                   | 25 +++++-------
> >  arch/Kconfig               | 19 +++++++++
> >  scripts/Makefile.lib       |  7 ++++
> >  scripts/Makefile.thinlto   | 40 ++++++++++++++++++
> >  scripts/Makefile.vmlinux_a | 83 ++++++++++++++++++++++++++++++++++++++
> >  scripts/mod/modpost.c      | 15 +++++--
> >  7 files changed, 174 insertions(+), 17 deletions(-)
> >  create mode 100644 scripts/Makefile.thinlto
> >  create mode 100644 scripts/Makefile.vmlinux_a
>
> Overall, this seems reasonable from a high level perspective. I have
> been testing it with my arm64 and x86_64 distribution configurations for
> the past couple of days and I have not noticed any issues.
>
> Did you take a look at the robot report from patch 2?
>
>   https://lore.kernel.org/202511052257.Bb85ptQG-lkp@intel.com/
>
> It seems like it could be caused by different optimizations?

I haven't tried to reproduce the issue in this report. However, based
on the warning messages, it looks fine to me: Distributed build mode
will have different optimizations from current in-process ThinLTO
build. This is mainly due to the fact that distributed build mode has
more precise option control. For a more detailed explanation, please
refer to my LLVM Discourse post:
https://discourse.llvm.org/t/rfc-distributed-thinlto-build-for-kernel/85934

Additionally, I will be presenting a talk on distributed ThinLTO
builds at the LPC next week and would be happy to discuss this topic
further with interested individuals in person.

Thanks,

-Rong

>
> I plan to take a more in-depth look at the implementation after I am
> back home from Plumbers in a couple of weeks (just to make sure I
> understand it from a maintainer's perspective). Based on that, I will
> either apply it to kbuild-next for 6.20/7.0 or ask for further
> interations, while still aiming to get it into that release.
>
> Cheers,
> Natha

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v5 0/2] kbuild: distributed build support for Clang ThinLTO
  2025-10-28 18:28 [PATCH v5 0/2] kbuild: distributed build support for Clang ThinLTO xur
                   ` (3 preceding siblings ...)
  2025-12-04 17:49 ` Nathan Chancellor
@ 2026-03-12  8:25 ` Nathan Chancellor
  2026-03-12 11:45   ` Nicolas Schier
  2026-03-13 14:50 ` Nicolas Schier
  5 siblings, 1 reply; 19+ messages in thread
From: Nathan Chancellor @ 2026-03-12  8:25 UTC (permalink / raw)
  To: xur, Nicolas Schier
  Cc: Masahiro Yamada, Nick Desaulniers, Bill Wendling, Justin Stitt,
	Miguel Ojeda, Thomas Gleixner, Alice Ryhl, Sami Tolvanen,
	Mike Rapoport (Microsoft), Rafael Aquini, Michael Ellerman,
	Stafford Horne, Christophe Leroy, Piotr Gorski, Teresa Johnson,
	linux-kernel, linux-kbuild, llvm

Hi Nicolas,

On Tue, Oct 28, 2025 at 06:28:20PM +0000, xur@google.com wrote:
> From: Rong Xu <xur@google.com>
> 
> ChangeLog:
> 
> V5: Handle single quote escaping in echo _c_flags to fix
>     Piotr Gorski's reported error.
> 
> V4: This is based on the reimplementation provided by Masahiro Yamada
> on May 26. The difference is that this version now saves the compiler
> flags (_c_flags) from the Front-End (FE) compilation and re-uses them
> for the subsequent Back-End (BE) compilation.
>  
> Rong Xu (2):
>   kbuild: move vmlinux.a build rule to scripts/Makefile.vmlinux_a
>   kbuild: distributed build support for Clang ThinLTO
> 
>  .gitignore                 |  2 +
>  Makefile                   | 25 +++++-------
>  arch/Kconfig               | 19 +++++++++
>  scripts/Makefile.lib       |  7 ++++
>  scripts/Makefile.thinlto   | 40 ++++++++++++++++++
>  scripts/Makefile.vmlinux_a | 83 ++++++++++++++++++++++++++++++++++++++
>  scripts/mod/modpost.c      | 15 +++++--
>  7 files changed, 174 insertions(+), 17 deletions(-)
>  create mode 100644 scripts/Makefile.thinlto
>  create mode 100644 scripts/Makefile.vmlinux_a

I have been running this for a few months locally and I have not noticed
any problems with it (at least for the two distribution configurations I
regularly test). I think this looks good overall and seems fairly well
contained thanks to Masahiro's work on the earlier revisions. Could this
be picked up in kbuild-next-unstable so it can bake in -next for a
little bit? It should still be applicable.

Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Tested-by: Nathan Chancellor <nathan@kernel.org>

Cheers,
Nathan

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v5 0/2] kbuild: distributed build support for Clang ThinLTO
  2026-03-12  8:25 ` Nathan Chancellor
@ 2026-03-12 11:45   ` Nicolas Schier
  2026-03-12 17:50     ` Rong Xu
  0 siblings, 1 reply; 19+ messages in thread
From: Nicolas Schier @ 2026-03-12 11:45 UTC (permalink / raw)
  To: Nathan Chancellor
  Cc: xur, Masahiro Yamada, Nick Desaulniers, Bill Wendling,
	Justin Stitt, Miguel Ojeda, Thomas Gleixner, Alice Ryhl,
	Sami Tolvanen, Mike Rapoport (Microsoft), Rafael Aquini,
	Michael Ellerman, Stafford Horne, Christophe Leroy, Piotr Gorski,
	Teresa Johnson, linux-kernel, linux-kbuild, llvm

On Thu, Mar 12, 2026 at 01:25:02AM -0700, Nathan Chancellor wrote:
> Hi Nicolas,
> 
> On Tue, Oct 28, 2025 at 06:28:20PM +0000, xur@google.com wrote:
> > From: Rong Xu <xur@google.com>
> > 
> > ChangeLog:
> > 
> > V5: Handle single quote escaping in echo _c_flags to fix
> >     Piotr Gorski's reported error.
> > 
> > V4: This is based on the reimplementation provided by Masahiro Yamada
> > on May 26. The difference is that this version now saves the compiler
> > flags (_c_flags) from the Front-End (FE) compilation and re-uses them
> > for the subsequent Back-End (BE) compilation.
> >  
> > Rong Xu (2):
> >   kbuild: move vmlinux.a build rule to scripts/Makefile.vmlinux_a
> >   kbuild: distributed build support for Clang ThinLTO
> > 
> >  .gitignore                 |  2 +
> >  Makefile                   | 25 +++++-------
> >  arch/Kconfig               | 19 +++++++++
> >  scripts/Makefile.lib       |  7 ++++
> >  scripts/Makefile.thinlto   | 40 ++++++++++++++++++
> >  scripts/Makefile.vmlinux_a | 83 ++++++++++++++++++++++++++++++++++++++
> >  scripts/mod/modpost.c      | 15 +++++--
> >  7 files changed, 174 insertions(+), 17 deletions(-)
> >  create mode 100644 scripts/Makefile.thinlto
> >  create mode 100644 scripts/Makefile.vmlinux_a
> 
> I have been running this for a few months locally and I have not noticed
> any problems with it (at least for the two distribution configurations I
> regularly test). I think this looks good overall and seems fairly well
> contained thanks to Masahiro's work on the earlier revisions. Could this
> be picked up in kbuild-next-unstable so it can bake in -next for a
> little bit? It should still be applicable.
> 
> Reviewed-by: Nathan Chancellor <nathan@kernel.org>
> Tested-by: Nathan Chancellor <nathan@kernel.org>

Thanks!  Sure, I will pick the series up.

Kind regards,
Nicolas

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v5 0/2] kbuild: distributed build support for Clang ThinLTO
  2026-03-12 11:45   ` Nicolas Schier
@ 2026-03-12 17:50     ` Rong Xu
  0 siblings, 0 replies; 19+ messages in thread
From: Rong Xu @ 2026-03-12 17:50 UTC (permalink / raw)
  To: Nicolas Schier
  Cc: Nathan Chancellor, Masahiro Yamada, Nick Desaulniers,
	Bill Wendling, Justin Stitt, Miguel Ojeda, Thomas Gleixner,
	Alice Ryhl, Sami Tolvanen, Mike Rapoport (Microsoft),
	Rafael Aquini, Michael Ellerman, Stafford Horne, Christophe Leroy,
	Piotr Gorski, Teresa Johnson, linux-kernel, linux-kbuild, llvm

I appreciate Nathan and Nicolas taking this on. Should there be
anything required from my side, please let me know.

-Rong

On Thu, Mar 12, 2026 at 4:46 AM Nicolas Schier <nsc@kernel.org> wrote:
>
> On Thu, Mar 12, 2026 at 01:25:02AM -0700, Nathan Chancellor wrote:
> > Hi Nicolas,
> >
> > On Tue, Oct 28, 2025 at 06:28:20PM +0000, xur@google.com wrote:
> > > From: Rong Xu <xur@google.com>
> > >
> > > ChangeLog:
> > >
> > > V5: Handle single quote escaping in echo _c_flags to fix
> > >     Piotr Gorski's reported error.
> > >
> > > V4: This is based on the reimplementation provided by Masahiro Yamada
> > > on May 26. The difference is that this version now saves the compiler
> > > flags (_c_flags) from the Front-End (FE) compilation and re-uses them
> > > for the subsequent Back-End (BE) compilation.
> > >
> > > Rong Xu (2):
> > >   kbuild: move vmlinux.a build rule to scripts/Makefile.vmlinux_a
> > >   kbuild: distributed build support for Clang ThinLTO
> > >
> > >  .gitignore                 |  2 +
> > >  Makefile                   | 25 +++++-------
> > >  arch/Kconfig               | 19 +++++++++
> > >  scripts/Makefile.lib       |  7 ++++
> > >  scripts/Makefile.thinlto   | 40 ++++++++++++++++++
> > >  scripts/Makefile.vmlinux_a | 83 ++++++++++++++++++++++++++++++++++++++
> > >  scripts/mod/modpost.c      | 15 +++++--
> > >  7 files changed, 174 insertions(+), 17 deletions(-)
> > >  create mode 100644 scripts/Makefile.thinlto
> > >  create mode 100644 scripts/Makefile.vmlinux_a
> >
> > I have been running this for a few months locally and I have not noticed
> > any problems with it (at least for the two distribution configurations I
> > regularly test). I think this looks good overall and seems fairly well
> > contained thanks to Masahiro's work on the earlier revisions. Could this
> > be picked up in kbuild-next-unstable so it can bake in -next for a
> > little bit? It should still be applicable.
> >
> > Reviewed-by: Nathan Chancellor <nathan@kernel.org>
> > Tested-by: Nathan Chancellor <nathan@kernel.org>
>
> Thanks!  Sure, I will pick the series up.
>
> Kind regards,
> Nicolas

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v5 0/2] kbuild: distributed build support for Clang ThinLTO
  2025-10-28 18:28 [PATCH v5 0/2] kbuild: distributed build support for Clang ThinLTO xur
                   ` (4 preceding siblings ...)
  2026-03-12  8:25 ` Nathan Chancellor
@ 2026-03-13 14:50 ` Nicolas Schier
  5 siblings, 0 replies; 19+ messages in thread
From: Nicolas Schier @ 2026-03-13 14:50 UTC (permalink / raw)
  To: Masahiro Yamada, Nathan Chancellor, Nicolas Schier,
	Nick Desaulniers, Bill Wendling, Justin Stitt, Miguel Ojeda,
	Alice Ryhl, Sami Tolvanen, Mike Rapoport (Microsoft),
	Rafael Aquini, Michael Ellerman, Stafford Horne, Piotr Gorski,
	Teresa Johnson, Thomas Gleixner, Christophe Leroy, xur
  Cc: Nicolas Schier, linux-kernel, linux-kbuild, llvm

On Tue, 28 Oct 2025 18:28:20 +0000, xur@google.com wrote:
> ChangeLog:
> 
> V5: Handle single quote escaping in echo _c_flags to fix
>     Piotr Gorski's reported error.
> 
> V4: This is based on the reimplementation provided by Masahiro Yamada
> on May 26. The difference is that this version now saves the compiler
> flags (_c_flags) from the Front-End (FE) compilation and re-uses them
> for the subsequent Back-End (BE) compilation.
> 
> [...]

Applied to kbuild/kbuild-next.git (kbuild-next-unstable), thanks!

[1/2] kbuild: move vmlinux.a build rule to scripts/Makefile.vmlinux_a
      https://git.kernel.org/kbuild/c/45192218
[2/2] kbuild: distributed build support for Clang ThinLTO
      https://git.kernel.org/kbuild/c/8e786d89

Please look out for regression or issue reports or other follow up
comments, as they may result in the patch/series getting dropped,
reverted or modified (e.g. trailers). Patches applied to the
kbuild-next-unstable branch are accepted pending wider testing in
linux-next and any post-commit review; they will generally be moved
to the kbuild-next branch in about a week if no issues are found.

Best regards,
-- 
Nicolas


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v5 1/2] kbuild: move vmlinux.a build rule to scripts/Makefile.vmlinux_a
  2025-10-28 18:28 ` [PATCH v5 1/2] kbuild: move vmlinux.a build rule to scripts/Makefile.vmlinux_a xur
@ 2026-03-16 13:33   ` Nicolas Schier
  2026-03-16 13:59     ` Miguel Ojeda
  0 siblings, 1 reply; 19+ messages in thread
From: Nicolas Schier @ 2026-03-16 13:33 UTC (permalink / raw)
  To: xur
  Cc: Masahiro Yamada, Nathan Chancellor, Nick Desaulniers,
	Bill Wendling, Justin Stitt, Miguel Ojeda, Thomas Gleixner,
	Alice Ryhl, Sami Tolvanen, Mike Rapoport (Microsoft),
	Rafael Aquini, Michael Ellerman, Stafford Horne, Christophe Leroy,
	Piotr Gorski, Teresa Johnson, linux-kernel, linux-kbuild, llvm

[-- Attachment #1: Type: text/plain, Size: 4006 bytes --]

On Tue, Oct 28, 2025 at 06:28:21PM +0000, xur@google.com wrote:
> From: Rong Xu <xur@google.com>
> 
> From: Masahiro Yamada <masahiroy@kernel.org>
> 
> Move the build rule for vmlinux.a to a separate file in preparation
> for supporting distributed builds with Clang ThinLTO.
> 
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> Tested-by: Rong Xu <xur@google.com>
> ---
>  Makefile                   | 16 +++++--------
>  scripts/Makefile.vmlinux_a | 46 ++++++++++++++++++++++++++++++++++++++
>  2 files changed, 52 insertions(+), 10 deletions(-)
>  create mode 100644 scripts/Makefile.vmlinux_a
> 
> diff --git a/Makefile b/Makefile
> index b34a1f4c03967..89a25bac2bbab 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1198,7 +1198,7 @@ export ARCH_DRIVERS	:= $(drivers-y) $(drivers-m)
>  KBUILD_VMLINUX_OBJS := built-in.a $(patsubst %/, %/lib.a, $(filter %/, $(libs-y)))
>  KBUILD_VMLINUX_LIBS := $(filter-out %/, $(libs-y))
>  
> -export KBUILD_VMLINUX_LIBS
> +export KBUILD_VMLINUX_OBJS KBUILD_VMLINUX_LIBS
>  export KBUILD_LDS          := arch/$(SRCARCH)/kernel/vmlinux.lds
>  
>  ifdef CONFIG_TRIM_UNUSED_KSYMS
> @@ -1207,16 +1207,12 @@ ifdef CONFIG_TRIM_UNUSED_KSYMS
>  KBUILD_MODULES := y
>  endif
>  
> -# '$(AR) mPi' needs 'T' to workaround the bug of llvm-ar <= 14
> -quiet_cmd_ar_vmlinux.a = AR      $@
> -      cmd_ar_vmlinux.a = \
> -	rm -f $@; \
> -	$(AR) cDPrST $@ $(KBUILD_VMLINUX_OBJS); \
> -	$(AR) mPiT $$($(AR) t $@ | sed -n 1p) $@ $$($(AR) t $@ | grep -F -f $(srctree)/scripts/head-object-list.txt)
> +PHONY += vmlinux_a
> +vmlinux_a: $(KBUILD_VMLINUX_OBJS) scripts/head-object-list.txt FORCE
> +	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.vmlinux_a
>  
> -targets += vmlinux.a
> -vmlinux.a: $(KBUILD_VMLINUX_OBJS) scripts/head-object-list.txt FORCE
> -	$(call if_changed,ar_vmlinux.a)
> +vmlinux.a: vmlinux_a
> +	@:
>  
>  PHONY += vmlinux_o
>  vmlinux_o: vmlinux.a $(KBUILD_VMLINUX_LIBS)
> diff --git a/scripts/Makefile.vmlinux_a b/scripts/Makefile.vmlinux_a
> new file mode 100644
> index 0000000000000..9774f02b43b2f
> --- /dev/null
> +++ b/scripts/Makefile.vmlinux_a
> @@ -0,0 +1,46 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +
> +PHONY := __default
> +__default: vmlinux.a
> +
> +include include/config/auto.conf
> +include $(srctree)/scripts/Kbuild.include
> +include $(srctree)/scripts/Makefile.lib
> +
> +# Link of built-in-fixup.a
> +# ---------------------------------------------------------------------------
> +
> +# '$(AR) mPi' needs --thin to workaround the bug of llvm-ar <= 14
> +quiet_cmd_ar_builtin_fixup = AR      $@
> +      cmd_ar_builtin_fixup = \
> +	rm -f $@; \
> +	$(AR) cDPrS --thin $@ $(KBUILD_VMLINUX_OBJS); \
> +	$(AR) mPi --thin $$($(AR) t $@ | sed -n 1p) $@ $$($(AR) t $@ | grep -F -f $(srctree)/scripts/head-object-list.txt)

Venkat Rao Bagalkote <venkat88@linux.ibm.com> reported [1] that this
breaks builds on system with binutils < 2.38.
Documentation/process/changes.rst state binutils 2.30 as build
dependency; binutils 2.38 has been released in 02/2022.

I removed the patch set from kbuild-next-unstable and kbuild-for-next
for now.

The simpliest solution would be to declare binutils 2.38 as build
dependency; but I am afraid that it is still "too new", right now.

Can someone elaborate a bit on the comment about --thin and "the bug of
llvm-ar <= 14'?

May it be an option to introduce $(ar-option) macro?
Something like this? (untested)

ar-option = $(call try-run, \
		( trap 'rm -f $(tmp-target).ar-option' EXIT INT HUP QUIT; \
		  $(AR) cr $(1) $(tmp-target).ar-option ),$(1),$(2) )
AR_THIN = $(call ar-option, --thin)

...

	$(AR) cDPrS $(AR_THIN) $@ $(KBUILD_VMLINUX_OBJS); \
	$(AR) mPi $(AR_THIN) $$($(AR) t $@ | sed -n 1p) $@ $$($(AR) t $@ | grep -F -f $(srctree)/scripts/head-object-list.txt)


Kind regards,
Nicolas


[1]: https://lore.kernel.org/linux-next/476507c9-a371-4864-9e87-572c1ecae82d@linux.ibm.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v5 1/2] kbuild: move vmlinux.a build rule to scripts/Makefile.vmlinux_a
  2026-03-16 13:33   ` Nicolas Schier
@ 2026-03-16 13:59     ` Miguel Ojeda
  2026-03-16 17:27       ` Rong Xu
  0 siblings, 1 reply; 19+ messages in thread
From: Miguel Ojeda @ 2026-03-16 13:59 UTC (permalink / raw)
  To: Nicolas Schier, xur, Masahiro Yamada, Nathan Chancellor,
	Nick Desaulniers, Bill Wendling, Justin Stitt, Miguel Ojeda,
	Thomas Gleixner, Alice Ryhl, Sami Tolvanen,
	Mike Rapoport (Microsoft), Rafael Aquini, Michael Ellerman,
	Stafford Horne, Christophe Leroy, Piotr Gorski, Teresa Johnson,
	linux-kernel, linux-kbuild, llvm, Arnd Bergmann

On Mon, Mar 16, 2026 at 2:40 PM Nicolas Schier <nsc@kernel.org> wrote:
>
> The simpliest solution would be to declare binutils 2.38 as build
> dependency; but I am afraid that it is still "too new", right now.

From the last bump at commit 118c40b7b503 ("kbuild: require gcc-8 and
binutils-2.30"), it seems binutils' minimum is supposed to go at
roughly the same pace as GCC's. Cc'ing Arnd.

Debian Old Stable has 2.40, so we would leave out Old Old Stable which
would have a recent enough GCC otherwise, so I guess that means we
cannot?

(I was taking a look at this minimum yesterday, because I was thinking
of removing the comment for a workaround related to binutils <= 2.36
in `Documentation/rust/quick-start.rst` when I bump the Rust minimum.
Of course, that comment is less critical, and those debugging a
Rust-enabled kernel likely have newer binutils anyway).

Cheers,
Miguel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v5 1/2] kbuild: move vmlinux.a build rule to scripts/Makefile.vmlinux_a
  2026-03-16 13:59     ` Miguel Ojeda
@ 2026-03-16 17:27       ` Rong Xu
  2026-03-16 17:52         ` Nicolas Schier
  2026-03-16 17:59         ` Nathan Chancellor
  0 siblings, 2 replies; 19+ messages in thread
From: Rong Xu @ 2026-03-16 17:27 UTC (permalink / raw)
  To: Miguel Ojeda
  Cc: Nicolas Schier, Masahiro Yamada, Nathan Chancellor,
	Nick Desaulniers, Bill Wendling, Justin Stitt, Miguel Ojeda,
	Thomas Gleixner, Alice Ryhl, Sami Tolvanen,
	Mike Rapoport (Microsoft), Rafael Aquini, Michael Ellerman,
	Stafford Horne, Christophe Leroy, Piotr Gorski, Teresa Johnson,
	linux-kernel, linux-kbuild, llvm, Arnd Bergmann

I think the problem was $(AR) mPi and llvm-ar <= 14 do work correctly
in preserving the orders for "thin" archives without --thin.

We can either
(1) Bump the LLVM version to 15 and remove the --thin flag, or
(2) Implement a condition, as the --thin flag is only required for
llvm-ar. I assume the reported error was using gcc? (can someone send
a link for the failure?)

I think (2) is less involved. I can prepare a updated patch.

-Rong

On Mon, Mar 16, 2026 at 1:59 PM Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
>
> On Mon, Mar 16, 2026 at 2:40 PM Nicolas Schier <nsc@kernel.org> wrote:
> >
> > The simpliest solution would be to declare binutils 2.38 as build
> > dependency; but I am afraid that it is still "too new", right now.
>
> From the last bump at commit 118c40b7b503 ("kbuild: require gcc-8 and
> binutils-2.30"), it seems binutils' minimum is supposed to go at
> roughly the same pace as GCC's. Cc'ing Arnd.
>
> Debian Old Stable has 2.40, so we would leave out Old Old Stable which
> would have a recent enough GCC otherwise, so I guess that means we
> cannot?
>
> (I was taking a look at this minimum yesterday, because I was thinking
> of removing the comment for a workaround related to binutils <= 2.36
> in `Documentation/rust/quick-start.rst` when I bump the Rust minimum.
> Of course, that comment is less critical, and those debugging a
> Rust-enabled kernel likely have newer binutils anyway).
>
> Cheers,
> Miguel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v5 1/2] kbuild: move vmlinux.a build rule to scripts/Makefile.vmlinux_a
  2026-03-16 17:27       ` Rong Xu
@ 2026-03-16 17:52         ` Nicolas Schier
  2026-03-16 17:59         ` Nathan Chancellor
  1 sibling, 0 replies; 19+ messages in thread
From: Nicolas Schier @ 2026-03-16 17:52 UTC (permalink / raw)
  To: Rong Xu
  Cc: Miguel Ojeda, Masahiro Yamada, Nathan Chancellor,
	Nick Desaulniers, Bill Wendling, Justin Stitt, Miguel Ojeda,
	Thomas Gleixner, Alice Ryhl, Sami Tolvanen,
	Mike Rapoport (Microsoft), Rafael Aquini, Michael Ellerman,
	Stafford Horne, Christophe Leroy, Piotr Gorski, Teresa Johnson,
	linux-kernel, linux-kbuild, llvm, Arnd Bergmann

On Mon, Mar 16, 2026 at 05:27:01PM +0000, Rong Xu wrote:
> I think the problem was $(AR) mPi and llvm-ar <= 14 do work correctly
> in preserving the orders for "thin" archives without --thin.
> 
> We can either
> (1) Bump the LLVM version to 15 and remove the --thin flag, or
> (2) Implement a condition, as the --thin flag is only required for
> llvm-ar. I assume the reported error was using gcc? (can someone send
> a link for the failure?)
> 
> I think (2) is less involved. I can prepare a updated patch.

Great, thanks!  Yes, the report was using gcc.

The report is at https://lore.kernel.org/linux-next/476507c9-a371-4864-9e87-572c1ecae82d@linux.ibm.com/

Kind regards,
Nicolas

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v5 1/2] kbuild: move vmlinux.a build rule to scripts/Makefile.vmlinux_a
  2026-03-16 17:27       ` Rong Xu
  2026-03-16 17:52         ` Nicolas Schier
@ 2026-03-16 17:59         ` Nathan Chancellor
  2026-03-16 18:16           ` Rong Xu
  1 sibling, 1 reply; 19+ messages in thread
From: Nathan Chancellor @ 2026-03-16 17:59 UTC (permalink / raw)
  To: Rong Xu
  Cc: Miguel Ojeda, Nicolas Schier, Masahiro Yamada, Nick Desaulniers,
	Bill Wendling, Justin Stitt, Miguel Ojeda, Thomas Gleixner,
	Alice Ryhl, Sami Tolvanen, Mike Rapoport (Microsoft),
	Rafael Aquini, Michael Ellerman, Stafford Horne, Christophe Leroy,
	Piotr Gorski, Teresa Johnson, linux-kernel, linux-kbuild, llvm,
	Arnd Bergmann

On Mon, Mar 16, 2026 at 05:27:01PM +0000, Rong Xu wrote:
> I think the problem was $(AR) mPi and llvm-ar <= 14 do work correctly
> in preserving the orders for "thin" archives without --thin.
> 
> We can either
> (1) Bump the LLVM version to 15 and remove the --thin flag, or

The minimum supported version of LLVM for building the kernel was
bumped to 15 in 6.18 with commit 20c098928356 ("kbuild: Bump minimum
version of LLVM for building the kernel to 15.0.0"), so it seems like we
can just do this?

Cheers,
Nathan

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v5 1/2] kbuild: move vmlinux.a build rule to scripts/Makefile.vmlinux_a
  2026-03-16 17:59         ` Nathan Chancellor
@ 2026-03-16 18:16           ` Rong Xu
  2026-03-16 20:43             ` Nathan Chancellor
  0 siblings, 1 reply; 19+ messages in thread
From: Rong Xu @ 2026-03-16 18:16 UTC (permalink / raw)
  To: Nathan Chancellor
  Cc: Miguel Ojeda, Nicolas Schier, Masahiro Yamada, Nick Desaulniers,
	Bill Wendling, Justin Stitt, Miguel Ojeda, Thomas Gleixner,
	Alice Ryhl, Sami Tolvanen, Mike Rapoport (Microsoft),
	Rafael Aquini, Michael Ellerman, Stafford Horne, Christophe Leroy,
	Piotr Gorski, Teresa Johnson, linux-kernel, linux-kbuild, llvm,
	Arnd Bergmann

If that's the case, we can just remove flag "--thin". Can we verify if
that works?

-Rong

On Mon, Mar 16, 2026 at 11:00 AM Nathan Chancellor <nathan@kernel.org> wrote:
>
> On Mon, Mar 16, 2026 at 05:27:01PM +0000, Rong Xu wrote:
> > I think the problem was $(AR) mPi and llvm-ar <= 14 do work correctly
> > in preserving the orders for "thin" archives without --thin.
> >
> > We can either
> > (1) Bump the LLVM version to 15 and remove the --thin flag, or
>
> The minimum supported version of LLVM for building the kernel was
> bumped to 15 in 6.18 with commit 20c098928356 ("kbuild: Bump minimum
> version of LLVM for building the kernel to 15.0.0"), so it seems like we
> can just do this?
>
> Cheers,
> Nathan

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v5 1/2] kbuild: move vmlinux.a build rule to scripts/Makefile.vmlinux_a
  2026-03-16 18:16           ` Rong Xu
@ 2026-03-16 20:43             ` Nathan Chancellor
  2026-03-16 20:53               ` Rong Xu
  0 siblings, 1 reply; 19+ messages in thread
From: Nathan Chancellor @ 2026-03-16 20:43 UTC (permalink / raw)
  To: Rong Xu
  Cc: Miguel Ojeda, Nicolas Schier, Masahiro Yamada, Nick Desaulniers,
	Bill Wendling, Justin Stitt, Miguel Ojeda, Thomas Gleixner,
	Alice Ryhl, Sami Tolvanen, Mike Rapoport (Microsoft),
	Rafael Aquini, Michael Ellerman, Stafford Horne, Christophe Leroy,
	Piotr Gorski, Teresa Johnson, linux-kernel, linux-kbuild, llvm,
	Arnd Bergmann

On Mon, Mar 16, 2026 at 11:16:41AM -0700, Rong Xu wrote:
> If that's the case, we can just remove flag "--thin". Can we verify if
> that works?

I think we can only remove it from the second llvm-ar invocation. I
tested

diff --git a/Makefile b/Makefile
index 2b15f0b4a0cb..fb001e02cc0f 100644
--- a/Makefile
+++ b/Makefile
@@ -1260,12 +1260,11 @@ ifdef CONFIG_TRIM_UNUSED_KSYMS
 KBUILD_MODULES := y
 endif
 
-# '$(AR) mPi' needs 'T' to workaround the bug of llvm-ar <= 14
 quiet_cmd_ar_vmlinux.a = AR      $@
       cmd_ar_vmlinux.a = \
 	rm -f $@; \
 	$(AR) cDPrST $@ $(KBUILD_VMLINUX_OBJS); \
-	$(AR) mPiT $$($(AR) t $@ | sed -n 1p) $@ $$($(AR) t $@ | grep -F -f $(srctree)/scripts/head-object-list.txt)
+	$(AR) mPi $$($(AR) t $@ | sed -n 1p) $@ $$($(AR) t $@ | grep -F -f $(srctree)/scripts/head-object-list.txt)
 
 targets += vmlinux.a
 vmlinux.a: $(KBUILD_VMLINUX_OBJS) scripts/head-object-list.txt FORCE
--

on 7.0-rc4, which showed no issues. This is basically a revert of my
suggested workaround for the original issue:

  https://lore.kernel.org/YyjjT5gQ2hGMH0ni@dev-arch.thelio-3990X/

You could add this as a prerequisite patch, I don't think it should be
squashed into the move change, especially since that appears to be why
we are in this situation. Why are we converting from the 'T' modifier to
'--thin' in the move patch? I know the documentation of llvm-ar and GNU
ar says that 'T' is deprecated in favor of '--thin' because it may do
different things on various ar implementations but the kernel only
supports these two implementations. I think we should just copy the
commands as they are and address the deprecation separately, perhaps
with an ar-option like Nicolas suggested upthread.

How about a v3 that looks like:

Patch 1: The diff above because all supported llvm-ar versions do the
same thing as GNU ar.

Patch 2: The move patch without changing 'T' into '--thin'.

Patch 3: The same as before, perhaps without '--thin' as well.

You'll need to base on kbuild-next-unstable [1] to address the conflict
with Yonghong's "kbuild: Reduce the number of compiler-generated
suffixes for clang thin-lto build" [2].

[1]: https://git.kernel.org/kbuild/l/kbuild-next-unstable
[2]: https://lore.kernel.org/20260307050250.3767489-1-yonghong.song@linux.dev/

Cheers,
Nathan

^ permalink raw reply related	[flat|nested] 19+ messages in thread

* Re: [PATCH v5 1/2] kbuild: move vmlinux.a build rule to scripts/Makefile.vmlinux_a
  2026-03-16 20:43             ` Nathan Chancellor
@ 2026-03-16 20:53               ` Rong Xu
  2026-03-16 21:31                 ` Rong Xu
  0 siblings, 1 reply; 19+ messages in thread
From: Rong Xu @ 2026-03-16 20:53 UTC (permalink / raw)
  To: Nathan Chancellor
  Cc: Miguel Ojeda, Nicolas Schier, Masahiro Yamada, Nick Desaulniers,
	Bill Wendling, Justin Stitt, Miguel Ojeda, Thomas Gleixner,
	Alice Ryhl, Sami Tolvanen, Mike Rapoport (Microsoft),
	Rafael Aquini, Michael Ellerman, Stafford Horne, Christophe Leroy,
	Piotr Gorski, Teresa Johnson, linux-kernel, linux-kbuild, llvm,
	Arnd Bergmann

Sorry that my previous email may have caused some confusion.
What I really meant was to keep -T (not change to '--thin').

I think Masahiro's change targeted llvm-ar (<= 14), where it uses -T
for "truncate", rather '--thin'. So he used ''--thin" explicitly. But
this "--thin" was not implemented for some old ar.

Since we the minimal llvm is now 15. The '-T' flag in legacy llvm-ar
is no longer a problem. We should keep '-T'.

-Rong


On Mon, Mar 16, 2026 at 8:43 PM Nathan Chancellor <nathan@kernel.org> wrote:
>
> On Mon, Mar 16, 2026 at 11:16:41AM -0700, Rong Xu wrote:
> > If that's the case, we can just remove flag "--thin". Can we verify if
> > that works?
>
> I think we can only remove it from the second llvm-ar invocation. I
> tested
>
> diff --git a/Makefile b/Makefile
> index 2b15f0b4a0cb..fb001e02cc0f 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1260,12 +1260,11 @@ ifdef CONFIG_TRIM_UNUSED_KSYMS
>  KBUILD_MODULES := y
>  endif
>
> -# '$(AR) mPi' needs 'T' to workaround the bug of llvm-ar <= 14
>  quiet_cmd_ar_vmlinux.a = AR      $@
>        cmd_ar_vmlinux.a = \
>         rm -f $@; \
>         $(AR) cDPrST $@ $(KBUILD_VMLINUX_OBJS); \
> -       $(AR) mPiT $$($(AR) t $@ | sed -n 1p) $@ $$($(AR) t $@ | grep -F -f $(srctree)/scripts/head-object-list.txt)
> +       $(AR) mPi $$($(AR) t $@ | sed -n 1p) $@ $$($(AR) t $@ | grep -F -f $(srctree)/scripts/head-object-list.txt)
>
>  targets += vmlinux.a
>  vmlinux.a: $(KBUILD_VMLINUX_OBJS) scripts/head-object-list.txt FORCE
> --
>
> on 7.0-rc4, which showed no issues. This is basically a revert of my
> suggested workaround for the original issue:
>
>   https://lore.kernel.org/YyjjT5gQ2hGMH0ni@dev-arch.thelio-3990X/
>
> You could add this as a prerequisite patch, I don't think it should be
> squashed into the move change, especially since that appears to be why
> we are in this situation. Why are we converting from the 'T' modifier to
> '--thin' in the move patch? I know the documentation of llvm-ar and GNU
> ar says that 'T' is deprecated in favor of '--thin' because it may do
> different things on various ar implementations but the kernel only
> supports these two implementations. I think we should just copy the
> commands as they are and address the deprecation separately, perhaps
> with an ar-option like Nicolas suggested upthread.
>
> How about a v3 that looks like:
>
> Patch 1: The diff above because all supported llvm-ar versions do the
> same thing as GNU ar.
>
> Patch 2: The move patch without changing 'T' into '--thin'.
>
> Patch 3: The same as before, perhaps without '--thin' as well.
>
> You'll need to base on kbuild-next-unstable [1] to address the conflict
> with Yonghong's "kbuild: Reduce the number of compiler-generated
> suffixes for clang thin-lto build" [2].
>
> [1]: https://git.kernel.org/kbuild/l/kbuild-next-unstable
> [2]: https://lore.kernel.org/20260307050250.3767489-1-yonghong.song@linux.dev/
>
> Cheers,
> Nathan

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v5 1/2] kbuild: move vmlinux.a build rule to scripts/Makefile.vmlinux_a
  2026-03-16 20:53               ` Rong Xu
@ 2026-03-16 21:31                 ` Rong Xu
  0 siblings, 0 replies; 19+ messages in thread
From: Rong Xu @ 2026-03-16 21:31 UTC (permalink / raw)
  To: Nathan Chancellor
  Cc: Miguel Ojeda, Nicolas Schier, Masahiro Yamada, Nick Desaulniers,
	Bill Wendling, Justin Stitt, Miguel Ojeda, Thomas Gleixner,
	Alice Ryhl, Sami Tolvanen, Mike Rapoport (Microsoft),
	Rafael Aquini, Michael Ellerman, Stafford Horne, Christophe Leroy,
	Piotr Gorski, Teresa Johnson, linux-kernel, linux-kbuild, llvm,
	Arnd Bergmann

I sent the following updated patch:

https://lore.kernel.org/all/20260316212930.120438-1-xur@google.com/T/#t

-Rong

On Mon, Mar 16, 2026 at 8:53 PM Rong Xu <xur@google.com> wrote:
>
> Sorry that my previous email may have caused some confusion.
> What I really meant was to keep -T (not change to '--thin').
>
> I think Masahiro's change targeted llvm-ar (<= 14), where it uses -T
> for "truncate", rather '--thin'. So he used ''--thin" explicitly. But
> this "--thin" was not implemented for some old ar.
>
> Since we the minimal llvm is now 15. The '-T' flag in legacy llvm-ar
> is no longer a problem. We should keep '-T'.
>
> -Rong
>
>
> On Mon, Mar 16, 2026 at 8:43 PM Nathan Chancellor <nathan@kernel.org> wrote:
> >
> > On Mon, Mar 16, 2026 at 11:16:41AM -0700, Rong Xu wrote:
> > > If that's the case, we can just remove flag "--thin". Can we verify if
> > > that works?
> >
> > I think we can only remove it from the second llvm-ar invocation. I
> > tested
> >
> > diff --git a/Makefile b/Makefile
> > index 2b15f0b4a0cb..fb001e02cc0f 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -1260,12 +1260,11 @@ ifdef CONFIG_TRIM_UNUSED_KSYMS
> >  KBUILD_MODULES := y
> >  endif
> >
> > -# '$(AR) mPi' needs 'T' to workaround the bug of llvm-ar <= 14
> >  quiet_cmd_ar_vmlinux.a = AR      $@
> >        cmd_ar_vmlinux.a = \
> >         rm -f $@; \
> >         $(AR) cDPrST $@ $(KBUILD_VMLINUX_OBJS); \
> > -       $(AR) mPiT $$($(AR) t $@ | sed -n 1p) $@ $$($(AR) t $@ | grep -F -f $(srctree)/scripts/head-object-list.txt)
> > +       $(AR) mPi $$($(AR) t $@ | sed -n 1p) $@ $$($(AR) t $@ | grep -F -f $(srctree)/scripts/head-object-list.txt)
> >
> >  targets += vmlinux.a
> >  vmlinux.a: $(KBUILD_VMLINUX_OBJS) scripts/head-object-list.txt FORCE
> > --
> >
> > on 7.0-rc4, which showed no issues. This is basically a revert of my
> > suggested workaround for the original issue:
> >
> >   https://lore.kernel.org/YyjjT5gQ2hGMH0ni@dev-arch.thelio-3990X/
> >
> > You could add this as a prerequisite patch, I don't think it should be
> > squashed into the move change, especially since that appears to be why
> > we are in this situation. Why are we converting from the 'T' modifier to
> > '--thin' in the move patch? I know the documentation of llvm-ar and GNU
> > ar says that 'T' is deprecated in favor of '--thin' because it may do
> > different things on various ar implementations but the kernel only
> > supports these two implementations. I think we should just copy the
> > commands as they are and address the deprecation separately, perhaps
> > with an ar-option like Nicolas suggested upthread.
> >
> > How about a v3 that looks like:
> >
> > Patch 1: The diff above because all supported llvm-ar versions do the
> > same thing as GNU ar.
> >
> > Patch 2: The move patch without changing 'T' into '--thin'.
> >
> > Patch 3: The same as before, perhaps without '--thin' as well.
> >
> > You'll need to base on kbuild-next-unstable [1] to address the conflict
> > with Yonghong's "kbuild: Reduce the number of compiler-generated
> > suffixes for clang thin-lto build" [2].
> >
> > [1]: https://git.kernel.org/kbuild/l/kbuild-next-unstable
> > [2]: https://lore.kernel.org/20260307050250.3767489-1-yonghong.song@linux.dev/
> >
> > Cheers,
> > Nathan

^ permalink raw reply	[flat|nested] 19+ messages in thread

end of thread, other threads:[~2026-03-16 21:31 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-10-28 18:28 [PATCH v5 0/2] kbuild: distributed build support for Clang ThinLTO xur
2025-10-28 18:28 ` [PATCH v5 1/2] kbuild: move vmlinux.a build rule to scripts/Makefile.vmlinux_a xur
2026-03-16 13:33   ` Nicolas Schier
2026-03-16 13:59     ` Miguel Ojeda
2026-03-16 17:27       ` Rong Xu
2026-03-16 17:52         ` Nicolas Schier
2026-03-16 17:59         ` Nathan Chancellor
2026-03-16 18:16           ` Rong Xu
2026-03-16 20:43             ` Nathan Chancellor
2026-03-16 20:53               ` Rong Xu
2026-03-16 21:31                 ` Rong Xu
2025-10-28 18:28 ` [PATCH v5 2/2] kbuild: distributed build support for Clang ThinLTO xur
2025-10-28 19:22 ` [PATCH v5 0/2] " Piotr Gorski
2025-12-04 17:49 ` Nathan Chancellor
2025-12-04 18:36   ` Rong Xu
2026-03-12  8:25 ` Nathan Chancellor
2026-03-12 11:45   ` Nicolas Schier
2026-03-12 17:50     ` Rong Xu
2026-03-13 14:50 ` Nicolas Schier

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox