* FAILED: patch "[PATCH] kbuild: Remove support for Clang's ThinLTO caching" failed to apply to 6.1-stable tree
@ 2024-06-13 7:12 gregkh
2024-06-13 18:33 ` [PATCH 6.1.y] kbuild: Remove support for Clang's ThinLTO caching Nathan Chancellor
0 siblings, 1 reply; 5+ messages in thread
From: gregkh @ 2024-06-13 7:12 UTC (permalink / raw)
To: nathan, elsk, masahiroy, mhiramat; +Cc: stable
The patch below does not apply to the 6.1-stable tree.
If someone wants it applied there, or to any other stable or longterm
tree, then please email the backport, including the original git commit
id to <stable@vger.kernel.org>.
To reproduce the conflict and resubmit, you may use the following commands:
git fetch https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ linux-6.1.y
git checkout FETCH_HEAD
git cherry-pick -x aba091547ef6159d52471f42a3ef531b7b660ed8
# <resolve conflicts, build, test, etc.>
git commit -s
git send-email --to '<stable@vger.kernel.org>' --in-reply-to '2024061340-troubling-automated-9989@gregkh' --subject-prefix 'PATCH 6.1.y' HEAD^..
Possible dependencies:
aba091547ef6 ("kbuild: Remove support for Clang's ThinLTO caching")
1db773da58df ("kbuild: remove old Rust docs output path")
7ea01d3169a2 ("rust: delete rust-project.json when running make clean")
thanks,
greg k-h
------------------ original commit in Linus's tree ------------------
From aba091547ef6159d52471f42a3ef531b7b660ed8 Mon Sep 17 00:00:00 2001
From: Nathan Chancellor <nathan@kernel.org>
Date: Wed, 1 May 2024 15:55:25 -0700
Subject: [PATCH] kbuild: Remove support for Clang's ThinLTO caching
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
There is an issue in clang's ThinLTO caching (enabled for the kernel via
'--thinlto-cache-dir') with .incbin, which the kernel occasionally uses
to include data within the kernel, such as the .config file for
/proc/config.gz. For example, when changing the .config and rebuilding
vmlinux, the copy of .config in vmlinux does not match the copy of
.config in the build folder:
$ echo 'CONFIG_LTO_NONE=n
CONFIG_LTO_CLANG_THIN=y
CONFIG_IKCONFIG=y
CONFIG_HEADERS_INSTALL=y' >kernel/configs/repro.config
$ make -skj"$(nproc)" ARCH=x86_64 LLVM=1 clean defconfig repro.config vmlinux
...
$ grep CONFIG_HEADERS_INSTALL .config
CONFIG_HEADERS_INSTALL=y
$ scripts/extract-ikconfig vmlinux | grep CONFIG_HEADERS_INSTALL
CONFIG_HEADERS_INSTALL=y
$ scripts/config -d HEADERS_INSTALL
$ make -kj"$(nproc)" ARCH=x86_64 LLVM=1 vmlinux
...
UPD kernel/config_data
GZIP kernel/config_data.gz
CC kernel/configs.o
...
LD vmlinux
...
$ grep CONFIG_HEADERS_INSTALL .config
# CONFIG_HEADERS_INSTALL is not set
$ scripts/extract-ikconfig vmlinux | grep CONFIG_HEADERS_INSTALL
CONFIG_HEADERS_INSTALL=y
Without '--thinlto-cache-dir' or when using full LTO, this issue does
not occur.
Benchmarking incremental builds on a few different machines with and
without the cache shows a 20% increase in incremental build time without
the cache when measured by touching init/main.c and running 'make all'.
ARCH=arm64 defconfig + CONFIG_LTO_CLANG_THIN=y on an arm64 host:
Benchmark 1: With ThinLTO cache
Time (mean ± σ): 56.347 s ± 0.163 s [User: 83.768 s, System: 24.661 s]
Range (min … max): 56.109 s … 56.594 s 10 runs
Benchmark 2: Without ThinLTO cache
Time (mean ± σ): 67.740 s ± 0.479 s [User: 718.458 s, System: 31.797 s]
Range (min … max): 67.059 s … 68.556 s 10 runs
Summary
With ThinLTO cache ran
1.20 ± 0.01 times faster than Without ThinLTO cache
ARCH=x86_64 defconfig + CONFIG_LTO_CLANG_THIN=y on an x86_64 host:
Benchmark 1: With ThinLTO cache
Time (mean ± σ): 85.772 s ± 0.252 s [User: 91.505 s, System: 8.408 s]
Range (min … max): 85.447 s … 86.244 s 10 runs
Benchmark 2: Without ThinLTO cache
Time (mean ± σ): 103.833 s ± 0.288 s [User: 232.058 s, System: 8.569 s]
Range (min … max): 103.286 s … 104.124 s 10 runs
Summary
With ThinLTO cache ran
1.21 ± 0.00 times faster than Without ThinLTO cache
While it is unfortunate to take this performance improvement off the
table, correctness is more important. If/when this is fixed in LLVM, it
can potentially be brought back in a conditional manner. Alternatively,
a developer can just disable LTO if doing incremental compiles quickly
is important, as a full compile cycle can still take over a minute even
with the cache and it is unlikely that LTO will result in functional
differences for a kernel change.
Cc: stable@vger.kernel.org
Fixes: dc5723b02e52 ("kbuild: add support for Clang LTO")
Reported-by: Yifan Hong <elsk@google.com>
Closes: https://github.com/ClangBuiltLinux/linux/issues/2021
Reported-by: Masami Hiramatsu <mhiramat@kernel.org>
Closes: https://lore.kernel.org/r/20220327115526.cc4b0ff55fc53c97683c3e4d@kernel.org/
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
diff --git a/Makefile b/Makefile
index 026971f9f6bc..12e1a792b8de 100644
--- a/Makefile
+++ b/Makefile
@@ -942,7 +942,6 @@ endif
ifdef CONFIG_LTO_CLANG
ifdef CONFIG_LTO_CLANG_THIN
CC_FLAGS_LTO := -flto=thin -fsplit-lto-unit
-KBUILD_LDFLAGS += --thinlto-cache-dir=$(extmod_prefix).thinlto-cache
else
CC_FLAGS_LTO := -flto
endif
@@ -1480,7 +1479,7 @@ endif # CONFIG_MODULES
# Directories & files removed with 'make clean'
CLEAN_FILES += vmlinux.symvers modules-only.symvers \
modules.builtin modules.builtin.modinfo modules.nsdeps \
- compile_commands.json .thinlto-cache rust/test \
+ compile_commands.json rust/test \
rust-project.json .vmlinux.objs .vmlinux.export.c
# Directories & files removed with 'make mrproper'
@@ -1787,7 +1786,7 @@ PHONY += compile_commands.json
clean-dirs := $(KBUILD_EXTMOD)
clean: rm-files := $(KBUILD_EXTMOD)/Module.symvers $(KBUILD_EXTMOD)/modules.nsdeps \
- $(KBUILD_EXTMOD)/compile_commands.json $(KBUILD_EXTMOD)/.thinlto-cache
+ $(KBUILD_EXTMOD)/compile_commands.json
PHONY += prepare
# now expand this into a simple variable to reduce the cost of shell evaluations
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 6.1.y] kbuild: Remove support for Clang's ThinLTO caching
2024-06-13 7:12 FAILED: patch "[PATCH] kbuild: Remove support for Clang's ThinLTO caching" failed to apply to 6.1-stable tree gregkh
@ 2024-06-13 18:33 ` Nathan Chancellor
2024-06-19 10:51 ` Greg KH
0 siblings, 1 reply; 5+ messages in thread
From: Nathan Chancellor @ 2024-06-13 18:33 UTC (permalink / raw)
To: gregkh; +Cc: nathan, stable
commit aba091547ef6159d52471f42a3ef531b7b660ed8 upstream.
There is an issue in clang's ThinLTO caching (enabled for the kernel via
'--thinlto-cache-dir') with .incbin, which the kernel occasionally uses
to include data within the kernel, such as the .config file for
/proc/config.gz. For example, when changing the .config and rebuilding
vmlinux, the copy of .config in vmlinux does not match the copy of
.config in the build folder:
$ echo 'CONFIG_LTO_NONE=n
CONFIG_LTO_CLANG_THIN=y
CONFIG_IKCONFIG=y
CONFIG_HEADERS_INSTALL=y' >kernel/configs/repro.config
$ make -skj"$(nproc)" ARCH=x86_64 LLVM=1 clean defconfig repro.config vmlinux
...
$ grep CONFIG_HEADERS_INSTALL .config
CONFIG_HEADERS_INSTALL=y
$ scripts/extract-ikconfig vmlinux | grep CONFIG_HEADERS_INSTALL
CONFIG_HEADERS_INSTALL=y
$ scripts/config -d HEADERS_INSTALL
$ make -kj"$(nproc)" ARCH=x86_64 LLVM=1 vmlinux
...
UPD kernel/config_data
GZIP kernel/config_data.gz
CC kernel/configs.o
...
LD vmlinux
...
$ grep CONFIG_HEADERS_INSTALL .config
# CONFIG_HEADERS_INSTALL is not set
$ scripts/extract-ikconfig vmlinux | grep CONFIG_HEADERS_INSTALL
CONFIG_HEADERS_INSTALL=y
Without '--thinlto-cache-dir' or when using full LTO, this issue does
not occur.
Benchmarking incremental builds on a few different machines with and
without the cache shows a 20% increase in incremental build time without
the cache when measured by touching init/main.c and running 'make all'.
ARCH=arm64 defconfig + CONFIG_LTO_CLANG_THIN=y on an arm64 host:
Benchmark 1: With ThinLTO cache
Time (mean ± σ): 56.347 s ± 0.163 s [User: 83.768 s, System: 24.661 s]
Range (min … max): 56.109 s … 56.594 s 10 runs
Benchmark 2: Without ThinLTO cache
Time (mean ± σ): 67.740 s ± 0.479 s [User: 718.458 s, System: 31.797 s]
Range (min … max): 67.059 s … 68.556 s 10 runs
Summary
With ThinLTO cache ran
1.20 ± 0.01 times faster than Without ThinLTO cache
ARCH=x86_64 defconfig + CONFIG_LTO_CLANG_THIN=y on an x86_64 host:
Benchmark 1: With ThinLTO cache
Time (mean ± σ): 85.772 s ± 0.252 s [User: 91.505 s, System: 8.408 s]
Range (min … max): 85.447 s … 86.244 s 10 runs
Benchmark 2: Without ThinLTO cache
Time (mean ± σ): 103.833 s ± 0.288 s [User: 232.058 s, System: 8.569 s]
Range (min … max): 103.286 s … 104.124 s 10 runs
Summary
With ThinLTO cache ran
1.21 ± 0.00 times faster than Without ThinLTO cache
While it is unfortunate to take this performance improvement off the
table, correctness is more important. If/when this is fixed in LLVM, it
can potentially be brought back in a conditional manner. Alternatively,
a developer can just disable LTO if doing incremental compiles quickly
is important, as a full compile cycle can still take over a minute even
with the cache and it is unlikely that LTO will result in functional
differences for a kernel change.
Cc: stable@vger.kernel.org
Fixes: dc5723b02e52 ("kbuild: add support for Clang LTO")
Reported-by: Yifan Hong <elsk@google.com>
Closes: https://github.com/ClangBuiltLinux/linux/issues/2021
Reported-by: Masami Hiramatsu <mhiramat@kernel.org>
Closes: https://lore.kernel.org/r/20220327115526.cc4b0ff55fc53c97683c3e4d@kernel.org/
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
[nathan: Address conflict in Makefile]
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
---
Makefile | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/Makefile b/Makefile
index c5147f1c46f8..abe7ba05155b 100644
--- a/Makefile
+++ b/Makefile
@@ -980,7 +980,6 @@ endif
ifdef CONFIG_LTO_CLANG
ifdef CONFIG_LTO_CLANG_THIN
CC_FLAGS_LTO := -flto=thin -fsplit-lto-unit
-KBUILD_LDFLAGS += --thinlto-cache-dir=$(extmod_prefix).thinlto-cache
else
CC_FLAGS_LTO := -flto
endif
@@ -1588,7 +1587,7 @@ endif # CONFIG_MODULES
# Directories & files removed with 'make clean'
CLEAN_FILES += include/ksym vmlinux.symvers modules-only.symvers \
modules.builtin modules.builtin.modinfo modules.nsdeps \
- compile_commands.json .thinlto-cache rust/test rust/doc \
+ compile_commands.json rust/test rust/doc \
.vmlinux.objs .vmlinux.export.c
# Directories & files removed with 'make mrproper'
@@ -1884,7 +1883,7 @@ PHONY += compile_commands.json
clean-dirs := $(KBUILD_EXTMOD)
clean: rm-files := $(KBUILD_EXTMOD)/Module.symvers $(KBUILD_EXTMOD)/modules.nsdeps \
- $(KBUILD_EXTMOD)/compile_commands.json $(KBUILD_EXTMOD)/.thinlto-cache
+ $(KBUILD_EXTMOD)/compile_commands.json
PHONY += prepare
# now expand this into a simple variable to reduce the cost of shell evaluations
base-commit: ae9f2a70d69e9c840ee1eda201f09662ca7e2038
--
2.45.2
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH 6.1.y] kbuild: Remove support for Clang's ThinLTO caching
2024-06-13 18:33 ` [PATCH 6.1.y] kbuild: Remove support for Clang's ThinLTO caching Nathan Chancellor
@ 2024-06-19 10:51 ` Greg KH
2024-06-19 14:23 ` Nathan Chancellor
0 siblings, 1 reply; 5+ messages in thread
From: Greg KH @ 2024-06-19 10:51 UTC (permalink / raw)
To: Nathan Chancellor; +Cc: stable
On Thu, Jun 13, 2024 at 11:33:22AM -0700, Nathan Chancellor wrote:
> commit aba091547ef6159d52471f42a3ef531b7b660ed8 upstream.
>
> There is an issue in clang's ThinLTO caching (enabled for the kernel via
> '--thinlto-cache-dir') with .incbin, which the kernel occasionally uses
> to include data within the kernel, such as the .config file for
> /proc/config.gz. For example, when changing the .config and rebuilding
> vmlinux, the copy of .config in vmlinux does not match the copy of
> .config in the build folder:
>
> $ echo 'CONFIG_LTO_NONE=n
> CONFIG_LTO_CLANG_THIN=y
> CONFIG_IKCONFIG=y
> CONFIG_HEADERS_INSTALL=y' >kernel/configs/repro.config
>
> $ make -skj"$(nproc)" ARCH=x86_64 LLVM=1 clean defconfig repro.config vmlinux
> ...
>
> $ grep CONFIG_HEADERS_INSTALL .config
> CONFIG_HEADERS_INSTALL=y
>
> $ scripts/extract-ikconfig vmlinux | grep CONFIG_HEADERS_INSTALL
> CONFIG_HEADERS_INSTALL=y
>
> $ scripts/config -d HEADERS_INSTALL
>
> $ make -kj"$(nproc)" ARCH=x86_64 LLVM=1 vmlinux
> ...
> UPD kernel/config_data
> GZIP kernel/config_data.gz
> CC kernel/configs.o
> ...
> LD vmlinux
> ...
>
> $ grep CONFIG_HEADERS_INSTALL .config
> # CONFIG_HEADERS_INSTALL is not set
>
> $ scripts/extract-ikconfig vmlinux | grep CONFIG_HEADERS_INSTALL
> CONFIG_HEADERS_INSTALL=y
>
> Without '--thinlto-cache-dir' or when using full LTO, this issue does
> not occur.
>
> Benchmarking incremental builds on a few different machines with and
> without the cache shows a 20% increase in incremental build time without
> the cache when measured by touching init/main.c and running 'make all'.
>
> ARCH=arm64 defconfig + CONFIG_LTO_CLANG_THIN=y on an arm64 host:
>
> Benchmark 1: With ThinLTO cache
> Time (mean ± σ): 56.347 s ± 0.163 s [User: 83.768 s, System: 24.661 s]
> Range (min … max): 56.109 s … 56.594 s 10 runs
>
> Benchmark 2: Without ThinLTO cache
> Time (mean ± σ): 67.740 s ± 0.479 s [User: 718.458 s, System: 31.797 s]
> Range (min … max): 67.059 s … 68.556 s 10 runs
>
> Summary
> With ThinLTO cache ran
> 1.20 ± 0.01 times faster than Without ThinLTO cache
>
> ARCH=x86_64 defconfig + CONFIG_LTO_CLANG_THIN=y on an x86_64 host:
>
> Benchmark 1: With ThinLTO cache
> Time (mean ± σ): 85.772 s ± 0.252 s [User: 91.505 s, System: 8.408 s]
> Range (min … max): 85.447 s … 86.244 s 10 runs
>
> Benchmark 2: Without ThinLTO cache
> Time (mean ± σ): 103.833 s ± 0.288 s [User: 232.058 s, System: 8.569 s]
> Range (min … max): 103.286 s … 104.124 s 10 runs
>
> Summary
> With ThinLTO cache ran
> 1.21 ± 0.00 times faster than Without ThinLTO cache
>
> While it is unfortunate to take this performance improvement off the
> table, correctness is more important. If/when this is fixed in LLVM, it
> can potentially be brought back in a conditional manner. Alternatively,
> a developer can just disable LTO if doing incremental compiles quickly
> is important, as a full compile cycle can still take over a minute even
> with the cache and it is unlikely that LTO will result in functional
> differences for a kernel change.
>
> Cc: stable@vger.kernel.org
> Fixes: dc5723b02e52 ("kbuild: add support for Clang LTO")
> Reported-by: Yifan Hong <elsk@google.com>
> Closes: https://github.com/ClangBuiltLinux/linux/issues/2021
> Reported-by: Masami Hiramatsu <mhiramat@kernel.org>
> Closes: https://lore.kernel.org/r/20220327115526.cc4b0ff55fc53c97683c3e4d@kernel.org/
> Signed-off-by: Nathan Chancellor <nathan@kernel.org>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> [nathan: Address conflict in Makefile]
> Signed-off-by: Nathan Chancellor <nathan@kernel.org>
> ---
> Makefile | 5 ++---
> 1 file changed, 2 insertions(+), 3 deletions(-)
This applied to 5.15.y, not 6.1.y :(
Can you rebase and resend a fix for 6.1.y?
thanks,
greg k-h
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH 6.1.y] kbuild: Remove support for Clang's ThinLTO caching
2024-06-19 10:51 ` Greg KH
@ 2024-06-19 14:23 ` Nathan Chancellor
2024-06-24 14:52 ` Greg KH
0 siblings, 1 reply; 5+ messages in thread
From: Nathan Chancellor @ 2024-06-19 14:23 UTC (permalink / raw)
To: Greg KH; +Cc: stable
On Wed, Jun 19, 2024 at 12:51:55PM +0200, Greg KH wrote:
> On Thu, Jun 13, 2024 at 11:33:22AM -0700, Nathan Chancellor wrote:
> > commit aba091547ef6159d52471f42a3ef531b7b660ed8 upstream.
> >
> > There is an issue in clang's ThinLTO caching (enabled for the kernel via
> > '--thinlto-cache-dir') with .incbin, which the kernel occasionally uses
> > to include data within the kernel, such as the .config file for
> > /proc/config.gz. For example, when changing the .config and rebuilding
> > vmlinux, the copy of .config in vmlinux does not match the copy of
> > .config in the build folder:
> >
> > $ echo 'CONFIG_LTO_NONE=n
> > CONFIG_LTO_CLANG_THIN=y
> > CONFIG_IKCONFIG=y
> > CONFIG_HEADERS_INSTALL=y' >kernel/configs/repro.config
> >
> > $ make -skj"$(nproc)" ARCH=x86_64 LLVM=1 clean defconfig repro.config vmlinux
> > ...
> >
> > $ grep CONFIG_HEADERS_INSTALL .config
> > CONFIG_HEADERS_INSTALL=y
> >
> > $ scripts/extract-ikconfig vmlinux | grep CONFIG_HEADERS_INSTALL
> > CONFIG_HEADERS_INSTALL=y
> >
> > $ scripts/config -d HEADERS_INSTALL
> >
> > $ make -kj"$(nproc)" ARCH=x86_64 LLVM=1 vmlinux
> > ...
> > UPD kernel/config_data
> > GZIP kernel/config_data.gz
> > CC kernel/configs.o
> > ...
> > LD vmlinux
> > ...
> >
> > $ grep CONFIG_HEADERS_INSTALL .config
> > # CONFIG_HEADERS_INSTALL is not set
> >
> > $ scripts/extract-ikconfig vmlinux | grep CONFIG_HEADERS_INSTALL
> > CONFIG_HEADERS_INSTALL=y
> >
> > Without '--thinlto-cache-dir' or when using full LTO, this issue does
> > not occur.
> >
> > Benchmarking incremental builds on a few different machines with and
> > without the cache shows a 20% increase in incremental build time without
> > the cache when measured by touching init/main.c and running 'make all'.
> >
> > ARCH=arm64 defconfig + CONFIG_LTO_CLANG_THIN=y on an arm64 host:
> >
> > Benchmark 1: With ThinLTO cache
> > Time (mean ± σ): 56.347 s ± 0.163 s [User: 83.768 s, System: 24.661 s]
> > Range (min … max): 56.109 s … 56.594 s 10 runs
> >
> > Benchmark 2: Without ThinLTO cache
> > Time (mean ± σ): 67.740 s ± 0.479 s [User: 718.458 s, System: 31.797 s]
> > Range (min … max): 67.059 s … 68.556 s 10 runs
> >
> > Summary
> > With ThinLTO cache ran
> > 1.20 ± 0.01 times faster than Without ThinLTO cache
> >
> > ARCH=x86_64 defconfig + CONFIG_LTO_CLANG_THIN=y on an x86_64 host:
> >
> > Benchmark 1: With ThinLTO cache
> > Time (mean ± σ): 85.772 s ± 0.252 s [User: 91.505 s, System: 8.408 s]
> > Range (min … max): 85.447 s … 86.244 s 10 runs
> >
> > Benchmark 2: Without ThinLTO cache
> > Time (mean ± σ): 103.833 s ± 0.288 s [User: 232.058 s, System: 8.569 s]
> > Range (min … max): 103.286 s … 104.124 s 10 runs
> >
> > Summary
> > With ThinLTO cache ran
> > 1.21 ± 0.00 times faster than Without ThinLTO cache
> >
> > While it is unfortunate to take this performance improvement off the
> > table, correctness is more important. If/when this is fixed in LLVM, it
> > can potentially be brought back in a conditional manner. Alternatively,
> > a developer can just disable LTO if doing incremental compiles quickly
> > is important, as a full compile cycle can still take over a minute even
> > with the cache and it is unlikely that LTO will result in functional
> > differences for a kernel change.
> >
> > Cc: stable@vger.kernel.org
> > Fixes: dc5723b02e52 ("kbuild: add support for Clang LTO")
> > Reported-by: Yifan Hong <elsk@google.com>
> > Closes: https://github.com/ClangBuiltLinux/linux/issues/2021
> > Reported-by: Masami Hiramatsu <mhiramat@kernel.org>
> > Closes: https://lore.kernel.org/r/20220327115526.cc4b0ff55fc53c97683c3e4d@kernel.org/
> > Signed-off-by: Nathan Chancellor <nathan@kernel.org>
> > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> > [nathan: Address conflict in Makefile]
> > Signed-off-by: Nathan Chancellor <nathan@kernel.org>
> > ---
> > Makefile | 5 ++---
> > 1 file changed, 2 insertions(+), 3 deletions(-)
>
> This applied to 5.15.y, not 6.1.y :(
>
> Can you rebase and resend a fix for 6.1.y?
I don't understand how that is possible, this was generated directly on
top of 6.1.93 (as evidenced by the base commit) and there were no
changes to Makefile in 6.1.94. It still applies cleanly for me?
$ curl -LSs https://lore.kernel.org/all/20240613183322.1088226-1-nathan@kernel.org/raw | patch -p1
patching file Makefile
Cheers,
Nathan
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH 6.1.y] kbuild: Remove support for Clang's ThinLTO caching
2024-06-19 14:23 ` Nathan Chancellor
@ 2024-06-24 14:52 ` Greg KH
0 siblings, 0 replies; 5+ messages in thread
From: Greg KH @ 2024-06-24 14:52 UTC (permalink / raw)
To: Nathan Chancellor; +Cc: stable
On Wed, Jun 19, 2024 at 07:23:39AM -0700, Nathan Chancellor wrote:
> On Wed, Jun 19, 2024 at 12:51:55PM +0200, Greg KH wrote:
> > On Thu, Jun 13, 2024 at 11:33:22AM -0700, Nathan Chancellor wrote:
> > > commit aba091547ef6159d52471f42a3ef531b7b660ed8 upstream.
> > >
> > > There is an issue in clang's ThinLTO caching (enabled for the kernel via
> > > '--thinlto-cache-dir') with .incbin, which the kernel occasionally uses
> > > to include data within the kernel, such as the .config file for
> > > /proc/config.gz. For example, when changing the .config and rebuilding
> > > vmlinux, the copy of .config in vmlinux does not match the copy of
> > > .config in the build folder:
> > >
> > > $ echo 'CONFIG_LTO_NONE=n
> > > CONFIG_LTO_CLANG_THIN=y
> > > CONFIG_IKCONFIG=y
> > > CONFIG_HEADERS_INSTALL=y' >kernel/configs/repro.config
> > >
> > > $ make -skj"$(nproc)" ARCH=x86_64 LLVM=1 clean defconfig repro.config vmlinux
> > > ...
> > >
> > > $ grep CONFIG_HEADERS_INSTALL .config
> > > CONFIG_HEADERS_INSTALL=y
> > >
> > > $ scripts/extract-ikconfig vmlinux | grep CONFIG_HEADERS_INSTALL
> > > CONFIG_HEADERS_INSTALL=y
> > >
> > > $ scripts/config -d HEADERS_INSTALL
> > >
> > > $ make -kj"$(nproc)" ARCH=x86_64 LLVM=1 vmlinux
> > > ...
> > > UPD kernel/config_data
> > > GZIP kernel/config_data.gz
> > > CC kernel/configs.o
> > > ...
> > > LD vmlinux
> > > ...
> > >
> > > $ grep CONFIG_HEADERS_INSTALL .config
> > > # CONFIG_HEADERS_INSTALL is not set
> > >
> > > $ scripts/extract-ikconfig vmlinux | grep CONFIG_HEADERS_INSTALL
> > > CONFIG_HEADERS_INSTALL=y
> > >
> > > Without '--thinlto-cache-dir' or when using full LTO, this issue does
> > > not occur.
> > >
> > > Benchmarking incremental builds on a few different machines with and
> > > without the cache shows a 20% increase in incremental build time without
> > > the cache when measured by touching init/main.c and running 'make all'.
> > >
> > > ARCH=arm64 defconfig + CONFIG_LTO_CLANG_THIN=y on an arm64 host:
> > >
> > > Benchmark 1: With ThinLTO cache
> > > Time (mean ± σ): 56.347 s ± 0.163 s [User: 83.768 s, System: 24.661 s]
> > > Range (min … max): 56.109 s … 56.594 s 10 runs
> > >
> > > Benchmark 2: Without ThinLTO cache
> > > Time (mean ± σ): 67.740 s ± 0.479 s [User: 718.458 s, System: 31.797 s]
> > > Range (min … max): 67.059 s … 68.556 s 10 runs
> > >
> > > Summary
> > > With ThinLTO cache ran
> > > 1.20 ± 0.01 times faster than Without ThinLTO cache
> > >
> > > ARCH=x86_64 defconfig + CONFIG_LTO_CLANG_THIN=y on an x86_64 host:
> > >
> > > Benchmark 1: With ThinLTO cache
> > > Time (mean ± σ): 85.772 s ± 0.252 s [User: 91.505 s, System: 8.408 s]
> > > Range (min … max): 85.447 s … 86.244 s 10 runs
> > >
> > > Benchmark 2: Without ThinLTO cache
> > > Time (mean ± σ): 103.833 s ± 0.288 s [User: 232.058 s, System: 8.569 s]
> > > Range (min … max): 103.286 s … 104.124 s 10 runs
> > >
> > > Summary
> > > With ThinLTO cache ran
> > > 1.21 ± 0.00 times faster than Without ThinLTO cache
> > >
> > > While it is unfortunate to take this performance improvement off the
> > > table, correctness is more important. If/when this is fixed in LLVM, it
> > > can potentially be brought back in a conditional manner. Alternatively,
> > > a developer can just disable LTO if doing incremental compiles quickly
> > > is important, as a full compile cycle can still take over a minute even
> > > with the cache and it is unlikely that LTO will result in functional
> > > differences for a kernel change.
> > >
> > > Cc: stable@vger.kernel.org
> > > Fixes: dc5723b02e52 ("kbuild: add support for Clang LTO")
> > > Reported-by: Yifan Hong <elsk@google.com>
> > > Closes: https://github.com/ClangBuiltLinux/linux/issues/2021
> > > Reported-by: Masami Hiramatsu <mhiramat@kernel.org>
> > > Closes: https://lore.kernel.org/r/20220327115526.cc4b0ff55fc53c97683c3e4d@kernel.org/
> > > Signed-off-by: Nathan Chancellor <nathan@kernel.org>
> > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> > > [nathan: Address conflict in Makefile]
> > > Signed-off-by: Nathan Chancellor <nathan@kernel.org>
> > > ---
> > > Makefile | 5 ++---
> > > 1 file changed, 2 insertions(+), 3 deletions(-)
> >
> > This applied to 5.15.y, not 6.1.y :(
> >
> > Can you rebase and resend a fix for 6.1.y?
>
> I don't understand how that is possible, this was generated directly on
> top of 6.1.93 (as evidenced by the base commit) and there were no
> changes to Makefile in 6.1.94. It still applies cleanly for me?
>
> $ curl -LSs https://lore.kernel.org/all/20240613183322.1088226-1-nathan@kernel.org/raw | patch -p1
> patching file Makefile
>
Very odd, I just tried it again and it worked, must have been a problem
on my side, sorry for the noise.
greg k-h
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2024-06-24 14:52 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-06-13 7:12 FAILED: patch "[PATCH] kbuild: Remove support for Clang's ThinLTO caching" failed to apply to 6.1-stable tree gregkh
2024-06-13 18:33 ` [PATCH 6.1.y] kbuild: Remove support for Clang's ThinLTO caching Nathan Chancellor
2024-06-19 10:51 ` Greg KH
2024-06-19 14:23 ` Nathan Chancellor
2024-06-24 14:52 ` Greg KH
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox