Revisiting c0a454b9044f

Revisiting c0a454b9044f

[Nathan Chancellor]

Hi all,

I am looking to potentially bump the minimum version of LLVM for
building the kernel to 15.0.0 after the next merge window. In my quest
to look for workarounds that can be dropped, I noticed that
CONFIG_ARM64_BTI_KERNEL was disabled unconditionally for GCC in commit
c0a454b9044f ("arm64/bti: Disable in kernel BTI when cross section
thunks are broken") as a result of [1]. Looking at that GCC report, it
seems like the AArch64 ABI now documents [2] the GNU toolchain's
behavior as expected and LLVM has been adjusted [3][4][5] to match. Do I
need to block CONFIG_ARM64_BTI_KERNEL from being selected with LLVM
21.0.0? Or should the kernel adjust its expectations now that the ABI
and toolchains all agree?

Cheers,
Nathan

[1]: https://gcc.gnu.org/pr106671
[2]: https://github.com/ARM-software/abi-aa/commit/606ce44fe4d3419c15cd9ed598f18fb5d520fcfc
[3]: https://github.com/llvm/llvm-project/commit/7af2b51e761f49974a64c3009882239cea618f2a
[4]: https://github.com/llvm/llvm-project/commit/edf21314c98a4fe05d48f83dfab2b201ed8bfe9c
[5]: https://github.com/llvm/llvm-project/commit/098b0d18add97dea94e16006486b2fded65e228d

Re: Revisiting c0a454b9044f

[Mark Rutland]

On Mon, Jul 14, 2025 at 12:52:05PM -0700, Nathan Chancellor wrote:
> Hi all,

Hi Nethan,

> I am looking to potentially bump the minimum version of LLVM for
> building the kernel to 15.0.0 after the next merge window. In my quest
> to look for workarounds that can be dropped, I noticed that
> CONFIG_ARM64_BTI_KERNEL was disabled unconditionally for GCC in commit
> c0a454b9044f ("arm64/bti: Disable in kernel BTI when cross section
> thunks are broken") as a result of [1]. Looking at that GCC report, it
> seems like the AArch64 ABI now documents [2] the GNU toolchain's
> behavior as expected 

For context, at the time of commit c0a454b9044f, GNU LD did not handle
this appropriately, leading to runtime BTI failures where two sections
were too far apart.

GNU LD was subsequently fixed, and the ABI documentation was updated,
but I'm not sure which specific versions of GNU LD have the fix, and we
hadn't chased that up to re-enable BTI with GCC.

> and LLVM has been adjusted [3][4][5] to match. Do I need to block
> CONFIG_ARM64_BTI_KERNEL from being selected with LLVM 21.0.0?

I'm missing something; why would we need to dsiable BTI in that case?

The concern from the kernel side is simply whether we get unexpected BTI
failures. IIUC so long as compiler and linker agree we should be good,
and we simply need to forbid broken combinations.

> Or should the kernel adjust its expectations now that the ABI and
> toolchains all agree?

Yes, we can probably rework this.

IIUC we'd need to forbid BTI with:

* GCC + old GNU LD
* GCC + old LLD
* new clang + old GNU LD
* new clang + old LLD

... and can enable BTI otherwise.

Does that make sense to you?

Mark.
 
> Cheers,
> Nathan
> 
> [1]: https://gcc.gnu.org/pr106671
> [2]: https://github.com/ARM-software/abi-aa/commit/606ce44fe4d3419c15cd9ed598f18fb5d520fcfc
> [3]: https://github.com/llvm/llvm-project/commit/7af2b51e761f49974a64c3009882239cea618f2a
> [4]: https://github.com/llvm/llvm-project/commit/edf21314c98a4fe05d48f83dfab2b201ed8bfe9c
> [5]: https://github.com/llvm/llvm-project/commit/098b0d18add97dea94e16006486b2fded65e228d

Re: Revisiting c0a454b9044f

[Nathan Chancellor]

On Tue, Jul 15, 2025 at 12:16:07PM +0100, Mark Rutland wrote:
> On Mon, Jul 14, 2025 at 12:52:05PM -0700, Nathan Chancellor wrote:
> > I am looking to potentially bump the minimum version of LLVM for
> > building the kernel to 15.0.0 after the next merge window. In my quest
> > to look for workarounds that can be dropped, I noticed that
> > CONFIG_ARM64_BTI_KERNEL was disabled unconditionally for GCC in commit
> > c0a454b9044f ("arm64/bti: Disable in kernel BTI when cross section
> > thunks are broken") as a result of [1]. Looking at that GCC report, it
> > seems like the AArch64 ABI now documents [2] the GNU toolchain's
> > behavior as expected 
> 
> For context, at the time of commit c0a454b9044f, GNU LD did not handle
> this appropriately, leading to runtime BTI failures where two sections
> were too far apart.
> 
> GNU LD was subsequently fixed, and the ABI documentation was updated,
> but I'm not sure which specific versions of GNU LD have the fix, and we
> hadn't chased that up to re-enable BTI with GCC.

Based on https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106671#c7 and
https://sourceware.org/bugzilla/show_bug.cgi?id=30076, it sounds like
binutils 2.41 would be the first fixed version.

> > and LLVM has been adjusted [3][4][5] to match. Do I need to block
> > CONFIG_ARM64_BTI_KERNEL from being selected with LLVM 21.0.0?
> 
> I'm missing something; why would we need to dsiable BTI in that case?

Nope, you are not missing anything, I missed the correlation between the
compiler and linker.

> The concern from the kernel side is simply whether we get unexpected BTI
> failures. IIUC so long as compiler and linker agree we should be good,
> and we simply need to forbid broken combinations.

Mark Brown did mention something about the module loader as well so I
was not sure if that was relevant here.

> > Or should the kernel adjust its expectations now that the ABI and
> > toolchains all agree?
> 
> Yes, we can probably rework this.
> 
> IIUC we'd need to forbid BTI with:
> 
> * GCC + old GNU LD
> * GCC + old LLD
> * new clang + old GNU LD
> * new clang + old LLD
> 
> ... and can enable BTI otherwise.
> 
> Does that make sense to you?

So something like this if I understand correctly?

Cheers,
Nathan

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 393d71124f5d..fe523f9f2d61 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -2097,7 +2097,11 @@ config ARM64_BTI_KERNEL
 	# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94697
 	depends on !CC_IS_GCC || GCC_VERSION >= 100100
 	# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106671
-	depends on !CC_IS_GCC
+	# https://sourceware.org/bugzilla/show_bug.cgi?id=30076
+	depends on !CC_IS_GCC || LD_VERSION >= 24100 || LLD_VERSION >= 210000
+	# https://github.com/llvm/llvm-project/commit/7af2b51e761f49974a64c3009882239cea618f2a
+	# https://github.com/llvm/llvm-project/commit/098b0d18add97dea94e16006486b2fded65e228d
+	depends on !CC_IS_CLANG || CLANG_VERSION < 210000 || (CLANG_VERSION >= 210000 && (LD_VERSION >= 24100 || LLD_VERSION >= 210000))
 	depends on (!FUNCTION_GRAPH_TRACER || DYNAMIC_FTRACE_WITH_ARGS)
 	help
 	  Build the kernel with Branch Target Identification annotations

Re: Revisiting c0a454b9044f

[Mark Rutland]

On Wed, Jul 16, 2025 at 12:26:41PM -0600, Nathan Chancellor wrote:
> On Tue, Jul 15, 2025 at 12:16:07PM +0100, Mark Rutland wrote:
> > The concern from the kernel side is simply whether we get unexpected BTI
> > failures. IIUC so long as compiler and linker agree we should be good,
> > and we simply need to forbid broken combinations.
> 
> Mark Brown did mention something about the module loader as well so I
> was not sure if that was relevant here.

Sorry, I had forgotten anout that, and that is a concern.

If a single module has executable sections placed more than 128MiB apart
we'd potentially have the same problem with any cross-section branch.
Truly handling that would be quite tricky and require a fair amount of
rework, so the best bet is probably to reject loading modules that are
too large (or where we specifically find such cross-section branches
needing veneers).

Note that exported symbols and address-taken functions should have a
BTI, so this only really matters for cross-section calls within a single
module.

I suspect it should be relatively simple but I'm not sure exactly where
to plumb that in. I can put that on my TODO list if no-one gets around
to it.

> > > Or should the kernel adjust its expectations now that the ABI and
> > > toolchains all agree?
> > 
> > Yes, we can probably rework this.
> > 
> > IIUC we'd need to forbid BTI with:
> > 
> > * GCC + old GNU LD
> > * GCC + old LLD
> > * new clang + old GNU LD
> > * new clang + old LLD
> > 
> > ... and can enable BTI otherwise.
> > 
> > Does that make sense to you?
> 
> So something like this if I understand correctly?
> 
> Cheers,
> Nathan
> 
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index 393d71124f5d..fe523f9f2d61 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -2097,7 +2097,11 @@ config ARM64_BTI_KERNEL
>  	# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94697
>  	depends on !CC_IS_GCC || GCC_VERSION >= 100100
>  	# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106671
> -	depends on !CC_IS_GCC
> +	# https://sourceware.org/bugzilla/show_bug.cgi?id=30076
> +	depends on !CC_IS_GCC || LD_VERSION >= 24100 || LLD_VERSION >= 210000
> +	# https://github.com/llvm/llvm-project/commit/7af2b51e761f49974a64c3009882239cea618f2a
> +	# https://github.com/llvm/llvm-project/commit/098b0d18add97dea94e16006486b2fded65e228d
> +	depends on !CC_IS_CLANG || CLANG_VERSION < 210000 || (CLANG_VERSION >= 210000 && (LD_VERSION >= 24100 || LLD_VERSION >= 210000))

Yep, something like that.

I was thinking that we could factor this out into a separate config,
like we have for BUILTIN_RETURN_ADDRESS_STRIPS_PAC, as that'll make it
easier to avoid duplication, e.g.

| config TOOLCHAIN_HAS_WORKING_BTI
| 	bool
| 	# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94697
| 	default n if CC_IS_GCC && GCC_VERSION < 100100
|	# Newer LD/LLD handle BTI in veneers automatically
| 	default y if LD_IS_LLD && LLD_VERSION >= 210000
| 	default y if LD_IS_GNU && LD_VERSION >= 24100
| 	# Newer clang requires newer LD/LLD above
| 	default y if CC_IS_CLANG && CLANG_VERSION < 210000
| 	default n

... and we can easily extend that to handle fixed stable versions, like with
BUILTIN_RETURN_ADDRESS_STRIPS_PAC.

Mark.