[PATCH 00/12] arm64/bti: Fix kernel BTI issues with livepatch, large kernels, toolchains

[PATCH 00/12] arm64/bti: Fix kernel BTI issues with livepatch, large kernels, toolchains

[Josh Poimboeuf]

These patches are a continuation of the following discussions about some
bugs related to kernel BTI:

  https://lore.kernel.org/ed4fe1f95071897859ec7fbe9176246cbd4962bf.1786138806.git.jpoimboe@kernel.org
  https://lore.kernel.org/20260812162058.612202-4-ardb@kernel.org

This started as a livepatch crash investigation but quickly ballooned
into a rabbit hole of latent bugs and toolchain quirks/crashes.

The most notable patch is probably the last one, which enables BTI on GCC.

I also hit a SIGSEGV in the GNU linker, for which this series has a
workaround.  The binutils bug is here:

  https://sourceware.org/bugzilla/show_bug.cgi?id=34525

Several of these patches probably wouldn't be needed if we had a flag
like -fno-omit-bti-pads or so for both compilers, but that currently
doesn't exist, and this at least fixes things for the existing
toolchains out there.

There is some toolchain discussion about that here:

 https://github.com/llvm/llvm-project/issues/215547

Josh Poimboeuf (12):
  arm64/bti: Add BTI landing pad to __sdei_asm_handler()
  arm64/module: Fix BTI exceptions caused by omitted landing pads in
    Clang 21
  arm64/bti: Fix BTI linker failures with long branches into .idmap.text
  arm64/bti: Work around ld crash caused by linker script aliases
  arm64/bti: Add link error for large kernels with BTI and unsupported
    toolchains
  arm64/bti: Add link error for large kernels with BTI and livepatch
  arm64/bti: Advertise BTI in assembly objects
  arm64/bti: Enable BTI in the pi/ startup code
  efi/libstub: Preserve the GNU property note
  efi/libstub: Remove obsolete .note.gnu.property workaround
  arm64/bti: Force-enable BTI linker veneers
  arm64/bti: Enable kernel BTI for GCC

 arch/arm64/Kconfig                    |   9 +-
 arch/arm64/Makefile                   |   5 +
 arch/arm64/include/asm/bti-note.h     |  32 +++++
 arch/arm64/include/asm/module.h       |   5 +
 arch/arm64/kernel/cpu-reset.S         |   2 +-
 arch/arm64/kernel/entry.S             |   1 +
 arch/arm64/kernel/head.S              |   8 +-
 arch/arm64/kernel/hyp-stub.S          |   1 +
 arch/arm64/kernel/image-vars.h        |  13 +-
 arch/arm64/kernel/module-plts.c       | 186 ++++++++++++++++++++++++++
 arch/arm64/kernel/pi/Makefile         |  11 +-
 arch/arm64/kernel/sleep.S             |   2 +-
 arch/arm64/kernel/vmlinux.lds.S       |  12 ++
 arch/arm64/lib/memcpy.S               |   3 +
 arch/arm64/lib/memset.S               |   2 +
 arch/arm64/mm/cache.S                 |   1 +
 arch/arm64/mm/proc.S                  |   8 +-
 drivers/firmware/efi/libstub/Makefile |  15 ++-
 18 files changed, 282 insertions(+), 34 deletions(-)
 create mode 100644 arch/arm64/include/asm/bti-note.h

-- 
2.55.0

[PATCH 01/12] arm64/bti: Add BTI landing pad to __sdei_asm_handler()

[Josh Poimboeuf]

With CONFIG_UNMAP_KERNEL_AT_EL0, __sdei_asm_entry_trampoline() indirect
jumps to __sdei_asm_handler().  Add "bti j" to prevent a Branch Target
exception.

Discovered by code inspection.

Fixes: c50d32859e70 ("arm64: Add types to indirect called assembly functions")
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
 arch/arm64/kernel/entry.S | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index e0db14e9c843a..16c21fcb41976 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -976,6 +976,7 @@ NOKPROBE(__sdei_asm_exit_trampoline)
  * want them.
  */
 SYM_CODE_START(__sdei_asm_handler)
+	bti	j
 	stp     x2, x3, [x1, #SDEI_EVENT_INTREGS + S_PC]
 	stp     x4, x5, [x1, #SDEI_EVENT_INTREGS + 16 * 2]
 	stp     x6, x7, [x1, #SDEI_EVENT_INTREGS + 16 * 3]
-- 
2.55.0

[PATCH 02/12] arm64/module: Fix BTI exceptions caused by omitted landing pads in Clang 21

[Josh Poimboeuf]

The following BTI exception was seen when loading a livepatch module:

  Internal error: Oops - BTI: 0000000036000001 [#1]  SMP
  pstate: 634004c9 (nZCv daIF +PAN -UAO +TCO +DIT -SSBS BTYPE=jc)
  pc : kill_orphaned_pgrp+0x0/0x150
  lr : do_exit+0x498/0xaf0 [livepatch_combined]

The problem is that the patch module's do_exit() is branching to a
static function in vmlinux using a module PLT veneer (indirect branch),
but the target function doesn't have a BTI landing pad.

Clang 21+ omits the landing pad for static functions which can only be
reached by a direct branch.  But livepatch modules use klp relocations
to reference arbitrary kernel symbols, and with
CONFIG_RANDOMIZE_MODULE_REGION_FULL the module is far enough away that
every call to vmlinux needs a PLT.

Note this problem is actually not specific to livepatch.  It's possible
for any module's .init section to be allocated > 128MB away from its
.text section.  So calls from .init to .text via a PLT can trigger a BTI
exception when the target function doesn't have a landing pad.

GCC has always omitted the landing pad when possible, so kernel BTI is
already considered incompatible with GCC since commit c0a454b9044f
("arm64/bti: Disable in kernel BTI when cross section thunks are
broken").

When missing landing pads are detected, allocate a page close to the
target which can be used to hold BTI veneers which receive PLT veneer
indirect branches and direct branch to the final target:

	bti	c
	b	<target>

Fixes: fd1e0fd71f65 ("arm64: Implement HAVE_LIVEPATCH")
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
 arch/arm64/include/asm/module.h |   5 +
 arch/arm64/kernel/module-plts.c | 186 ++++++++++++++++++++++++++++++++
 2 files changed, 191 insertions(+)

diff --git a/arch/arm64/include/asm/module.h b/arch/arm64/include/asm/module.h
index fb9b88eebeb15..f705cbec4463f 100644
--- a/arch/arm64/include/asm/module.h
+++ b/arch/arm64/include/asm/module.h
@@ -13,10 +13,15 @@ struct mod_plt_sec {
 	int			plt_max_entries;
 };
 
+struct bti_veneer_page;
+
 struct mod_arch_specific {
 	struct mod_plt_sec	core;
 	struct mod_plt_sec	init;
 
+	/* for CONFIG_ARM64_BTI_KERNEL */
+	struct bti_veneer_page	*bti_veneers;
+
 	/* for CONFIG_DYNAMIC_FTRACE */
 	struct plt_entry	*ftrace_trampolines;
 	struct plt_entry	*init_ftrace_trampolines;
diff --git a/arch/arm64/kernel/module-plts.c b/arch/arm64/kernel/module-plts.c
index 7afd370da9f48..4ba31e336deb6 100644
--- a/arch/arm64/kernel/module-plts.c
+++ b/arch/arm64/kernel/module-plts.c
@@ -3,12 +3,18 @@
  * Copyright (C) 2014-2017 Linaro Ltd. <ard.biesheuvel@linaro.org>
  */
 
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
 #include <linux/elf.h>
 #include <linux/ftrace.h>
 #include <linux/kernel.h>
 #include <linux/module.h>
 #include <linux/moduleloader.h>
+#include <linux/slab.h>
 #include <linux/sort.h>
+#include <linux/vmalloc.h>
+#include <asm/cpufeature.h>
+#include <asm/text-patching.h>
 
 static struct plt_entry __get_adrp_add_pair(u64 dst, u64 pc,
 					    enum aarch64_insn_register reg)
@@ -66,6 +72,180 @@ static bool plt_entries_equal(const struct plt_entry *a,
 	       (q + aarch64_insn_adrp_get_offset(le32_to_cpu(b->adrp)));
 }
 
+/*
+ * The compiler may omit a function's BTI landing pad if it's a static function
+ * with no pointers referencing it.  That breaks two cases where a PLT might be
+ * needed to call such a function:
+ *
+ *   1) module cross-section call (e.g., .init to .text)
+ *
+ *   2) livepatch module using a klp relocation
+ *
+ * Fix up such cases with a second veneer which lives close to the target.
+ */
+struct bti_veneer {
+	__le32	bti_c;
+	__le32	b;
+};
+
+struct bti_veneer_page {
+	struct bti_veneer_page	*next;
+	struct bti_veneer	*veneers;
+	unsigned int		used;
+};
+
+#define BTI_VENEERS_PER_PAGE	(PAGE_SIZE / sizeof(struct bti_veneer))
+
+static bool plt_target_has_landing_pad(u64 target)
+{
+	u32 insn;
+
+	if (!system_supports_bti_kernel())
+		return true;
+
+	if (aarch64_insn_read((void *)target, &insn))
+		return true;
+
+	if (!aarch64_insn_is_hint(insn))
+		return false;
+
+	switch (insn & 0xFE0) {
+	case AARCH64_INSN_HINT_BTIC:
+	case AARCH64_INSN_HINT_BTIJ:
+	case AARCH64_INSN_HINT_BTIJC:
+	case AARCH64_INSN_HINT_PACIASP:
+	case AARCH64_INSN_HINT_PACIBSP:
+		return true;
+	}
+
+	return false;
+}
+
+static void *bti_veneer_vmalloc(u64 start, u64 end, gfp_t gfp)
+{
+	pgprot_t prot = __pgprot(pgprot_val(PAGE_KERNEL_ROX) | PTE_MAYBE_GP);
+
+	return __vmalloc_node_range(PAGE_SIZE, PAGE_SIZE, PAGE_ALIGN(start),
+				    ALIGN_DOWN(end, PAGE_SIZE), gfp, prot,
+				    VM_FLUSH_RESET_PERMS, NUMA_NO_NODE,
+				    __builtin_return_address(0));
+}
+
+static bool bti_veneer_in_range(const struct bti_veneer *veneer, u64 target)
+{
+	s64 offset = (s64)target - (s64)&veneer->b;
+
+	return offset >= -SZ_128M && offset < SZ_128M;
+}
+
+static struct bti_veneer_page *bti_veneer_page_alloc(struct module *mod,
+						     u64 target)
+{
+	struct bti_veneer_page *page;
+	void *p;
+
+	/*
+	 * vmalloc allocates at the lowest free address in the given range, so
+	 * try the range above the target first so it will be as close as
+	 * possible, making it more likely to be reusable for other targets in
+	 * the same object.
+	 */
+	p = bti_veneer_vmalloc(target, target + SZ_128M,
+			       GFP_KERNEL | __GFP_NOWARN);
+	if (!p)
+		p = bti_veneer_vmalloc(target - SZ_128M, target,
+				       GFP_KERNEL | __GFP_NOWARN);
+	if (!p) {
+		pr_err("%s: no address space within branch range of %pS for a BTI veneer\n",
+		       mod->name, (void *)target);
+		return NULL;
+	}
+
+	/* Don't leave unused slots executable */
+	aarch64_insn_set(p, AARCH64_BREAK_FAULT, PAGE_SIZE);
+
+	page = kzalloc_obj(*page, GFP_KERNEL);
+	if (!page) {
+		vfree(p);
+		return NULL;
+	}
+
+	page->veneers = p;
+	page->next = mod->arch.bti_veneers;
+	mod->arch.bti_veneers = page;
+
+	return page;
+}
+
+static u64 module_emit_bti_veneer(struct module *mod, u64 target)
+{
+	struct bti_veneer_page *page;
+	struct bti_veneer *veneer, insns;
+	u32 insn;
+
+	/* Look for an existing veneer for the target */
+	for (page = mod->arch.bti_veneers; page; page = page->next) {
+		for (unsigned int i = 0; i < page->used; i++) {
+			s32 offset;
+
+			veneer = &page->veneers[i];
+			insn = le32_to_cpu(veneer->b);
+			offset = aarch64_get_branch_offset(insn);
+
+			if ((u64)&veneer->b + offset == target)
+				return (u64)veneer;
+		}
+	}
+
+	/* Look for a free slot in range of the target */
+	for (page = mod->arch.bti_veneers; page; page = page->next) {
+		if (page->used == BTI_VENEERS_PER_PAGE)
+			continue;
+
+		veneer = &page->veneers[page->used];
+		if (bti_veneer_in_range(veneer, target))
+			goto emit;
+	}
+
+	page = bti_veneer_page_alloc(mod, target);
+	if (!page)
+		return 0;
+
+	veneer = &page->veneers[0];
+
+emit:
+	insn = aarch64_insn_gen_branch_imm((u64)&veneer->b, target,
+					   AARCH64_INSN_BRANCH_NOLINK);
+	if (WARN_ON(insn == AARCH64_BREAK_FAULT))
+		return 0;
+
+	insns.bti_c = cpu_to_le32(aarch64_insn_gen_hint(AARCH64_INSN_HINT_BTIC));
+	insns.b	    = cpu_to_le32(insn);
+
+	if (!aarch64_insn_copy(veneer, &insns, sizeof(insns))) {
+		pr_err("%s: failed to write BTI veneer for %pS\n",
+		       mod->name, (void *)target);
+		return 0;
+	}
+
+	page->used++;
+
+	return (u64)veneer;
+}
+
+void module_arch_cleanup(struct module *mod)
+{
+	struct bti_veneer_page *page, *next;
+
+	for (page = mod->arch.bti_veneers; page; page = next) {
+		next = page->next;
+		vfree(page->veneers);
+		kfree(page);
+	}
+
+	mod->arch.bti_veneers = NULL;
+}
+
 u64 module_emit_plt_entry(struct module *mod, Elf64_Shdr *sechdrs,
 			  void *loc, const Elf64_Rela *rela,
 			  Elf64_Sym *sym)
@@ -77,6 +257,12 @@ u64 module_emit_plt_entry(struct module *mod, Elf64_Shdr *sechdrs,
 	int j = i - 1;
 	u64 val = sym->st_value + rela->r_addend;
 
+	if (!plt_target_has_landing_pad(val)) {
+		val = module_emit_bti_veneer(mod, val);
+		if (!val)
+			return 0;
+	}
+
 	if (is_forbidden_offset_for_adrp(&plt[i].adrp))
 		i++;
 
-- 
2.55.0

[PATCH 03/12] arm64/bti: Fix BTI linker failures with long branches into .idmap.text

[Josh Poimboeuf]

On a kernel whose text exceeds the +/128MB direct branch range, the
linker inserts veneers.  With BTI enabled, the veneers' indirect branch
targets need a BTI landing pad, which not all functions have starting
with Clang 21 (and for all versions of GCC).

In such cases the linker can emit a second veneer close to the target
which has the landing pad along with a direct branch to the target.  But
a long branch to .idmap.text never gets one because it's missing the
executable section flag.

With the LLVM linker, it's a silent failure, presumably only discovered
by a BTI exception at runtime.  With the GNU linker it's even worse, as
it dereferences the missing stub/veneer group entry and seg faults (this
was how I discovered it).

Make sure the section is executable by adding the "x" flag to all the
creators of the input section.

Also manually add "bti c" to primary_entry() and enter_vhe(), otherwise
the linker-generated veneer page pushes the .idmap.text past its
asserted 4KB size:

  ld.bfd: ID map text too big or misaligned

Link: https://sourceware.org/bugzilla/show_bug.cgi?id=34525
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
 arch/arm64/kernel/cpu-reset.S | 2 +-
 arch/arm64/kernel/head.S      | 7 ++++---
 arch/arm64/kernel/hyp-stub.S  | 1 +
 arch/arm64/kernel/sleep.S     | 2 +-
 arch/arm64/mm/proc.S          | 8 ++++----
 5 files changed, 11 insertions(+), 9 deletions(-)

diff --git a/arch/arm64/kernel/cpu-reset.S b/arch/arm64/kernel/cpu-reset.S
index c87445dde6745..9943a7c70f6b0 100644
--- a/arch/arm64/kernel/cpu-reset.S
+++ b/arch/arm64/kernel/cpu-reset.S
@@ -14,7 +14,7 @@
 #include <asm/virt.h>
 
 .text
-.pushsection    .idmap.text, "a"
+.pushsection    .idmap.text, "ax"
 
 /*
  * cpu_soft_restart(el2_switch, entry, arg0, arg1, arg2)
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 87a822e5c4ca8..541721488bef9 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -71,7 +71,7 @@
 
 	__EFI_PE_HEADER
 
-	.section ".idmap.text","a"
+	.section ".idmap.text","ax"
 
 	/*
 	 * The following callee saved general purpose registers are used on the
@@ -83,6 +83,7 @@
 	 *  x21        primary_entry() .. start_kernel()        FDT pointer passed at boot in x0
 	 */
 SYM_CODE_START(primary_entry)
+	bti	c
 	bl	record_mmu_state
 	bl	preserve_boot_args
 
@@ -251,7 +252,7 @@ SYM_FUNC_END(__primary_switched)
  * end early head section, begin head code that is also used for
  * hotplug and needs to have the same protections as the text region
  */
-	.section ".idmap.text","a"
+	.section ".idmap.text","ax"
 
 /*
  * Starting from EL2 or EL1, configure the CPU to execute at the highest
@@ -455,7 +456,7 @@ SYM_FUNC_END(set_cpu_boot_mode_flag)
  * Checks if the selected granule size is supported by the CPU.
  * If it isn't, park the CPU
  */
-	.section ".idmap.text","a"
+	.section ".idmap.text","ax"
 SYM_FUNC_START(__enable_mmu)
 	mrs	x3, ID_AA64MMFR0_EL1
 	ubfx	x3, x3, #ID_AA64MMFR0_EL1_TGRAN_SHIFT, 4
diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S
index 37c6976e44a4c..5f9c5ffb4afc1 100644
--- a/arch/arm64/kernel/hyp-stub.S
+++ b/arch/arm64/kernel/hyp-stub.S
@@ -167,6 +167,7 @@ SYM_CODE_END(__finalise_el2)
 	.pushsection	.idmap.text, "ax"
 
 SYM_CODE_START_LOCAL(enter_vhe)
+	bti	c
 	// Invalidate TLBs before enabling the MMU
 	tlbi	vmalle1
 	dsb	nsh
diff --git a/arch/arm64/kernel/sleep.S b/arch/arm64/kernel/sleep.S
index f093cdf71be11..dfb5b8bc8cdb5 100644
--- a/arch/arm64/kernel/sleep.S
+++ b/arch/arm64/kernel/sleep.S
@@ -97,7 +97,7 @@ SYM_FUNC_START(__cpu_suspend_enter)
 	ret
 SYM_FUNC_END(__cpu_suspend_enter)
 
-	.pushsection ".idmap.text", "a"
+	.pushsection ".idmap.text", "ax"
 SYM_CODE_START(cpu_resume)
 	mov	x0, xzr
 	bl	init_kernel_el
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 22866b49be372..b8ff93c748bf1 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -175,7 +175,7 @@ alternative_else_nop_endif
 SYM_FUNC_END(cpu_do_resume)
 #endif
 
-	.pushsection ".idmap.text", "a"
+	.pushsection ".idmap.text", "ax"
 
 .macro	__idmap_cpu_set_reserved_ttbr1, tmp1, tmp2
 	adrp	\tmp1, reserved_pg_dir
@@ -211,7 +211,7 @@ SYM_FUNC_ALIAS(__pi_idmap_cpu_replace_ttbr1, idmap_cpu_replace_ttbr1)
 #define KPTI_NG_PTE_FLAGS	(PTE_ATTRINDX(MT_NORMAL) | PTE_TYPE_PAGE | \
 				 PTE_AF | PTE_SHARED | PTE_UXN | PTE_WRITE)
 
-	.pushsection ".idmap.text", "a"
+	.pushsection ".idmap.text", "ax"
 
 	.macro	pte_to_phys, phys, pte
 	and	\phys, \pte, #PTE_ADDR_LOW
@@ -438,7 +438,7 @@ SYM_FUNC_END(idmap_kpti_install_ng_mappings)
 	.popsection
 #endif
 
-	.pushsection ".idmap.text", "a"
+	.pushsection ".idmap.text", "ax"
 SYM_TYPED_FUNC_START(wait_linear_map_split_to_ptes)
 	/* Must be same registers as in idmap_kpti_install_ng_mappings */
 	swapper_ttb	.req	x3
@@ -479,7 +479,7 @@ SYM_FUNC_END(wait_linear_map_split_to_ptes)
  * Output:
  *	Return in x0 the value of the SCTLR_EL1 register.
  */
-	.pushsection ".idmap.text", "a"
+	.pushsection ".idmap.text", "ax"
 SYM_FUNC_START(__cpu_setup)
 	tlbi	vmalle1				// Invalidate local TLB
 	dsb	nsh
-- 
2.55.0

[PATCH 04/12] arm64/bti: Work around ld crash caused by linker script aliases

[Josh Poimboeuf]

Linking an allyesconfig kernel (~700MB of text) with the GNU linker and
CONFIG_ARM64_BTI_KERNEL triggers a seg fault in the linker:

  #1  elf64_aarch64_stub_name (input_section=0x100000040, sym_sec=..., hash=..., rel=...) at ../../bfd/elfnn-aarch64.c:3042
  #2  _bfd_aarch64_add_call_stub_entries (...) at ../../bfd/elfnn-aarch64.c:4674
  #3  elf64_aarch64_size_stubs (...) at ../../bfd/elfnn-aarch64.c:4839

On a kernel whose text exceeds the +/128MB direct branch range, the
linker inserts veneers.  With BTI enabled, the veneers' indirect branch
targets need a BTI landing pad, which not all functions have starting
with Clang 21 (and for all versions of GCC).

In such cases the linker can emit a second veneer close to the target
which has the landing pad along with a direct branch to the target.  The
GNU linker assumes each target has an input section.  But linker-defined
symbols don't have input sections, and ld crashes trying to access their
veneer data.

vmlinux.lds.S (via image-vars.h) uses PROVIDE() to alias some function
symbols into the __efistub_ and __pi_ namespaces, which are used by the
EFI stub and the position-independent startup code, respectively.

Work around the crash by defining those aliases in code.  Note this
might also end up being a permanent fix, depending on whether the ld fix
ends up reporting an error for such cases, or whether it will infer the
alias's section from the aliasee.

Define the five affected aliases in the objects which define their
aliasees so they live in a real input section.

Link: https://sourceware.org/bugzilla/show_bug.cgi?id=34525
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
 arch/arm64/kernel/head.S       |  1 +
 arch/arm64/kernel/image-vars.h | 13 +------------
 arch/arm64/lib/memcpy.S        |  3 +++
 arch/arm64/lib/memset.S        |  2 ++
 arch/arm64/mm/cache.S          |  1 +
 5 files changed, 8 insertions(+), 12 deletions(-)

diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 541721488bef9..e96af16b421d8 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -130,6 +130,7 @@ SYM_CODE_START(primary_entry)
 	bl	__cpu_setup			// initialise processor
 	b	__primary_switch
 SYM_CODE_END(primary_entry)
+SYM_FUNC_ALIAS(__efistub_primary_entry, primary_entry)
 
 	__INIT
 SYM_CODE_START_LOCAL(record_mmu_state)
diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h
index d4c7d45ae6bc8..713b96cf5dc89 100644
--- a/arch/arm64/kernel/image-vars.h
+++ b/arch/arm64/kernel/image-vars.h
@@ -20,19 +20,12 @@
 	PROVIDE(pisym = sym);		\
 	ASSERT((sym - KIMAGE_VADDR) < (__bss_start - KIMAGE_VADDR), #msg)
 
-PROVIDE(__efistub_primary_entry		= primary_entry);
-
 /*
  * The EFI stub has its own symbol namespace prefixed by __efistub_, to
  * isolate it from the kernel proper. The following symbols are legally
  * accessed by the stub, so provide some aliases to make them accessible.
- * Only include data symbols here, or text symbols of functions that are
- * guaranteed to be safe when executed at another offset than they were
- * linked at. The routines below are all implemented in assembler in a
- * position independent manner
+ * Only include data symbols here.
  */
-PROVIDE(__efistub_caches_clean_inval_pou = __pi_caches_clean_inval_pou);
-
 PROVIDE(__efistub__text			= _text);
 PROVIDE(__efistub__end			= _end);
 PROVIDE(__efistub___inittext_end       	= __inittext_end);
@@ -42,10 +35,6 @@ PROVIDE(__efistub_sysfb_primary_display	= sysfb_primary_display);
 #endif
 PROVIDE(__efistub__ctype		= _ctype);
 
-PROVIDE(__pi___memcpy			= __pi_memcpy);
-PROVIDE(__pi___memmove			= __pi_memmove);
-PROVIDE(__pi___memset			= __pi_memset);
-
 PI_EXPORT_SYM(id_aa64isar1_override);
 PI_EXPORT_SYM(id_aa64isar2_override);
 PI_EXPORT_SYM(id_aa64mmfr0_override);
diff --git a/arch/arm64/lib/memcpy.S b/arch/arm64/lib/memcpy.S
index 9b99106fb95f1..90dbb0d3acdef 100644
--- a/arch/arm64/lib/memcpy.S
+++ b/arch/arm64/lib/memcpy.S
@@ -268,3 +268,6 @@ SYM_FUNC_ALIAS(__memmove, __pi_memmove)
 EXPORT_SYMBOL(__memmove)
 SYM_FUNC_ALIAS_WEAK(memmove, __memmove)
 EXPORT_SYMBOL(memmove)
+
+SYM_FUNC_ALIAS(__pi___memcpy, __pi_memcpy)
+SYM_FUNC_ALIAS(__pi___memmove, __pi_memmove)
diff --git a/arch/arm64/lib/memset.S b/arch/arm64/lib/memset.S
index 97157da65ec6b..31e514f020aa3 100644
--- a/arch/arm64/lib/memset.S
+++ b/arch/arm64/lib/memset.S
@@ -226,3 +226,5 @@ EXPORT_SYMBOL(__memset)
 
 SYM_FUNC_ALIAS_WEAK(memset, __pi_memset)
 EXPORT_SYMBOL(memset)
+
+SYM_FUNC_ALIAS(__pi___memset, __pi_memset)
diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S
index ab75c050f5590..ab2cd6c524073 100644
--- a/arch/arm64/mm/cache.S
+++ b/arch/arm64/mm/cache.S
@@ -57,6 +57,7 @@ SYM_FUNC_START(caches_clean_inval_pou)
 	ret
 SYM_FUNC_END(caches_clean_inval_pou)
 SYM_FUNC_ALIAS(__pi_caches_clean_inval_pou, caches_clean_inval_pou)
+SYM_FUNC_ALIAS(__efistub_caches_clean_inval_pou, caches_clean_inval_pou)
 
 /*
  *	caches_clean_inval_user_pou(start,end)
-- 
2.55.0

[PATCH 05/12] arm64/bti: Add link error for large kernels with BTI and unsupported toolchains

[Josh Poimboeuf]

On a kernel whose text exceeds the +/128MB direct branch range, the
linker inserts veneers.  With BTI enabled, the veneers' indirect branch
targets need a BTI landing pad, which not all functions have starting
with Clang 21 (and for all versions of GCC).

In such cases the linker can emit a second veneer close to the target
which has the landing pad along with a direct branch to the target.
However, that's currently not being done, so GCC and Clang 21+ are
broken with BTI on large kernels.

Only newer linkers have proper support for adding the second veneer,
starting with binutils 2.42 and LLVM 20.

Only allow a large kernel with the right combination of compiler and
linker.

Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
 arch/arm64/Kconfig              | 7 +++++++
 arch/arm64/kernel/vmlinux.lds.S | 5 +++++
 2 files changed, 12 insertions(+)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index b3afe0688919b..2687b71438661 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -2123,6 +2123,13 @@ config ARM64_BTI_KERNEL
 	  is enabled and the system supports BTI all kernel code including
 	  modular code must have BTI enabled.
 
+config CC_OMITS_BTI_LANDING_PADS
+	def_bool !CC_IS_CLANG || CLANG_VERSION >= 210000
+
+config LD_HAS_BTI_STUBS
+	def_bool (LD_IS_BFD && LD_VERSION >= 24200) || \
+		 (LD_IS_LLD && LLD_VERSION >= 200000)
+
 config CC_HAS_BRANCH_PROT_PAC_RET_BTI
 	# GCC 9 or later, clang 8 or later
 	def_bool $(cc-option,-mbranch-protection=pac-ret+leaf+bti)
diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
index af1d720209764..3a88da9212831 100644
--- a/arch/arm64/kernel/vmlinux.lds.S
+++ b/arch/arm64/kernel/vmlinux.lds.S
@@ -414,6 +414,11 @@ ASSERT((__entry_tramp_text_end - __entry_tramp_text_start) <= 3*PAGE_SIZE,
 #ifdef CONFIG_KVM
 ASSERT(__hyp_bss_start == __bss_start, "HYP and Host BSS are misaligned")
 #endif
+
+#if defined(CONFIG_ARM64_BTI_KERNEL) && !defined(CONFIG_COMPILE_TEST) && \
+    defined(CONFIG_CC_OMITS_BTI_LANDING_PADS) && !defined(CONFIG_LD_HAS_BTI_STUBS)
+ASSERT(__exittext_end - _text <= SZ_128M, "Kernel text too big for BTI")
+#endif
 /*
  * If padding is applied before .head.text, virt<->phys conversions will fail.
  */
-- 
2.55.0

[PATCH 06/12] arm64/bti: Add link error for large kernels with BTI and livepatch

[Josh Poimboeuf]

A livepatch module uses klp relocations to reference static functions in
vmlinux.  If the livepatch module is placed at an address far away from
vmlinux, it needs PLT veneers to call those functions.

With BTI enabled, the veneers' indirect branch targets need a BTI
landing pad, which not all functions have starting with Clang 21 (and
for all versions of GCC).

In such cases the module loader attempts to allocate a page close to the
target for emitting a second veneer which has the landing pad along with
a direct branch to the target.

That page has to come from either the region below _text or the region
above _end, as there's no way to allocate memory in the middle of the
vmlinux image.  For a target near the end of a >128MB text region
there's no address space below _text within branch range at all, leaving
only whatever room happens to remain above _end after rodata, data and
bss.

If the branch target is out of range from any available free pages, the
livepatch module load fails with -ENOEXEC and a "no address space within
branch range" error.

Add an assertion so it fails the build instead.  Only count the module
region below _text; counting the space above _end would make a text size
limit depend on the size of rodata, data and bss, and doesn't help
unless text is bigger than the rest of the image.  The span is _text to
_etext rather than __exittext_end, as init and exit text are freed
before any module loads and can't be klp targets.

Distro kernels don't seem to come close to hitting the 128MB text mark,
so this should hopefully be more of a safety backstop than something
people are actually hitting.

Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
 arch/arm64/kernel/vmlinux.lds.S | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
index 3a88da9212831..f5bd5ad24e19f 100644
--- a/arch/arm64/kernel/vmlinux.lds.S
+++ b/arch/arm64/kernel/vmlinux.lds.S
@@ -419,6 +419,13 @@ ASSERT(__hyp_bss_start == __bss_start, "HYP and Host BSS are misaligned")
     defined(CONFIG_CC_OMITS_BTI_LANDING_PADS) && !defined(CONFIG_LD_HAS_BTI_STUBS)
 ASSERT(__exittext_end - _text <= SZ_128M, "Kernel text too big for BTI")
 #endif
+
+#if defined(CONFIG_ARM64_BTI_KERNEL) && !defined(CONFIG_COMPILE_TEST) &&	\
+    defined(CONFIG_CC_OMITS_BTI_LANDING_PADS) && defined(CONFIG_LIVEPATCH)
+ASSERT(_etext - _text <= SZ_128M - PAGE_SIZE,
+       "Kernel text too big for BTI live patching")
+#endif
+
 /*
  * If padding is applied before .head.text, virt<->phys conversions will fail.
  */
-- 
2.55.0

[PATCH 07/12] arm64/bti: Advertise BTI in assembly objects

[Josh Poimboeuf]

On a kernel whose text exceeds the +/-128MB direct branch range, the
linker inserts veneers.  With BTI enabled, the veneers' indirect branch
targets need a BTI landing pad, which not all functions have starting
with Clang 21 (and for all versions of GCC).

In such cases the linker can emit a second veneer close to the target
which has the landing pad along with a direct branch to the target.
However, that's currently not being done, so GCC and Clang 21+ are
broken with BTI on large kernels.

The linker only emits the BTI veneer if *all* input objects have
GNU_PROPERTY_AARCH64_FEATURE_1_BTI, which is not being done for
hand-written asm.

Force-include a property note with the BTI bit into every assembly
translation unit, which is accurate as SYM_FUNC_START*() already emits a
"bti c" landing pad for callable assembly functions.

Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
 arch/arm64/Makefile               |  4 ++++
 arch/arm64/include/asm/bti-note.h | 32 +++++++++++++++++++++++++++++++
 2 files changed, 36 insertions(+)
 create mode 100644 arch/arm64/include/asm/bti-note.h

diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 6b005c8fef706..4eee721c0b278 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -23,6 +23,10 @@ ifeq ($(CONFIG_ARM64_ERRATUM_843419),y)
 LDFLAGS_vmlinux	+= --fix-cortex-a53-843419
 endif
 
+ifeq ($(CONFIG_ARM64_BTI_KERNEL),y)
+KBUILD_AFLAGS	+= -include $(srctree)/arch/arm64/include/asm/bti-note.h
+endif
+
 cc_has_k_constraint := $(call try-run,echo				\
 	'int main(void) {						\
 		asm volatile("and w0, w0, %w0" :: "K" (4294967295));	\
diff --git a/arch/arm64/include/asm/bti-note.h b/arch/arm64/include/asm/bti-note.h
new file mode 100644
index 0000000000000..17ef5692987f7
--- /dev/null
+++ b/arch/arm64/include/asm/bti-note.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Emit a GNU_PROPERTY_AARCH64_FEATURE_1_BTI note.  This is force-included in
+ * every assembly file so the linker emits BTI veneers for >128MB kernels.
+ *
+ * Clang has -mmark-bti-property, but there's no equivalent for GCC/GAS.
+ *
+ * Binutils 2.44+ and LLVM 22+ support a much more compact version:
+ *
+ *   .aeabi_subsection aeabi_feature_and_bits, optional, ULEB128
+ *   .aeabi_attribute Tag_Feature_BTI, 1
+ */
+#ifndef __ASM_BTI_NOTE_H
+#define __ASM_BTI_NOTE_H
+
+	.pushsection .note.gnu.property, "a"
+	.align	3
+	.long	2f - 1f
+	.long	6f - 3f
+	.long	5		/* NT_GNU_PROPERTY_TYPE_0 */
+1:	.string	"GNU"
+2:
+	.align	3
+3:	.long	0xc0000000	/* GNU_PROPERTY_AARCH64_FEATURE_1_AND */
+	.long	5f - 4f
+4:	.long	1		/* GNU_PROPERTY_AARCH64_FEATURE_1_BTI */
+5:
+	.align	3
+6:
+	.popsection
+
+#endif /* __ASM_BTI_NOTE_H */
-- 
2.55.0

[PATCH 08/12] arm64/bti: Enable BTI in the pi/ startup code

[Josh Poimboeuf]

The early startup code in kernel/pi/ is built with
-mbranch-protection=none and has its property note stripped, which
prevents the linker from generating BTI veneers in >128MB kernels.

Unlike the assembly objects, which only lacked the note because
SYM_FUNC_START*() already emits the landing pads, pi/ needs the real
thing.  Some of it runs with the MMU on, where the kernel text is mapped
PTE_GP and BTI is enforced: __pi_scs_patch() is called when loading a
module, and __pi_map_range() is called from create_idmap().

Add -mbranch-protection=bti.  The added "bti c" landing pads execute as
NOPs in the startup code wherever BTI isn't implemented or isn't enforced.

Also stop stripping the objects' property notes, as the linker only
marks the output BTI-compatible if *all* objects advertise it.  Keeping
the note as-is isn't enough either: --prefix-alloc-sections=.init
renames it to .init.note.gnu.property, which GNU ld still parses but LLD
ignores.  Rename it back afterwards so both linkers see it.

Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
 arch/arm64/kernel/pi/Makefile | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/kernel/pi/Makefile b/arch/arm64/kernel/pi/Makefile
index be92d73c25b21..2101d96d754e5 100644
--- a/arch/arm64/kernel/pi/Makefile
+++ b/arch/arm64/kernel/pi/Makefile
@@ -4,7 +4,7 @@
 KBUILD_CFLAGS	:= $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) -fpie \
 		   -Os -DDISABLE_BRANCH_PROFILING $(DISABLE_KSTACK_ERASE) \
 		   $(DISABLE_LATENT_ENTROPY_PLUGIN) \
-		   $(call cc-option,-mbranch-protection=none) \
+		   $(call cc-option,-mbranch-protection=bti) \
 		   -I$(srctree)/scripts/dtc/libfdt -fno-stack-protector \
 		   -include $(srctree)/include/linux/hidden.h \
 		   -D__DISABLE_EXPORTS -ffreestanding -D__NO_FORTIFY \
@@ -21,11 +21,14 @@ KBUILD_CFLAGS	:= $(filter-out $(CC_FLAGS_LTO), $(KBUILD_CFLAGS))
 
 hostprogs	:= relacheck
 
+# --prefix-alloc-sections=.init also renames .note.gnu.property, which LLD then
+# ignores, dropping the BTI property.  Rename it back.
 quiet_cmd_piobjcopy = $(quiet_cmd_objcopy)
-      cmd_piobjcopy = $(cmd_objcopy) && $(obj)/relacheck $(@) $(<)
+      cmd_piobjcopy = $(cmd_objcopy) && \
+		      $(OBJCOPY) --rename-section .init.note.gnu.property=.note.gnu.property $(@) && \
+		      $(obj)/relacheck $(@) $(<)
 
-$(obj)/%.pi.o: OBJCOPYFLAGS := --prefix-symbols=__pi_ \
-			       --remove-section=.note.gnu.property
+$(obj)/%.pi.o: OBJCOPYFLAGS := --prefix-symbols=__pi_
 $(obj)/%.pi.o: $(obj)/%.o $(obj)/relacheck FORCE
 	$(call if_changed,piobjcopy)
 
-- 
2.55.0

[PATCH 09/12] efi/libstub: Preserve the GNU property note

[Josh Poimboeuf]

With "-z force-bti" the linker warns (or errors with CONFIG_WERROR) for
every libstub object:

  warning: BTI is required by -z force-bti, but this input object file lacks the necessary property note.

On arm64 the EFI stub is part of vmlinux, so its objects are annotated
as __init at the section level by running objcopy with
--prefix-alloc-sections=.init.  That has the side effect of renaming
.note.gnu.property to .init.note.gnu.property, which LLD ignores,
resulting in the BTI bit getting removed.

That's only a problem for toolchains which advertise the branch
protection features solely through .note.gnu.property.  GCC 16 and Clang
20 also record them in .ARM.attributes, which objcopy leaves alone, but
older compilers emit only the note.

Rename the section back to its original name so the linker sees it, and
so the generic NOTES macro can discard it.  objcopy applies
--rename-section before --prefix-alloc-sections regardless of their
order on the command line, so this needs a second objcopy invocation
rather than another flag on the existing one.

RISC-V and LoongArch apply the same .init prefix, so key the rename off
that rather than on arm64 alone.  Neither emits .note.gnu.property
today, and renaming a section which isn't present is a no-op, so this
changes nothing for them for now, but it prevents a future silent
failure mode.

The note is still stripped by --remove-section=.note.gnu.property, so
this change is inert until that section removal goes away in a
subsequent patch.

Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
 drivers/firmware/efi/libstub/Makefile | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile
index 77a2b2d74f3f6..b1c95f69e807d 100644
--- a/drivers/firmware/efi/libstub/Makefile
+++ b/drivers/firmware/efi/libstub/Makefile
@@ -155,6 +155,12 @@ STUBCOPY_FLAGS-$(CONFIG_LOONGARCH)	+= --prefix-alloc-sections=.init \
 					   --prefix-symbols=__efistub_
 STUBCOPY_RELOC-$(CONFIG_LOONGARCH)	:= R_LARCH_MARK_LA
 
+# --prefix-alloc-sections=.init also renames .note.gnu.property, which the
+# linker then ignores, dropping the branch protection properties.  Rename it
+# back on any architecture which applies the prefix.
+STUBCOPY_RENAME-y = $(if $(findstring --prefix-alloc-sections,$(STUBCOPY_FLAGS-y)), \
+		      --rename-section .init.note.gnu.property=.note.gnu.property)
+
 $(obj)/%.stub.o: $(obj)/%.o FORCE
 	$(call if_changed,stubcopy)
 
@@ -171,4 +177,5 @@ quiet_cmd_stubcopy = STUBCPY $@
 		echo "$@: absolute symbol references not allowed in the EFI stub" >&2; \
 		/bin/false;						\
 	fi;								\
-	$(OBJCOPY) $(STUBCOPY_FLAGS-y) $< $@
+	$(OBJCOPY) $(STUBCOPY_FLAGS-y) $< $@				\
+	$(if $(STUBCOPY_RENAME-y),; $(OBJCOPY) $(STUBCOPY_RENAME-y) $@)
-- 
2.55.0

[PATCH 10/12] efi/libstub: Remove obsolete .note.gnu.property workaround

[Josh Poimboeuf]

Commit e2179a09ab08 ("efi/libstub: Disable -mbranch-protection") added
--remove-section=.note.gnu.property to the stub objcopy invocation to
work around a Clang bug where the note was emitted for code-less object
files even with -mbranch-protection=none.

That was fixed by LLVM commit a48f6079f288 ("[AArch64] Generate
.note.gnu.property based on module flags") which was released with
Clang 12.  The minimum Clang version is now Clang 17, so this workaround
is no longer needed.

On arm64, the stub also no longer builds with -mbranch-protection=none,
as it has inherited the kernel's flags since commit 8358098b9787
("arm64: efi: Enable BTI codegen and add PE/COFF annotation").

Remove the workaround.  This fixes BTI on arm64, and is a no-op on
RISC-V and LoongArch where the vmlinux generic NOTES macro discards it,
and the x86 and ARM decompressors discard .note.* explicitly.

Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
 drivers/firmware/efi/libstub/Makefile | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile
index b1c95f69e807d..e18e124a89acf 100644
--- a/drivers/firmware/efi/libstub/Makefile
+++ b/drivers/firmware/efi/libstub/Makefile
@@ -106,12 +106,6 @@ lib-$(CONFIG_UNACCEPTED_MEMORY) += unaccepted_memory.o bitmap.o find.o
 targets				:= $(lib-y)
 lib-y				:= $(patsubst %.o,%.stub.o,$(lib-y))
 
-# Even when -mbranch-protection=none is set, Clang will generate a
-# .note.gnu.property for code-less object files (like lib/ctype.c),
-# so work around this by explicitly removing the unwanted section.
-# https://llvm.org/pr46480
-STUBCOPY_FLAGS-y		+= --remove-section=.note.gnu.property
-
 STUBCOPY_RELOC-$(CONFIG_X86_32)	:= R_386_32
 STUBCOPY_RELOC-$(CONFIG_X86_64)	:= R_X86_64_64
 
-- 
2.55.0

[PATCH 11/12] arm64/bti: Force-enable BTI linker veneers

[Josh Poimboeuf]

The linker only emits the BTI veneers if *all* input objects have
GNU_PROPERTY_AARCH64_FEATURE_1_BTI.  All the kernel's objects now
advertise it, but if some object stops doing so, BTI veneers silently
stop getting created on >128MB kernels, resulting in BTI exceptions at
runtime.

Force BTI veneers enabled with "-z force-bti" to prevent a single bad
object from ruining things.  This also emits a warning (or error with
CONFIG_WERROR) if any objects are missing the bit.

Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
 arch/arm64/Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 4eee721c0b278..d0db9a6766a2f 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -25,6 +25,7 @@ endif
 
 ifeq ($(CONFIG_ARM64_BTI_KERNEL),y)
 KBUILD_AFLAGS	+= -include $(srctree)/arch/arm64/include/asm/bti-note.h
+LDFLAGS_vmlinux	+= $(call ld-option,-z force-bti)
 endif
 
 cc_has_k_constraint := $(call try-run,echo				\
-- 
2.55.0

[PATCH 12/12] arm64/bti: Enable kernel BTI for GCC

[Josh Poimboeuf]

Commit c0a454b9044f ("arm64/bti: Disable in kernel BTI when cross
section thunks are broken") disabled in-kernel BTI for GCC because it
omits a function's BTI landing pad when it determines that a function
can only ever reached by a direct branch.  The observed failure was a
module whose init text landed far enough from its core text to need a
PLT, whose "br x16" then hit a function with no "bti c".

Note that behavior isn't GCC-specific: Clang 21 has started ommitting
landing pads as well.

Now that all the known BTI bugs have been sorted out, re-enable it for
GCC.

Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
 arch/arm64/Kconfig | 2 --
 1 file changed, 2 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 2687b71438661..e150807b03016 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -2114,8 +2114,6 @@ config ARM64_BTI_KERNEL
 	depends on CC_HAS_BRANCH_PROT_PAC_RET_BTI
 	# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94697
 	depends on !CC_IS_GCC || GCC_VERSION >= 100100
-	# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106671
-	depends on !CC_IS_GCC
 	depends on (!FUNCTION_GRAPH_TRACER || DYNAMIC_FTRACE_WITH_ARGS)
 	help
 	  Build the kernel with Branch Target Identification annotations
-- 
2.55.0

Re: [PATCH 02/12] arm64/module: Fix BTI exceptions caused by omitted landing pads in Clang 21

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [High] Direct memory assignment to the PLT section bypasses text patching, causing a Data Abort when a livepatch module applies relocations to a newly loaded target module.
- [High] The `aarch64_insn_set()` function incorrectly writes A64 instructions in host byte order, causing BTI veneer pages to be filled with corrupt instruction encodings on Big Endian systems.
--

--- Patch [2]: [PATCH 02/12] arm64/module: Fix BTI exceptions caused by omitted landing pads in Clang 21 ---
commit 74ebe0a6ff8b7ada8b741ca9247269a7b1b55fb6
Author: Josh Poimboeuf <jpoimboe@kernel.org>
arm64/module: Fix BTI exceptions caused by omitted landing pads in Clang 21

This commit fixes BTI exceptions caused by Clang 21+ omitting landing pads
for static functions. It allocates a page close to the target to hold BTI
veneers which receive PLT indirect branches and direct branch to the final
target.

> diff --git a/arch/arm64/kernel/module-plts.c b/arch/arm64/kernel/module-plts.c
> index 7afd370da9f48..4ba31e336deb6 100644
> --- a/arch/arm64/kernel/module-plts.c
> +++ b/arch/arm64/kernel/module-plts.c
[ ... ]
> @@ -66,6 +72,180 @@ static bool plt_entries_equal(const struct plt_entry *a,
[ ... ]
> +static struct bti_veneer_page *bti_veneer_page_alloc(struct module *mod,
> +						     u64 target)
> +{
[ ... ]
> +	if (!p) {
> +		pr_err("%s: no address space within branch range of %pS for a BTI veneer\n",
> +		       mod->name, (void *)target);
> +		return NULL;
> +	}
> +
> +	/* Don't leave unused slots executable */
> +	aarch64_insn_set(p, AARCH64_BREAK_FAULT, PAGE_SIZE);

[Severity: High]
This is a pre-existing issue with aarch64_insn_set(), but does this new
usage write the AARCH64_BREAK_FAULT instruction in the wrong byte order on
Big Endian systems?

Looking at aarch64_insn_set(), it passes the host-endian integer to
memset32(), which performs native 32-bit stores without cpu_to_le32()
conversion:

arch/arm64/kernel/patching.c:aarch64_insn_set() {
    ...
    return __text_poke(text_poke_memset, dst, &insn, len);
}

Since ARMv8 instruction fetches are architecturally always Little Endian,
wouldn't the CPU fetch a byte-reversed, corrupt instruction (reading
0x000020d4) on Big Endian systems, resulting in a fatal Unknown instruction
exception rather than a BRK exception if an unused slot is executed?

> +
> +	page = kzalloc_obj(*page, GFP_KERNEL);
> +	if (!page) {
[ ... ]
> @@ -77,6 +257,12 @@ u64 module_emit_plt_entry(struct module *mod, Elf64_Shdr *sechdrs,
>  			  void *loc, const Elf64_Rela *rela,
>  			  Elf64_Sym *sym)
>  {
[ ... ]
>  	u64 val = sym->st_value + rela->r_addend;
>  
> +	if (!plt_target_has_landing_pad(val)) {
> +		val = module_emit_bti_veneer(mod, val);
> +		if (!val)
> +			return 0;
> +	}
> +
>  	if (is_forbidden_offset_for_adrp(&plt[i].adrp))
>  		i++;
>  
>  	plt[i] = get_plt_entry(val, &plt[i]);

[Severity: High]
This isn't a bug introduced by this patch, but can this direct memory
assignment cause a Data Abort kernel panic when a livepatch module applies
relocations to a newly loaded target module?

If a livepatch module is already loaded and active, its sections (including
the PLT) are marked ROX. When a new target module loads,
klp_module_coming() calls apply_relocate_add() to apply the patch module's
relocations for this new target.

If an out-of-range jump requires a new PLT entry, it seems this direct
assignment to the ROX plt memory array bypasses text patching mechanisms
like patch_map() or aarch64_insn_copy().

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1786768375.git.jpoimboe@kernel.org?part=2

Re: [PATCH 11/12] arm64/bti: Force-enable BTI linker veneers

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] ld-option incorrectly evaluates to true for -z force-bti on older GNU ld versions, causing build failures.
--

--- Patch [11]: [PATCH 11/12] arm64/bti: Force-enable BTI linker veneers ---
commit dc92fd3fe21010a76154b8b30a8e4ff7c0d55416
Author: Josh Poimboeuf <jpoimboe@kernel.org>

arm64/bti: Force-enable BTI linker veneers

This commit force-enables BTI veneers using the "-z force-bti" linker
option to ensure they are created even if some input objects are missing
the BTI feature bit. This aims to prevent silent runtime BTI exceptions
on larger kernels.

> diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
> index 4eee721c0b278..d0db9a6766a2f 100644
> --- a/arch/arm64/Makefile
> +++ b/arch/arm64/Makefile
> @@ -25,6 +25,7 @@ endif
>  
>  ifeq ($(CONFIG_ARM64_BTI_KERNEL),y)
>  KBUILD_AFLAGS	+= -include $(srctree)/arch/arm64/include/asm/bti-note.h
> +LDFLAGS_vmlinux	+= $(call ld-option,-z force-bti)

[Severity: Medium]
Does this ld-option check reliably detect if the linker supports "-z force-bti"?

When Kbuild's ld-option tests linker support, it executes the linker with "-v".
However, when GNU ld is invoked with "-v" and no input files, it prints its
version information and immediately exits with a 0 status code, completely
bypassing any validation of keyword arguments like "-z".

Because of this, ld-option will falsely report that "-z force-bti" is supported
on all versions of GNU ld, even older versions (prior to 2.34) where the option
was not yet introduced. 

During the actual kernel link, ld parses the "-z" list, fails to recognize
"force-bti", and emits an ignored warning. If CONFIG_WERROR=y is set, the linker
escalates this warning to a fatal error and breaks the build. On builds without
CONFIG_WERROR, the warning is ignored and the patch silently fails to enforce
the intended BTI veneers.

>  endif
>  
>  cc_has_k_constraint := $(call try-run,echo				\

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1786768375.git.jpoimboe@kernel.org?part=11

Re: [PATCH 01/12] arm64/bti: Add BTI landing pad to __sdei_asm_handler()

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [High] Missing BTI landing pads (`bti j`) on indirect branch targets in the KPTI trampoline (`__sdei_asm_exit_trampoline` and `tramp_exit`).
- [High] The KPTI entry trampoline mapping at `TRAMP_VALIAS` lacks the BTI Guarded Page (`PTE_GP`) attribute, disabling hardware CFI checks.
--

--- Patch [1]: [PATCH 01/12] arm64/bti: Add BTI landing pad to __sdei_asm_handler() ---
commit 59fadd16326c687a5994c4bec81d9484edcc08f9
Author: Josh Poimboeuf <jpoimboe@kernel.org>

arm64/bti: Add BTI landing pad to __sdei_asm_handler()

With CONFIG_UNMAP_KERNEL_AT_EL0, __sdei_asm_entry_trampoline() indirect
jumps to __sdei_asm_handler(). Add "bti j" to prevent a Branch Target
exception.

> diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
> index e0db14e9c843a..16c21fcb41976 100644
> --- a/arch/arm64/kernel/entry.S
> +++ b/arch/arm64/kernel/entry.S
> @@ -976,6 +976,7 @@ NOKPROBE(__sdei_asm_exit_trampoline)

[Severity: High]
This is a pre-existing issue, but do we also need bti j landing pads on the
indirect branch targets for the exit paths?

For example, __sdei_asm_exit_trampoline() is called via an indirect branch
(br x5):

arch/arm64/kernel/entry.S:__sdei_asm_exit_trampoline() {
    SYM_CODE_START(__sdei_asm_exit_trampoline)
        ldr x4, [x4, #(SDEI_EVENT_INTREGS + S_SDEI_TTBR1)]
        ...
}

Similarly, tramp_exit() in the same file lacks a landing pad despite being
called via an indirect branch (br x29) on the EL0 exit path:

arch/arm64/kernel/entry.S:tramp_exit() {
    SYM_CODE_START_LOCAL(tramp_exit)
        tramp_unmap_kernel x29
        ...
}

Would it be better to add bti j annotations to these functions as well to
prevent Branch Target exceptions?


[Severity: High]
This isn't a bug introduced by this patch, but does the KPTI trampoline mapping
need the BTI Guarded Page (PTE_GP) attribute?

Looking at map_entry_trampoline() in arch/arm64/mm/mmu.c, it uses
kernel_exec_prot() to map the trampoline text:

arch/arm64/mm/mmu.c:map_entry_trampoline() {
    pgprot_t prot = kernel_exec_prot();
    ...
    early_create_pgd_mapping(tramp_pg_dir, pa_start, TRAMP_VALIAS,
                             entry_tramp_text_size(), prot,
                             pgd_pgtable_alloc_init_mm, NO_BLOCK_MAPPINGS);
}

The prot variable is derived from PAGE_KERNEL_ROX, which does not appear to
contain the PTE_GP bit. Does this inadvertently disable hardware BTI enforcement
for the trampoline pages? If so, could this architectural gap be masking the
missing bti j annotations on the trampoline exit paths mentioned above?

>   * want them.
>   */
>  SYM_CODE_START(__sdei_asm_handler)
> +	bti	j
>  	stp     x2, x3, [x1, #SDEI_EVENT_INTREGS + S_PC]
>  	stp     x4, x5, [x1, #SDEI_EVENT_INTREGS + 16 * 2]
>  	stp     x6, x7, [x1, #SDEI_EVENT_INTREGS + 16 * 3]

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1786768375.git.jpoimboe@kernel.org?part=1

Re: [PATCH 07/12] arm64/bti: Advertise BTI in assembly objects

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] The patch fails to enable BTI veneers for the kernel because EFI stub and PI object files explicitly strip the property note.
--

--- Patch [7]: [PATCH 07/12] arm64/bti: Advertise BTI in assembly objects ---
commit f368db6f176a29dc1fd3803f6ac93282119ab6aa
Author: Josh Poimboeuf <jpoimboe@kernel.org>

arm64/bti: Advertise BTI in assembly objects

This commit forces the inclusion of a GNU property note with the BTI bit
into every assembly file. This is intended to ensure the linker emits BTI
veneers for >128MB kernels by ensuring all input objects have the property.

> diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
> index 6b005c8fef706..4eee721c0b278 100644
> --- a/arch/arm64/Makefile
> +++ b/arch/arm64/Makefile
> @@ -23,6 +23,10 @@ ifeq ($(CONFIG_ARM64_ERRATUM_843419),y)
>  LDFLAGS_vmlinux	+= --fix-cortex-a53-843419
>  endif
>  
> +ifeq ($(CONFIG_ARM64_BTI_KERNEL),y)
> +KBUILD_AFLAGS	+= -include $(srctree)/arch/arm64/include/asm/bti-note.h
> +endif

[Severity: High]
This isn't a bug introduced by this patch, but will this approach still
fail to emit BTI veneers for the kernel?

The GNU linker requires all input objects to have the BTI property, but
the EFI stub and PI object files explicitly strip the property note before
they are linked into vmlinux.

In drivers/firmware/efi/libstub/Makefile:

STUBCOPY_FLAGS-y		+= --remove-section=.note.gnu.property

And in arch/arm64/kernel/pi/Makefile:

$(obj)/%.pi.o: OBJCOPYFLAGS := --prefix-symbols=__pi_ \
			       --remove-section=.note.gnu.property

Since these objects are stripped of the note and linked into vmlinux,
will the linker continue to evaluate the global link with objects lacking
the BTI property, leaving the original large-kernel crash unresolved?

[ ... ]

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1786768375.git.jpoimboe@kernel.org?part=7

Re: [PATCH 02/12] arm64/module: Fix BTI exceptions caused by omitted landing pads in Clang 21

[Ard Biesheuvel]

Hi Josh,

On Sat, 15 Aug 2026, at 07:45, Josh Poimboeuf wrote:
> The following BTI exception was seen when loading a livepatch module:
>
>   Internal error: Oops - BTI: 0000000036000001 [#1]  SMP
>   pstate: 634004c9 (nZCv daIF +PAN -UAO +TCO +DIT -SSBS BTYPE=jc)
>   pc : kill_orphaned_pgrp+0x0/0x150
>   lr : do_exit+0x498/0xaf0 [livepatch_combined]
>
> The problem is that the patch module's do_exit() is branching to a
> static function in vmlinux using a module PLT veneer (indirect branch),
> but the target function doesn't have a BTI landing pad.
>
> Clang 21+ omits the landing pad for static functions which can only be
> reached by a direct branch.  But livepatch modules use klp relocations
> to reference arbitrary kernel symbols, and with
> CONFIG_RANDOMIZE_MODULE_REGION_FULL the module is far enough away that
> every call to vmlinux needs a PLT.
>
> Note this problem is actually not specific to livepatch.  It's possible
> for any module's .init section to be allocated > 128MB away from its
> .text section.  So calls from .init to .text via a PLT can trigger a BTI
> exception when the target function doesn't have a landing pad.
>
> GCC has always omitted the landing pad when possible, so kernel BTI is
> already considered incompatible with GCC since commit c0a454b9044f
> ("arm64/bti: Disable in kernel BTI when cross section thunks are
> broken").
>
> When missing landing pads are detected, allocate a page close to the
> target which can be used to hold BTI veneers which receive PLT veneer
> indirect branches and direct branch to the final target:
>

This does not work for cross-section calls from .init.text to .text.

If .init.text is far away from .text, it is likely because .text
ended up in the 128M 'near' module region, and .init.text did not.
(They tend to end up in direct branching range of each otherwise.)

Given that the module init code is typically small, I don't think
it is safe to assume that allocating a single page close enough to
.text is going to be possible if allocating the space for .init.*
was not.

IOW, the fix I proposed for cross-section calls is still needed
with this approach.

Re: [PATCH 02/12] arm64/module: Fix BTI exceptions caused by omitted landing pads in Clang 21

[Josh Poimboeuf]

On Sat, Aug 15, 2026 at 12:56:11PM +0300, Ard Biesheuvel wrote:
> Hi Josh,
> 
> On Sat, 15 Aug 2026, at 07:45, Josh Poimboeuf wrote:
> > The following BTI exception was seen when loading a livepatch module:
> >
> >   Internal error: Oops - BTI: 0000000036000001 [#1]  SMP
> >   pstate: 634004c9 (nZCv daIF +PAN -UAO +TCO +DIT -SSBS BTYPE=jc)
> >   pc : kill_orphaned_pgrp+0x0/0x150
> >   lr : do_exit+0x498/0xaf0 [livepatch_combined]
> >
> > The problem is that the patch module's do_exit() is branching to a
> > static function in vmlinux using a module PLT veneer (indirect branch),
> > but the target function doesn't have a BTI landing pad.
> >
> > Clang 21+ omits the landing pad for static functions which can only be
> > reached by a direct branch.  But livepatch modules use klp relocations
> > to reference arbitrary kernel symbols, and with
> > CONFIG_RANDOMIZE_MODULE_REGION_FULL the module is far enough away that
> > every call to vmlinux needs a PLT.
> >
> > Note this problem is actually not specific to livepatch.  It's possible
> > for any module's .init section to be allocated > 128MB away from its
> > .text section.  So calls from .init to .text via a PLT can trigger a BTI
> > exception when the target function doesn't have a landing pad.
> >
> > GCC has always omitted the landing pad when possible, so kernel BTI is
> > already considered incompatible with GCC since commit c0a454b9044f
> > ("arm64/bti: Disable in kernel BTI when cross section thunks are
> > broken").
> >
> > When missing landing pads are detected, allocate a page close to the
> > target which can be used to hold BTI veneers which receive PLT veneer
> > indirect branches and direct branch to the final target:
> >
> 
> This does not work for cross-section calls from .init.text to .text.
> 
> If .init.text is far away from .text, it is likely because .text
> ended up in the 128M 'near' module region, and .init.text did not.
> (They tend to end up in direct branching range of each otherwise.)
> 
> Given that the module init code is typically small, I don't think
> it is safe to assume that allocating a single page close enough to
> .text is going to be possible if allocating the space for .init.*
> was not.
> 
> IOW, the fix I proposed for cross-section calls is still needed
> with this approach.

But the BTI veneer page is allocated from a *256MB* window, of which the
near region is only a 128MB subset.

There is a theoretical case where the 256MB window around the target is
completely full without any fragmentation, but I would think that there
would almost always be some fragmentation.  If that window is modules
stacked together, most modules have at least .init.plt and .init.text,
and many have .init.data.

Right now it needs two pages (because of the default guard page) but we
could maybe fall back to VM_NO_GUARD in case of emergency.

-- 
Josh