[PATCH RFC 0/8] arm64: ftrace: support CALL_OPS on kernels built with kCFI

[PATCH RFC 0/8] arm64: ftrace: support CALL_OPS on kernels built with kCFI

[Jose Fernandez (Anthropic)]

CALL_OPS and kCFI have been mutually exclusive on arm64 since commit
baaf553d3bc3 ("arm64: Implement HAVE_DYNAMIC_FTRACE_WITH_CALL_OPS").
kCFI checks the type hash at a fixed offset before the entry point, so
every function needs the same number of prefix NOPs. Before clang 21 the
compiler could not emit prefix NOPs without also adding the function to
ftrace's table. Since commit 9315e22b0c0a ("arm64: ftrace: allow
DIRECT_CALLS without CALL_OPS") a kCFI kernel can attach BPF
trampolines. A trampoline out of BL range is reached through
ftrace_caller. That path costs ~10 ns more per call than on a CALL_OPS
kernel, plus ~4 ns for every extra ftrace_ops registered. Clang 21 takes
a section name as a third argument of -fpatchable-function-entry [1].
This series uses it to give every function three prefix NOPs while
keeping untraced functions out of ftrace's table, and enables CALL_OPS
under kCFI. The per-call cost drops by ~10 ns and stops growing with the
number of ftrace_ops registered.

Patch 1 lets core ftrace skip a patch site the architecture rejects,
which patch 4 relies on. Patch 2 takes the sorttable entry offset from
Kconfig. Patch 3 lets notrace keep the prefix NOPs. Patch 4 makes the
arm64 prefix layout configurable and refuses patch sites that do not
fit. Patch 5 applies the prefix count where the type hash is emitted or
read by hand. Patch 6 adds the compiler probe and enables CALL_OPS
under CFI. Patch 7 uses five NOPs on ThinLTO kernels with BTI. Patch 8
gives Rust functions the same prefix NOPs.

The cost is text size, 4.8% of .text on a defconfig-based build with
CFI and 7.4% with ThinLTO and BTI. CALL_OPS under CFI needs clang 21 or
later, and Rust 1.98 or later with RUST=y. Commit d3359af21fc9e ("arm64:
bti: Disable in-kernel BTI with recent versions of Clang") turns BTI
off on clang 21 or later, so a mainline build has one or the other
today. A module must be built with the same prefix count as its kernel,
since vermagic does not carry the count.

Numbers below are from a KVM guest on a Graviton4 host, v7.3-rc1 with
and without this series, clang 21.1.4. Cost of a fentry program on a
syscall, per call, the call with the program attached minus the call
without it:

  Kernel            | Trampoline   | Extra      || Per-call cost (ns)
                    | vs. BL range | ftrace_ops || Median | Min  | Max
  ==================+==============+============++========+======+======
  kCFI, base        | out of range |          0 ||   32.9 | 32.6 | 33.4
  kCFI, base        | out of range |         16 ||   92.6 | 92.4 | 96.3
  kCFI, base        | in range     |          0 ||   21.3 | 21.0 | 21.8
  ------------------+--------------+------------++--------+------+------
  kCFI, this series | out of range |          0 ||   22.7 | 22.0 | 22.9
  kCFI, this series | out of range |         16 ||   22.1 | 21.9 | 22.4
  kCFI, this series | in range     |          0 ||   22.4 | 22.1 | 22.6
  ------------------+--------------+------------++--------+------+------

Mark Rutland discussed the uniform prefix with a section for untraced
entries in 2022 [2]. His review of the DIRECT_CALLS series raised the
cost of the ftrace_caller path [3]. Josh Poimboeuf's objtool support
for arm64 livepatch [4] detects the prefix by the CALL_OPS layout and
treats kCFI and CALL_OPS as never meeting, which this series changes.

This is an RFC because it moves the kCFI type hash relative to the entry
point on arm64 and every function carries the prefix NOPs. We would like
the arm64 and CFI maintainers' view on that layout.

[1] https://github.com/llvm/llvm-project/pull/131230
[2] https://lore.kernel.org/all/Y1LBGZPMfCZ8A1bl@FVFF77S0Q05N/
[3] https://lore.kernel.org/all/amjnf5gz0xP5PTSB@J2N7QTR9R3/
[4] https://lore.kernel.org/all/cover.1786230311.git.jpoimboe@kernel.org/

---
Jose Fernandez (Anthropic) (8):
      ftrace: Let ftrace_call_adjust() reject a patch site
      scripts/sorttable: Make the arm64 before_func offset configurable
      compiler_types: Let notrace keep the function prefix NOPs
      arm64: ftrace: Make the CALL_OPS prefix layout configurable
      arm64: cfi: Use CONFIG_ARM64_FUNCTION_PREFIX_NOPS for the type hash offset
      arm64: ftrace: Support CALL_OPS on kernels built with kCFI
      arm64: ftrace: Use five prefix NOPs on ThinLTO kernels with BTI
      arm64: ftrace: Allow CALL_OPS on kCFI kernels built with Rust

 arch/arm64/Kconfig                    |  27 +++++++-
 arch/arm64/Makefile                   |  15 ++++-
 arch/arm64/include/asm/cfi.h          |   8 +++
 arch/arm64/include/asm/ftrace.h       |  16 +++++
 arch/arm64/include/asm/linkage.h      |  11 ++++
 arch/arm64/kernel/entry-ftrace.S      |  16 ++---
 arch/arm64/kernel/ftrace.c            | 120 +++++++++++++++++++---------------
 arch/arm64/kernel/pi/Makefile         |   2 +
 arch/arm64/kernel/vdso/Makefile       |   2 +-
 arch/arm64/net/bpf_jit_comp.c         |   8 ++-
 drivers/firmware/efi/libstub/Makefile |   2 +-
 include/linux/compiler_types.h        |   6 ++
 kernel/trace/ftrace.c                 |   4 ++
 rust/Makefile                         |   5 +-
 scripts/Makefile                      |   3 +
 scripts/sorttable.c                   |   7 +-
 16 files changed, 185 insertions(+), 67 deletions(-)
---
base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
change-id: 20260904-b4-arm64-callops-kcfi-40016eed4fb0

Best regards,
--  
Jose Fernandez (Anthropic) <jose.fernandez@linux.dev>

[PATCH RFC 1/8] ftrace: Let ftrace_call_adjust() reject a patch site

[Jose Fernandez (Anthropic)]

When ftrace_call_adjust() returns 0 for a patch site,
ftrace_process_locs() does not skip the site. The NULL test runs before
the adjustment since commit 6eeca746fa5f ("ftrace: Test mcount_loc
addr before calling ftrace_call_addr()"), so the 0 is stored as the
record's ip and ftrace_init_nop() later hits the BUG_ON() in arm64's
patch_map(). Test the adjusted address too and count the site as
skipped.

Reviewed-by: Ben Cressey <ben@cressey.dev>
Reviewed-by: Florent Revest (Anthropic) <florent.revest@linux.dev>
Assisted-by: LLM
Signed-off-by: Jose Fernandez (Anthropic) <jose.fernandez@linux.dev>
---
 kernel/trace/ftrace.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
index f9d80c7bd9f16..394f82230e0c3 100644
--- a/kernel/trace/ftrace.c
+++ b/kernel/trace/ftrace.c
@@ -7676,6 +7676,10 @@ static int ftrace_process_locs(struct module *mod,
 		}
 
 		addr = ftrace_call_adjust(addr);
+		if (!addr) {
+			skipped++;
+			continue;
+		}
 
 		end_offset = (pg->index+1) * sizeof(pg->records[0]);
 		if (end_offset > PAGE_SIZE << pg->order) {

-- 
2.52.0

[PATCH RFC 2/8] scripts/sorttable: Make the arm64 before_func offset configurable

[Jose Fernandez (Anthropic)]

sorttable hardcodes before_func to 8 bytes on arm64 (two prefix NOPs).
Make it configurable from CONFIG_ARM64_FUNCTION_PREFIX_NOPS instead.
kCFI kernels with CALL_OPS need three NOPs. The symbol does not exist
yet, so no functional change.

Reviewed-by: Ben Cressey <ben@cressey.dev>
Reviewed-by: Florent Revest (Anthropic) <florent.revest@linux.dev>
Assisted-by: LLM
Signed-off-by: Jose Fernandez (Anthropic) <jose.fernandez@linux.dev>
---
 scripts/Makefile    | 3 +++
 scripts/sorttable.c | 7 ++++++-
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/scripts/Makefile b/scripts/Makefile
index 3434a82a119f0..366c6a35141a6 100644
--- a/scripts/Makefile
+++ b/scripts/Makefile
@@ -51,6 +51,9 @@ endif
 
 ifdef CONFIG_BUILDTIME_MCOUNT_SORT
 HOSTCFLAGS_sorttable.o += -DMCOUNT_SORT_ENABLED
+ifneq ($(filter-out 0,$(CONFIG_ARM64_FUNCTION_PREFIX_NOPS)),)
+HOSTCFLAGS_sorttable.o += -DFUNCTION_PREFIX_BYTES=$(shell expr 4 \* $(CONFIG_ARM64_FUNCTION_PREFIX_NOPS))
+endif
 endif
 
 # The following programs are only built on demand
diff --git a/scripts/sorttable.c b/scripts/sorttable.c
index d8dc2a1b7c312..925382bb23dcc 100644
--- a/scripts/sorttable.c
+++ b/scripts/sorttable.c
@@ -281,6 +281,11 @@ static int add_field(uint64_t addr, uint64_t size)
 /* Used for when mcount/fentry is before the function entry */
 static int before_func;
 
+/* How far before the function entry a patchable entry may be recorded */
+#ifndef FUNCTION_PREFIX_BYTES
+#define FUNCTION_PREFIX_BYTES	8
+#endif
+
 /* Only return match if the address lies inside the function size */
 static int cmp_func_addr(const void *K, const void *A)
 {
@@ -899,7 +904,7 @@ static int do_file(char const *const fname, void *addr)
 		/* fallthrough */
 	case EM_RISCV:
 		/* arm64 and RISC-V place patchable entries before the function. */
-		before_func = 8;
+		before_func = FUNCTION_PREFIX_BYTES;
 #else
 	case EM_AARCH64:
 	case EM_RISCV:

-- 
2.52.0

[PATCH RFC 3/8] compiler_types: Let notrace keep the function prefix NOPs

[Jose Fernandez (Anthropic)]

Let notrace keep the prefix NOPs when the architecture defines
CC_USING_PATCHABLE_FUNCTION_PREFIX. kCFI callers check the type hash
at a fixed offset before the entry point, so a notrace function needs
the same prefix NOPs as any other. Record the entry in a section the
linker drops, so ftrace never sees it. Clang 21 takes a section name
as a third argument of the attribute. Nothing defines the macro yet,
so no functional change.

Reviewed-by: Ben Cressey <ben@cressey.dev>
Reviewed-by: Florent Revest (Anthropic) <florent.revest@linux.dev>
Assisted-by: LLM
Signed-off-by: Jose Fernandez (Anthropic) <jose.fernandez@linux.dev>
---
 include/linux/compiler_types.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h
index c5921f1390079..87e10920acba7 100644
--- a/include/linux/compiler_types.h
+++ b/include/linux/compiler_types.h
@@ -211,6 +211,12 @@ struct ftrace_likely_data {
 
 #if defined(CC_USING_HOTPATCH)
 #define notrace			__attribute__((hotpatch(0, 0)))
+#elif defined(CC_USING_PATCHABLE_FUNCTION_PREFIX) && !defined(__BINDGEN__)
+/* bindgen generates no code and its libclang may not take the section argument. */
+#define notrace								\
+	__attribute__((patchable_function_entry(CC_USING_PATCHABLE_FUNCTION_PREFIX, \
+						 CC_USING_PATCHABLE_FUNCTION_PREFIX, \
+						 ".discard.patchable_function_entries")))
 #elif defined(CC_USING_PATCHABLE_FUNCTION_ENTRY)
 #define notrace			__attribute__((patchable_function_entry(0, 0)))
 #else

-- 
2.52.0

[PATCH RFC 4/8] arm64: ftrace: Make the CALL_OPS prefix layout configurable

[Jose Fernandez (Anthropic)]

The arm64 ftrace code assumes two NOPs before every function in
several places. Replace the hardcoded count with a Kconfig value named
CONFIG_ARM64_FUNCTION_PREFIX_NOPS that defaults to 2. Compute the ops
literal address with one formula in both the assembly that reads it
and the C code that writes it. kCFI kernels with CALL_OPS need three
NOPs, or five with ThinLTO and BTI, and the reader and the writer stay
in sync either way.

Also make ftrace_call_adjust() look at each site's bytes and refuse
the ones that do not match the expected layout instead of patching
them. The two instructions to patch must be NOPs and the literal must
fall within the prefix NOPs. Log a refused site and leave it untraced.

No functional change for the layouts compilers generate today.

Reviewed-by: Ben Cressey <ben@cressey.dev>
Reviewed-by: Florent Revest (Anthropic) <florent.revest@linux.dev>
Assisted-by: LLM
Signed-off-by: Jose Fernandez (Anthropic) <jose.fernandez@linux.dev>
---
 arch/arm64/Kconfig               |   5 ++
 arch/arm64/include/asm/ftrace.h  |  16 ++++++
 arch/arm64/kernel/entry-ftrace.S |  16 +++---
 arch/arm64/kernel/ftrace.c       | 120 ++++++++++++++++++++++-----------------
 4 files changed, 98 insertions(+), 59 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index b5a51b0ef9440..9fbcf12a3808b 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -276,6 +276,11 @@ config GCC_SUPPORTS_DYNAMIC_FTRACE_WITH_ARGS
 	def_bool CC_IS_GCC
 	depends on $(cc-option,-fpatchable-function-entry=2)
 
+config ARM64_FUNCTION_PREFIX_NOPS
+	int
+	default 2 if DYNAMIC_FTRACE_WITH_CALL_OPS
+	default 0
+
 config 64BIT
 	def_bool y
 
diff --git a/arch/arm64/include/asm/ftrace.h b/arch/arm64/include/asm/ftrace.h
index 1621c84f44b32..5a7a515bf7cdc 100644
--- a/arch/arm64/include/asm/ftrace.h
+++ b/arch/arm64/include/asm/ftrace.h
@@ -24,6 +24,22 @@
 #define FTRACE_PLT_IDX		0
 #define NR_FTRACE_PLTS		1
 
+#ifdef CONFIG_DYNAMIC_FTRACE_WITH_CALL_OPS
+/*
+ * ftrace_caller reads the callsite's ftrace_ops literal at
+ * ALIGN_DOWN(LR, 8) - FTRACE_CALL_OPS_BIAS. See ftrace_call_ops_adjust()
+ * for the layouts this serves.
+ */
+#define FTRACE_PREFIX_NOPS	CONFIG_ARM64_FUNCTION_PREFIX_NOPS
+#if FTRACE_PREFIX_NOPS == 2 || FTRACE_PREFIX_NOPS == 3
+#define FTRACE_CALL_OPS_BIAS	16
+#elif FTRACE_PREFIX_NOPS == 5
+#define FTRACE_CALL_OPS_BIAS	24
+#else
+#error "Unsupported number of function prefix NOPs"
+#endif
+#endif /* CONFIG_DYNAMIC_FTRACE_WITH_CALL_OPS */
+
 /*
  * Currently, gcc tends to save the link register after the local variables
  * on the stack. This causes the max stack tracer to report the function
diff --git a/arch/arm64/kernel/entry-ftrace.S b/arch/arm64/kernel/entry-ftrace.S
index 025140caafe74..95727fee3cc55 100644
--- a/arch/arm64/kernel/entry-ftrace.S
+++ b/arch/arm64/kernel/entry-ftrace.S
@@ -39,17 +39,17 @@ SYM_CODE_START(ftrace_caller)
 #ifdef CONFIG_DYNAMIC_FTRACE_WITH_CALL_OPS
 	/*
 	 * The literal pointer to the ops is at an 8-byte aligned boundary
-	 * which is either 12 or 16 bytes before the BL instruction in the call
-	 * site. See ftrace_call_adjust() for details.
+	 * before the BL instruction in the call site. See ftrace_call_adjust()
+	 * and <asm/ftrace.h> for details.
 	 *
-	 * Therefore here the LR points at `literal + 16` or `literal + 20`,
-	 * and we can find the address of the literal in either case by
-	 * aligning to an 8-byte boundary and subtracting 16. We do the
-	 * alignment first as this allows us to fold the subtraction into the
-	 * LDR.
+	 * Therefore here the LR points at `literal + FTRACE_CALL_OPS_BIAS` or
+	 * `literal + FTRACE_CALL_OPS_BIAS + 4`, and we can find the address of
+	 * the literal in either case by aligning to an 8-byte boundary and
+	 * subtracting FTRACE_CALL_OPS_BIAS. We do the alignment first as this
+	 * allows us to fold the subtraction into the LDR.
 	 */
 	bic	x11, x30, 0x7
-	ldr	x11, [x11, #-(4 * AARCH64_INSN_SIZE)]		// op
+	ldr	x11, [x11, #-FTRACE_CALL_OPS_BIAS]		// op
 
 #ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS
 	/*
diff --git a/arch/arm64/kernel/ftrace.c b/arch/arm64/kernel/ftrace.c
index e1a3c0b3a0514..0d922fa63f71b 100644
--- a/arch/arm64/kernel/ftrace.c
+++ b/arch/arm64/kernel/ftrace.c
@@ -60,6 +60,73 @@ int ftrace_regs_query_register_offset(const char *name)
 }
 #endif
 
+#ifdef CONFIG_DYNAMIC_FTRACE_WITH_CALL_OPS
+/*
+ * The address of the ops literal for a callsite whose `BL <caller>` is at
+ * @pc, exactly as ftrace_caller derives it from the return address.
+ */
+static unsigned long ftrace_call_ops_literal(unsigned long pc)
+{
+	return ALIGN_DOWN(pc + AARCH64_INSN_SIZE, 8) - FTRACE_CALL_OPS_BIAS;
+}
+
+/*
+ * addr is the first of the FTRACE_PREFIX_NOPS NOPs before a function:
+ *
+ * addr-04:		.word		// kCFI type hash, address-taken and non-local only
+ * addr+00:		NOP		// FTRACE_PREFIX_NOPS of these, two hold the literal
+ * func+00:	func:	BTI	C	// only if BTI and the function may be called indirectly
+ * func+04:		NOP		// patched to MOV X9, LR
+ * func+08:		NOP		// patched to BL <caller>
+ *
+ * The compiler aligns whatever comes first to 8 bytes. ftrace_caller reads
+ * the literal at ALIGN_DOWN(LR, 8) - FTRACE_CALL_OPS_BIAS, with LR at
+ * func+8, or func+12 after a BTI:
+ *
+ *	NOPs	BIAS	literal at
+ *	2	16	func - 8
+ *	3	16	func - 8, or func - 12 if func % 8 == 4 (no hash, no BTI)
+ *	5	24	func - 16, or func - 12 / func - 20 if func % 8 == 4
+ *
+ * Return the address of the NOP to be patched to BL, or 0 when the two
+ * NOPs are not there or the literal would fall outside the prefix.
+ */
+static unsigned long ftrace_call_ops_adjust(unsigned long addr)
+{
+	unsigned long func = addr + FTRACE_PREFIX_NOPS * AARCH64_INSN_SIZE;
+	unsigned long pc = func + AARCH64_INSN_SIZE;
+	unsigned long literal;
+	u32 nop = aarch64_insn_gen_nop();
+
+	if (IS_ENABLED(CONFIG_ARM64_BTI_KERNEL) &&
+	    aarch64_insn_is_bti(le32_to_cpu(*(__le32 *)func)))
+		pc += AARCH64_INSN_SIZE;
+
+	/* Objects built with only the prefix NOPs have nothing to patch. */
+	if (le32_to_cpu(*(__le32 *)(pc - AARCH64_INSN_SIZE)) != nop ||
+	    le32_to_cpu(*(__le32 *)pc) != nop) {
+		pr_warn_ratelimited("ftrace: no patchable entry at %ps\n",
+				    (void *)func);
+		return 0;
+	}
+
+	/* The literal would overlap the type hash or the entry point. */
+	literal = ftrace_call_ops_literal(pc);
+	if (literal < addr || literal + sizeof(u64) > func) {
+		pr_warn_ratelimited("ftrace: cannot trace %ps: no room for the ops literal\n",
+				    (void *)func);
+		return 0;
+	}
+
+	return pc;
+}
+#else
+static unsigned long ftrace_call_ops_adjust(unsigned long addr)
+{
+	return 0;
+}
+#endif /* CONFIG_DYNAMIC_FTRACE_WITH_CALL_OPS */
+
 unsigned long ftrace_call_adjust(unsigned long addr)
 {
 	/*
@@ -91,56 +158,7 @@ unsigned long ftrace_call_adjust(unsigned long addr)
 	if (!IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_CALL_OPS))
 		return addr + AARCH64_INSN_SIZE;
 
-	/*
-	 * When using patchable-function-entry with pre-function NOPs, addr is
-	 * the address of the first pre-function NOP.
-	 *
-	 * Starting from an 8-byte aligned base, the compiler has either
-	 * generated:
-	 *
-	 * addr+00:		NOP		// Literal (first 32 bits)
-	 * addr+04:		NOP		// Literal (last 32 bits)
-	 * addr+08:	func:	NOP		// To be patched to MOV X9, LR
-	 * addr+12:		NOP		// To be patched to BL <caller>
-	 *
-	 * Or:
-	 *
-	 * addr+00:		NOP		// Literal (first 32 bits)
-	 * addr+04:		NOP		// Literal (last 32 bits)
-	 * addr+08:	func:	BTI	C
-	 * addr+12:		NOP		// To be patched to MOV X9, LR
-	 * addr+16:		NOP		// To be patched to BL <caller>
-	 *
-	 * We must adjust addr to the address of the NOP which will be patched
-	 * to `BL <caller>`, which is at either addr+12 or addr+16 depending on
-	 * whether there is a BTI.
-	 */
-
-	if (!IS_ALIGNED(addr, sizeof(unsigned long))) {
-		WARN_RATELIMIT(1, "Misaligned patch-site %pS\n",
-			       (void *)(addr + 8));
-		return 0;
-	}
-
-	/* Skip the NOPs placed before the function entry point */
-	addr += 2 * AARCH64_INSN_SIZE;
-
-	/* Skip any BTI */
-	if (IS_ENABLED(CONFIG_ARM64_BTI_KERNEL)) {
-		u32 insn = le32_to_cpu(*(__le32 *)addr);
-
-		if (aarch64_insn_is_bti(insn)) {
-			addr += AARCH64_INSN_SIZE;
-		} else if (insn != aarch64_insn_gen_nop()) {
-			WARN_RATELIMIT(1, "unexpected insn in patch-site %pS: 0x%08x\n",
-				       (void *)addr, insn);
-		}
-	}
-
-	/* Skip the first NOP after function entry */
-	addr += AARCH64_INSN_SIZE;
-
-	return addr;
+	return ftrace_call_ops_adjust(addr);
 }
 
 /* Convert fentry_ip to the symbol address without kallsyms */
@@ -368,7 +386,7 @@ static const struct ftrace_ops *arm64_rec_get_ops(struct dyn_ftrace *rec)
 static int ftrace_rec_set_ops(const struct dyn_ftrace *rec,
 			      const struct ftrace_ops *ops)
 {
-	unsigned long literal = ALIGN_DOWN(rec->ip - 12, 8);
+	unsigned long literal = ftrace_call_ops_literal(rec->ip);
 	return aarch64_insn_write_literal_u64((void *)literal,
 					      (unsigned long)ops);
 }

-- 
2.52.0

[PATCH RFC 5/8] arm64: cfi: Use CONFIG_ARM64_FUNCTION_PREFIX_NOPS for the type hash offset

[Jose Fernandez (Anthropic)]

Make cfi_get_offset(), SYM_TYPED_FUNC_START and the BPF JIT use
CONFIG_ARM64_FUNCTION_PREFIX_NOPS instead of hardcoding four bytes. kCFI
kernels with CALL_OPS need three NOPs, or five with ThinLTO and BTI.

CONFIG_ARM64_FUNCTION_PREFIX_NOPS is 0 on every CFI kernel today, so no
functional change.

Reviewed-by: Ben Cressey <ben@cressey.dev>
Reviewed-by: Florent Revest (Anthropic) <florent.revest@linux.dev>
Assisted-by: LLM
Signed-off-by: Jose Fernandez (Anthropic) <jose.fernandez@linux.dev>
---
 arch/arm64/include/asm/cfi.h     |  8 ++++++++
 arch/arm64/include/asm/linkage.h | 11 +++++++++++
 arch/arm64/net/bpf_jit_comp.c    |  8 ++++++--
 3 files changed, 25 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/include/asm/cfi.h b/arch/arm64/include/asm/cfi.h
index ab90f0351b7ae..03f02d4614c23 100644
--- a/arch/arm64/include/asm/cfi.h
+++ b/arch/arm64/include/asm/cfi.h
@@ -4,4 +4,12 @@
 
 #define __bpfcall
 
+#ifdef CONFIG_CFI
+static inline int cfi_get_offset(void)
+{
+	return 4 + 4 * CONFIG_ARM64_FUNCTION_PREFIX_NOPS;
+}
+#define cfi_get_offset cfi_get_offset
+#endif /* CONFIG_CFI */
+
 #endif /* _ASM_ARM64_CFI_H */
diff --git a/arch/arm64/include/asm/linkage.h b/arch/arm64/include/asm/linkage.h
index d1f7a16729d25..14bc3c67c38b3 100644
--- a/arch/arm64/include/asm/linkage.h
+++ b/arch/arm64/include/asm/linkage.h
@@ -39,6 +39,17 @@
 	SYM_START(name, SYM_L_WEAK, SYM_A_NONE)		\
 	bti c ;
 
+/*
+ * The compiler emits CONFIG_ARM64_FUNCTION_PREFIX_NOPS NOPs between a C
+ * function's kCFI type hash and its entry point. Callers check the hash
+ * at that offset.
+ */
+#define __CFI_TYPE(name)				\
+	.4byte __kcfi_typeid_##name ASM_NL		\
+	.rept CONFIG_ARM64_FUNCTION_PREFIX_NOPS ASM_NL	\
+	nop ASM_NL					\
+	.endr
+
 #define SYM_TYPED_FUNC_START(name)				\
 	SYM_TYPED_START(name, SYM_L_GLOBAL, SYM_A_ALIGN)	\
 	bti c ;
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index c18e005a41dbe..7ad14d9847138 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -185,8 +185,12 @@ static inline void emit_bti(u32 insn, struct jit_ctx *ctx)
 
 static inline void emit_kcfi(u32 hash, struct jit_ctx *ctx)
 {
-	if (IS_ENABLED(CONFIG_CFI))
-		emit_u32_data(hash, ctx);
+	if (!IS_ENABLED(CONFIG_CFI))
+		return;
+
+	emit_u32_data(hash, ctx);
+	for (int i = 0; i < CONFIG_ARM64_FUNCTION_PREFIX_NOPS; i++)
+		emit(A64_NOP, ctx);
 }
 
 /*

-- 
2.52.0

[PATCH RFC 6/8] arm64: ftrace: Support CALL_OPS on kernels built with kCFI

[Jose Fernandez (Anthropic)]

CALL_OPS and kCFI have been mutually exclusive on arm64 since commit
baaf553d3bc3 ("arm64: Implement HAVE_DYNAMIC_FTRACE_WITH_CALL_OPS").
kCFI checks the type hash at a fixed offset before the entry point, so
every function needs the same number of prefix NOPs. Before clang 21,
prefix NOPs came only with an entry in ftrace's table, and untraced
functions had none. Since commit 9315e22b0c0a ("arm64: ftrace: allow
DIRECT_CALLS without CALL_OPS") a kCFI kernel can attach BPF
trampolines. But without CALL_OPS, a trampoline out of BL range is
reached through ftrace_caller, ~10 ns more per call than on a CALL_OPS
kernel, plus ~4 ns for every extra ftrace_ops registered.

Clang 21 added a third argument to -fpatchable-function-entry that names
the section the entries go to [1]. Point it at a section the linker
scripts already drop and a function gets prefix NOPs with no entry in
ftrace's table. When CFI and CALL_OPS are both enabled, compile every
object with
-fpatchable-function-entry=M,M,.discard.patchable_function_entries,
where M is CONFIG_ARM64_FUNCTION_PREFIX_NOPS.

Gate CALL_OPS under CFI on ARM64_CFI_PERMITS_CALL_OPS, which needs
clang 21 or newer and RUST off. rustc has its own flag for the prefix
NOPs and this patch does not pass it, so Rust functions would have no
prefix and a different hash offset.

M is 3 because the ops pointer needs an aligned 8-byte slot inside the
prefix NOPs, and the 4-byte hash puts the NOPs at offset 4. A traced
function, with the two entry NOPs from CC_FLAGS_FTRACE, then looks
like this:

  func-16:            .word <hash>    // kCFI type hash
  func-12:            NOP
  func-08:            NOP             // ops pointer, 8 bytes
  func-04:            NOP             //
  func+00:    func:   NOP             // To be patched to MOV X9, LR
  func+04:            NOP             // To be patched to BL <caller>

Size of .text in vmlinux on a defconfig-based config with clang 21.1.4:

  Kernel                 || .text bytes | Delta       | Delta
  =======================++=============+=============+=======
  kCFI, base             ||  21,023,888 |             |
  kCFI, M = 3            ||  22,031,504 |  +1,007,616 | +4.8%
  -----------------------++-------------+-------------+-------

Numbers below are from a KVM guest on a Graviton4 host, v7.3-rc1 with
and without this series. The workload is a fentry program on
__arm64_sys_getpid, 20M calls per pass, six passes per boot, six
boots, each cell the call with the program attached minus the call
without it. The in-range rows are the same kernels booted with nokaslr.

  Kernel            | Trampoline   | Extra      || Per-call cost (ns)
                    | vs. BL range | ftrace_ops || Median | Min  | Max
  ==================+==============+============++========+======+======
  kCFI, base        | out of range |          0 ||   32.9 | 32.6 | 33.4
  kCFI, base        | out of range |         16 ||   92.6 | 92.4 | 96.3
  kCFI, base        | in range     |          0 ||   21.3 | 21.0 | 21.8
  ------------------+--------------+------------++--------+------+------
  kCFI, this series | out of range |          0 ||   22.7 | 22.0 | 22.9
  kCFI, this series | out of range |         16 ||   22.1 | 21.9 | 22.4
  kCFI, this series | in range     |          0 ||   22.4 | 22.1 | 22.6
  ------------------+--------------+------------++--------+------+------

Out of range this saves ~10 ns per call and the cost stops growing
with extra ftrace_ops. In range both kernels branch straight to
the trampoline and the numbers roughly match.

ftracetest and the BPF trampoline, struct_ops and kprobe_multi
selftests pass with CFI enforcing.

[1] https://github.com/llvm/llvm-project/pull/131230

Reviewed-by: Ben Cressey <ben@cressey.dev>
Reviewed-by: Florent Revest (Anthropic) <florent.revest@linux.dev>
Assisted-by: LLM
Signed-off-by: Jose Fernandez (Anthropic) <jose.fernandez@linux.dev>
---
 arch/arm64/Kconfig                    | 15 ++++++++++++++-
 arch/arm64/Makefile                   | 12 +++++++++++-
 arch/arm64/kernel/pi/Makefile         |  2 ++
 arch/arm64/kernel/vdso/Makefile       |  2 +-
 drivers/firmware/efi/libstub/Makefile |  2 +-
 5 files changed, 29 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 9fbcf12a3808b..6e5fa009c11af 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -190,7 +190,8 @@ config ARM64
 	select HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS \
 		if DYNAMIC_FTRACE_WITH_ARGS
 	select HAVE_DYNAMIC_FTRACE_WITH_CALL_OPS \
-		if (DYNAMIC_FTRACE_WITH_ARGS && !CFI && \
+		if (DYNAMIC_FTRACE_WITH_ARGS && \
+		    (!CFI || ARM64_CFI_PERMITS_CALL_OPS) && \
 		    (CC_IS_CLANG || !CC_OPTIMIZE_FOR_SIZE))
 	select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY \
 		if DYNAMIC_FTRACE_WITH_ARGS
@@ -276,8 +277,20 @@ config GCC_SUPPORTS_DYNAMIC_FTRACE_WITH_ARGS
 	def_bool CC_IS_GCC
 	depends on $(cc-option,-fpatchable-function-entry=2)
 
+config CC_HAS_PATCHABLE_FUNCTION_ENTRY_SECTION
+	# The section argument needs clang 21 or later:
+	# https://github.com/llvm/llvm-project/pull/131230
+	def_bool $(cc-option,-fpatchable-function-entry=1$(comma)1$(comma).discard.patchable_function_entries)
+
+config ARM64_CFI_PERMITS_CALL_OPS
+	def_bool y
+	depends on CFI
+	depends on CC_HAS_PATCHABLE_FUNCTION_ENTRY_SECTION
+	depends on !RUST
+
 config ARM64_FUNCTION_PREFIX_NOPS
 	int
+	default 3 if DYNAMIC_FTRACE_WITH_CALL_OPS && CFI
 	default 2 if DYNAMIC_FTRACE_WITH_CALL_OPS
 	default 0
 
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 6b005c8fef706..653c1ed6ef38f 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -142,7 +142,17 @@ CHECKFLAGS	+= -D__aarch64__
 
 ifeq ($(CONFIG_DYNAMIC_FTRACE_WITH_CALL_OPS),y)
   KBUILD_CPPFLAGS += -DCC_USING_PATCHABLE_FUNCTION_ENTRY
-  CC_FLAGS_FTRACE := -fpatchable-function-entry=4,2
+  prefix_nops := $(CONFIG_ARM64_FUNCTION_PREFIX_NOPS)
+  CC_FLAGS_FTRACE := -fpatchable-function-entry=$(shell expr $(prefix_nops) + 2),$(prefix_nops)
+  ifeq ($(CONFIG_CFI),y)
+    # Every object gets the prefix NOPs and no ftrace entry by default.
+    # CC_FLAGS_FTRACE comes later for traced objects and overrides the
+    # -fpatchable-function-entry.
+    CFI_PREFIX_CFLAGS := -fpatchable-function-entry=$(prefix_nops),$(prefix_nops),.discard.patchable_function_entries \
+			 -DCC_USING_PATCHABLE_FUNCTION_PREFIX=$(prefix_nops)
+    KBUILD_CFLAGS += $(CFI_PREFIX_CFLAGS)
+    export CFI_PREFIX_CFLAGS
+  endif
 else ifeq ($(CONFIG_DYNAMIC_FTRACE_WITH_ARGS),y)
   KBUILD_CPPFLAGS += -DCC_USING_PATCHABLE_FUNCTION_ENTRY
   CC_FLAGS_FTRACE := -fpatchable-function-entry=2
diff --git a/arch/arm64/kernel/pi/Makefile b/arch/arm64/kernel/pi/Makefile
index be92d73c25b21..df9f78dd9daec 100644
--- a/arch/arm64/kernel/pi/Makefile
+++ b/arch/arm64/kernel/pi/Makefile
@@ -18,6 +18,8 @@ CFLAGS_map_range.o += -mstrict-align
 KBUILD_CFLAGS	:= $(filter-out $(CC_FLAGS_SCS), $(KBUILD_CFLAGS))
 # disable LTO
 KBUILD_CFLAGS	:= $(filter-out $(CC_FLAGS_LTO), $(KBUILD_CFLAGS))
+# the discard section holds absolute relocations, which relacheck rejects
+KBUILD_CFLAGS	:= $(filter-out $(CFI_PREFIX_CFLAGS), $(KBUILD_CFLAGS))
 
 hostprogs	:= relacheck
 
diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
index 7dec05dd33b70..44f4a6622ccd2 100644
--- a/arch/arm64/kernel/vdso/Makefile
+++ b/arch/arm64/kernel/vdso/Makefile
@@ -38,7 +38,7 @@ ccflags-y += -DDISABLE_BRANCH_PROFILING -DBUILD_VDSO
 CC_FLAGS_REMOVE_VDSO := $(CC_FLAGS_FTRACE) -Os $(CC_FLAGS_SCS) \
 			$(RANDSTRUCT_CFLAGS) $(KSTACK_ERASE_CFLAGS) \
 			$(GCC_PLUGINS_CFLAGS) \
-			$(CC_FLAGS_LTO) $(CC_FLAGS_CFI) \
+			$(CC_FLAGS_LTO) $(CC_FLAGS_CFI) $(CFI_PREFIX_CFLAGS) \
 			-Wmissing-prototypes -Wmissing-declarations
 
 CC_FLAGS_ADD_VDSO := -O2 -mcmodel=tiny -fasynchronous-unwind-tables
diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile
index 77a2b2d74f3f6..9fb4e23da4535 100644
--- a/drivers/firmware/efi/libstub/Makefile
+++ b/drivers/firmware/efi/libstub/Makefile
@@ -53,7 +53,7 @@ KBUILD_CFLAGS := $(filter-out $(RANDSTRUCT_CFLAGS), $(KBUILD_CFLAGS))
 # remove SCS flags from all objects in this directory
 KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_SCS), $(KBUILD_CFLAGS))
 # disable CFI
-KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_CFI), $(KBUILD_CFLAGS))
+KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_CFI) $(CFI_PREFIX_CFLAGS), $(KBUILD_CFLAGS))
 # disable LTO
 KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_LTO), $(KBUILD_CFLAGS))
 

-- 
2.52.0

[PATCH RFC 7/8] arm64: ftrace: Use five prefix NOPs on ThinLTO kernels with BTI

[Jose Fernandez (Anthropic)]

Default CONFIG_ARM64_FUNCTION_PREFIX_NOPS to 5 on ThinLTO kernels with
BTI. Many functions there have a landing pad but no hash. That moves the
entry point 4 bytes. The 8-byte slot lands on the landing pad and ftrace
refuses the site. With five NOPs every function has a slot.

Commit d3359af21fc9e ("arm64: bti: Disable in-kernel BTI with recent
versions of Clang") keeps BTI off on clang 21 or later, so this default
does not trigger on a mainline build today. Josh Poimboeuf's series to
bring it back is under review [1].

With five NOPs the series costs 7.4% of .text on a defconfig-based
ThinLTO kernel with clang 21.1.4:

  Kernel                 || .text bytes | Delta       | Delta
  =======================++=============+=============+=======
  ThinLTO kCFI, base     ||  21,466,256 |             |
  ThinLTO kCFI, M = 5    ||  23,047,312 |  +1,581,056 | +7.4%
  -----------------------++-------------+-------------+-------

ftracetest results match with and without this series. The three
failures on a ThinLTO kernel are pre-existing. The BPF trampoline,
struct_ops and kprobe_multi selftests pass with CFI enforcing.

[1] https://lore.kernel.org/all/cover.1786768375.git.jpoimboe@kernel.org/

Reviewed-by: Ben Cressey <ben@cressey.dev>
Reviewed-by: Florent Revest (Anthropic) <florent.revest@linux.dev>
Assisted-by: LLM
Signed-off-by: Jose Fernandez (Anthropic) <jose.fernandez@linux.dev>
---
 arch/arm64/Kconfig | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 6e5fa009c11af..afa566e0fa86c 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -290,6 +290,8 @@ config ARM64_CFI_PERMITS_CALL_OPS
 
 config ARM64_FUNCTION_PREFIX_NOPS
 	int
+	default 5 if DYNAMIC_FTRACE_WITH_CALL_OPS && CFI && \
+		     (LTO_CLANG_THIN || LTO_CLANG_THIN_DIST) && ARM64_BTI_KERNEL
 	default 3 if DYNAMIC_FTRACE_WITH_CALL_OPS && CFI
 	default 2 if DYNAMIC_FTRACE_WITH_CALL_OPS
 	default 0

-- 
2.52.0

[PATCH RFC 8/8] arm64: ftrace: Allow CALL_OPS on kCFI kernels built with Rust

[Jose Fernandez (Anthropic)]

Pass -Zpatchable-function-entry=M,M,.discard.patchable_function_entries
to rustc when CFI and CALL_OPS are both enabled, so Rust functions get
the same prefix NOPs as C functions. The section argument needs
Rust 1.98 [1]. Probe for it, and let ARM64_CFI_PERMITS_CALL_OPS
accept RUST=y when the probe passes.

[1] https://github.com/rust-lang/rust/pull/157445

Suggested-by: Ben Cressey <ben@cressey.dev>
Reviewed-by: Ben Cressey <ben@cressey.dev>
Reviewed-by: Florent Revest (Anthropic) <florent.revest@linux.dev>
Assisted-by: LLM
Signed-off-by: Jose Fernandez (Anthropic) <jose.fernandez@linux.dev>
---
 arch/arm64/Kconfig  | 7 ++++++-
 arch/arm64/Makefile | 3 +++
 rust/Makefile       | 5 ++++-
 3 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index afa566e0fa86c..cad2bdf3456ff 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -282,11 +282,16 @@ config CC_HAS_PATCHABLE_FUNCTION_ENTRY_SECTION
 	# https://github.com/llvm/llvm-project/pull/131230
 	def_bool $(cc-option,-fpatchable-function-entry=1$(comma)1$(comma).discard.patchable_function_entries)
 
+config RUSTC_HAS_PATCHABLE_FUNCTION_ENTRY_SECTION
+	# The section argument needs Rust 1.98.0 or later:
+	# https://github.com/rust-lang/rust/pull/157445
+	def_bool $(rustc-option,-Zpatchable-function-entry=1$(comma)1$(comma).discard.patchable_function_entries)
+
 config ARM64_CFI_PERMITS_CALL_OPS
 	def_bool y
 	depends on CFI
 	depends on CC_HAS_PATCHABLE_FUNCTION_ENTRY_SECTION
-	depends on !RUST
+	depends on !RUST || RUSTC_HAS_PATCHABLE_FUNCTION_ENTRY_SECTION
 
 config ARM64_FUNCTION_PREFIX_NOPS
 	int
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 653c1ed6ef38f..b23430e2b8567 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -152,6 +152,9 @@ ifeq ($(CONFIG_DYNAMIC_FTRACE_WITH_CALL_OPS),y)
 			 -DCC_USING_PATCHABLE_FUNCTION_PREFIX=$(prefix_nops)
     KBUILD_CFLAGS += $(CFI_PREFIX_CFLAGS)
     export CFI_PREFIX_CFLAGS
+    # Rust functions are not patched by ftrace, but they are called indirectly
+    # from C and make indirect calls themselves, so they need the same prefix.
+    KBUILD_RUSTFLAGS += -Zpatchable-function-entry=$(prefix_nops),$(prefix_nops),.discard.patchable_function_entries
   endif
 else ifeq ($(CONFIG_DYNAMIC_FTRACE_WITH_ARGS),y)
   KBUILD_CPPFLAGS += -DCC_USING_PATCHABLE_FUNCTION_ENTRY
diff --git a/rust/Makefile b/rust/Makefile
index da1a7409d9845..0036cefbed76f 100644
--- a/rust/Makefile
+++ b/rust/Makefile
@@ -432,7 +432,10 @@ rusttest-macros: $(src)/macros/lib.rs \
 	+$(call if_changed,rustdoc_test)
 
 ifdef CONFIG_CC_IS_CLANG
-bindgen_c_flags = $(c_flags)
+# bindgen generates no code, and its libclang may predate the section
+# argument of -fpatchable-function-entry that a kCFI kernel with CALL_OPS
+# passes on arm64.
+bindgen_c_flags = $(filter-out -fpatchable-function-entry=%,$(c_flags))
 else
 # bindgen relies on libclang to parse C. Ideally, bindgen would support a GCC
 # plugin backend and/or the Clang driver would be perfectly compatible with GCC.

-- 
2.52.0

Re: [PATCH RFC 1/8] ftrace: Let ftrace_call_adjust() reject a patch site

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

New issues:
- [Medium] Permanent memory leak of ftrace page group when all tracepoints in a module are rejected.

Pre-existing issues:
- [High] Lockless readers can observe uninitialized ftrace pages due to a missing write memory barrier.
--

commit 47d688aa11de52022b8f09bf86665d475e816eb6
Author: Jose Fernandez (Anthropic) <jose.fernandez@linux.dev>
Subject: ftrace: Let ftrace_call_adjust() reject a patch site

This commit addresses an issue where ftrace_process_locs() does not skip a
patch site when ftrace_call_adjust() returns 0. It tests the adjusted address
and skips the site if it is NULL, preventing a bug_on() during ftrace_init_nop
on arm64.

> diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
> index f9d80c7bd9f16..394f82230e0c3 100644
> --- a/kernel/trace/ftrace.c
> +++ b/kernel/trace/ftrace.c
> @@ -7676,6 +7676,10 @@ static int ftrace_process_locs(struct module *mod,
>  		}
>  
>  		addr = ftrace_call_adjust(addr);
> +		if (!addr) {
> +			skipped++;
> +			continue;
> +		}

[Severity: Medium]
If all tracepoints in a module are rejected and skipped here, the allocated
start_pg retains an index of 0. When the module is later unloaded, does this
leave the empty page permanently in the global ftrace_pages list?

During module unload, ftrace_release_mod() checks if the first record's ip
is within the module:

kernel/trace/ftrace.c:ftrace_release_mod() {
    ...
	for (pg = ftrace_pages_start; pg; pg = pg->next) {
		rec = &pg->records[0];
		if (within_module(rec->ip, mod)) {
			/*
			 * pg->index could be zero if we only allocated
			 * a page but did not store any records
			 */
            ...
}

Since the record ip is 0, within_module(0, mod) fails, and the page group
would not be freed. Can this regression cause a memory leak of the ftrace
page group for each incompatible module load?

[Severity: High]
This isn't a bug introduced by this patch, but while reviewing this function,
I noticed that the new ftrace page is published to the lockless list without
a write memory barrier earlier in ftrace_process_locs():

kernel/trace/ftrace.c:ftrace_process_locs() {
    ...
		if (WARN_ON(ftrace_pages->next)) {
			/* Hmm, we have free pages? */
			while (ftrace_pages->next)
				ftrace_pages = ftrace_pages->next;
		}

		ftrace_pages->next = start_pg;
    ...
}

Because the cpu can reorder the kzalloc initialization of start_pg after its
publication to the ftrace_pages list, lockless readers like
ftrace_location_range() iterating the list under rcu_read_lock() might read
uninitialized data:

kernel/trace/ftrace.c:lookup_rec() {
    ...
	for (pg = ftrace_pages_start; pg; pg = pg->next) {
		if (pg->index == 0 ||
		    rec->ip < pg->records[0].ip ||
		    rec->ip > pg->records[pg->index - 1].ip)
			continue;
    ...
}

Could this lead to a panic or memory corruption due to an out-of-bounds
access or a garbage pointer dereference in bsearch() if the reader observes an
uninitialized pg->index?

>  
>  		end_offset = (pg->index+1) * sizeof(pg->records[0]);
>  		if (end_offset > PAGE_SIZE << pg->order) {

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260904-b4-arm64-callops-kcfi-v1-0-ce6687739b0c@linux.dev?part=1

Re: [PATCH RFC 5/8] arm64: cfi: Use CONFIG_ARM64_FUNCTION_PREFIX_NOPS for the type hash offset

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [Medium] bpf_prog_fill_jited_linfo() calculates incorrect line info addresses because ctx.offset includes the CFI prefix size.
--

commit 1909b8cf9ec25c69a7f49627fb1a30b3d8e545d4
Author: Jose Fernandez (Anthropic) <jose.fernandez@linux.dev>
arm64: cfi: Use CONFIG_ARM64_FUNCTION_PREFIX_NOPS for the type hash offset

This patch modifies the CFI prefix to use CONFIG_ARM64_FUNCTION_PREFIX_NOPS
instead of hardcoding four bytes. It adjusts cfi_get_offset(),
SYM_TYPED_FUNC_START, and the BPF JIT to account for the dynamic number of
NOPs required by kCFI kernels with CALL_OPS or ThinLTO and BTI.

> diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
> index c18e005a41dbe..7ad14d9847138 100644
> --- a/arch/arm64/net/bpf_jit_comp.c
> +++ b/arch/arm64/net/bpf_jit_comp.c
[ ... ]
> @@ -185,8 +185,12 @@ static inline void emit_bti(u32 insn, struct jit_ctx *ctx)
>  
>  static inline void emit_kcfi(u32 hash, struct jit_ctx *ctx)
>  {
> -	if (IS_ENABLED(CONFIG_CFI))
> -		emit_u32_data(hash, ctx);
> +	if (!IS_ENABLED(CONFIG_CFI))
> +		return;
> +
> +	emit_u32_data(hash, ctx);
> +	for (int i = 0; i < CONFIG_ARM64_FUNCTION_PREFIX_NOPS; i++)
> +		emit(A64_NOP, ctx);
>  }

[Severity: Medium]
This is a pre-existing issue, but does expanding the CFI prefix length
cause bpf_prog_fill_jited_linfo() to calculate incorrect line info addresses
by adding the CFI prefix twice?

When loading any BPF program with line info on an arm64 kernel built with
kCFI, ctx.offset measures from the start of the JIT buffer, which includes
the CFI prefix emitted here.

However, in bpf_int_jit_compile() in arch/arm64/net/bpf_jit_comp.c, the
offsets are passed to bpf_prog_fill_jited_linfo():

arch/arm64/net/bpf_jit_comp.c:bpf_int_jit_compile() {
    ...
    for (i = 0; i <= prog->len; i++)
        ctx.offset[i] *= AARCH64_INSN_SIZE;
    bpf_prog_fill_jited_linfo(prog, ctx.offset + 1);
    ...
}

Since bpf_prog_fill_jited_linfo() adds these offsets to prog->bpf_func,
which is already shifted past the CFI prefix by cfi_get_offset(), the CFI
prefix length ends up being added twice. The bug existed prior to this patch
(off by 4 bytes) but this patch exacerbates it to 16 or 24 bytes depending
on the number of NOPs.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260904-b4-arm64-callops-kcfi-v1-0-ce6687739b0c@linux.dev?part=5

Re: [PATCH RFC 2/8] scripts/sorttable: Make the arm64 before_func offset configurable

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 1337 bytes --]

> diff --git a/scripts/Makefile b/scripts/Makefile
> index 3434a82a119f0..366c6a35141a6 100644
> --- a/scripts/Makefile
> +++ b/scripts/Makefile
> @@ -51,6 +51,9 @@ endif
>
>  ifdef CONFIG_BUILDTIME_MCOUNT_SORT
>  HOSTCFLAGS_sorttable.o += -DMCOUNT_SORT_ENABLED
> +ifneq ($(filter-out 0,$(CONFIG_ARM64_FUNCTION_PREFIX_NOPS)),)
> +HOSTCFLAGS_sorttable.o += -DFUNCTION_PREFIX_BYTES=$(shell expr 4 \* $(CONFIG_ARM64_FUNCTION_PREFIX_NOPS))
> +endif
>  endif

This isn't a bug, but would it be worth letting a configured 0 through
here (or noting why it deliberately keeps the historical 8-byte
tolerance), so the block doesn't read as honouring a value it actually
filters out? As written, when CONFIG_ARM64_FUNCTION_PREFIX_NOPS is set
to 0, the guard treats it the same as the symbol being absent, so both
fall back to sorttable.c's built-in 8. This is behaviour-preserving
today and avoids passing an empty value to expr, but once the symbol
lands with a default of 0 for the non-CALL_OPS case, the one configured
value that differs from the legacy constant is the one the guard
discards.

[ ... ]


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/33826319138

Re: [PATCH RFC 6/8] arm64: ftrace: Support CALL_OPS on kernels built with kCFI

[Miguel Ojeda]

On Fri, Sep 4, 2026 at 3:20 AM Jose Fernandez (Anthropic)
<jose.fernandez@linux.dev> wrote:
>
> Gate CALL_OPS under CFI on ARM64_CFI_PERMITS_CALL_OPS, which needs
> clang 21 or newer and RUST off. rustc has its own flag for the prefix
> NOPs and this patch does not pass it, so Rust functions would have no
> prefix and a different hash offset.

Why doesn't the patch pass the flag to `rustc`?

(Looking at later patches) Ah, is it because you defer it to the last patch?

Thanks!

Cheers,
Miguel

Re: [PATCH RFC 1/8] ftrace: Let ftrace_call_adjust() reject a patch site

[Steven Rostedt]

On Fri, 04 Sep 2026 01:18:33 +0000
"Jose Fernandez (Anthropic)" <jose.fernandez@linux.dev> wrote:

> diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
> index f9d80c7bd9f16..394f82230e0c3 100644
> --- a/kernel/trace/ftrace.c
> +++ b/kernel/trace/ftrace.c
> @@ -7676,6 +7676,10 @@ static int ftrace_process_locs(struct module *mod,
>  		}
>  
>  		addr = ftrace_call_adjust(addr);
> +		if (!addr) {
> +			skipped++;
> +			continue;
> +		}
>  

The above could use a comment to why addr could be zero (I didn't even
realize some architectures do that).

	/* Some architectures may return zero for mis-aligned addresses */

-- Steve