[PATCH bpf-next 0/1] bpf, arm64: fix fp initialization for exception boundary

[PATCH bpf-next 0/1] bpf, arm64: fix fp initialization for exception boundary

[Puranjay Mohan]

In the ARM64 BPF JIT when prog->aux->exception_boundary is set for a BPF
program, find_used_callee_regs() is not called because for a program acting
as exception boundary, all callee saved registers are saved.
find_used_callee_regs() sets `ctx->fp_used = true;` when it sees FP being
used in any of the instructions.
For programs acting as exception boundary, ctx->fp_used always remains
false and therefore, BPF frame pointer is never set-up for such programs in
the prologue.

This can cause crashes like:

With the following BPF program loaded and attached:

    static __noinline int static_func(u64 i)
    {
            bpf_throw(0);
            return i;
    }

    SEC("fentry/do_unlinkat")
    int BPF_PROG(do_unlinkat, int dfd, struct filename *name)
    {
            volatile u64 a[2] = {0};

            a[1] = __sync_fetch_and_add(&a[0], 1);

            static_func(23);
            return 0;
    }

Triggering it causes a page fault because the FP register is not
initialised.

[root@localhost ~]# touch test
[root@localhost ~]# rm test
 Unable to handle kernel paging request at virtual address fffffffffffffff0
 Mem abort info:
   ESR = 0x0000000096000006
   EC = 0x25: DABT (current EL), IL = 32 bits
   SET = 0, FnV = 0
   EA = 0, S1PTW = 0
   FSC = 0x06: level 2 translation fault
 Data abort info:
   ISV = 0, ISS = 0x00000006, ISS2 = 0x00000000
   CM = 0, WnR = 0, TnD = 0, TagAccess = 0
   GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
 swapper pgtable: 4k pages, 48-bit VAs, pgdp=0000000043783000
 [fffffffffffffff0] pgd=0000000000000000, p4d=00000000450a0403, pud=00000000450a1403, pmd=0000000000000000
 Internal error: Oops: 0000000096000006 [#1]  SMP
 Modules linked in:
 CPU: 12 UID: 0 PID: 487 Comm: rm Not tainted 6.16.0-rc6-00212-g7abc678e3084 #7 PREEMPT
 Hardware name: linux,dummy-virt (DT)
 pstate: 60400005 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
 pc : bpf_prog_5148f2d6554f3ab3_do_unlinkat+0x48/0x90
 lr : bpf_trampoline_6442562433+0x68/0x168
 sp : ffff80008c8b3d10
 x29: ffff80008c8b3d80 x28: ffff0000d66d0000 x27: 0000000000000000
 x26: ffff80008c8b3d70 x25: 0000000000000000 x24: 0000000000000000
 x23: 0000000060001000 x22: 0000ffffaea95b0c x21: 00000000ffffffff
 x20: 0000000000000001 x19: ffff80008c1bd000 x18: 0000000000000000
 x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000
 x14: 0000000000000000 x13: 0000000000020000 x12: 0000000000000015
 x11: 0000000000000000 x10: fffffffffffffff0 x9 : ffff8000871d65e8
 x8 : ffff800084eb1d30 x7 : 0000000000000000 x6 : 0000000000000001
 x5 : 00000000c3affd79 x4 : 000000001055937d x3 : ffff801809627000
 x2 : ffff0000d66d0000 x1 : 0000000100000000 x0 : 0000000000000001
 Call trace:
  bpf_prog_5148f2d6554f3ab3_do_unlinkat+0x48/0x90 (P)
  bpf_trampoline_6442562433+0x68/0x168
  do_unlinkat+0x8/0x290
  __arm64_sys_unlinkat+0x44/0x90
  invoke_syscall+0x50/0x120
  el0_svc_common.constprop.0+0xc8/0xf0
  do_el0_svc+0x24/0x38
  el0_svc+0x48/0xf0
  el0t_64_sync_handler+0xc8/0xd0
  el0t_64_sync+0x198/0x1a0
 Code: f90007e0 f90003e0 d2800020 d100432a (f8e00140)
 ---[ end trace 0000000000000000 ]---
 Kernel panic - not syncing: Oops: Fatal exception
 SMP: stopping secondary CPUs
 Kernel Offset: disabled
 CPU features: 0x2000,000081c0,02004ca1,04407a0b
 Memory Limit: none
 ---[ end Kernel panic - not syncing: Oops: Fatal exception ]---

Please take it into the bpf tree if needed, I sent it to bpf-next as
this bug is multiple months old.

Puranjay Mohan (1):
  bpf, arm64: fix fp initialization for exception boundary

 arch/arm64/net/bpf_jit_comp.c | 1 +
 1 file changed, 1 insertion(+)

-- 
2.47.1

[PATCH bpf-next 1/1] bpf, arm64: fix fp initialization for exception boundary

[Puranjay Mohan]

In the ARM64 BPF JIT when prog->aux->exception_boundary is set for a BPF
program, find_used_callee_regs() is not called because for a program
acting as exception boundary, all callee saved registers are saved.
find_used_callee_regs() sets `ctx->fp_used = true;` when it sees FP
being used in any of the instructions.

For programs acting as exception boundary, ctx->fp_used remains false
even if frame pointer is used by the program and therefore, FP is not
set-up for such programs in the prologue. This can cause the kernel to
crash due to a pagefault.

Fix it by setting ctx->fp_used = true for exception boundary programs as
fp is always saved in such programs.

Fixes: 5d4fa9ec5643 ("bpf, arm64: Avoid blindly saving/restoring all callee-saved registers")
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
---
 arch/arm64/net/bpf_jit_comp.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index 89b1b8c248c62..97ab651c0bd5d 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -412,6 +412,7 @@ static void push_callee_regs(struct jit_ctx *ctx)
 		emit(A64_PUSH(A64_R(23), A64_R(24), A64_SP), ctx);
 		emit(A64_PUSH(A64_R(25), A64_R(26), A64_SP), ctx);
 		emit(A64_PUSH(A64_R(27), A64_R(28), A64_SP), ctx);
+		ctx->fp_used = true;
 	} else {
 		find_used_callee_regs(ctx);
 		for (i = 0; i + 1 < ctx->nr_used_callee_reg; i += 2) {
-- 
2.47.1

Re: [PATCH bpf-next 1/1] bpf, arm64: fix fp initialization for exception boundary

[Xu Kuohai]

On 7/22/2025 9:34 PM, Puranjay Mohan wrote:
> In the ARM64 BPF JIT when prog->aux->exception_boundary is set for a BPF
> program, find_used_callee_regs() is not called because for a program
> acting as exception boundary, all callee saved registers are saved.
> find_used_callee_regs() sets `ctx->fp_used = true;` when it sees FP
> being used in any of the instructions.
> 
> For programs acting as exception boundary, ctx->fp_used remains false
> even if frame pointer is used by the program and therefore, FP is not
> set-up for such programs in the prologue. This can cause the kernel to
> crash due to a pagefault.
> 
> Fix it by setting ctx->fp_used = true for exception boundary programs as
> fp is always saved in such programs.
> 
> Fixes: 5d4fa9ec5643 ("bpf, arm64: Avoid blindly saving/restoring all callee-saved registers")
> Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
> ---
>   arch/arm64/net/bpf_jit_comp.c | 1 +
>   1 file changed, 1 insertion(+)
> 
> diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
> index 89b1b8c248c62..97ab651c0bd5d 100644
> --- a/arch/arm64/net/bpf_jit_comp.c
> +++ b/arch/arm64/net/bpf_jit_comp.c
> @@ -412,6 +412,7 @@ static void push_callee_regs(struct jit_ctx *ctx)
>   		emit(A64_PUSH(A64_R(23), A64_R(24), A64_SP), ctx);
>   		emit(A64_PUSH(A64_R(25), A64_R(26), A64_SP), ctx);
>   		emit(A64_PUSH(A64_R(27), A64_R(28), A64_SP), ctx);
> +		ctx->fp_used = true;
>   	} else {
>   		find_used_callee_regs(ctx);
>   		for (i = 0; i + 1 < ctx->nr_used_callee_reg; i += 2) {

Acked-by: Xu Kuohai <xukuohai@huawei.com>

Re: [PATCH bpf-next 0/1] bpf, arm64: fix fp initialization for exception boundary

[patchwork-bot+netdevbpf]

Hello:

This patch was applied to bpf/bpf-next.git (master)
by Daniel Borkmann <daniel@iogearbox.net>:

On Tue, 22 Jul 2025 13:34:08 +0000 you wrote:
> In the ARM64 BPF JIT when prog->aux->exception_boundary is set for a BPF
> program, find_used_callee_regs() is not called because for a program acting
> as exception boundary, all callee saved registers are saved.
> find_used_callee_regs() sets `ctx->fp_used = true;` when it sees FP being
> used in any of the instructions.
> For programs acting as exception boundary, ctx->fp_used always remains
> false and therefore, BPF frame pointer is never set-up for such programs in
> the prologue.
> 
> [...]

Here is the summary with links:
  - [bpf-next,1/1] bpf, arm64: fix fp initialization for exception boundary
    https://git.kernel.org/bpf/bpf-next/c/b114fcee766d

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html