Re: [BUG] bpf: verifier: False warning for helpers in speculative branches

[Luis Gerhorst <luis.gerhorst@fau.de>]

Yinhao Hu <dddddd@hust.edu.cn> writes:

> Our fuzzer discovered a verifier bug in the BPF subsystem. The warning
> triggers when Spectre mitigation is enabled and a write-performing
> helper call is placed in a speculatively-executed branch.
>
> The BPF verifier assumes `insn_aux->nospec_result` is only set for
> direct memory writes (e.g., `*(u32*)(r1+off) = r2`). However, it fails
> to account for helper calls (e.g., `bpf_skb_load_bytes_relative`) that
> perform writes to stack memory.
>
> The problem: `BPF_CALL` instructions have `BPF_CLASS(insn->code) ==
> BPF_JMP`, which triggers the warning check. The code comment states:
>
> ```c
> /* "This can currently never happen because nospec_result is only
>  *  used for the write-ops `*(size*)(dst_reg+off)=src_reg|imm32`
>  *  which must never skip the following insn."
>  */
> ```
>
> However, helper calls break this assumption:
> - Helpers like `bpf_skb_load_bytes_relative` write to stack memory
> - `check_helper_call()` loops through `meta.access_size`, calling
> `check_mem_access(..., BPF_WRITE)`
> - `check_stack_write()` sets `insn_aux->nospec_result = 1`
> - Since `BPF_CALL` is encoded as `BPF_JMP | BPF_CALL`, the warning fires

Thank you very much for the report. I think we just have to make the
check more precise as this is a false-positive warning. The nospec after
the helper call should still have the desired effect.

I can check the call graph to make sure there are no other ways
check_stack_write() can be called and send a patch in the new year.

> Reported-by: Yinhao Hu <dddddd@hust.edu.cn>
> Reported-by: Kaiyan Mei <M202472210@hust.edu.cn>
> Reviewed-by: Dongliang Mu <dzm91@hust.edu.cn>
>
> ### Trigger Condition
>
> The warning occurs when both flags are set:
> 1. `state->speculative = 1` — Verifier processes a branch that won't
> execute (marked during `check_cond_jmp_op`)
> 2. `insn_aux->nospec_result = 1` — A helper performs stack writes (set
> during `check_helper_call`)
>
> ### Execution Flow
>
> ```
> 1. Drop capabilities → Enable Spectre mitigation
> 2. Load BPF program
>    └─> do_check()
>        ├─> check_cond_jmp_op() → Marks dead branch as speculative
>        │   └─> push_stack(..., speculative=true)
>        ├─> pop_stack() → state->speculative = 1
>        ├─> check_helper_call() → Processes helper in dead branch
>        │   └─> check_mem_access(..., BPF_WRITE)
>        │       └─> insn_aux->nospec_result = 1
>        └─> Checks: state->speculative && insn_aux->nospec_result
>            └─> BPF_CLASS(insn->code) == BPF_JMP → WARNING
> ```
>
> ### Warning
>
> ```yaml
> ------------[ cut here ]------------
> verifier bug: speculation barrier after jump instruction may not have
> the desired effect (BPF_CLASS(insn->code) == BPF_JMP ||
> BPF_CLASS(insn->code) == BPF_JMP32)
> WARNING: CPU: 0 PID: 9956 at kernel/bpf/verifier.c:20536 do_check
> kernel/bpf/verifier.c:20536 [inline]
> WARNING: CPU: 0 PID: 9956 at kernel/bpf/verifier.c:20536
> do_check_common+0xac7b/0xb200 kernel/bpf/verifier.c:23784
> Modules linked in:
> CPU: 0 UID: 0 PID: 9956 Comm: syz-executor206 Not tainted
> 6.18.0-rc4-g93ce3bee311d #3 PREEMPT(full)
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1
> 04/01/2014
> RIP: 0010:do_check kernel/bpf/verifier.c:20536 [inline]
> RIP: 0010:do_check_common+0xac7b/0xb200 kernel/bpf/verifier.c:23784
> Code: 00 e9 2b 84 ff ff e8 f4 ea 4c 00 e9 31 83 ff ff e8 6a 47 e0 ff c6
> 05 b3 8d 6c 0f 01 90 48 c7 c7 c0 ab 76 8b e8 a6 64 9f ff 90 <0f> 0b 90
> 90 e9 96 83 ff ff e8 c7 ea 4c 00 e9 29 89 ff ff e8 1d eb
> RSP: 0018:ffa00000080df5e0 EFLAGS: 00010282
> RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffffff817acafe
> RDX: ff11000108f0ca00 RSI: ffffffff817acb0b RDI: 0000000000000001
> RBP: 0000000000000017 R08: 0000000000000001 R09: ffe21c00142c4841
> R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
> R13: 0000000000000000 R14: ff11000024320000 R15: dffffc0000000000
> FS:  000055558abb53c0(0000) GS:ff1100010ccd0000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 0000200000000040 CR3: 0000000028fc5000 CR4: 0000000000753ef0
> PKRU: 55555554
> Call Trace:
>  <TASK>
>  do_check_main kernel/bpf/verifier.c:23867 [inline]
>  bpf_check+0x9382/0xb930 kernel/bpf/verifier.c:25174
>  bpf_prog_load+0x17a6/0x2960 kernel/bpf/syscall.c:3095
>  __sys_bpf+0x1971/0x5390 kernel/bpf/syscall.c:6171
>  __do_sys_bpf kernel/bpf/syscall.c:6281 [inline]
>  __se_sys_bpf kernel/bpf/syscall.c:6279 [inline]
>  __x64_sys_bpf+0x7d/0xc0 kernel/bpf/syscall.c:6279
>  do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
>  do_syscall_64+0xcb/0xfa0 arch/x86/entry/syscall_64.c:94
>  entry_SYSCALL_64_after_hwframe+0x77/0x7f
> RIP: 0033:0x7f13824ac64d
> Code: 28 c3 e8 46 1e 00 00 66 0f 1f 44 00 00 f3 0f 1e fa 48 89 f8 48 89
> f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01
> f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
> RSP: 002b:00007ffc6d73d488 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
> RAX: ffffffffffffffda RBX: 00007ffc6d73d698 RCX: 00007f13824ac64d
> RDX: 0000000000000094 RSI: 0000200000000a00 RDI: 0000000000000005
> RBP: 0000000000000001 R08: 0000000000000000 R09: 0000000000000000
> R10: 0000000000000002 R11: 0000000000000246 R12: 0000000000000001
> R13: 00007ffc6d73d688 R14: 00007f1382529530 R15: 0000000000000001
>  </TASK>
> ```
>
> ### Proof of Concept
>
> Tested on:
> - Linux next 6.19.0-rc1-next-20251219 (commit
> cc3aa43b44bdb43dfbac0fcb51c56594a11338a8)
> - bpf next (commit ac1c5bc7c4c7e20e2070e6eaa673fc3e11619dbb)
>
> ```c
> #define _GNU_SOURCE
> #include <linux/bpf.h>
> #include <linux/filter.h>
> #include <stdio.h>
> #include <string.h>
> #include <sys/syscall.h>
> #include <unistd.h>
> #include <stdint.h>
>
> int main(void)
> {
>     /* Setup memory for capset (optional for most systems) */
>     syscall(__NR_mmap, 0x200000000000ul, 0x1000000ul, 7, 0x32, -1, 0);
>
>     /* Drop capabilities to enable Spectre mitigation */
>     *(uint32_t*)0x200000000040 = 0x20080522;  /*
> _LINUX_CAPABILITY_VERSION_3 */
>     *(uint32_t*)0x200000000044 = 0;
>     memset((void*)0x200000000080, 0, 24);
>     syscall(__NR_capset, 0x200000000040ul, 0x200000000080ul);
>
>     /* BPF program: write-performing helper in dead branch */
>     struct bpf_insn prog[] = {
>         /* r0 = 0 */
>         { .code = BPF_ALU64 | BPF_MOV | BPF_K, .dst_reg = BPF_REG_0,
> .imm = 0,},
>         /* if r0 != 1 goto +6 */
>         {.code = BPF_JMP | BPF_JNE | BPF_K, .dst_reg = BPF_REG_0, .imm =
> 1, .off = 6,},
>         /* R2 = offset */
>         {.code = BPF_ALU64 | BPF_MOV | BPF_K, .dst_reg = BPF_REG_2, .imm
> = 0,},
>         /* R3 = R10 - 16 */
>         {.code = BPF_ALU64 | BPF_MOV | BPF_X, .dst_reg = BPF_REG_3,
> .src_reg = BPF_REG_10,},
>         {.code = BPF_ALU64 | BPF_ADD | BPF_K, .dst_reg = BPF_REG_3, .imm
> = -16,},
>         /* R4 = 4 */
>         {.code = BPF_ALU64 | BPF_MOV | BPF_K, .dst_reg = BPF_REG_4, .imm
> = 4,},
>         /* R5 = flags */
>         {.code = BPF_ALU64 | BPF_MOV | BPF_K, .dst_reg = BPF_REG_5, .imm
> = 0,},
>         /* call helper 68 */
>         {.code = BPF_JMP | BPF_CALL, .imm =
> BPF_FUNC_skb_load_bytes_relative,},
>         /* exit */
>         {.code = BPF_JMP | BPF_EXIT,},
>     };
>
>     char log_buf[65536] = {0};
>     union bpf_attr attr = {
>         .prog_type = BPF_PROG_TYPE_SOCKET_FILTER,
>         .insns = (uint64_t)prog,
>         .insn_cnt = sizeof(prog) / sizeof(prog[0]),
>         .license = (uint64_t)"GPL",
>         .log_buf = (uint64_t)log_buf,
>         .log_size = sizeof(log_buf),
>         .log_level = 2,
>     };
>
>     int fd = syscall(__NR_bpf, BPF_PROG_LOAD, &attr, sizeof(attr));
>     if (fd < 0) {
>         perror("bpf");
>         fprintf(stderr, "\nVerifier log:\n%s\n", log_buf);
>         return 1;
>     }
>
>     printf("Loaded (fd=%d) — Check dmesg for WARNING\n", fd);
>     close(fd);
>     return 0;
> }
> ```
>
> [2. text/plain; config-linux-next]...