Re: [BUG] bpf: verifier: False warning for helpers in speculative branches

[Eduard Zingerman <eddyz87@gmail.com>]

On Sat, 2025-12-27 at 16:01 +0100, Luis Gerhorst wrote:
> Yinhao Hu <dddddd@hust.edu.cn> writes:
> 
> > Our fuzzer discovered a verifier bug in the BPF subsystem. The warning
> > triggers when Spectre mitigation is enabled and a write-performing
> > helper call is placed in a speculatively-executed branch.
> > 
> > The BPF verifier assumes `insn_aux->nospec_result` is only set for
> > direct memory writes (e.g., `*(u32*)(r1+off) = r2`). However, it fails
> > to account for helper calls (e.g., `bpf_skb_load_bytes_relative`) that
> > perform writes to stack memory.
> > 
> > The problem: `BPF_CALL` instructions have `BPF_CLASS(insn->code) ==
> > BPF_JMP`, which triggers the warning check. The code comment states:
> > 
> > ```c
> > /* "This can currently never happen because nospec_result is only
> >  *  used for the write-ops `*(size*)(dst_reg+off)=src_reg|imm32`
> >  *  which must never skip the following insn."
> >  */
> > ```
> > 
> > However, helper calls break this assumption:
> > - Helpers like `bpf_skb_load_bytes_relative` write to stack memory
> > - `check_helper_call()` loops through `meta.access_size`, calling
> > `check_mem_access(..., BPF_WRITE)`
> > - `check_stack_write()` sets `insn_aux->nospec_result = 1`
> > - Since `BPF_CALL` is encoded as `BPF_JMP | BPF_CALL`, the warning fires
> 
> Thank you very much for the report. I think we just have to make the
> check more precise as this is a false-positive warning. The nospec after
> the helper call should still have the desired effect.
> 
> I can check the call graph to make sure there are no other ways
> check_stack_write() can be called and send a patch in the new year.

Fixing it this way makes sense to me.
Beside matching exact opcode, another option might be checking the
result of the bpf_insn_successors() call to see if there are multiple
successors. (But please measure on some selftest if that is not too
expensive).

> > Reported-by: Yinhao Hu <dddddd@hust.edu.cn>
> > Reported-by: Kaiyan Mei <M202472210@hust.edu.cn>
> > Reviewed-by: Dongliang Mu <dzm91@hust.edu.cn>
> > 
> > ### Trigger Condition
> > 
> > The warning occurs when both flags are set:
> > 1. `state->speculative = 1` — Verifier processes a branch that won't
> > execute (marked during `check_cond_jmp_op`)
> > 2. `insn_aux->nospec_result = 1` — A helper performs stack writes (set
> > during `check_helper_call`)
> > 
> > ### Execution Flow
> > 
> > ```
> > 1. Drop capabilities → Enable Spectre mitigation
> > 2. Load BPF program
> >    └─> do_check()
> >        ├─> check_cond_jmp_op() → Marks dead branch as speculative
> >        │   └─> push_stack(..., speculative=true)
> >        ├─> pop_stack() → state->speculative = 1
> >        ├─> check_helper_call() → Processes helper in dead branch
> >        │   └─> check_mem_access(..., BPF_WRITE)
> >        │       └─> insn_aux->nospec_result = 1
> >        └─> Checks: state->speculative && insn_aux->nospec_result
> >            └─> BPF_CLASS(insn->code) == BPF_JMP → WARNING
> > ```
> > 
> > ### Warning
> > 
> > ```yaml
> > ------------[ cut here ]------------
> > verifier bug: speculation barrier after jump instruction may not have
> > the desired effect (BPF_CLASS(insn->code) == BPF_JMP ||
> > BPF_CLASS(insn->code) == BPF_JMP32)
> > WARNING: CPU: 0 PID: 9956 at kernel/bpf/verifier.c:20536 do_check
> > kernel/bpf/verifier.c:20536 [inline]
> > WARNING: CPU: 0 PID: 9956 at kernel/bpf/verifier.c:20536
> > do_check_common+0xac7b/0xb200 kernel/bpf/verifier.c:23784
> > Modules linked in:
> > CPU: 0 UID: 0 PID: 9956 Comm: syz-executor206 Not tainted
> > 6.18.0-rc4-g93ce3bee311d #3 PREEMPT(full)
> > Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1
> > 04/01/2014
> > RIP: 0010:do_check kernel/bpf/verifier.c:20536 [inline]
> > RIP: 0010:do_check_common+0xac7b/0xb200 kernel/bpf/verifier.c:23784
> > Code: 00 e9 2b 84 ff ff e8 f4 ea 4c 00 e9 31 83 ff ff e8 6a 47 e0 ff c6
> > 05 b3 8d 6c 0f 01 90 48 c7 c7 c0 ab 76 8b e8 a6 64 9f ff 90 <0f> 0b 90
> > 90 e9 96 83 ff ff e8 c7 ea 4c 00 e9 29 89 ff ff e8 1d eb
> > RSP: 0018:ffa00000080df5e0 EFLAGS: 00010282
> > RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffffff817acafe
> > RDX: ff11000108f0ca00 RSI: ffffffff817acb0b RDI: 0000000000000001
> > RBP: 0000000000000017 R08: 0000000000000001 R09: ffe21c00142c4841
> > R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
> > R13: 0000000000000000 R14: ff11000024320000 R15: dffffc0000000000
> > FS:  000055558abb53c0(0000) GS:ff1100010ccd0000(0000) knlGS:0000000000000000
> > CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > CR2: 0000200000000040 CR3: 0000000028fc5000 CR4: 0000000000753ef0
> > PKRU: 55555554
> > Call Trace:
> >  <TASK>
> >  do_check_main kernel/bpf/verifier.c:23867 [inline]
> >  bpf_check+0x9382/0xb930 kernel/bpf/verifier.c:25174
> >  bpf_prog_load+0x17a6/0x2960 kernel/bpf/syscall.c:3095
> >  __sys_bpf+0x1971/0x5390 kernel/bpf/syscall.c:6171
> >  __do_sys_bpf kernel/bpf/syscall.c:6281 [inline]
> >  __se_sys_bpf kernel/bpf/syscall.c:6279 [inline]
> >  __x64_sys_bpf+0x7d/0xc0 kernel/bpf/syscall.c:6279
> >  do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
> >  do_syscall_64+0xcb/0xfa0 arch/x86/entry/syscall_64.c:94
> >  entry_SYSCALL_64_after_hwframe+0x77/0x7f
> > RIP: 0033:0x7f13824ac64d
> > Code: 28 c3 e8 46 1e 00 00 66 0f 1f 44 00 00 f3 0f 1e fa 48 89 f8 48 89
> > f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01
> > f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
> > RSP: 002b:00007ffc6d73d488 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
> > RAX: ffffffffffffffda RBX: 00007ffc6d73d698 RCX: 00007f13824ac64d
> > RDX: 0000000000000094 RSI: 0000200000000a00 RDI: 0000000000000005
> > RBP: 0000000000000001 R08: 0000000000000000 R09: 0000000000000000
> > R10: 0000000000000002 R11: 0000000000000246 R12: 0000000000000001
> > R13: 00007ffc6d73d688 R14: 00007f1382529530 R15: 0000000000000001
> >  </TASK>
> > ```
> > 
> > ### Proof of Concept
> > 
> > Tested on:
> > - Linux next 6.19.0-rc1-next-20251219 (commit
> > cc3aa43b44bdb43dfbac0fcb51c56594a11338a8)
> > - bpf next (commit ac1c5bc7c4c7e20e2070e6eaa673fc3e11619dbb)
> > 
> > ```c
> > #define _GNU_SOURCE
> > #include <linux/bpf.h>
> > #include <linux/filter.h>
> > #include <stdio.h>
> > #include <string.h>
> > #include <sys/syscall.h>
> > #include <unistd.h>
> > #include <stdint.h>
> > 
> > int main(void)
> > {
> >     /* Setup memory for capset (optional for most systems) */
> >     syscall(__NR_mmap, 0x200000000000ul, 0x1000000ul, 7, 0x32, -1, 0);
> > 
> >     /* Drop capabilities to enable Spectre mitigation */
> >     *(uint32_t*)0x200000000040 = 0x20080522;  /*
> > _LINUX_CAPABILITY_VERSION_3 */
> >     *(uint32_t*)0x200000000044 = 0;
> >     memset((void*)0x200000000080, 0, 24);
> >     syscall(__NR_capset, 0x200000000040ul, 0x200000000080ul);
> > 
> >     /* BPF program: write-performing helper in dead branch */
> >     struct bpf_insn prog[] = {
> >         /* r0 = 0 */
> >         { .code = BPF_ALU64 | BPF_MOV | BPF_K, .dst_reg = BPF_REG_0,
> > .imm = 0,},
> >         /* if r0 != 1 goto +6 */
> >         {.code = BPF_JMP | BPF_JNE | BPF_K, .dst_reg = BPF_REG_0, .imm =
> > 1, .off = 6,},
> >         /* R2 = offset */
> >         {.code = BPF_ALU64 | BPF_MOV | BPF_K, .dst_reg = BPF_REG_2, .imm
> > = 0,},
> >         /* R3 = R10 - 16 */
> >         {.code = BPF_ALU64 | BPF_MOV | BPF_X, .dst_reg = BPF_REG_3,
> > .src_reg = BPF_REG_10,},
> >         {.code = BPF_ALU64 | BPF_ADD | BPF_K, .dst_reg = BPF_REG_3, .imm
> > = -16,},
> >         /* R4 = 4 */
> >         {.code = BPF_ALU64 | BPF_MOV | BPF_K, .dst_reg = BPF_REG_4, .imm
> > = 4,},
> >         /* R5 = flags */
> >         {.code = BPF_ALU64 | BPF_MOV | BPF_K, .dst_reg = BPF_REG_5, .imm
> > = 0,},
> >         /* call helper 68 */
> >         {.code = BPF_JMP | BPF_CALL, .imm =
> > BPF_FUNC_skb_load_bytes_relative,},
> >         /* exit */
> >         {.code = BPF_JMP | BPF_EXIT,},
> >     };
> > 
> >     char log_buf[65536] = {0};
> >     union bpf_attr attr = {
> >         .prog_type = BPF_PROG_TYPE_SOCKET_FILTER,
> >         .insns = (uint64_t)prog,
> >         .insn_cnt = sizeof(prog) / sizeof(prog[0]),
> >         .license = (uint64_t)"GPL",
> >         .log_buf = (uint64_t)log_buf,
> >         .log_size = sizeof(log_buf),
> >         .log_level = 2,
> >     };
> > 
> >     int fd = syscall(__NR_bpf, BPF_PROG_LOAD, &attr, sizeof(attr));
> >     if (fd < 0) {
> >         perror("bpf");
> >         fprintf(stderr, "\nVerifier log:\n%s\n", log_buf);
> >         return 1;
> >     }
> > 
> >     printf("Loaded (fd=%d) — Check dmesg for WARNING\n", fd);
> >     close(fd);
> >     return 0;
> > }
> > ```
> > 
> > [2. text/plain; config-linux-next]...