* [PATCH bpf-next v2 1/2] bpf: Preserve scalar zero spills for var-offset stack reads
2026-06-13 11:16 [PATCH bpf-next v2 0/2] bpf: Preserve scalar zero spills for var-offset stack reads Woojin Ji
@ 2026-06-13 11:16 ` Woojin Ji
2026-06-13 11:54 ` bot+bpf-ci
2026-06-13 11:16 ` [PATCH bpf-next v2 2/2] selftests/bpf: Cover var-offset stack reads from zero spills Woojin Ji
2026-06-21 15:29 ` [PATCH bpf-next v2 0/2] bpf: Preserve scalar zero spills for var-offset stack reads Yonghong Song
2 siblings, 1 reply; 5+ messages in thread
From: Woojin Ji @ 2026-06-13 11:16 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Martin KaFai Lau,
Song Liu, Yonghong Song, Jiri Olsa, Emil Tsalapatis,
John Fastabend
Cc: bpf, Woojin Ji
Variable-offset stack reads can read back bytes that belong to a
previously spilled scalar constant zero. Today mark_reg_stack_read()
only treats STACK_ZERO bytes as known zero bytes, so the destination
register becomes unknown for such reads even though every byte in the
read range is known to be zero. This can lead to rejecting otherwise
valid programs once the loaded byte is used as a pointer offset.
This pattern is not limited to hand-written verifier tests. clang
22.1.6 at -O2 and -O3 emits it from a small helper-based BPF C
reproducer; the resulting program is rejected before this change and
accepted afterwards. No deployed-program regression is currently known,
so target bpf-next rather than bpf.
Teach the variable-offset stack read path to also consider STACK_SPILL
bytes backed by a spilled scalar constant zero as zero bytes. When a
zero result depends on such a spill, mark the contributing stack slots
precise before accepting the const-zero result so pruning cannot reuse a
zero-spill state for a later non-zero spill state.
Keep fixed-offset stack reads out of this eager precision marking path,
as the new behaviour is only needed when filling a register from a
variable-offset stack byte range.
Assisted-by: opencode:gpt-5.5
Signed-off-by: Woojin Ji <random6.xyz@gmail.com>
---
include/linux/bpf_verifier.h | 5 ++++
kernel/bpf/verifier.c | 55 ++++++++++++++++++++++++++++++++++++--------
2 files changed, 50 insertions(+), 10 deletions(-)
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index d57b339a8..c2e665044 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -1242,6 +1242,11 @@ static inline void bpf_bt_set_frame_slot(struct backtrack_state *bt, u32 frame,
bt->stack_masks[frame] |= 1ull << slot;
}
+static inline void bpf_bt_set_frame_slot_mask(struct backtrack_state *bt, u32 frame, u64 mask)
+{
+ bt->stack_masks[frame] |= mask;
+}
+
static inline void bt_set_frame_stack_arg_slot(struct backtrack_state *bt, u32 frame, u32 slot)
{
bt->stack_arg_masks[frame] |= 1 << slot;
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index eb46a81a8..46cadb83f 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -3696,14 +3696,23 @@ static int check_stack_write_var_off(struct bpf_verifier_env *env,
* SCALAR. This function does not deal with register filling; the caller must
* ensure that all spilled registers in the stack range have been marked as
* read.
+ *
+ * If requested, STACK_SPILL bytes backed by spilled scalar const zeroes are
+ * also considered zero bytes. In that case, mark the contributing stack slots
+ * precise so pruning cannot reuse a zero-spill state for a later non-zero
+ * spill state.
+ *
+ * Returns an error if precision backtracking fails.
*/
-static void mark_reg_stack_read(struct bpf_verifier_env *env,
- /* func where src register points to */
- struct bpf_func_state *ptr_state,
- int min_off, int max_off, int dst_regno)
+static int mark_reg_stack_read(struct bpf_verifier_env *env,
+ /* func where src register points to */
+ struct bpf_func_state *ptr_state,
+ int min_off, int max_off, int dst_regno,
+ bool mark_zero_spills)
{
struct bpf_verifier_state *vstate = env->cur_state;
struct bpf_func_state *state = vstate->frame[vstate->curframe];
+ u64 zero_spill_mask = 0;
int i, slot, spi;
u8 *stype;
int zeros = 0;
@@ -3713,19 +3722,37 @@ static void mark_reg_stack_read(struct bpf_verifier_env *env,
spi = slot / BPF_REG_SIZE;
mark_stack_slot_scratched(env, spi);
stype = ptr_state->stack[spi].slot_type;
- if (stype[slot % BPF_REG_SIZE] != STACK_ZERO)
- break;
- zeros++;
+ if (stype[slot % BPF_REG_SIZE] == STACK_ZERO) {
+ zeros++;
+ continue;
+ }
+ if (mark_zero_spills &&
+ stype[slot % BPF_REG_SIZE] == STACK_SPILL &&
+ bpf_is_spilled_scalar_reg(&ptr_state->stack[spi]) &&
+ tnum_is_const(ptr_state->stack[spi].spilled_ptr.var_off) &&
+ ptr_state->stack[spi].spilled_ptr.var_off.value == 0) {
+ zero_spill_mask |= 1ull << spi;
+ zeros++;
+ continue;
+ }
+ break;
}
if (zeros == max_off - min_off) {
/* Any access_size read into register is zero extended,
* so the whole register == const_zero.
*/
__mark_reg_const_zero(env, &state->regs[dst_regno]);
+ if (zero_spill_mask) {
+ bpf_bt_set_frame_slot_mask(&env->bt, ptr_state->frameno,
+ zero_spill_mask);
+ return mark_chain_precision_batch(env, env->cur_state);
+ }
} else {
/* have read misc data from the stack */
mark_reg_unknown(env, state->regs, dst_regno);
}
+
+ return 0;
}
/* Read the stack at 'off' and put the results into the register indicated by
@@ -3747,6 +3774,7 @@ static int check_stack_read_fixed_off(struct bpf_verifier_env *env,
int i, slot = -off - 1, spi = slot / BPF_REG_SIZE;
struct bpf_reg_state *reg;
u8 *stype, type;
+ int err;
int insn_flags = INSN_F_STACK_ACCESS;
int hist_spi = spi, hist_frame = reg_state->frameno;
@@ -3874,8 +3902,12 @@ static int check_stack_read_fixed_off(struct bpf_verifier_env *env,
}
return -EACCES;
}
- if (dst_regno >= 0)
- mark_reg_stack_read(env, reg_state, off, off + size, dst_regno);
+ if (dst_regno >= 0) {
+ err = mark_reg_stack_read(env, reg_state, off, off + size,
+ dst_regno, false);
+ if (err)
+ return err;
+ }
insn_flags = 0; /* we are not restoring spilled register */
}
if (insn_flags)
@@ -3929,7 +3961,10 @@ static int check_stack_read_var_off(struct bpf_verifier_env *env, struct bpf_reg
min_off = reg_smin(reg) + off;
max_off = reg_smax(reg) + off;
- mark_reg_stack_read(env, ptr_state, min_off, max_off + size, dst_regno);
+ err = mark_reg_stack_read(env, ptr_state, min_off, max_off + size,
+ dst_regno, true);
+ if (err)
+ return err;
check_fastcall_stack_contract(env, ptr_state, env->insn_idx, min_off);
return 0;
}
--
2.54.0
^ permalink raw reply related [flat|nested] 5+ messages in thread* [PATCH bpf-next v2 2/2] selftests/bpf: Cover var-offset stack reads from zero spills
2026-06-13 11:16 [PATCH bpf-next v2 0/2] bpf: Preserve scalar zero spills for var-offset stack reads Woojin Ji
2026-06-13 11:16 ` [PATCH bpf-next v2 1/2] " Woojin Ji
@ 2026-06-13 11:16 ` Woojin Ji
2026-06-21 15:29 ` [PATCH bpf-next v2 0/2] bpf: Preserve scalar zero spills for var-offset stack reads Yonghong Song
2 siblings, 0 replies; 5+ messages in thread
From: Woojin Ji @ 2026-06-13 11:16 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Martin KaFai Lau,
Song Liu, Yonghong Song, Jiri Olsa, Emil Tsalapatis,
John Fastabend
Cc: bpf, Woojin Ji
Add verifier_var_off coverage for variable-offset stack reads from
spilled scalar constant zero values.
Cover single-slot and cross-slot spilled zero reads, a sub-8-byte spill
with neighbouring STACK_ZERO bytes, and a sub-8-byte spill with
neighbouring STACK_MISC bytes that must not be treated as zero. Add a
state-pruning negative test to ensure the verifier marks the spilled
zero stack slot precise before using it to derive a const-zero register.
Use verifier log assertions to check both the zero result and the
precision backtracking trail.
Assisted-by: opencode:gpt-5.5
Signed-off-by: Woojin Ji <random6.xyz@gmail.com>
---
.../testing/selftests/bpf/progs/verifier_var_off.c | 148 +++++++++++++++++++++
1 file changed, 148 insertions(+)
diff --git a/tools/testing/selftests/bpf/progs/verifier_var_off.c b/tools/testing/selftests/bpf/progs/verifier_var_off.c
index f345466bc..f0052d831 100644
--- a/tools/testing/selftests/bpf/progs/verifier_var_off.c
+++ b/tools/testing/selftests/bpf/progs/verifier_var_off.c
@@ -59,6 +59,154 @@ __naked void stack_read_priv_vs_unpriv(void)
" ::: __clobber_all);
}
+SEC("cgroup/skb")
+__description("variable-offset stack read preserves spilled zero")
+__success
+__log_level(2)
+__msg("mark_precise: frame0: regs= stack=-8")
+__msg("R3=0")
+__failure_unpriv __msg_unpriv("R2 variable stack access prohibited for !root")
+__retval(0)
+__naked void stack_read_var_off_preserves_spilled_zero(void)
+{
+ asm volatile ("\
+ r0 = 0;\
+ *(u64*)(r10 - 8) = r0;\
+ r2 = *(u32*)(r1 + 0);\
+ r2 &= 7;\
+ r2 -= 8;\
+ r2 += r10;\
+ r3 = *(u8*)(r2 + 0);\
+ r1 = r10;\
+ r1 += -1;\
+ r1 += r3;\
+ *(u8*)(r1 + 0) = r3;\
+ r0 = 0;\
+ exit;\
+" ::: __clobber_all);
+}
+
+SEC("cgroup/skb")
+__description("variable-offset stack read preserves spilled zero across slots")
+__success
+__log_level(2)
+__msg("mark_precise: frame0: regs= stack=-8,-16")
+__msg("R3=0")
+__failure_unpriv __msg_unpriv("R2 variable stack access prohibited for !root")
+__retval(0)
+__naked void stack_read_var_off_preserves_spilled_zero_across_slots(void)
+{
+ asm volatile ("\
+ r0 = 0;\
+ *(u64*)(r10 - 8) = r0;\
+ *(u64*)(r10 - 16) = r0;\
+ r2 = *(u32*)(r1 + 0);\
+ r2 &= 15;\
+ r2 -= 16;\
+ r2 += r10;\
+ r3 = *(u8*)(r2 + 0);\
+ r1 = r10;\
+ r1 += -1;\
+ r1 += r3;\
+ *(u8*)(r1 + 0) = r3;\
+ r0 = 0;\
+ exit;\
+" ::: __clobber_all);
+}
+
+SEC("cgroup/skb")
+__description("variable-offset stack read preserves partial spilled zero")
+__success
+__log_level(2)
+__msg("mark_precise: frame0: regs= stack=-8")
+__msg("R3=0")
+__failure_unpriv __msg_unpriv("R2 variable stack access prohibited for !root")
+__retval(0)
+__naked void stack_read_var_off_preserves_partial_spilled_zero(void)
+{
+ asm volatile ("\
+ r0 = 0;\
+ *(u8*)(r10 - 9) = r0;\
+ *(u8*)(r10 - 10) = r0;\
+ *(u8*)(r10 - 11) = r0;\
+ *(u8*)(r10 - 12) = r0;\
+ *(u8*)(r10 - 13) = r0;\
+ *(u8*)(r10 - 14) = r0;\
+ *(u8*)(r10 - 15) = r0;\
+ *(u32*)(r10 - 8) = r0;\
+ r2 = *(u32*)(r1 + 0);\
+ r2 &= 15;\
+ if r2 > 10 goto l0_%=;\
+ r2 -= 15;\
+ r2 += r10;\
+ r3 = *(u8*)(r2 + 0);\
+ r1 = r10;\
+ r1 += -1;\
+ r1 += r3;\
+ *(u8*)(r1 + 0) = r3;\
+l0_%=: r0 = 0;\
+ exit;\
+" ::: __clobber_all);
+}
+
+SEC("cgroup/skb")
+__description("variable-offset stack read partial spill with misc data")
+__failure
+__msg("invalid variable-offset write to stack R1")
+__failure_unpriv __msg_unpriv("R2 variable stack access prohibited for !root")
+__naked void stack_read_var_off_partial_spill_with_misc_data(void)
+{
+ asm volatile ("\
+ r0 = 0;\
+ *(u32*)(r10 - 8) = r0;\
+ r2 = *(u32*)(r1 + 0);\
+ r2 &= 7;\
+ r2 -= 8;\
+ r2 += r10;\
+ r3 = *(u8*)(r2 + 0);\
+ r1 = r10;\
+ r1 += -1;\
+ r1 += r3;\
+ *(u8*)(r1 + 0) = 0;\
+ r0 = 0;\
+ exit;\
+" ::: __clobber_all);
+}
+
+SEC("cgroup/skb")
+__description("variable-offset stack read tracks spilled zero precisely")
+__failure
+__flag(BPF_F_TEST_STATE_FREQ)
+__log_level(2)
+__msg("mark_precise: frame0: regs= stack=-8")
+__msg("invalid variable-offset write to stack R1")
+__failure_unpriv __msg_unpriv("R2 variable stack access prohibited for !root")
+__naked void stack_read_var_off_tracks_spilled_zero_precisely(void)
+{
+ asm volatile ("\
+ r6 = *(u32*)(r1 + 0);\
+ r6 &= 1;\
+ r0 = 0;\
+ if r6 != 0 goto l0_%=;\
+ *(u64*)(r10 - 8) = r0;\
+ goto l1_%=;\
+l0_%=: r0 = 1;\
+ *(u64*)(r10 - 8) = r0;\
+l1_%=: r0 = 0;\
+ r2 = *(u32*)(r1 + 4);\
+ r2 &= 7;\
+ r2 -= 8;\
+ r2 += r10;\
+ r3 = *(u8*)(r2 + 0);\
+ r1 = r10;\
+ r1 += -1;\
+ r1 += r3;\
+ *(u8*)(r1 + 0) = 0;\
+ r0 = 0;\
+ exit;\
+" ::: __clobber_all);
+}
+
SEC("cgroup/skb")
__description("variable-offset stack read, uninitialized")
__success
--
2.54.0
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [PATCH bpf-next v2 0/2] bpf: Preserve scalar zero spills for var-offset stack reads
2026-06-13 11:16 [PATCH bpf-next v2 0/2] bpf: Preserve scalar zero spills for var-offset stack reads Woojin Ji
2026-06-13 11:16 ` [PATCH bpf-next v2 1/2] " Woojin Ji
2026-06-13 11:16 ` [PATCH bpf-next v2 2/2] selftests/bpf: Cover var-offset stack reads from zero spills Woojin Ji
@ 2026-06-21 15:29 ` Yonghong Song
2 siblings, 0 replies; 5+ messages in thread
From: Yonghong Song @ 2026-06-21 15:29 UTC (permalink / raw)
To: Woojin Ji, Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Martin KaFai Lau,
Song Liu, Jiri Olsa, Emil Tsalapatis, John Fastabend
Cc: bpf
On 6/13/26 4:16 AM, Woojin Ji wrote:
> Variable-offset stack reads currently lose the known-zero fact when the
> loaded byte comes from a spilled scalar constant zero rather than from a
> STACK_ZERO byte. This series teaches the var-offset stack read path to
> preserve that zero fact while marking the contributing spill slots
> precise, and adds verifier_var_off coverage for the new behaviour and
> the pruning-sensitive negative case.
>
> I don't have a confirmed deployed-program regression, so this is targeted
> at bpf-next. I did confirm the pattern is reachable from normal C codegen
> with a small helper-based BPF C reproducer: clang 22.1.6 -O2/-O3 can
> produce a spilled scalar-zero plus variable-offset stack byte load pattern
> that the unpatched verifier rejects and the patched verifier accepts.
Could you share your C code to desmonstrate patch 1? We can then check
whether the pattern is common and whether the code can be easliy worked
around with source code (e.g. barrier_var() etc.).
>
> Changes in v2:
> - Rebased onto bpf-next.
> - Split verifier and selftests changes into separate patches.
> - Added bpf_bt_set_frame_slot_mask() instead of open-coding a slot loop.
> - Kept the new eager precision marking on the variable-offset read path.
> - Added verifier log assertions for the zero result and mark_precise trail.
> - Added sub-8-byte spill coverage with STACK_ZERO and STACK_MISC neighbours.
> - Cleaned up inline asm formatting and local labels.
>
> Tested with:
> - make O=../../out/kernel olddefconfig
> - make O=../../out/kernel -j$(nproc) kernel/bpf/verifier.o
> - make O=../../out/kernel LLVM=1 -j$(nproc) bzImage
> - ./test_progs -t verifier_var_off -v
> Summary: 1/30 PASSED, 0 SKIPPED, 0 FAILED
> - ./test_progs -t verifier_spill_fill -t verifier_live_stack \
> -t verifier_search_pruning -v
> Summary: 3/127 PASSED, 0 SKIPPED, 0 FAILED
> - veristat -o csv verifier_var_off.bpf.o
>
> Assisted-by: opencode:gpt-5.5
> Signed-off-by: Woojin Ji <random6.xyz@gmail.com>
> ---
> Woojin Ji (2):
> bpf: Preserve scalar zero spills for var-offset stack reads
> selftests/bpf: Cover var-offset stack reads from zero spills
>
> include/linux/bpf_verifier.h | 5 +
> kernel/bpf/verifier.c | 55 ++++++--
> .../testing/selftests/bpf/progs/verifier_var_off.c | 148 +++++++++++++++++++++
> 3 files changed, 198 insertions(+), 10 deletions(-)
> ---
> base-commit: 7bfb93e3475be9de894f1cecd3a727d3e1649b03
> change-id: 20260610-bpf-stack-var-off-zero-v1-34ad1bc3b533
>
> Best regards,
> --
> Woojin Ji <random6.xyz@gmail.com>
>
>
^ permalink raw reply [flat|nested] 5+ messages in thread