[PATCH bpf-next 1/2] bpf: Check load-acquire src ptr type before the load

[PATCH bpf-next 1/2] bpf: Check load-acquire src ptr type before the load

[Daniel Borkmann]

check_atomic_load() calls check_load_mem() before atomic_ptr_type_ok().
For a load-acquire that fetches into its own source register (dst_reg ==
src_reg), check_load_mem() overwrites src_reg's type with the type of the
loaded value, so the subsequent atomic_ptr_type_ok() no longer sees the
source pointer and fails to reject the disallowed types (ctx, pkt,
flow_keys, sock).

Since bpf_convert_ctx_accesses() does not rewrite atomic loads, the raw
access to the underlying kernel object is left in place. The destination
type is taken from the ctx access itself, so a load-acquire of the sk
field of struct __sk_buff for example leaves the register typed as
PTR_TO_SOCK_COMMON_OR_NULL, which type_is_sk_pointer() does not match
either, while it actually holds unconverted struct sk_buff bytes. Once
the NULL check has passed this is a type confusion, not just a leak of
kernel data.

Validate src_reg with check_reg_arg() and check the source pointer type
with atomic_ptr_type_ok() before the load again, mirroring
check_atomic_rmw(). Out-of-range register numbers are already rejected
earlier by check_and_resolve_insns() (commit 503d21ef8eac ("bpf: Do
register range validation early")), and the only exemption there,
is_stack_arg_ldx(), requires BPF_LDX | BPF_MEM | BPF_DW and thus never
matches a BPF_ATOMIC insn. atomic_ptr_type_ok() can therefore not
dereference register state out of bounds, that is, the out-of-bounds
read addressed by the Fixes commit below does not reappear (as proven
also via selftest).

Fixes: c03bb2fa327e ("bpf: Fix out-of-bounds read in check_atomic_load/store()")
Reported-by: STAR Labs SG <info@starlabs.sg>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
---
 kernel/bpf/verifier.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index b274004fccfd..9513e18836c2 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -6616,7 +6616,7 @@ static int check_atomic_load(struct bpf_verifier_env *env,
 {
 	int err;
 
-	err = check_load_mem(env, insn, true, false, false, "atomic_load");
+	err = check_reg_arg(env, insn->src_reg, SRC_OP);
 	if (err)
 		return err;
 
@@ -6627,7 +6627,7 @@ static int check_atomic_load(struct bpf_verifier_env *env,
 		return -EACCES;
 	}
 
-	return 0;
+	return check_load_mem(env, insn, true, false, false, "atomic_load");
 }
 
 static int check_atomic_store(struct bpf_verifier_env *env,
-- 
2.43.0

[PATCH bpf-next 2/2] selftests/bpf: Add load-acquire test for dst_reg == src_reg from ctx

[Daniel Borkmann]

Add a verifier test that a load-acquire fetching into its own source
register (dst_reg == src_reg) from a ctx pointer is rejected.

  # LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t verifier_load_acquire
  [...]
  #614/1   verifier_load_acquire/load-acquire, 8-bit:OK
  #614/2   verifier_load_acquire/load-acquire, 8-bit @unpriv:OK
  #614/3   verifier_load_acquire/load-acquire, 16-bit:OK
  #614/4   verifier_load_acquire/load-acquire, 16-bit @unpriv:OK
  #614/5   verifier_load_acquire/load-acquire, 32-bit:OK
  #614/6   verifier_load_acquire/load-acquire, 32-bit @unpriv:OK
  #614/7   verifier_load_acquire/load-acquire, 64-bit:OK
  #614/8   verifier_load_acquire/load-acquire, 64-bit @unpriv:OK
  #614/9   verifier_load_acquire/load-acquire with uninitialized src_reg:OK
  #614/10  verifier_load_acquire/load-acquire with uninitialized src_reg @unpriv:OK
  #614/11  verifier_load_acquire/load-acquire with non-pointer src_reg:OK
  #614/12  verifier_load_acquire/load-acquire with non-pointer src_reg @unpriv:OK
  #614/13  verifier_load_acquire/misaligned load-acquire:OK
  #614/14  verifier_load_acquire/misaligned load-acquire @unpriv:OK
  #614/15  verifier_load_acquire/load-acquire from ctx pointer:OK
  #614/16  verifier_load_acquire/load-acquire from ctx pointer @unpriv:OK
  #614/17  verifier_load_acquire/load-acquire from ctx pointer, same dst and src register:OK
  #614/18  verifier_load_acquire/load-acquire from ctx pointer, same dst and src register @unpriv:OK
  #614/19  verifier_load_acquire/load-acquire with invalid register R15:OK
  #614/20  verifier_load_acquire/load-acquire with invalid register R15 @unpriv:OK
  #614/21  verifier_load_acquire/load-acquire from pkt pointer:OK
  #614/22  verifier_load_acquire/load-acquire from flow_keys pointer:OK
  #614/23  verifier_load_acquire/load-acquire from sock pointer:OK
  #614     verifier_load_acquire:OK
  Summary: 1/23 PASSED, 0 SKIPPED, 0 FAILED

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
---
 .../selftests/bpf/progs/verifier_load_acquire.c  | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/tools/testing/selftests/bpf/progs/verifier_load_acquire.c b/tools/testing/selftests/bpf/progs/verifier_load_acquire.c
index 74f4f19c10b8..ae1dab1b0cbb 100644
--- a/tools/testing/selftests/bpf/progs/verifier_load_acquire.c
+++ b/tools/testing/selftests/bpf/progs/verifier_load_acquire.c
@@ -148,6 +148,22 @@ __naked void load_acquire_from_ctx_pointer(void)
 	: __clobber_all);
 }
 
+SEC("socket")
+__description("load-acquire from ctx pointer, same dst and src register")
+__failure __failure_unpriv __msg("BPF_ATOMIC loads from R6 ctx is not allowed")
+__naked void load_acquire_ctx_same_dst_src(void)
+{
+	asm volatile (
+	"r6 = r1;"
+	".8byte %[load_acquire_insn];"	// w6 = load_acquire((u32 *)(r6 + 0));
+	"r0 = 0;"
+	"exit;"
+	:
+	: __imm_insn(load_acquire_insn,
+		     BPF_ATOMIC_OP(BPF_W, BPF_LOAD_ACQ, BPF_REG_6, BPF_REG_6, 0))
+	: __clobber_all);
+}
+
 SEC("xdp")
 __description("load-acquire from pkt pointer")
 __failure __msg("BPF_ATOMIC loads from R2 pkt is not allowed")
-- 
2.43.0

Re: [PATCH bpf-next 2/2] selftests/bpf: Add load-acquire test for dst_reg == src_reg from ctx

[Eduard Zingerman]

On Tue, 2026-08-04 at 22:19 +0200, Daniel Borkmann wrote:
> Add a verifier test that a load-acquire fetching into its own source
> register (dst_reg == src_reg) from a ctx pointer is rejected.
> 
>   # LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t verifier_load_acquire
>   [...]
>   #614/1   verifier_load_acquire/load-acquire, 8-bit:OK
>   #614/2   verifier_load_acquire/load-acquire, 8-bit @unpriv:OK
>   #614/3   verifier_load_acquire/load-acquire, 16-bit:OK
>   #614/4   verifier_load_acquire/load-acquire, 16-bit @unpriv:OK
>   #614/5   verifier_load_acquire/load-acquire, 32-bit:OK
>   #614/6   verifier_load_acquire/load-acquire, 32-bit @unpriv:OK
>   #614/7   verifier_load_acquire/load-acquire, 64-bit:OK
>   #614/8   verifier_load_acquire/load-acquire, 64-bit @unpriv:OK
>   #614/9   verifier_load_acquire/load-acquire with uninitialized src_reg:OK
>   #614/10  verifier_load_acquire/load-acquire with uninitialized src_reg @unpriv:OK
>   #614/11  verifier_load_acquire/load-acquire with non-pointer src_reg:OK
>   #614/12  verifier_load_acquire/load-acquire with non-pointer src_reg @unpriv:OK
>   #614/13  verifier_load_acquire/misaligned load-acquire:OK
>   #614/14  verifier_load_acquire/misaligned load-acquire @unpriv:OK
>   #614/15  verifier_load_acquire/load-acquire from ctx pointer:OK
>   #614/16  verifier_load_acquire/load-acquire from ctx pointer @unpriv:OK
>   #614/17  verifier_load_acquire/load-acquire from ctx pointer, same dst and src register:OK
>   #614/18  verifier_load_acquire/load-acquire from ctx pointer, same dst and src register @unpriv:OK
>   #614/19  verifier_load_acquire/load-acquire with invalid register R15:OK
>   #614/20  verifier_load_acquire/load-acquire with invalid register R15 @unpriv:OK
>   #614/21  verifier_load_acquire/load-acquire from pkt pointer:OK
>   #614/22  verifier_load_acquire/load-acquire from flow_keys pointer:OK
>   #614/23  verifier_load_acquire/load-acquire from sock pointer:OK
>   #614     verifier_load_acquire:OK
>   Summary: 1/23 PASSED, 0 SKIPPED, 0 FAILED
> 
> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
> ---

Acked-by: Eduard Zingerman <eddyz87@gmail.com>

...

Re: [PATCH bpf-next 1/2] bpf: Check load-acquire src ptr type before the load

[Eduard Zingerman]

On Tue, 2026-08-04 at 22:19 +0200, Daniel Borkmann wrote:
> check_atomic_load() calls check_load_mem() before atomic_ptr_type_ok().
> For a load-acquire that fetches into its own source register (dst_reg ==
> src_reg), check_load_mem() overwrites src_reg's type with the type of the
> loaded value, so the subsequent atomic_ptr_type_ok() no longer sees the
> source pointer and fails to reject the disallowed types (ctx, pkt,
> flow_keys, sock).
> 
> Since bpf_convert_ctx_accesses() does not rewrite atomic loads, the raw
> access to the underlying kernel object is left in place. The destination
> type is taken from the ctx access itself, so a load-acquire of the sk
> field of struct __sk_buff for example leaves the register typed as
> PTR_TO_SOCK_COMMON_OR_NULL, which type_is_sk_pointer() does not match
> either, while it actually holds unconverted struct sk_buff bytes. Once
> the NULL check has passed this is a type confusion, not just a leak of
> kernel data.
> 
> Validate src_reg with check_reg_arg() and check the source pointer type
> with atomic_ptr_type_ok() before the load again, mirroring
> check_atomic_rmw(). Out-of-range register numbers are already rejected
> earlier by check_and_resolve_insns() (commit 503d21ef8eac ("bpf: Do
> register range validation early")), and the only exemption there,
> is_stack_arg_ldx(), requires BPF_LDX | BPF_MEM | BPF_DW and thus never
> matches a BPF_ATOMIC insn. atomic_ptr_type_ok() can therefore not
> dereference register state out of bounds, that is, the out-of-bounds
> read addressed by the Fixes commit below does not reappear (as proven
> also via selftest).
> 
> Fixes: c03bb2fa327e ("bpf: Fix out-of-bounds read in check_atomic_load/store()")
> Reported-by: STAR Labs SG <info@starlabs.sg>
> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
> ---

Acked-by: Eduard Zingerman <eddyz87@gmail.com>

...

Re: [PATCH bpf-next 1/2] bpf: Check load-acquire src ptr type before the load

[patchwork-bot+netdevbpf]

Hello:

This series was applied to bpf/bpf-next.git (master)
by Kumar Kartikeya Dwivedi <memxor@gmail.com>:

On Tue,  4 Aug 2026 22:19:16 +0200 you wrote:
> check_atomic_load() calls check_load_mem() before atomic_ptr_type_ok().
> For a load-acquire that fetches into its own source register (dst_reg ==
> src_reg), check_load_mem() overwrites src_reg's type with the type of the
> loaded value, so the subsequent atomic_ptr_type_ok() no longer sees the
> source pointer and fails to reject the disallowed types (ctx, pkt,
> flow_keys, sock).
> 
> [...]

Here is the summary with links:
  - [bpf-next,1/2] bpf: Check load-acquire src ptr type before the load
    https://git.kernel.org/bpf/bpf-next/c/b87803391baa
  - [bpf-next,2/2] selftests/bpf: Add load-acquire test for dst_reg == src_reg from ctx
    https://git.kernel.org/bpf/bpf-next/c/363b15d8551e

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html