* [PATCH bpf 1/3] bpf: zero extend the result of an arena 32-bit cmpxchg
@ 2026-09-03 17:15 Eduard Zingerman
2026-09-03 17:15 ` [PATCH bpf 2/3] bpf: update disasm.c to print BPF_PROBE_ATOMIC as atomics Eduard Zingerman
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Eduard Zingerman @ 2026-09-03 17:15 UTC (permalink / raw)
To: bpf, ast, andrii
Cc: daniel, martin.lau, kernel-team, yonghong.song, eddyz87, memxor,
npc
bpf_convert_ctx_accesses() rewrites an atomic on an arena pointer from
BPF_STX | BPF_ATOMIC to BPF_STX | BPF_PROBE_ATOMIC, and it runs before
bpf_opt_subreg_zext_lo32_rnd_hi32().
That pass emits an explicit zero extension for a 32-bit cmpxchg even
when bpf_jit_needs_zext() is false. This is done because on some
architectures 32-bit cmpxchg requires explicit zero extension for the
dst register. E.g. on x86-64 'lock cmpxchg' does not change the %eax
if comparison is successful, while BPF semantics declare that each
operation on a 32-bit register zero extends it's upper half.
is_cmpxchg_insn() matches BPF_MODE == BPF_ATOMIC only, so an arena
cmpxchg misses said zero extension adjustment. This patch adjusts
is_cmpxchg_insn() to match BPF_PROBE_ATOMIC alongside BPF_ATOMIC.
Fixes: d503a04f8bc0 ("bpf: Add support for certain atomics in bpf_arena to x86 JIT")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
---
kernel/bpf/fixups.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
index 65b441e4a351..52d3cec33672 100644
--- a/kernel/bpf/fixups.c
+++ b/kernel/bpf/fixups.c
@@ -13,10 +13,15 @@
#define verbose(env, fmt, args...) bpf_verifier_log_write(env, fmt, ##args)
+/*
+ * Matches BPF_PROBE_ATOMIC too: bpf_convert_ctx_accesses() rewrites arena
+ * atomics before bpf_opt_subreg_zext_lo32_rnd_hi32() runs.
+ */
static bool is_cmpxchg_insn(const struct bpf_insn *insn)
{
return BPF_CLASS(insn->code) == BPF_STX &&
- BPF_MODE(insn->code) == BPF_ATOMIC &&
+ (BPF_MODE(insn->code) == BPF_ATOMIC ||
+ BPF_MODE(insn->code) == BPF_PROBE_ATOMIC) &&
insn->imm == BPF_CMPXCHG;
}
--
2.55.0
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH bpf 2/3] bpf: update disasm.c to print BPF_PROBE_ATOMIC as atomics 2026-09-03 17:15 [PATCH bpf 1/3] bpf: zero extend the result of an arena 32-bit cmpxchg Eduard Zingerman @ 2026-09-03 17:15 ` Eduard Zingerman 2026-09-03 18:11 ` bot+bpf-ci 2026-09-03 17:15 ` [PATCH bpf 3/3] selftests/bpf: check zero extension of an arena 32-bit cmpxchg Eduard Zingerman 2026-09-04 2:00 ` [PATCH bpf 1/3] bpf: zero extend the result " patchwork-bot+netdevbpf 2 siblings, 1 reply; 6+ messages in thread From: Eduard Zingerman @ 2026-09-03 17:15 UTC (permalink / raw) To: bpf, ast, andrii Cc: daniel, martin.lau, kernel-team, yonghong.song, eddyz87, memxor, npc bpf_convert_ctx_accesses() rewrites an atomic on an arena pointer from BPF_STX | BPF_ATOMIC to BPF_STX | BPF_PROBE_ATOMIC, this patch adjusts print_bpf_insn() to print such instructions as regular atomics with a 'probe_' prefix (instead of printing them as BUG_XX). Signed-off-by: Eduard Zingerman <eddyz87@gmail.com> --- kernel/bpf/disasm.c | 47 ++++++++++++++++++++++++--------------------- 1 file changed, 25 insertions(+), 22 deletions(-) diff --git a/kernel/bpf/disasm.c b/kernel/bpf/disasm.c index b1a3fbe3fda5..3ce8d74b0e40 100644 --- a/kernel/bpf/disasm.c +++ b/kernel/bpf/disasm.c @@ -7,6 +7,9 @@ #include "disasm.h" +/* Only defined by the non-UAPI linux/filter.h, which this file cannot use. */ +#define BPF_PROBE_ATOMIC 0xe0 + #define __BPF_FUNC_STR_FN(x) [BPF_FUNC_ ## x] = __stringify(bpf_ ## x) static const char * const func_id_str[] = { __BPF_FUNC_MAPPER(__BPF_FUNC_STR_FN) @@ -226,57 +229,57 @@ void print_bpf_insn(const struct bpf_insn_cbs *cbs, insn->imm); } } else if (class == BPF_STX) { + const char *probe_pfx = BPF_MODE(insn->code) == BPF_PROBE_ATOMIC ? "probe " : ""; + bool atomic = BPF_MODE(insn->code) == BPF_ATOMIC || + BPF_MODE(insn->code) == BPF_PROBE_ATOMIC; + if (BPF_MODE(insn->code) == BPF_MEM) verbose(cbs->private_data, "(%02x) *(%s *)(r%d %+d) = r%d", insn->code, bpf_ldst_string[BPF_SIZE(insn->code) >> 3], insn->dst_reg, insn->off, insn->src_reg); - else if (BPF_MODE(insn->code) == BPF_ATOMIC && + else if (atomic && (insn->imm == BPF_ADD || insn->imm == BPF_AND || insn->imm == BPF_OR || insn->imm == BPF_XOR)) { - verbose(cbs->private_data, "(%02x) lock *(%s *)(r%d %+d) %s r%d", - insn->code, + verbose(cbs->private_data, "(%02x) %slock *(%s *)(r%d %+d) %s r%d", + insn->code, probe_pfx, bpf_ldst_string[BPF_SIZE(insn->code) >> 3], insn->dst_reg, insn->off, bpf_alu_string[BPF_OP(insn->imm) >> 4], insn->src_reg); - } else if (BPF_MODE(insn->code) == BPF_ATOMIC && + } else if (atomic && (insn->imm == (BPF_ADD | BPF_FETCH) || insn->imm == (BPF_AND | BPF_FETCH) || insn->imm == (BPF_OR | BPF_FETCH) || insn->imm == (BPF_XOR | BPF_FETCH))) { - verbose(cbs->private_data, "(%02x) r%d = atomic%s_fetch_%s((%s *)(r%d %+d), r%d)", - insn->code, insn->src_reg, + verbose(cbs->private_data, "(%02x) %sr%d = atomic%s_fetch_%s((%s *)(r%d %+d), r%d)", + insn->code, probe_pfx, insn->src_reg, BPF_SIZE(insn->code) == BPF_DW ? "64" : "", bpf_atomic_alu_string[BPF_OP(insn->imm) >> 4], bpf_ldst_string[BPF_SIZE(insn->code) >> 3], insn->dst_reg, insn->off, insn->src_reg); - } else if (BPF_MODE(insn->code) == BPF_ATOMIC && - insn->imm == BPF_CMPXCHG) { - verbose(cbs->private_data, "(%02x) r0 = atomic%s_cmpxchg((%s *)(r%d %+d), r0, r%d)", - insn->code, + } else if (atomic && insn->imm == BPF_CMPXCHG) { + verbose(cbs->private_data, "(%02x) %sr0 = atomic%s_cmpxchg((%s *)(r%d %+d), r0, r%d)", + insn->code, probe_pfx, BPF_SIZE(insn->code) == BPF_DW ? "64" : "", bpf_ldst_string[BPF_SIZE(insn->code) >> 3], insn->dst_reg, insn->off, insn->src_reg); - } else if (BPF_MODE(insn->code) == BPF_ATOMIC && - insn->imm == BPF_XCHG) { - verbose(cbs->private_data, "(%02x) r%d = atomic%s_xchg((%s *)(r%d %+d), r%d)", - insn->code, insn->src_reg, + } else if (atomic && insn->imm == BPF_XCHG) { + verbose(cbs->private_data, "(%02x) %sr%d = atomic%s_xchg((%s *)(r%d %+d), r%d)", + insn->code, probe_pfx, insn->src_reg, BPF_SIZE(insn->code) == BPF_DW ? "64" : "", bpf_ldst_string[BPF_SIZE(insn->code) >> 3], insn->dst_reg, insn->off, insn->src_reg); - } else if (BPF_MODE(insn->code) == BPF_ATOMIC && - insn->imm == BPF_LOAD_ACQ) { - verbose(cbs->private_data, "(%02x) r%d = load_acquire((%s *)(r%d %+d))", - insn->code, insn->dst_reg, + } else if (atomic && insn->imm == BPF_LOAD_ACQ) { + verbose(cbs->private_data, "(%02x) %sr%d = load_acquire((%s *)(r%d %+d))", + insn->code, probe_pfx, insn->dst_reg, bpf_ldst_string[BPF_SIZE(insn->code) >> 3], insn->src_reg, insn->off); - } else if (BPF_MODE(insn->code) == BPF_ATOMIC && - insn->imm == BPF_STORE_REL) { - verbose(cbs->private_data, "(%02x) store_release((%s *)(r%d %+d), r%d)", - insn->code, + } else if (atomic && insn->imm == BPF_STORE_REL) { + verbose(cbs->private_data, "(%02x) %sstore_release((%s *)(r%d %+d), r%d)", + insn->code, probe_pfx, bpf_ldst_string[BPF_SIZE(insn->code) >> 3], insn->dst_reg, insn->off, insn->src_reg); } else { -- 2.55.0 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH bpf 2/3] bpf: update disasm.c to print BPF_PROBE_ATOMIC as atomics 2026-09-03 17:15 ` [PATCH bpf 2/3] bpf: update disasm.c to print BPF_PROBE_ATOMIC as atomics Eduard Zingerman @ 2026-09-03 18:11 ` bot+bpf-ci 2026-09-03 19:38 ` Eduard Zingerman 0 siblings, 1 reply; 6+ messages in thread From: bot+bpf-ci @ 2026-09-03 18:11 UTC (permalink / raw) To: eddyz87, bpf, ast, andrii Cc: daniel, martin.lau, kernel-team, yonghong.song, eddyz87, memxor, npc, ast, andrii, daniel, martin.lau, eddyz87, yonghong.song, mason, ihor.solodrai [-- Attachment #1: Type: text/plain, Size: 1614 bytes --] > bpf: update disasm.c to print BPF_PROBE_ATOMIC as atomics > > bpf_convert_ctx_accesses() rewrites an atomic on an arena pointer from > BPF_STX | BPF_ATOMIC to BPF_STX | BPF_PROBE_ATOMIC, this patch adjusts > print_bpf_insn() to print such instructions as regular atomics with a > 'probe_' prefix (instead of printing them as BUG_XX). The changelog says the instructions are printed "as regular atomics with a 'probe_' prefix", but looking at the implementation: const char *probe_pfx = BPF_MODE(insn->code) == BPF_PROBE_ATOMIC ? "probe " : ""; ... verbose(cbs->private_data, "(%02x) %sr0 = atomic%s_cmpxchg((%s *)(r%d %+d), r0, r%d)", insn->code, probe_pfx, ...); the code emits a separate word "probe " (trailing space, no underscore) at the start of the rendered instruction, producing output like: probe r0 = atomic_cmpxchg((u32 *)(r1 +0), r0, r2) rather than: r0 = probe_atomic_cmpxchg(...) as the message suggests. For the fetch/cmpxchg/xchg/load_acquire/ store_release forms, this also detaches the qualifier from the operation it qualifies (e.g. "probe r%d = atomic..._fetch_...", reading as if "probe" modified the destination register). Should the changelog describe the emitted "probe " prefix, or should the prefix be attached to the operation name (probe_atomic_cmpxchg, probe_load_acquire, etc.) to match the message? --- AI reviewed your patch. Please fix the bug or email reply why it's not a bug. See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md CI run summary: https://github.com/kernel-patches/bpf/actions/runs/33784588558 ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH bpf 2/3] bpf: update disasm.c to print BPF_PROBE_ATOMIC as atomics 2026-09-03 18:11 ` bot+bpf-ci @ 2026-09-03 19:38 ` Eduard Zingerman 0 siblings, 0 replies; 6+ messages in thread From: Eduard Zingerman @ 2026-09-03 19:38 UTC (permalink / raw) To: bot+bpf-ci, bpf, ast, andrii Cc: daniel, martin.lau, kernel-team, yonghong.song, memxor, npc, martin.lau, mason, ihor.solodrai On Thu, 2026-09-03 at 18:11 +0000, bot+bpf-ci@kernel.org wrote: > > bpf: update disasm.c to print BPF_PROBE_ATOMIC as atomics > > > > bpf_convert_ctx_accesses() rewrites an atomic on an arena pointer from > > BPF_STX | BPF_ATOMIC to BPF_STX | BPF_PROBE_ATOMIC, this patch adjusts > > print_bpf_insn() to print such instructions as regular atomics with a > > 'probe_' prefix (instead of printing them as BUG_XX). > > The changelog says the instructions are printed "as regular atomics with > a 'probe_' prefix", but looking at the implementation: > > const char *probe_pfx = BPF_MODE(insn->code) == BPF_PROBE_ATOMIC ? "probe " : ""; > ... > verbose(cbs->private_data, "(%02x) %sr0 = atomic%s_cmpxchg((%s *)(r%d %+d), r0, r%d)", > insn->code, probe_pfx, ...); > > the code emits a separate word "probe " (trailing space, no underscore) > at the start of the rendered instruction, producing output like: > > probe r0 = atomic_cmpxchg((u32 *)(r1 +0), r0, r2) > > rather than: > > r0 = probe_atomic_cmpxchg(...) > > as the message suggests. For the fetch/cmpxchg/xchg/load_acquire/ > store_release forms, this also detaches the qualifier from the operation > it qualifies (e.g. "probe r%d = atomic..._fetch_...", reading as if > "probe" modified the destination register). > > Should the changelog describe the emitted "probe " prefix, or should the > prefix be attached to the operation name (probe_atomic_cmpxchg, > probe_load_acquire, etc.) to match the message? Yes, I should have updated the log. The intent is to print: (hex) probe ...something-something... will respin v2, waiting for sashiko. ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH bpf 3/3] selftests/bpf: check zero extension of an arena 32-bit cmpxchg 2026-09-03 17:15 [PATCH bpf 1/3] bpf: zero extend the result of an arena 32-bit cmpxchg Eduard Zingerman 2026-09-03 17:15 ` [PATCH bpf 2/3] bpf: update disasm.c to print BPF_PROBE_ATOMIC as atomics Eduard Zingerman @ 2026-09-03 17:15 ` Eduard Zingerman 2026-09-04 2:00 ` [PATCH bpf 1/3] bpf: zero extend the result " patchwork-bot+netdevbpf 2 siblings, 0 replies; 6+ messages in thread From: Eduard Zingerman @ 2026-09-03 17:15 UTC (permalink / raw) To: bpf, ast, andrii Cc: daniel, martin.lau, kernel-team, yonghong.song, eddyz87, memxor, npc Add a test to verify that destination register of a 32-bit cmpxchg operating on an arena pointer is explicitly zero extended. W/o patch #1 this did not happen. Reported-by: Nicholas Carlini <npc@anthropic.com> Suggested-by: Nicholas Carlini <npc@anthropic.com> Signed-off-by: Eduard Zingerman <eddyz87@gmail.com> --- .../selftests/bpf/progs/verifier_zext.c | 26 +++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/tools/testing/selftests/bpf/progs/verifier_zext.c b/tools/testing/selftests/bpf/progs/verifier_zext.c index 8f2362da91d6..572017fe28fb 100644 --- a/tools/testing/selftests/bpf/progs/verifier_zext.c +++ b/tools/testing/selftests/bpf/progs/verifier_zext.c @@ -356,6 +356,32 @@ __naked void arena_ptr(void) : __clobber_all); } +/* + * Result of a 32-bit cmpxchg is always explicitly zero extended. + * Check that this holds for arenas (BPF_PROBE_ATOMIC instruction flavor). + */ +SEC("socket") +__success +__xlated("probe r0 = atomic_cmpxchg((u32 *)(r1 +0), r0, r2)") +__xlated("w0 = w0") +__naked void zext_arena_cmpxchg32(void) +{ + asm volatile (" \ + r9 = %[arena] ll; /* associate the arena with the program */ \ + r1 = 0; \ + r1 = addr_space_cast(r1, 0, 1); \ + r0 = 0; \ + r2 = 0; \ + .8byte %[cmpxchg32]; \ + r0 >>= 32; /* make the upper half live */ \ + exit; \ +" : + : __imm_addr(arena), + __imm_insn(cmpxchg32, + BPF_ATOMIC_OP(BPF_W, BPF_CMPXCHG, BPF_REG_1, BPF_REG_2, 0)) + : __clobber_all); +} + #endif /* Check if probe mem loads keep their zero extension. */ -- 2.55.0 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH bpf 1/3] bpf: zero extend the result of an arena 32-bit cmpxchg 2026-09-03 17:15 [PATCH bpf 1/3] bpf: zero extend the result of an arena 32-bit cmpxchg Eduard Zingerman 2026-09-03 17:15 ` [PATCH bpf 2/3] bpf: update disasm.c to print BPF_PROBE_ATOMIC as atomics Eduard Zingerman 2026-09-03 17:15 ` [PATCH bpf 3/3] selftests/bpf: check zero extension of an arena 32-bit cmpxchg Eduard Zingerman @ 2026-09-04 2:00 ` patchwork-bot+netdevbpf 2 siblings, 0 replies; 6+ messages in thread From: patchwork-bot+netdevbpf @ 2026-09-04 2:00 UTC (permalink / raw) To: Eduard Zingerman Cc: bpf, ast, andrii, daniel, martin.lau, kernel-team, yonghong.song, memxor, npc Hello: This series was applied to bpf/bpf.git (master) by Alexei Starovoitov <ast@kernel.org>: On Thu, 3 Sep 2026 10:15:39 -0700 you wrote: > bpf_convert_ctx_accesses() rewrites an atomic on an arena pointer from > BPF_STX | BPF_ATOMIC to BPF_STX | BPF_PROBE_ATOMIC, and it runs before > bpf_opt_subreg_zext_lo32_rnd_hi32(). > > That pass emits an explicit zero extension for a 32-bit cmpxchg even > when bpf_jit_needs_zext() is false. This is done because on some > architectures 32-bit cmpxchg requires explicit zero extension for the > dst register. E.g. on x86-64 'lock cmpxchg' does not change the %eax > if comparison is successful, while BPF semantics declare that each > operation on a 32-bit register zero extends it's upper half. > > [...] Here is the summary with links: - [bpf,1/3] bpf: zero extend the result of an arena 32-bit cmpxchg https://git.kernel.org/bpf/bpf/c/4814ed6406f3 - [bpf,2/3] bpf: update disasm.c to print BPF_PROBE_ATOMIC as atomics https://git.kernel.org/bpf/bpf/c/1f3cd9719c40 - [bpf,3/3] selftests/bpf: check zero extension of an arena 32-bit cmpxchg https://git.kernel.org/bpf/bpf/c/54ed91950363 You are awesome, thank you! -- Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2026-09-04 2:01 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-09-03 17:15 [PATCH bpf 1/3] bpf: zero extend the result of an arena 32-bit cmpxchg Eduard Zingerman 2026-09-03 17:15 ` [PATCH bpf 2/3] bpf: update disasm.c to print BPF_PROBE_ATOMIC as atomics Eduard Zingerman 2026-09-03 18:11 ` bot+bpf-ci 2026-09-03 19:38 ` Eduard Zingerman 2026-09-03 17:15 ` [PATCH bpf 3/3] selftests/bpf: check zero extension of an arena 32-bit cmpxchg Eduard Zingerman 2026-09-04 2:00 ` [PATCH bpf 1/3] bpf: zero extend the result " patchwork-bot+netdevbpf
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox