* [PATCH bpf-next v3 0/3] Signed loads from Arena
@ 2025-09-15 16:28 Puranjay Mohan
2025-09-15 16:28 ` [PATCH bpf-next v3 1/3] bpf, x86: Add support for signed arena loads Puranjay Mohan
` (3 more replies)
0 siblings, 4 replies; 12+ messages in thread
From: Puranjay Mohan @ 2025-09-15 16:28 UTC (permalink / raw)
To: bpf
Cc: Puranjay Mohan, Alexei Starovoitov, Andrii Nakryiko,
Daniel Borkmann, Martin KaFai Lau, Eduard Zingerman, kkd,
kernel-team
Changelog:
v1 -> v2:
v1: https://lore.kernel.org/bpf/20250509194956.1635207-1-memxor@gmail.com
- Use bpf_jit_supports_insn. (Alexei)
v2 -> v3:
v2: https://lore.kernel.org/bpf/20250514175415.2045783-1-memxor@gmail.com/
- Fix encoding for the generated instructions in x86 JIT (Eduard)
The patch in v2 was generating instructions like:
42 63 44 20 f8 movslq -0x8(%rax,%r12), %eax
This doesn't make sense because movslq outputs a 64-bit result, but
the destination register here is set to eax (32-bit). The fix it to
set the REX.W bit in the opcode, that means changing
EMIT2(add_3mod(0x40, ...)) to EMIT2(add_3mod(0x48, ...))
- Add arm64 support
- Add selftests signed laods from arena.
Currently, signed load instructions into arena memory are unsupported.
The compiler is free to generate these, and on GCC-14 we see a
corresponding error when it happens. The hurdle in supporting them is
deciding which unused opcode to use to mark them for the JIT's own
consumption. After much thinking, it appears 0xc0 / BPF_NOSPEC can be
combined with load instructions to identify signed arena loads. Use
this to recognize and JIT them appropriately, and remove the verifier
side limitation on the program if the JIT supports them.
Kumar Kartikeya Dwivedi (1):
bpf, x86: Add support for signed arena loads
Puranjay Mohan (2):
bpf, arm64: Add support for signed arena loads
selftests: bpf: Add tests for signed loads from arena
arch/arm64/net/bpf_jit_comp.c | 25 ++-
arch/s390/net/bpf_jit_comp.c | 5 +
arch/x86/net/bpf_jit_comp.c | 35 +++-
include/linux/filter.h | 3 +
kernel/bpf/verifier.c | 11 +-
.../selftests/bpf/progs/verifier_ldsx.c | 176 ++++++++++++++++++
6 files changed, 243 insertions(+), 12 deletions(-)
--
2.47.3
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH bpf-next v3 1/3] bpf, x86: Add support for signed arena loads
2025-09-15 16:28 [PATCH bpf-next v3 0/3] Signed loads from Arena Puranjay Mohan
@ 2025-09-15 16:28 ` Puranjay Mohan
2025-09-17 1:05 ` Eduard Zingerman
2025-09-15 16:28 ` [PATCH bpf-next v3 2/3] bpf, arm64: " Puranjay Mohan
` (2 subsequent siblings)
3 siblings, 1 reply; 12+ messages in thread
From: Puranjay Mohan @ 2025-09-15 16:28 UTC (permalink / raw)
To: bpf
Cc: Puranjay Mohan, Alexei Starovoitov, Andrii Nakryiko,
Daniel Borkmann, Martin KaFai Lau, Eduard Zingerman, kkd,
kernel-team, Kumar Kartikeya Dwivedi
From: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Currently, signed load instructions into arena memory are unsupported.
The compiler is free to generate these, and on GCC-14 we see a
corresponding error when it happens. The hurdle in supporting them is
deciding which unused opcode to use to mark them for the JIT's own
consumption. After much thinking, it appears 0xc0 / BPF_NOSPEC can be
combined with load instructions to identify signed arena loads. Use
this to recognize and JIT them appropriately, and remove the verifier
side limitation on the program if the JIT supports them.
Co-developed-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
---
arch/arm64/net/bpf_jit_comp.c | 5 +++++
arch/s390/net/bpf_jit_comp.c | 5 +++++
arch/x86/net/bpf_jit_comp.c | 35 ++++++++++++++++++++++++++++++++++-
include/linux/filter.h | 3 +++
kernel/bpf/verifier.c | 11 ++++++++---
5 files changed, 55 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index 008273a53e04..f2b85a10add2 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -3064,6 +3064,11 @@ bool bpf_jit_supports_insn(struct bpf_insn *insn, bool in_arena)
if (!bpf_atomic_is_load_store(insn) &&
!cpus_have_cap(ARM64_HAS_LSE_ATOMICS))
return false;
+ break;
+ case BPF_LDX | BPF_MEMSX | BPF_B:
+ case BPF_LDX | BPF_MEMSX | BPF_H:
+ case BPF_LDX | BPF_MEMSX | BPF_W:
+ return false;
}
return true;
}
diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c
index 8b57d8532f36..cf461d76e9da 100644
--- a/arch/s390/net/bpf_jit_comp.c
+++ b/arch/s390/net/bpf_jit_comp.c
@@ -2967,6 +2967,11 @@ bool bpf_jit_supports_insn(struct bpf_insn *insn, bool in_arena)
case BPF_STX | BPF_ATOMIC | BPF_DW:
if (bpf_atomic_is_load_store(insn))
return false;
+ break;
+ case BPF_LDX | BPF_MEMSX | BPF_B:
+ case BPF_LDX | BPF_MEMSX | BPF_H:
+ case BPF_LDX | BPF_MEMSX | BPF_W:
+ return false;
}
return true;
}
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 8d34a9400a5e..a6550da34268 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -1152,11 +1152,38 @@ static void emit_ldx_index(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, u32 i
*pprog = prog;
}
+static void emit_ldsx_index(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, u32 index_reg, int off)
+{
+ u8 *prog = *pprog;
+
+ switch (size) {
+ case BPF_B:
+ /* movsx rax, byte ptr [rax + r12 + off] */
+ EMIT3(add_3mod(0x48, src_reg, dst_reg, index_reg), 0x0F, 0xBE);
+ break;
+ case BPF_H:
+ /* movsx rax, word ptr [rax + r12 + off] */
+ EMIT3(add_3mod(0x48, src_reg, dst_reg, index_reg), 0x0F, 0xBF);
+ break;
+ case BPF_W:
+ /* movsx rax, dword ptr [rax + r12 + off] */
+ EMIT2(add_3mod(0x48, src_reg, dst_reg, index_reg), 0x63);
+ break;
+ }
+ emit_insn_suffix_SIB(&prog, src_reg, dst_reg, index_reg, off);
+ *pprog = prog;
+}
+
static void emit_ldx_r12(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, int off)
{
emit_ldx_index(pprog, size, dst_reg, src_reg, X86_REG_R12, off);
}
+static void emit_ldsx_r12(u8 **prog, u32 size, u32 dst_reg, u32 src_reg, int off)
+{
+ emit_ldsx_index(prog, size, dst_reg, src_reg, X86_REG_R12, off);
+}
+
/* STX: *(u8*)(dst_reg + off) = src_reg */
static void emit_stx(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, int off)
{
@@ -2109,13 +2136,19 @@ st: if (is_imm8(insn->off))
case BPF_LDX | BPF_PROBE_MEM32 | BPF_H:
case BPF_LDX | BPF_PROBE_MEM32 | BPF_W:
case BPF_LDX | BPF_PROBE_MEM32 | BPF_DW:
+ case BPF_LDX | BPF_PROBE_MEM32SX | BPF_B:
+ case BPF_LDX | BPF_PROBE_MEM32SX | BPF_H:
+ case BPF_LDX | BPF_PROBE_MEM32SX | BPF_W:
case BPF_STX | BPF_PROBE_MEM32 | BPF_B:
case BPF_STX | BPF_PROBE_MEM32 | BPF_H:
case BPF_STX | BPF_PROBE_MEM32 | BPF_W:
case BPF_STX | BPF_PROBE_MEM32 | BPF_DW:
start_of_ldx = prog;
if (BPF_CLASS(insn->code) == BPF_LDX)
- emit_ldx_r12(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn->off);
+ if (BPF_MODE(insn->code) == BPF_PROBE_MEM32SX)
+ emit_ldsx_r12(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn->off);
+ else
+ emit_ldx_r12(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn->off);
else
emit_stx_r12(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn->off);
populate_extable:
diff --git a/include/linux/filter.h b/include/linux/filter.h
index 4241a885975f..f5c859b8131a 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -78,6 +78,9 @@ struct ctl_table_header;
/* unused opcode to mark special atomic instruction */
#define BPF_PROBE_ATOMIC 0xe0
+/* unused opcode to mark special ldsx instruction. Same as BPF_NOSPEC */
+#define BPF_PROBE_MEM32SX 0xc0
+
/* unused opcode to mark call to interpreter with arguments */
#define BPF_CALL_ARGS 0xe0
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 17fe623400a5..411a3980d7b7 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -21423,10 +21423,14 @@ static int convert_ctx_accesses(struct bpf_verifier_env *env)
continue;
case PTR_TO_ARENA:
if (BPF_MODE(insn->code) == BPF_MEMSX) {
- verbose(env, "sign extending loads from arena are not supported yet\n");
- return -EOPNOTSUPP;
+ if (!bpf_jit_supports_insn(insn, true)) {
+ verbose(env, "sign extending loads from arena are not supported yet\n");
+ return -EOPNOTSUPP;
+ }
+ insn->code = BPF_CLASS(insn->code) | BPF_PROBE_MEM32SX | BPF_SIZE(insn->code);
+ } else {
+ insn->code = BPF_CLASS(insn->code) | BPF_PROBE_MEM32 | BPF_SIZE(insn->code);
}
- insn->code = BPF_CLASS(insn->code) | BPF_PROBE_MEM32 | BPF_SIZE(insn->code);
env->prog->aux->num_exentries++;
continue;
default:
@@ -21632,6 +21636,7 @@ static int jit_subprogs(struct bpf_verifier_env *env)
if (BPF_CLASS(insn->code) == BPF_LDX &&
(BPF_MODE(insn->code) == BPF_PROBE_MEM ||
BPF_MODE(insn->code) == BPF_PROBE_MEM32 ||
+ BPF_MODE(insn->code) == BPF_PROBE_MEM32SX ||
BPF_MODE(insn->code) == BPF_PROBE_MEMSX))
num_exentries++;
if ((BPF_CLASS(insn->code) == BPF_STX ||
--
2.47.3
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH bpf-next v3 2/3] bpf, arm64: Add support for signed arena loads
2025-09-15 16:28 [PATCH bpf-next v3 0/3] Signed loads from Arena Puranjay Mohan
2025-09-15 16:28 ` [PATCH bpf-next v3 1/3] bpf, x86: Add support for signed arena loads Puranjay Mohan
@ 2025-09-15 16:28 ` Puranjay Mohan
2025-09-20 10:37 ` Xu Kuohai
2025-09-15 16:28 ` [PATCH bpf-next v3 3/3] selftests: bpf: Add tests for signed loads from arena Puranjay Mohan
2025-09-17 1:03 ` [PATCH bpf-next v3 0/3] Signed loads from Arena Kumar Kartikeya Dwivedi
3 siblings, 1 reply; 12+ messages in thread
From: Puranjay Mohan @ 2025-09-15 16:28 UTC (permalink / raw)
To: bpf
Cc: Puranjay Mohan, Alexei Starovoitov, Andrii Nakryiko,
Daniel Borkmann, Martin KaFai Lau, Eduard Zingerman, kkd,
kernel-team
Add support for signed loads from arena which are internally converted
to loads with mode set BPF_PROBE_MEM32SX by the verifier. The
implementation is similar to BPF_PROBE_MEMSX and BPF_MEMSX but for
BPF_PROBE_MEM32SX, arena_vm_base is added to the src register to form
the address.
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
---
arch/arm64/net/bpf_jit_comp.c | 30 +++++++++++++++++-------------
1 file changed, 17 insertions(+), 13 deletions(-)
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index f2b85a10add2..7233acec69ce 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -1133,12 +1133,14 @@ static int add_exception_handler(const struct bpf_insn *insn,
return 0;
if (BPF_MODE(insn->code) != BPF_PROBE_MEM &&
- BPF_MODE(insn->code) != BPF_PROBE_MEMSX &&
- BPF_MODE(insn->code) != BPF_PROBE_MEM32 &&
- BPF_MODE(insn->code) != BPF_PROBE_ATOMIC)
+ BPF_MODE(insn->code) != BPF_PROBE_MEMSX &&
+ BPF_MODE(insn->code) != BPF_PROBE_MEM32 &&
+ BPF_MODE(insn->code) != BPF_PROBE_MEM32SX &&
+ BPF_MODE(insn->code) != BPF_PROBE_ATOMIC)
return 0;
is_arena = (BPF_MODE(insn->code) == BPF_PROBE_MEM32) ||
+ (BPF_MODE(insn->code) == BPF_PROBE_MEM32SX) ||
(BPF_MODE(insn->code) == BPF_PROBE_ATOMIC);
if (!ctx->prog->aux->extable ||
@@ -1659,7 +1661,11 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
case BPF_LDX | BPF_PROBE_MEM32 | BPF_H:
case BPF_LDX | BPF_PROBE_MEM32 | BPF_W:
case BPF_LDX | BPF_PROBE_MEM32 | BPF_DW:
- if (BPF_MODE(insn->code) == BPF_PROBE_MEM32) {
+ case BPF_LDX | BPF_PROBE_MEM32SX | BPF_B:
+ case BPF_LDX | BPF_PROBE_MEM32SX | BPF_H:
+ case BPF_LDX | BPF_PROBE_MEM32SX | BPF_W:
+ if (BPF_MODE(insn->code) == BPF_PROBE_MEM32 ||
+ BPF_MODE(insn->code) == BPF_PROBE_MEM32SX) {
emit(A64_ADD(1, tmp2, src, arena_vm_base), ctx);
src = tmp2;
}
@@ -1671,7 +1677,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
off_adj = off;
}
sign_extend = (BPF_MODE(insn->code) == BPF_MEMSX ||
- BPF_MODE(insn->code) == BPF_PROBE_MEMSX);
+ BPF_MODE(insn->code) == BPF_PROBE_MEMSX ||
+ BPF_MODE(insn->code) == BPF_PROBE_MEM32SX);
switch (BPF_SIZE(code)) {
case BPF_W:
if (is_lsi_offset(off_adj, 2)) {
@@ -1879,9 +1886,11 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
if (ret)
return ret;
- ret = add_exception_handler(insn, ctx, dst);
- if (ret)
- return ret;
+ if (BPF_MODE(insn->code) == BPF_PROBE_ATOMIC) {
+ ret = add_exception_handler(insn, ctx, dst);
+ if (ret)
+ return ret;
+ }
break;
default:
@@ -3064,11 +3073,6 @@ bool bpf_jit_supports_insn(struct bpf_insn *insn, bool in_arena)
if (!bpf_atomic_is_load_store(insn) &&
!cpus_have_cap(ARM64_HAS_LSE_ATOMICS))
return false;
- break;
- case BPF_LDX | BPF_MEMSX | BPF_B:
- case BPF_LDX | BPF_MEMSX | BPF_H:
- case BPF_LDX | BPF_MEMSX | BPF_W:
- return false;
}
return true;
}
--
2.47.3
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH bpf-next v3 3/3] selftests: bpf: Add tests for signed loads from arena
2025-09-15 16:28 [PATCH bpf-next v3 0/3] Signed loads from Arena Puranjay Mohan
2025-09-15 16:28 ` [PATCH bpf-next v3 1/3] bpf, x86: Add support for signed arena loads Puranjay Mohan
2025-09-15 16:28 ` [PATCH bpf-next v3 2/3] bpf, arm64: " Puranjay Mohan
@ 2025-09-15 16:28 ` Puranjay Mohan
2025-09-17 1:03 ` [PATCH bpf-next v3 0/3] Signed loads from Arena Kumar Kartikeya Dwivedi
3 siblings, 0 replies; 12+ messages in thread
From: Puranjay Mohan @ 2025-09-15 16:28 UTC (permalink / raw)
To: bpf
Cc: Puranjay Mohan, Alexei Starovoitov, Andrii Nakryiko,
Daniel Borkmann, Martin KaFai Lau, Eduard Zingerman, kkd,
kernel-team
Add tests for loading 8, 16, and 32 bits with sign extension from arena,
also verify that exception handling is working correctly and correct
assembly is being generated by the x86 and arm64 JITs.
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
---
.../selftests/bpf/progs/verifier_ldsx.c | 176 ++++++++++++++++++
1 file changed, 176 insertions(+)
diff --git a/tools/testing/selftests/bpf/progs/verifier_ldsx.c b/tools/testing/selftests/bpf/progs/verifier_ldsx.c
index 52edee41caf6..6c16904d2afb 100644
--- a/tools/testing/selftests/bpf/progs/verifier_ldsx.c
+++ b/tools/testing/selftests/bpf/progs/verifier_ldsx.c
@@ -3,6 +3,7 @@
#include <linux/bpf.h>
#include <bpf/bpf_helpers.h>
#include "bpf_misc.h"
+#include "bpf_arena_common.h"
#if (defined(__TARGET_ARCH_arm64) || defined(__TARGET_ARCH_x86) || \
(defined(__TARGET_ARCH_riscv) && __riscv_xlen == 64) || \
@@ -10,6 +11,12 @@
defined(__TARGET_ARCH_loongarch)) && \
__clang_major__ >= 18
+struct {
+ __uint(type, BPF_MAP_TYPE_ARENA);
+ __uint(map_flags, BPF_F_MMAPABLE);
+ __uint(max_entries, 1);
+} arena SEC(".maps");
+
SEC("socket")
__description("LDSX, S8")
__success __success_unpriv __retval(-2)
@@ -256,6 +263,175 @@ __naked void ldsx_ctx_8(void)
: __clobber_all);
}
+SEC("syscall")
+__description("Arena LDSX Disasm")
+__success
+__arch_x86_64
+__jited("movslq 0x10(%rax,%r12), %r14")
+__jited("movswq 0x18(%rax,%r12), %r14")
+__jited("movsbq 0x20(%rax,%r12), %r14")
+__jited("movslq 0x10(%rdi,%r12), %r15")
+__jited("movswq 0x18(%rdi,%r12), %r15")
+__jited("movsbq 0x20(%rdi,%r12), %r15")
+__arch_arm64
+__jited("add x11, x7, x28")
+__jited("ldrsw x21, [x11, #0x10]")
+__jited("add x11, x7, x28")
+__jited("ldrsh x21, [x11, #0x18]")
+__jited("add x11, x7, x28")
+__jited("ldrsb x21, [x11, #0x20]")
+__jited("add x11, x0, x28")
+__jited("ldrsw x22, [x11, #0x10]")
+__jited("add x11, x0, x28")
+__jited("ldrsh x22, [x11, #0x18]")
+__jited("add x11, x0, x28")
+__jited("ldrsb x22, [x11, #0x20]")
+__naked void arena_ldsx_disasm(void *ctx)
+{
+ asm volatile (
+ "r1 = %[arena] ll;"
+ "r2 = 0;"
+ "r3 = 1;"
+ "r4 = %[numa_no_node];"
+ "r5 = 0;"
+ "call %[bpf_arena_alloc_pages];"
+ "r0 = addr_space_cast(r0, 0x0, 0x1);"
+ "r1 = r0;"
+ "r8 = *(s32 *)(r0 + 16);"
+ "r8 = *(s16 *)(r0 + 24);"
+ "r8 = *(s8 *)(r0 + 32);"
+ "r9 = *(s32 *)(r1 + 16);"
+ "r9 = *(s16 *)(r1 + 24);"
+ "r9 = *(s8 *)(r1 + 32);"
+ "r0 = 0;"
+ "exit;"
+ :: __imm(bpf_arena_alloc_pages),
+ __imm_addr(arena),
+ __imm_const(numa_no_node, NUMA_NO_NODE)
+ : __clobber_all
+ );
+}
+
+SEC("syscall")
+__description("Arena LDSX Exception")
+__success __retval(0)
+__arch_x86_64
+__arch_arm64
+__naked void arena_ldsx_exception(void *ctx)
+{
+ asm volatile (
+ "r1 = %[arena] ll;"
+ "r0 = 0xdeadbeef;"
+ "r0 = addr_space_cast(r0, 0x0, 0x1);"
+ "r1 = 0x3fe;"
+ "*(u64 *)(r0 + 0) = r1;"
+ "r0 = *(s8 *)(r0 + 0);"
+ "exit;"
+ :
+ : __imm_addr(arena)
+ : __clobber_all
+ );
+}
+
+SEC("syscall")
+__description("Arena LDSX, S8")
+__success __retval(-1)
+__arch_x86_64
+__arch_arm64
+__naked void arena_ldsx_s8(void *ctx)
+{
+ asm volatile (
+ "r1 = %[arena] ll;"
+ "r2 = 0;"
+ "r3 = 1;"
+ "r4 = %[numa_no_node];"
+ "r5 = 0;"
+ "call %[bpf_arena_alloc_pages];"
+ "r0 = addr_space_cast(r0, 0x0, 0x1);"
+ "r1 = 0x3fe;"
+ "*(u64 *)(r0 + 0) = r1;"
+#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
+ "r0 = *(s8 *)(r0 + 0);"
+#else
+ "r0 = *(s8 *)(r0 + 7);"
+#endif
+ "r0 >>= 1;"
+ "exit;"
+ :: __imm(bpf_arena_alloc_pages),
+ __imm_addr(arena),
+ __imm_const(numa_no_node, NUMA_NO_NODE)
+ : __clobber_all
+ );
+}
+
+SEC("syscall")
+__description("Arena LDSX, S16")
+__success __retval(-1)
+__arch_x86_64
+__arch_arm64
+__naked void arena_ldsx_s16(void *ctx)
+{
+ asm volatile (
+ "r1 = %[arena] ll;"
+ "r2 = 0;"
+ "r3 = 1;"
+ "r4 = %[numa_no_node];"
+ "r5 = 0;"
+ "call %[bpf_arena_alloc_pages];"
+ "r0 = addr_space_cast(r0, 0x0, 0x1);"
+ "r1 = 0x3fffe;"
+ "*(u64 *)(r0 + 0) = r1;"
+#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
+ "r0 = *(s16 *)(r0 + 0);"
+#else
+ "r0 = *(s16 *)(r0 + 6);"
+#endif
+ "r0 >>= 1;"
+ "exit;"
+ :: __imm(bpf_arena_alloc_pages),
+ __imm_addr(arena),
+ __imm_const(numa_no_node, NUMA_NO_NODE)
+ : __clobber_all
+ );
+}
+
+SEC("syscall")
+__description("Arena LDSX, S32")
+__success __retval(-1)
+__arch_x86_64
+__arch_arm64
+__naked void arena_ldsx_s32(void *ctx)
+{
+ asm volatile (
+ "r1 = %[arena] ll;"
+ "r2 = 0;"
+ "r3 = 1;"
+ "r4 = %[numa_no_node];"
+ "r5 = 0;"
+ "call %[bpf_arena_alloc_pages];"
+ "r0 = addr_space_cast(r0, 0x0, 0x1);"
+ "r1 = 0xfffffffe;"
+ "*(u64 *)(r0 + 0) = r1;"
+#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
+ "r0 = *(s32 *)(r0 + 0);"
+#else
+ "r0 = *(s32 *)(r0 + 4);"
+#endif
+ "r0 >>= 1;"
+ "exit;"
+ :: __imm(bpf_arena_alloc_pages),
+ __imm_addr(arena),
+ __imm_const(numa_no_node, NUMA_NO_NODE)
+ : __clobber_all
+ );
+}
+
+/* to retain debug info for BTF generation */
+void kfunc_root(void)
+{
+ bpf_arena_alloc_pages(0, 0, 0, 0, 0);
+}
+
#else
SEC("socket")
--
2.47.3
^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH bpf-next v3 0/3] Signed loads from Arena
2025-09-15 16:28 [PATCH bpf-next v3 0/3] Signed loads from Arena Puranjay Mohan
` (2 preceding siblings ...)
2025-09-15 16:28 ` [PATCH bpf-next v3 3/3] selftests: bpf: Add tests for signed loads from arena Puranjay Mohan
@ 2025-09-17 1:03 ` Kumar Kartikeya Dwivedi
2025-09-17 3:41 ` Xu Kuohai
3 siblings, 1 reply; 12+ messages in thread
From: Kumar Kartikeya Dwivedi @ 2025-09-17 1:03 UTC (permalink / raw)
To: Puranjay Mohan, Xu Kuohai
Cc: bpf, Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Martin KaFai Lau, Eduard Zingerman, kkd, kernel-team
On Mon, 15 Sept 2025 at 18:29, Puranjay Mohan <puranjay@kernel.org> wrote:
>
> Changelog:
> v1 -> v2:
> v1: https://lore.kernel.org/bpf/20250509194956.1635207-1-memxor@gmail.com
> - Use bpf_jit_supports_insn. (Alexei)
>
> v2 -> v3:
> v2: https://lore.kernel.org/bpf/20250514175415.2045783-1-memxor@gmail.com/
> - Fix encoding for the generated instructions in x86 JIT (Eduard)
> The patch in v2 was generating instructions like:
> 42 63 44 20 f8 movslq -0x8(%rax,%r12), %eax
> This doesn't make sense because movslq outputs a 64-bit result, but
> the destination register here is set to eax (32-bit). The fix it to
> set the REX.W bit in the opcode, that means changing
> EMIT2(add_3mod(0x40, ...)) to EMIT2(add_3mod(0x48, ...))
> - Add arm64 support
> - Add selftests signed laods from arena.
>
> Currently, signed load instructions into arena memory are unsupported.
> The compiler is free to generate these, and on GCC-14 we see a
> corresponding error when it happens. The hurdle in supporting them is
> deciding which unused opcode to use to mark them for the JIT's own
> consumption. After much thinking, it appears 0xc0 / BPF_NOSPEC can be
> combined with load instructions to identify signed arena loads. Use
> this to recognize and JIT them appropriately, and remove the verifier
> side limitation on the program if the JIT supports them.
>
> Kumar Kartikeya Dwivedi (1):
> bpf, x86: Add support for signed arena loads
>
> Puranjay Mohan (2):
> bpf, arm64: Add support for signed arena loads
+Cc Xu, could you please ack arm64 bits before we merge this?
Thanks a lot.
> [...]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH bpf-next v3 1/3] bpf, x86: Add support for signed arena loads
2025-09-15 16:28 ` [PATCH bpf-next v3 1/3] bpf, x86: Add support for signed arena loads Puranjay Mohan
@ 2025-09-17 1:05 ` Eduard Zingerman
2025-09-17 1:08 ` Kumar Kartikeya Dwivedi
2025-09-17 14:16 ` Puranjay Mohan
0 siblings, 2 replies; 12+ messages in thread
From: Eduard Zingerman @ 2025-09-17 1:05 UTC (permalink / raw)
To: Puranjay Mohan, bpf
Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Martin KaFai Lau, kkd, kernel-team, Kumar Kartikeya Dwivedi
On Mon, 2025-09-15 at 16:28 +0000, Puranjay Mohan wrote:
[...]
> diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
> index 008273a53e04..f2b85a10add2 100644
> --- a/arch/arm64/net/bpf_jit_comp.c
> +++ b/arch/arm64/net/bpf_jit_comp.c
> @@ -3064,6 +3064,11 @@ bool bpf_jit_supports_insn(struct bpf_insn *insn, bool in_arena)
> if (!bpf_atomic_is_load_store(insn) &&
> !cpus_have_cap(ARM64_HAS_LSE_ATOMICS))
> return false;
> + break;
> + case BPF_LDX | BPF_MEMSX | BPF_B:
> + case BPF_LDX | BPF_MEMSX | BPF_H:
> + case BPF_LDX | BPF_MEMSX | BPF_W:
> + return false;
> }
> return true;
> }
Is the same hunk necessary in riscv/net/bpf_jit_comp64.c?
[...]
> diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
> index 8d34a9400a5e..a6550da34268 100644
> --- a/arch/x86/net/bpf_jit_comp.c
> +++ b/arch/x86/net/bpf_jit_comp.c
> @@ -1152,11 +1152,38 @@ static void emit_ldx_index(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, u32 i
> *pprog = prog;
> }
>
> +static void emit_ldsx_index(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, u32 index_reg, int off)
> +{
> + u8 *prog = *pprog;
> +
> + switch (size) {
> + case BPF_B:
> + /* movsx rax, byte ptr [rax + r12 + off] */
> + EMIT3(add_3mod(0x48, src_reg, dst_reg, index_reg), 0x0F, 0xBE);
> + break;
> + case BPF_H:
> + /* movsx rax, word ptr [rax + r12 + off] */
> + EMIT3(add_3mod(0x48, src_reg, dst_reg, index_reg), 0x0F, 0xBF);
> + break;
> + case BPF_W:
> + /* movsx rax, dword ptr [rax + r12 + off] */
> + EMIT2(add_3mod(0x48, src_reg, dst_reg, index_reg), 0x63);
> + break;
> + }
> + emit_insn_suffix_SIB(&prog, src_reg, dst_reg, index_reg, off);
> + *pprog = prog;
> +}
> +
Encoding looks correct.
[...]
> @@ -2109,13 +2136,19 @@ st: if (is_imm8(insn->off))
> case BPF_LDX | BPF_PROBE_MEM32 | BPF_H:
> case BPF_LDX | BPF_PROBE_MEM32 | BPF_W:
> case BPF_LDX | BPF_PROBE_MEM32 | BPF_DW:
> + case BPF_LDX | BPF_PROBE_MEM32SX | BPF_B:
> + case BPF_LDX | BPF_PROBE_MEM32SX | BPF_H:
> + case BPF_LDX | BPF_PROBE_MEM32SX | BPF_W:
> case BPF_STX | BPF_PROBE_MEM32 | BPF_B:
> case BPF_STX | BPF_PROBE_MEM32 | BPF_H:
> case BPF_STX | BPF_PROBE_MEM32 | BPF_W:
> case BPF_STX | BPF_PROBE_MEM32 | BPF_DW:
> start_of_ldx = prog;
> if (BPF_CLASS(insn->code) == BPF_LDX)
> - emit_ldx_r12(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn->off);
> + if (BPF_MODE(insn->code) == BPF_PROBE_MEM32SX)
> + emit_ldsx_r12(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn->off);
> + else
> + emit_ldx_r12(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn->off);
> else
> emit_stx_r12(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn->off);
Heh, apparently this is correct C code. Dangling else is associated
with the closest 'if' statement. Didn't know that.
[...]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH bpf-next v3 1/3] bpf, x86: Add support for signed arena loads
2025-09-17 1:05 ` Eduard Zingerman
@ 2025-09-17 1:08 ` Kumar Kartikeya Dwivedi
2025-09-17 14:16 ` Puranjay Mohan
1 sibling, 0 replies; 12+ messages in thread
From: Kumar Kartikeya Dwivedi @ 2025-09-17 1:08 UTC (permalink / raw)
To: Eduard Zingerman
Cc: Puranjay Mohan, bpf, Alexei Starovoitov, Andrii Nakryiko,
Daniel Borkmann, Martin KaFai Lau, kkd, kernel-team
On Wed, 17 Sept 2025 at 03:05, Eduard Zingerman <eddyz87@gmail.com> wrote:
>
> On Mon, 2025-09-15 at 16:28 +0000, Puranjay Mohan wrote:
>
> [...]
>
> > diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
> > index 008273a53e04..f2b85a10add2 100644
> > --- a/arch/arm64/net/bpf_jit_comp.c
> > +++ b/arch/arm64/net/bpf_jit_comp.c
> > @@ -3064,6 +3064,11 @@ bool bpf_jit_supports_insn(struct bpf_insn *insn, bool in_arena)
> > if (!bpf_atomic_is_load_store(insn) &&
> > !cpus_have_cap(ARM64_HAS_LSE_ATOMICS))
> > return false;
> > + break;
> > + case BPF_LDX | BPF_MEMSX | BPF_B:
> > + case BPF_LDX | BPF_MEMSX | BPF_H:
> > + case BPF_LDX | BPF_MEMSX | BPF_W:
> > + return false;
> > }
> > return true;
> > }
>
> Is the same hunk necessary in riscv/net/bpf_jit_comp64.c?
>
> [...]
>
> > diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
> > index 8d34a9400a5e..a6550da34268 100644
> > --- a/arch/x86/net/bpf_jit_comp.c
> > +++ b/arch/x86/net/bpf_jit_comp.c
> > @@ -1152,11 +1152,38 @@ static void emit_ldx_index(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, u32 i
> > *pprog = prog;
> > }
> >
> > +static void emit_ldsx_index(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, u32 index_reg, int off)
> > +{
> > + u8 *prog = *pprog;
> > +
> > + switch (size) {
> > + case BPF_B:
> > + /* movsx rax, byte ptr [rax + r12 + off] */
> > + EMIT3(add_3mod(0x48, src_reg, dst_reg, index_reg), 0x0F, 0xBE);
> > + break;
> > + case BPF_H:
> > + /* movsx rax, word ptr [rax + r12 + off] */
> > + EMIT3(add_3mod(0x48, src_reg, dst_reg, index_reg), 0x0F, 0xBF);
> > + break;
> > + case BPF_W:
> > + /* movsx rax, dword ptr [rax + r12 + off] */
> > + EMIT2(add_3mod(0x48, src_reg, dst_reg, index_reg), 0x63);
> > + break;
> > + }
> > + emit_insn_suffix_SIB(&prog, src_reg, dst_reg, index_reg, off);
> > + *pprog = prog;
> > +}
> > +
>
> Encoding looks correct.
>
> [...]
>
> > @@ -2109,13 +2136,19 @@ st: if (is_imm8(insn->off))
> > case BPF_LDX | BPF_PROBE_MEM32 | BPF_H:
> > case BPF_LDX | BPF_PROBE_MEM32 | BPF_W:
> > case BPF_LDX | BPF_PROBE_MEM32 | BPF_DW:
> > + case BPF_LDX | BPF_PROBE_MEM32SX | BPF_B:
> > + case BPF_LDX | BPF_PROBE_MEM32SX | BPF_H:
> > + case BPF_LDX | BPF_PROBE_MEM32SX | BPF_W:
> > case BPF_STX | BPF_PROBE_MEM32 | BPF_B:
> > case BPF_STX | BPF_PROBE_MEM32 | BPF_H:
> > case BPF_STX | BPF_PROBE_MEM32 | BPF_W:
> > case BPF_STX | BPF_PROBE_MEM32 | BPF_DW:
> > start_of_ldx = prog;
> > if (BPF_CLASS(insn->code) == BPF_LDX)
> > - emit_ldx_r12(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn->off);
> > + if (BPF_MODE(insn->code) == BPF_PROBE_MEM32SX)
> > + emit_ldsx_r12(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn->off);
> > + else
> > + emit_ldx_r12(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn->off);
> > else
> > emit_stx_r12(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn->off);
>
> Heh, apparently this is correct C code. Dangling else is associated
> with the closest 'if' statement. Didn't know that.
I was going to say; let's use braces instead of doing this.
>
>
> [...]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH bpf-next v3 0/3] Signed loads from Arena
2025-09-17 1:03 ` [PATCH bpf-next v3 0/3] Signed loads from Arena Kumar Kartikeya Dwivedi
@ 2025-09-17 3:41 ` Xu Kuohai
0 siblings, 0 replies; 12+ messages in thread
From: Xu Kuohai @ 2025-09-17 3:41 UTC (permalink / raw)
To: Kumar Kartikeya Dwivedi, Puranjay Mohan
Cc: bpf, Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Martin KaFai Lau, Eduard Zingerman, kkd, kernel-team
On 9/17/2025 9:03 AM, Kumar Kartikeya Dwivedi wrote:
> On Mon, 15 Sept 2025 at 18:29, Puranjay Mohan <puranjay@kernel.org> wrote:
>>
>> Changelog:
>> v1 -> v2:
>> v1: https://lore.kernel.org/bpf/20250509194956.1635207-1-memxor@gmail.com
>> - Use bpf_jit_supports_insn. (Alexei)
>>
>> v2 -> v3:
>> v2: https://lore.kernel.org/bpf/20250514175415.2045783-1-memxor@gmail.com/
>> - Fix encoding for the generated instructions in x86 JIT (Eduard)
>> The patch in v2 was generating instructions like:
>> 42 63 44 20 f8 movslq -0x8(%rax,%r12), %eax
>> This doesn't make sense because movslq outputs a 64-bit result, but
>> the destination register here is set to eax (32-bit). The fix it to
>> set the REX.W bit in the opcode, that means changing
>> EMIT2(add_3mod(0x40, ...)) to EMIT2(add_3mod(0x48, ...))
>> - Add arm64 support
>> - Add selftests signed laods from arena.
>>
>> Currently, signed load instructions into arena memory are unsupported.
>> The compiler is free to generate these, and on GCC-14 we see a
>> corresponding error when it happens. The hurdle in supporting them is
>> deciding which unused opcode to use to mark them for the JIT's own
>> consumption. After much thinking, it appears 0xc0 / BPF_NOSPEC can be
>> combined with load instructions to identify signed arena loads. Use
>> this to recognize and JIT them appropriately, and remove the verifier
>> side limitation on the program if the JIT supports them.
>>
>> Kumar Kartikeya Dwivedi (1):
>> bpf, x86: Add support for signed arena loads
>>
>> Puranjay Mohan (2):
>> bpf, arm64: Add support for signed arena loads
>
> +Cc Xu, could you please ack arm64 bits before we merge this?
> Thanks a lot.
>
Sure, I'll look into it a bit later.
>> [...]
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH bpf-next v3 1/3] bpf, x86: Add support for signed arena loads
2025-09-17 1:05 ` Eduard Zingerman
2025-09-17 1:08 ` Kumar Kartikeya Dwivedi
@ 2025-09-17 14:16 ` Puranjay Mohan
1 sibling, 0 replies; 12+ messages in thread
From: Puranjay Mohan @ 2025-09-17 14:16 UTC (permalink / raw)
To: Eduard Zingerman, bpf
Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Martin KaFai Lau, kkd, kernel-team, Kumar Kartikeya Dwivedi
Eduard Zingerman <eddyz87@gmail.com> writes:
> On Mon, 2025-09-15 at 16:28 +0000, Puranjay Mohan wrote:
>
> [...]
>
>> diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
>> index 008273a53e04..f2b85a10add2 100644
>> --- a/arch/arm64/net/bpf_jit_comp.c
>> +++ b/arch/arm64/net/bpf_jit_comp.c
>> @@ -3064,6 +3064,11 @@ bool bpf_jit_supports_insn(struct bpf_insn *insn, bool in_arena)
>> if (!bpf_atomic_is_load_store(insn) &&
>> !cpus_have_cap(ARM64_HAS_LSE_ATOMICS))
>> return false;
>> + break;
>> + case BPF_LDX | BPF_MEMSX | BPF_B:
>> + case BPF_LDX | BPF_MEMSX | BPF_H:
>> + case BPF_LDX | BPF_MEMSX | BPF_W:
>> + return false;
>> }
>> return true;
>> }
>
> Is the same hunk necessary in riscv/net/bpf_jit_comp64.c?
Yes, will add in next version.
> [...]
>
>> diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
>> index 8d34a9400a5e..a6550da34268 100644
>> --- a/arch/x86/net/bpf_jit_comp.c
>> +++ b/arch/x86/net/bpf_jit_comp.c
>> @@ -1152,11 +1152,38 @@ static void emit_ldx_index(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, u32 i
>> *pprog = prog;
>> }
>>
>> +static void emit_ldsx_index(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, u32 index_reg, int off)
>> +{
>> + u8 *prog = *pprog;
>> +
>> + switch (size) {
>> + case BPF_B:
>> + /* movsx rax, byte ptr [rax + r12 + off] */
>> + EMIT3(add_3mod(0x48, src_reg, dst_reg, index_reg), 0x0F, 0xBE);
>> + break;
>> + case BPF_H:
>> + /* movsx rax, word ptr [rax + r12 + off] */
>> + EMIT3(add_3mod(0x48, src_reg, dst_reg, index_reg), 0x0F, 0xBF);
>> + break;
>> + case BPF_W:
>> + /* movsx rax, dword ptr [rax + r12 + off] */
>> + EMIT2(add_3mod(0x48, src_reg, dst_reg, index_reg), 0x63);
>> + break;
>> + }
>> + emit_insn_suffix_SIB(&prog, src_reg, dst_reg, index_reg, off);
>> + *pprog = prog;
>> +}
>> +
>
> Encoding looks correct.
>
> [...]
>
>> @@ -2109,13 +2136,19 @@ st: if (is_imm8(insn->off))
>> case BPF_LDX | BPF_PROBE_MEM32 | BPF_H:
>> case BPF_LDX | BPF_PROBE_MEM32 | BPF_W:
>> case BPF_LDX | BPF_PROBE_MEM32 | BPF_DW:
>> + case BPF_LDX | BPF_PROBE_MEM32SX | BPF_B:
>> + case BPF_LDX | BPF_PROBE_MEM32SX | BPF_H:
>> + case BPF_LDX | BPF_PROBE_MEM32SX | BPF_W:
>> case BPF_STX | BPF_PROBE_MEM32 | BPF_B:
>> case BPF_STX | BPF_PROBE_MEM32 | BPF_H:
>> case BPF_STX | BPF_PROBE_MEM32 | BPF_W:
>> case BPF_STX | BPF_PROBE_MEM32 | BPF_DW:
>> start_of_ldx = prog;
>> if (BPF_CLASS(insn->code) == BPF_LDX)
>> - emit_ldx_r12(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn->off);
>> + if (BPF_MODE(insn->code) == BPF_PROBE_MEM32SX)
>> + emit_ldsx_r12(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn->off);
>> + else
>> + emit_ldx_r12(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn->off);
>> else
>> emit_stx_r12(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn->off);
>
> Heh, apparently this is correct C code. Dangling else is associated
> with the closest 'if' statement. Didn't know that.
This code goes against kernel coding style, I should have used braces.
Will fix it in next version.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH bpf-next v3 2/3] bpf, arm64: Add support for signed arena loads
2025-09-15 16:28 ` [PATCH bpf-next v3 2/3] bpf, arm64: " Puranjay Mohan
@ 2025-09-20 10:37 ` Xu Kuohai
2025-09-23 10:52 ` Puranjay Mohan
0 siblings, 1 reply; 12+ messages in thread
From: Xu Kuohai @ 2025-09-20 10:37 UTC (permalink / raw)
To: Puranjay Mohan, bpf
Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Martin KaFai Lau, Eduard Zingerman, kkd, kernel-team
On 9/16/2025 12:28 AM, Puranjay Mohan wrote:
> Add support for signed loads from arena which are internally converted
> to loads with mode set BPF_PROBE_MEM32SX by the verifier. The
> implementation is similar to BPF_PROBE_MEMSX and BPF_MEMSX but for
> BPF_PROBE_MEM32SX, arena_vm_base is added to the src register to form
> the address.
>
> Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
> ---
> arch/arm64/net/bpf_jit_comp.c | 30 +++++++++++++++++-------------
> 1 file changed, 17 insertions(+), 13 deletions(-)
>
> diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
> index f2b85a10add2..7233acec69ce 100644
> --- a/arch/arm64/net/bpf_jit_comp.c
> +++ b/arch/arm64/net/bpf_jit_comp.c
> @@ -1133,12 +1133,14 @@ static int add_exception_handler(const struct bpf_insn *insn,
> return 0;
>
> if (BPF_MODE(insn->code) != BPF_PROBE_MEM &&
> - BPF_MODE(insn->code) != BPF_PROBE_MEMSX &&
> - BPF_MODE(insn->code) != BPF_PROBE_MEM32 &&
> - BPF_MODE(insn->code) != BPF_PROBE_ATOMIC)
> + BPF_MODE(insn->code) != BPF_PROBE_MEMSX &&
> + BPF_MODE(insn->code) != BPF_PROBE_MEM32 &&
> + BPF_MODE(insn->code) != BPF_PROBE_MEM32SX &&
> + BPF_MODE(insn->code) != BPF_PROBE_ATOMIC)
> return 0;
>
> is_arena = (BPF_MODE(insn->code) == BPF_PROBE_MEM32) ||
> + (BPF_MODE(insn->code) == BPF_PROBE_MEM32SX) ||
> (BPF_MODE(insn->code) == BPF_PROBE_ATOMIC);
>
> if (!ctx->prog->aux->extable ||
> @@ -1659,7 +1661,11 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
> case BPF_LDX | BPF_PROBE_MEM32 | BPF_H:
> case BPF_LDX | BPF_PROBE_MEM32 | BPF_W:
> case BPF_LDX | BPF_PROBE_MEM32 | BPF_DW:
> - if (BPF_MODE(insn->code) == BPF_PROBE_MEM32) {
> + case BPF_LDX | BPF_PROBE_MEM32SX | BPF_B:
> + case BPF_LDX | BPF_PROBE_MEM32SX | BPF_H:
> + case BPF_LDX | BPF_PROBE_MEM32SX | BPF_W:
> + if (BPF_MODE(insn->code) == BPF_PROBE_MEM32 ||
> + BPF_MODE(insn->code) == BPF_PROBE_MEM32SX) {
> emit(A64_ADD(1, tmp2, src, arena_vm_base), ctx);
> src = tmp2;
> }
> @@ -1671,7 +1677,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
> off_adj = off;
> }
> sign_extend = (BPF_MODE(insn->code) == BPF_MEMSX ||
> - BPF_MODE(insn->code) == BPF_PROBE_MEMSX);
> + BPF_MODE(insn->code) == BPF_PROBE_MEMSX ||
> + BPF_MODE(insn->code) == BPF_PROBE_MEM32SX);
> switch (BPF_SIZE(code)) {
> case BPF_W:
> if (is_lsi_offset(off_adj, 2)) {
> @@ -1879,9 +1886,11 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
> if (ret)
> return ret;
>
> - ret = add_exception_handler(insn, ctx, dst);
> - if (ret)
> - return ret;
> + if (BPF_MODE(insn->code) == BPF_PROBE_ATOMIC) {
add_exception_handler already checked this condition, why add a check here?
> + ret = add_exception_handler(insn, ctx, dst);
> + if (ret)
> + return ret;
> + }
> break;
>
> default:
> @@ -3064,11 +3073,6 @@ bool bpf_jit_supports_insn(struct bpf_insn *insn, bool in_arena)
> if (!bpf_atomic_is_load_store(insn) &&
> !cpus_have_cap(ARM64_HAS_LSE_ATOMICS))
> return false;
> - break;
> - case BPF_LDX | BPF_MEMSX | BPF_B:
> - case BPF_LDX | BPF_MEMSX | BPF_H:
> - case BPF_LDX | BPF_MEMSX | BPF_W:
> - return false;
> }
> return true;
> }
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH bpf-next v3 2/3] bpf, arm64: Add support for signed arena loads
2025-09-20 10:37 ` Xu Kuohai
@ 2025-09-23 10:52 ` Puranjay Mohan
2025-09-25 13:10 ` Xu Kuohai
0 siblings, 1 reply; 12+ messages in thread
From: Puranjay Mohan @ 2025-09-23 10:52 UTC (permalink / raw)
To: Xu Kuohai
Cc: Puranjay Mohan, bpf, Alexei Starovoitov, Andrii Nakryiko,
Daniel Borkmann, Martin KaFai Lau, Eduard Zingerman, kkd,
kernel-team
On Sat, Sep 20, 2025 at 12:37 PM Xu Kuohai <xukuohai@huaweicloud.com> wrote:
>
> On 9/16/2025 12:28 AM, Puranjay Mohan wrote:
> > Add support for signed loads from arena which are internally converted
> > to loads with mode set BPF_PROBE_MEM32SX by the verifier. The
> > implementation is similar to BPF_PROBE_MEMSX and BPF_MEMSX but for
> > BPF_PROBE_MEM32SX, arena_vm_base is added to the src register to form
> > the address.
> >
> > Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
> > ---
> > arch/arm64/net/bpf_jit_comp.c | 30 +++++++++++++++++-------------
> > 1 file changed, 17 insertions(+), 13 deletions(-)
> >
> > diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
> > index f2b85a10add2..7233acec69ce 100644
> > --- a/arch/arm64/net/bpf_jit_comp.c
> > +++ b/arch/arm64/net/bpf_jit_comp.c
> > @@ -1133,12 +1133,14 @@ static int add_exception_handler(const struct bpf_insn *insn,
> > return 0;
> >
> > if (BPF_MODE(insn->code) != BPF_PROBE_MEM &&
> > - BPF_MODE(insn->code) != BPF_PROBE_MEMSX &&
> > - BPF_MODE(insn->code) != BPF_PROBE_MEM32 &&
> > - BPF_MODE(insn->code) != BPF_PROBE_ATOMIC)
> > + BPF_MODE(insn->code) != BPF_PROBE_MEMSX &&
> > + BPF_MODE(insn->code) != BPF_PROBE_MEM32 &&
> > + BPF_MODE(insn->code) != BPF_PROBE_MEM32SX &&
> > + BPF_MODE(insn->code) != BPF_PROBE_ATOMIC)
> > return 0;
> >
> > is_arena = (BPF_MODE(insn->code) == BPF_PROBE_MEM32) ||
> > + (BPF_MODE(insn->code) == BPF_PROBE_MEM32SX) ||
> > (BPF_MODE(insn->code) == BPF_PROBE_ATOMIC);
> >
> > if (!ctx->prog->aux->extable ||
> > @@ -1659,7 +1661,11 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
> > case BPF_LDX | BPF_PROBE_MEM32 | BPF_H:
> > case BPF_LDX | BPF_PROBE_MEM32 | BPF_W:
> > case BPF_LDX | BPF_PROBE_MEM32 | BPF_DW:
> > - if (BPF_MODE(insn->code) == BPF_PROBE_MEM32) {
> > + case BPF_LDX | BPF_PROBE_MEM32SX | BPF_B:
> > + case BPF_LDX | BPF_PROBE_MEM32SX | BPF_H:
> > + case BPF_LDX | BPF_PROBE_MEM32SX | BPF_W:
> > + if (BPF_MODE(insn->code) == BPF_PROBE_MEM32 ||
> > + BPF_MODE(insn->code) == BPF_PROBE_MEM32SX) {
> > emit(A64_ADD(1, tmp2, src, arena_vm_base), ctx);
> > src = tmp2;
> > }
> > @@ -1671,7 +1677,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
> > off_adj = off;
> > }
> > sign_extend = (BPF_MODE(insn->code) == BPF_MEMSX ||
> > - BPF_MODE(insn->code) == BPF_PROBE_MEMSX);
> > + BPF_MODE(insn->code) == BPF_PROBE_MEMSX ||
> > + BPF_MODE(insn->code) == BPF_PROBE_MEM32SX);
> > switch (BPF_SIZE(code)) {
> > case BPF_W:
> > if (is_lsi_offset(off_adj, 2)) {
> > @@ -1879,9 +1886,11 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
> > if (ret)
> > return ret;
> >
> > - ret = add_exception_handler(insn, ctx, dst);
> > - if (ret)
> > - return ret;
> > + if (BPF_MODE(insn->code) == BPF_PROBE_ATOMIC) {
>
> add_exception_handler already checked this condition, why add a check here?
>
If I don't check it here then add_exception_handler() will be called
even for BPF_ATOMIC (0xc0), earlier
add_exception_handler() would have rejected it but now
BPF_PROBE_MEM32SX is also defined as 0xc0, so
add_exception_handler() will confuse BPF_ATOMIC for BPF_PROBE_MEM32SX
and allow it, I felt it was better to
just add a check here.
Thanks,
Puranjay
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH bpf-next v3 2/3] bpf, arm64: Add support for signed arena loads
2025-09-23 10:52 ` Puranjay Mohan
@ 2025-09-25 13:10 ` Xu Kuohai
0 siblings, 0 replies; 12+ messages in thread
From: Xu Kuohai @ 2025-09-25 13:10 UTC (permalink / raw)
To: Puranjay Mohan
Cc: Puranjay Mohan, bpf, Alexei Starovoitov, Andrii Nakryiko,
Daniel Borkmann, Martin KaFai Lau, Eduard Zingerman, kkd,
kernel-team
On 9/23/2025 6:52 PM, Puranjay Mohan wrote:
> On Sat, Sep 20, 2025 at 12:37 PM Xu Kuohai <xukuohai@huaweicloud.com> wrote:
>>
>> On 9/16/2025 12:28 AM, Puranjay Mohan wrote:
>>> Add support for signed loads from arena which are internally converted
>>> to loads with mode set BPF_PROBE_MEM32SX by the verifier. The
>>> implementation is similar to BPF_PROBE_MEMSX and BPF_MEMSX but for
>>> BPF_PROBE_MEM32SX, arena_vm_base is added to the src register to form
>>> the address.
>>>
>>> Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
>>> ---
>>> arch/arm64/net/bpf_jit_comp.c | 30 +++++++++++++++++-------------
>>> 1 file changed, 17 insertions(+), 13 deletions(-)
>>>
>>> diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
>>> index f2b85a10add2..7233acec69ce 100644
>>> --- a/arch/arm64/net/bpf_jit_comp.c
>>> +++ b/arch/arm64/net/bpf_jit_comp.c
>>> @@ -1133,12 +1133,14 @@ static int add_exception_handler(const struct bpf_insn *insn,
>>> return 0;
>>>
>>> if (BPF_MODE(insn->code) != BPF_PROBE_MEM &&
>>> - BPF_MODE(insn->code) != BPF_PROBE_MEMSX &&
>>> - BPF_MODE(insn->code) != BPF_PROBE_MEM32 &&
>>> - BPF_MODE(insn->code) != BPF_PROBE_ATOMIC)
>>> + BPF_MODE(insn->code) != BPF_PROBE_MEMSX &&
>>> + BPF_MODE(insn->code) != BPF_PROBE_MEM32 &&
>>> + BPF_MODE(insn->code) != BPF_PROBE_MEM32SX &&
>>> + BPF_MODE(insn->code) != BPF_PROBE_ATOMIC)
>>> return 0;
>>>
>>> is_arena = (BPF_MODE(insn->code) == BPF_PROBE_MEM32) ||
>>> + (BPF_MODE(insn->code) == BPF_PROBE_MEM32SX) ||
>>> (BPF_MODE(insn->code) == BPF_PROBE_ATOMIC);
>>>
>>> if (!ctx->prog->aux->extable ||
>>> @@ -1659,7 +1661,11 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
>>> case BPF_LDX | BPF_PROBE_MEM32 | BPF_H:
>>> case BPF_LDX | BPF_PROBE_MEM32 | BPF_W:
>>> case BPF_LDX | BPF_PROBE_MEM32 | BPF_DW:
>>> - if (BPF_MODE(insn->code) == BPF_PROBE_MEM32) {
>>> + case BPF_LDX | BPF_PROBE_MEM32SX | BPF_B:
>>> + case BPF_LDX | BPF_PROBE_MEM32SX | BPF_H:
>>> + case BPF_LDX | BPF_PROBE_MEM32SX | BPF_W:
>>> + if (BPF_MODE(insn->code) == BPF_PROBE_MEM32 ||
>>> + BPF_MODE(insn->code) == BPF_PROBE_MEM32SX) {
>>> emit(A64_ADD(1, tmp2, src, arena_vm_base), ctx);
>>> src = tmp2;
>>> }
>>> @@ -1671,7 +1677,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
>>> off_adj = off;
>>> }
>>> sign_extend = (BPF_MODE(insn->code) == BPF_MEMSX ||
>>> - BPF_MODE(insn->code) == BPF_PROBE_MEMSX);
>>> + BPF_MODE(insn->code) == BPF_PROBE_MEMSX ||
>>> + BPF_MODE(insn->code) == BPF_PROBE_MEM32SX);
>>> switch (BPF_SIZE(code)) {
>>> case BPF_W:
>>> if (is_lsi_offset(off_adj, 2)) {
>>> @@ -1879,9 +1886,11 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
>>> if (ret)
>>> return ret;
>>>
>>> - ret = add_exception_handler(insn, ctx, dst);
>>> - if (ret)
>>> - return ret;
>>> + if (BPF_MODE(insn->code) == BPF_PROBE_ATOMIC) {
>>
>> add_exception_handler already checked this condition, why add a check here?
>>
>
> If I don't check it here then add_exception_handler() will be called
> even for BPF_ATOMIC (0xc0), earlier
> add_exception_handler() would have rejected it but now
> BPF_PROBE_MEM32SX is also defined as 0xc0, so
> add_exception_handler() will confuse BPF_ATOMIC for BPF_PROBE_MEM32SX
> and allow it, I felt it was better to
> just add a check here.
>
I see. Thanks for the explanation. I think it’s better to include this
explanation as a comment here.
Other than that, this patch looks good to me. Thanks.
> Thanks,
> Puranjay
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2025-09-25 13:10 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-09-15 16:28 [PATCH bpf-next v3 0/3] Signed loads from Arena Puranjay Mohan
2025-09-15 16:28 ` [PATCH bpf-next v3 1/3] bpf, x86: Add support for signed arena loads Puranjay Mohan
2025-09-17 1:05 ` Eduard Zingerman
2025-09-17 1:08 ` Kumar Kartikeya Dwivedi
2025-09-17 14:16 ` Puranjay Mohan
2025-09-15 16:28 ` [PATCH bpf-next v3 2/3] bpf, arm64: " Puranjay Mohan
2025-09-20 10:37 ` Xu Kuohai
2025-09-23 10:52 ` Puranjay Mohan
2025-09-25 13:10 ` Xu Kuohai
2025-09-15 16:28 ` [PATCH bpf-next v3 3/3] selftests: bpf: Add tests for signed loads from arena Puranjay Mohan
2025-09-17 1:03 ` [PATCH bpf-next v3 0/3] Signed loads from Arena Kumar Kartikeya Dwivedi
2025-09-17 3:41 ` Xu Kuohai
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.