* [PATCH bpf-next 0/2] bpf,riscv: Add support for BPF Arena @ 2024-03-23 15:46 ` Puranjay Mohan 0 siblings, 0 replies; 16+ messages in thread From: Puranjay Mohan @ 2024-03-23 15:46 UTC (permalink / raw) To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, Björn Töpel, Luke Nelson, Xi Wang, Paul Walmsley, Palmer Dabbelt, Albert Ou, bpf, linux-riscv, linux-kernel Cc: puranjay12 This series adds the support for PROBE_MEM32 and bpf_addr_space_cast instructions to the RISCV BPF JIT. These two instructions allow the enablement of BPF Arena. All arena related selftests are passing: root@rv-tester:~/bpf# uname -p riscv64 root@rv-tester:~/bpf# ./test_progs -a "*arena*" #3/1 arena_htab/arena_htab_llvm:OK #3/2 arena_htab/arena_htab_asm:OK #3 arena_htab:OK #4/1 arena_list/arena_list_1:OK #4/2 arena_list/arena_list_1000:OK #4 arena_list:OK #434/1 verifier_arena/basic_alloc1:OK #434/2 verifier_arena/basic_alloc2:OK #434/3 verifier_arena/basic_alloc3:OK #434/4 verifier_arena/iter_maps1:OK #434/5 verifier_arena/iter_maps2:OK #434/6 verifier_arena/iter_maps3:OK #434 verifier_arena:OK Summary: 3/10 PASSED, 0 SKIPPED, 0 FAILED This needs the following commit in bpf/bpf.git to work correctly: https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/commit/?id=f7f5d1808b1b66935a24dd796dd1a0612ca9c147 Puranjay Mohan (2): bpf,riscv: Implement PROBE_MEM32 pseudo instructions bpf,riscv: Implement bpf_addr_space_cast instruction arch/riscv/net/bpf_jit.h | 2 + arch/riscv/net/bpf_jit_comp64.c | 208 +++++++++++++++++++++++++++++++- arch/riscv/net/bpf_jit_core.c | 3 + 3 files changed, 210 insertions(+), 3 deletions(-) -- 2.40.1 ^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH bpf-next 0/2] bpf,riscv: Add support for BPF Arena @ 2024-03-23 15:46 ` Puranjay Mohan 0 siblings, 0 replies; 16+ messages in thread From: Puranjay Mohan @ 2024-03-23 15:46 UTC (permalink / raw) To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, Björn Töpel, Luke Nelson, Xi Wang, Paul Walmsley, Palmer Dabbelt, Albert Ou, bpf, linux-riscv, linux-kernel Cc: puranjay12 This series adds the support for PROBE_MEM32 and bpf_addr_space_cast instructions to the RISCV BPF JIT. These two instructions allow the enablement of BPF Arena. All arena related selftests are passing: root@rv-tester:~/bpf# uname -p riscv64 root@rv-tester:~/bpf# ./test_progs -a "*arena*" #3/1 arena_htab/arena_htab_llvm:OK #3/2 arena_htab/arena_htab_asm:OK #3 arena_htab:OK #4/1 arena_list/arena_list_1:OK #4/2 arena_list/arena_list_1000:OK #4 arena_list:OK #434/1 verifier_arena/basic_alloc1:OK #434/2 verifier_arena/basic_alloc2:OK #434/3 verifier_arena/basic_alloc3:OK #434/4 verifier_arena/iter_maps1:OK #434/5 verifier_arena/iter_maps2:OK #434/6 verifier_arena/iter_maps3:OK #434 verifier_arena:OK Summary: 3/10 PASSED, 0 SKIPPED, 0 FAILED This needs the following commit in bpf/bpf.git to work correctly: https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/commit/?id=f7f5d1808b1b66935a24dd796dd1a0612ca9c147 Puranjay Mohan (2): bpf,riscv: Implement PROBE_MEM32 pseudo instructions bpf,riscv: Implement bpf_addr_space_cast instruction arch/riscv/net/bpf_jit.h | 2 + arch/riscv/net/bpf_jit_comp64.c | 208 +++++++++++++++++++++++++++++++- arch/riscv/net/bpf_jit_core.c | 3 + 3 files changed, 210 insertions(+), 3 deletions(-) -- 2.40.1 _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv ^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH bpf-next 1/2] bpf,riscv: Implement PROBE_MEM32 pseudo instructions 2024-03-23 15:46 ` Puranjay Mohan @ 2024-03-23 15:46 ` Puranjay Mohan -1 siblings, 0 replies; 16+ messages in thread From: Puranjay Mohan @ 2024-03-23 15:46 UTC (permalink / raw) To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, Björn Töpel, Luke Nelson, Xi Wang, Paul Walmsley, Palmer Dabbelt, Albert Ou, bpf, linux-riscv, linux-kernel Cc: puranjay12 Add support for [LDX | STX | ST], PROBE_MEM32, [B | H | W | DW] instructions. They are similar to PROBE_MEM instructions with the following differences: - PROBE_MEM32 supports store. - PROBE_MEM32 relies on the verifier to clear upper 32-bit of the src/dst register - PROBE_MEM32 adds 64-bit kern_vm_start address (which is stored in S11 in the prologue). Due to bpf_arena constructions such S11 + reg + off16 access is guaranteed to be within arena virtual range, so no address check at run-time. - S11 is a free callee-saved register, so it is used to store kern_vm_start - PROBE_MEM32 allows STX and ST. If they fault the store is a nop. When LDX faults the destination register is zeroed. To support these on riscv, we do tmp = S11 + src/dst reg and then use tmp2 as the new src/dst register. This allows us to reuse most of the code for normal [LDX | STX | ST]. Signed-off-by: Puranjay Mohan <puranjay12@gmail.com> --- arch/riscv/net/bpf_jit.h | 1 + arch/riscv/net/bpf_jit_comp64.c | 193 +++++++++++++++++++++++++++++++- arch/riscv/net/bpf_jit_core.c | 2 + 3 files changed, 193 insertions(+), 3 deletions(-) diff --git a/arch/riscv/net/bpf_jit.h b/arch/riscv/net/bpf_jit.h index f4b6b3b9edda..8a47da08dd9c 100644 --- a/arch/riscv/net/bpf_jit.h +++ b/arch/riscv/net/bpf_jit.h @@ -81,6 +81,7 @@ struct rv_jit_context { int nexentries; unsigned long flags; int stack_size; + u64 arena_vm_start; }; /* Convert from ninsns to bytes. */ diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c index aac190085472..f51b832eafb6 100644 --- a/arch/riscv/net/bpf_jit_comp64.c +++ b/arch/riscv/net/bpf_jit_comp64.c @@ -255,6 +255,10 @@ static void __build_epilogue(bool is_tail_call, struct rv_jit_context *ctx) emit_ld(RV_REG_S6, store_offset, RV_REG_SP, ctx); store_offset -= 8; } + if (ctx->arena_vm_start) { + emit_ld(RV_REG_S11, store_offset, RV_REG_SP, ctx); + store_offset -= 8; + } emit_addi(RV_REG_SP, RV_REG_SP, stack_adjust, ctx); /* Set return value. */ @@ -548,6 +552,7 @@ static void emit_atomic(u8 rd, u8 rs, s16 off, s32 imm, bool is64, #define BPF_FIXUP_OFFSET_MASK GENMASK(26, 0) #define BPF_FIXUP_REG_MASK GENMASK(31, 27) +#define DONT_CLEAR 16 /* RV_REG_A6 unused in BPF */ bool ex_handler_bpf(const struct exception_table_entry *ex, struct pt_regs *regs) @@ -555,7 +560,8 @@ bool ex_handler_bpf(const struct exception_table_entry *ex, off_t offset = FIELD_GET(BPF_FIXUP_OFFSET_MASK, ex->fixup); int regs_offset = FIELD_GET(BPF_FIXUP_REG_MASK, ex->fixup); - *(unsigned long *)((void *)regs + pt_regmap[regs_offset]) = 0; + if (regs_offset != DONT_CLEAR) + *(unsigned long *)((void *)regs + pt_regmap[regs_offset]) = 0; regs->epc = (unsigned long)&ex->fixup - offset; return true; @@ -572,7 +578,8 @@ static int add_exception_handler(const struct bpf_insn *insn, off_t fixup_offset; if (!ctx->insns || !ctx->ro_insns || !ctx->prog->aux->extable || - (BPF_MODE(insn->code) != BPF_PROBE_MEM && BPF_MODE(insn->code) != BPF_PROBE_MEMSX)) + (BPF_MODE(insn->code) != BPF_PROBE_MEM && BPF_MODE(insn->code) != BPF_PROBE_MEMSX && + BPF_MODE(insn->code) != BPF_PROBE_MEM32)) return 0; if (WARN_ON_ONCE(ctx->nexentries >= ctx->prog->aux->num_exentries)) @@ -622,6 +629,9 @@ static int add_exception_handler(const struct bpf_insn *insn, ex->insn = ins_offset; + if (BPF_CLASS(insn->code) != BPF_LDX) + dst_reg = DONT_CLEAR; + ex->fixup = FIELD_PREP(BPF_FIXUP_OFFSET_MASK, fixup_offset) | FIELD_PREP(BPF_FIXUP_REG_MASK, dst_reg); ex->type = EX_TYPE_BPF; @@ -1063,7 +1073,7 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, BPF_CLASS(insn->code) == BPF_JMP; int s, e, rvoff, ret, i = insn - ctx->prog->insnsi; struct bpf_prog_aux *aux = ctx->prog->aux; - u8 rd = -1, rs = -1, code = insn->code; + u8 rd = -1, rs = -1, code = insn->code, reg_arena_vm_start = RV_REG_S11; s16 off = insn->off; s32 imm = insn->imm; @@ -1523,6 +1533,11 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, case BPF_LDX | BPF_PROBE_MEMSX | BPF_B: case BPF_LDX | BPF_PROBE_MEMSX | BPF_H: case BPF_LDX | BPF_PROBE_MEMSX | BPF_W: + /* LDX | PROBE_MEM32: dst = *(unsigned size *)(src + S11 + off)*/ + case BPF_LDX | BPF_PROBE_MEM32 | BPF_B: + case BPF_LDX | BPF_PROBE_MEM32 | BPF_H: + case BPF_LDX | BPF_PROBE_MEM32 | BPF_W: + case BPF_LDX | BPF_PROBE_MEM32 | BPF_DW: { int insn_len, insns_start; bool sign_ext; @@ -1530,6 +1545,11 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, sign_ext = BPF_MODE(insn->code) == BPF_MEMSX || BPF_MODE(insn->code) == BPF_PROBE_MEMSX; + if (BPF_MODE(insn->code) == BPF_PROBE_MEM32) { + emit_add(RV_REG_T2, rs, reg_arena_vm_start, ctx); + rs = RV_REG_T2; + } + switch (BPF_SIZE(code)) { case BPF_B: if (is_12b_int(off)) { @@ -1666,6 +1686,87 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, emit_sd(RV_REG_T2, 0, RV_REG_T1, ctx); break; + case BPF_ST | BPF_PROBE_MEM32 | BPF_B: + case BPF_ST | BPF_PROBE_MEM32 | BPF_H: + case BPF_ST | BPF_PROBE_MEM32 | BPF_W: + case BPF_ST | BPF_PROBE_MEM32 | BPF_DW: + { + int insn_len, insns_start; + + emit_add(RV_REG_T3, rd, reg_arena_vm_start, ctx); + rd = RV_REG_T3; + + /* Load imm to a register then store it */ + emit_imm(RV_REG_T1, imm, ctx); + + switch (BPF_SIZE(code)) { + case BPF_B: + if (is_12b_int(off)) { + insns_start = ctx->ninsns; + emit(rv_sb(rd, off, RV_REG_T1), ctx); + insn_len = ctx->ninsns - insns_start; + break; + } + + emit_imm(RV_REG_T2, off, ctx); + emit_add(RV_REG_T2, RV_REG_T2, rd, ctx); + insns_start = ctx->ninsns; + emit(rv_sb(RV_REG_T2, 0, RV_REG_T1), ctx); + insn_len = ctx->ninsns - insns_start; + + break; + + case BPF_H: + if (is_12b_int(off)) { + insns_start = ctx->ninsns; + emit(rv_sh(rd, off, RV_REG_T1), ctx); + insn_len = ctx->ninsns - insns_start; + break; + } + + emit_imm(RV_REG_T2, off, ctx); + emit_add(RV_REG_T2, RV_REG_T2, rd, ctx); + insns_start = ctx->ninsns; + emit(rv_sh(RV_REG_T2, 0, RV_REG_T1), ctx); + insn_len = ctx->ninsns - insns_start; + break; + case BPF_W: + if (is_12b_int(off)) { + insns_start = ctx->ninsns; + emit_sw(rd, off, RV_REG_T1, ctx); + insn_len = ctx->ninsns - insns_start; + break; + } + + emit_imm(RV_REG_T2, off, ctx); + emit_add(RV_REG_T2, RV_REG_T2, rd, ctx); + insns_start = ctx->ninsns; + emit_sw(RV_REG_T2, 0, RV_REG_T1, ctx); + insn_len = ctx->ninsns - insns_start; + break; + case BPF_DW: + if (is_12b_int(off)) { + insns_start = ctx->ninsns; + emit_sd(rd, off, RV_REG_T1, ctx); + insn_len = ctx->ninsns - insns_start; + break; + } + + emit_imm(RV_REG_T2, off, ctx); + emit_add(RV_REG_T2, RV_REG_T2, rd, ctx); + insns_start = ctx->ninsns; + emit_sd(RV_REG_T2, 0, RV_REG_T1, ctx); + insn_len = ctx->ninsns - insns_start; + break; + } + + ret = add_exception_handler(insn, ctx, rd, insn_len); + if (ret) + return ret; + + break; + } + /* STX: *(size *)(dst + off) = src */ case BPF_STX | BPF_MEM | BPF_B: if (is_12b_int(off)) { @@ -1712,6 +1813,83 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, emit_atomic(rd, rs, off, imm, BPF_SIZE(code) == BPF_DW, ctx); break; + + case BPF_STX | BPF_PROBE_MEM32 | BPF_B: + case BPF_STX | BPF_PROBE_MEM32 | BPF_H: + case BPF_STX | BPF_PROBE_MEM32 | BPF_W: + case BPF_STX | BPF_PROBE_MEM32 | BPF_DW: + { + int insn_len, insns_start; + + emit_add(RV_REG_T2, rd, reg_arena_vm_start, ctx); + rd = RV_REG_T2; + + switch (BPF_SIZE(code)) { + case BPF_B: + if (is_12b_int(off)) { + insns_start = ctx->ninsns; + emit(rv_sb(rd, off, rs), ctx); + insn_len = ctx->ninsns - insns_start; + break; + } + + emit_imm(RV_REG_T1, off, ctx); + emit_add(RV_REG_T1, RV_REG_T1, rd, ctx); + insns_start = ctx->ninsns; + emit(rv_sb(RV_REG_T1, 0, rs), ctx); + insn_len = ctx->ninsns - insns_start; + break; + case BPF_H: + if (is_12b_int(off)) { + insns_start = ctx->ninsns; + emit(rv_sh(rd, off, rs), ctx); + insn_len = ctx->ninsns - insns_start; + break; + } + + emit_imm(RV_REG_T1, off, ctx); + emit_add(RV_REG_T1, RV_REG_T1, rd, ctx); + insns_start = ctx->ninsns; + emit(rv_sh(RV_REG_T1, 0, rs), ctx); + insn_len = ctx->ninsns - insns_start; + break; + case BPF_W: + if (is_12b_int(off)) { + insns_start = ctx->ninsns; + emit_sw(rd, off, rs, ctx); + insn_len = ctx->ninsns - insns_start; + break; + } + + emit_imm(RV_REG_T1, off, ctx); + emit_add(RV_REG_T1, RV_REG_T1, rd, ctx); + insns_start = ctx->ninsns; + emit_sw(RV_REG_T1, 0, rs, ctx); + insn_len = ctx->ninsns - insns_start; + break; + case BPF_DW: + if (is_12b_int(off)) { + insns_start = ctx->ninsns; + emit_sd(rd, off, rs, ctx); + insn_len = ctx->ninsns - insns_start; + break; + } + + emit_imm(RV_REG_T1, off, ctx); + emit_add(RV_REG_T1, RV_REG_T1, rd, ctx); + insns_start = ctx->ninsns; + emit_sd(RV_REG_T1, 0, rs, ctx); + insn_len = ctx->ninsns - insns_start; + break; + } + + ret = add_exception_handler(insn, ctx, rd, insn_len); + if (ret) + return ret; + + break; + } + default: pr_err("bpf-jit: unknown opcode %02x\n", code); return -EINVAL; @@ -1743,6 +1921,8 @@ void bpf_jit_build_prologue(struct rv_jit_context *ctx, bool is_subprog) stack_adjust += 8; if (seen_reg(RV_REG_S6, ctx)) stack_adjust += 8; + if (ctx->arena_vm_start) + stack_adjust += 8; stack_adjust = round_up(stack_adjust, 16); stack_adjust += bpf_stack_adjust; @@ -1794,6 +1974,10 @@ void bpf_jit_build_prologue(struct rv_jit_context *ctx, bool is_subprog) emit_sd(RV_REG_SP, store_offset, RV_REG_S6, ctx); store_offset -= 8; } + if (ctx->arena_vm_start) { + emit_sd(RV_REG_SP, store_offset, RV_REG_S11, ctx); + store_offset -= 8; + } emit_addi(RV_REG_FP, RV_REG_SP, stack_adjust, ctx); @@ -1807,6 +1991,9 @@ void bpf_jit_build_prologue(struct rv_jit_context *ctx, bool is_subprog) emit_mv(RV_REG_TCC_SAVED, RV_REG_TCC, ctx); ctx->stack_size = stack_adjust; + + if (ctx->arena_vm_start) + emit_imm(RV_REG_S11, ctx->arena_vm_start, ctx); } void bpf_jit_build_epilogue(struct rv_jit_context *ctx) diff --git a/arch/riscv/net/bpf_jit_core.c b/arch/riscv/net/bpf_jit_core.c index 6b3acac30c06..9b6696b1290a 100644 --- a/arch/riscv/net/bpf_jit_core.c +++ b/arch/riscv/net/bpf_jit_core.c @@ -50,6 +50,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) int pass = 0, prev_ninsns = 0, i; struct rv_jit_data *jit_data; struct rv_jit_context *ctx; + u64 arena_vm_start; if (!prog->jit_requested) return orig_prog; @@ -80,6 +81,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) goto skip_init_ctx; } + ctx->arena_vm_start = bpf_arena_get_kern_vm_start(prog->aux->arena); ctx->prog = prog; ctx->offset = kcalloc(prog->len, sizeof(int), GFP_KERNEL); if (!ctx->offset) { -- 2.40.1 ^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH bpf-next 1/2] bpf,riscv: Implement PROBE_MEM32 pseudo instructions @ 2024-03-23 15:46 ` Puranjay Mohan 0 siblings, 0 replies; 16+ messages in thread From: Puranjay Mohan @ 2024-03-23 15:46 UTC (permalink / raw) To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, Björn Töpel, Luke Nelson, Xi Wang, Paul Walmsley, Palmer Dabbelt, Albert Ou, bpf, linux-riscv, linux-kernel Cc: puranjay12 Add support for [LDX | STX | ST], PROBE_MEM32, [B | H | W | DW] instructions. They are similar to PROBE_MEM instructions with the following differences: - PROBE_MEM32 supports store. - PROBE_MEM32 relies on the verifier to clear upper 32-bit of the src/dst register - PROBE_MEM32 adds 64-bit kern_vm_start address (which is stored in S11 in the prologue). Due to bpf_arena constructions such S11 + reg + off16 access is guaranteed to be within arena virtual range, so no address check at run-time. - S11 is a free callee-saved register, so it is used to store kern_vm_start - PROBE_MEM32 allows STX and ST. If they fault the store is a nop. When LDX faults the destination register is zeroed. To support these on riscv, we do tmp = S11 + src/dst reg and then use tmp2 as the new src/dst register. This allows us to reuse most of the code for normal [LDX | STX | ST]. Signed-off-by: Puranjay Mohan <puranjay12@gmail.com> --- arch/riscv/net/bpf_jit.h | 1 + arch/riscv/net/bpf_jit_comp64.c | 193 +++++++++++++++++++++++++++++++- arch/riscv/net/bpf_jit_core.c | 2 + 3 files changed, 193 insertions(+), 3 deletions(-) diff --git a/arch/riscv/net/bpf_jit.h b/arch/riscv/net/bpf_jit.h index f4b6b3b9edda..8a47da08dd9c 100644 --- a/arch/riscv/net/bpf_jit.h +++ b/arch/riscv/net/bpf_jit.h @@ -81,6 +81,7 @@ struct rv_jit_context { int nexentries; unsigned long flags; int stack_size; + u64 arena_vm_start; }; /* Convert from ninsns to bytes. */ diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c index aac190085472..f51b832eafb6 100644 --- a/arch/riscv/net/bpf_jit_comp64.c +++ b/arch/riscv/net/bpf_jit_comp64.c @@ -255,6 +255,10 @@ static void __build_epilogue(bool is_tail_call, struct rv_jit_context *ctx) emit_ld(RV_REG_S6, store_offset, RV_REG_SP, ctx); store_offset -= 8; } + if (ctx->arena_vm_start) { + emit_ld(RV_REG_S11, store_offset, RV_REG_SP, ctx); + store_offset -= 8; + } emit_addi(RV_REG_SP, RV_REG_SP, stack_adjust, ctx); /* Set return value. */ @@ -548,6 +552,7 @@ static void emit_atomic(u8 rd, u8 rs, s16 off, s32 imm, bool is64, #define BPF_FIXUP_OFFSET_MASK GENMASK(26, 0) #define BPF_FIXUP_REG_MASK GENMASK(31, 27) +#define DONT_CLEAR 16 /* RV_REG_A6 unused in BPF */ bool ex_handler_bpf(const struct exception_table_entry *ex, struct pt_regs *regs) @@ -555,7 +560,8 @@ bool ex_handler_bpf(const struct exception_table_entry *ex, off_t offset = FIELD_GET(BPF_FIXUP_OFFSET_MASK, ex->fixup); int regs_offset = FIELD_GET(BPF_FIXUP_REG_MASK, ex->fixup); - *(unsigned long *)((void *)regs + pt_regmap[regs_offset]) = 0; + if (regs_offset != DONT_CLEAR) + *(unsigned long *)((void *)regs + pt_regmap[regs_offset]) = 0; regs->epc = (unsigned long)&ex->fixup - offset; return true; @@ -572,7 +578,8 @@ static int add_exception_handler(const struct bpf_insn *insn, off_t fixup_offset; if (!ctx->insns || !ctx->ro_insns || !ctx->prog->aux->extable || - (BPF_MODE(insn->code) != BPF_PROBE_MEM && BPF_MODE(insn->code) != BPF_PROBE_MEMSX)) + (BPF_MODE(insn->code) != BPF_PROBE_MEM && BPF_MODE(insn->code) != BPF_PROBE_MEMSX && + BPF_MODE(insn->code) != BPF_PROBE_MEM32)) return 0; if (WARN_ON_ONCE(ctx->nexentries >= ctx->prog->aux->num_exentries)) @@ -622,6 +629,9 @@ static int add_exception_handler(const struct bpf_insn *insn, ex->insn = ins_offset; + if (BPF_CLASS(insn->code) != BPF_LDX) + dst_reg = DONT_CLEAR; + ex->fixup = FIELD_PREP(BPF_FIXUP_OFFSET_MASK, fixup_offset) | FIELD_PREP(BPF_FIXUP_REG_MASK, dst_reg); ex->type = EX_TYPE_BPF; @@ -1063,7 +1073,7 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, BPF_CLASS(insn->code) == BPF_JMP; int s, e, rvoff, ret, i = insn - ctx->prog->insnsi; struct bpf_prog_aux *aux = ctx->prog->aux; - u8 rd = -1, rs = -1, code = insn->code; + u8 rd = -1, rs = -1, code = insn->code, reg_arena_vm_start = RV_REG_S11; s16 off = insn->off; s32 imm = insn->imm; @@ -1523,6 +1533,11 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, case BPF_LDX | BPF_PROBE_MEMSX | BPF_B: case BPF_LDX | BPF_PROBE_MEMSX | BPF_H: case BPF_LDX | BPF_PROBE_MEMSX | BPF_W: + /* LDX | PROBE_MEM32: dst = *(unsigned size *)(src + S11 + off)*/ + case BPF_LDX | BPF_PROBE_MEM32 | BPF_B: + case BPF_LDX | BPF_PROBE_MEM32 | BPF_H: + case BPF_LDX | BPF_PROBE_MEM32 | BPF_W: + case BPF_LDX | BPF_PROBE_MEM32 | BPF_DW: { int insn_len, insns_start; bool sign_ext; @@ -1530,6 +1545,11 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, sign_ext = BPF_MODE(insn->code) == BPF_MEMSX || BPF_MODE(insn->code) == BPF_PROBE_MEMSX; + if (BPF_MODE(insn->code) == BPF_PROBE_MEM32) { + emit_add(RV_REG_T2, rs, reg_arena_vm_start, ctx); + rs = RV_REG_T2; + } + switch (BPF_SIZE(code)) { case BPF_B: if (is_12b_int(off)) { @@ -1666,6 +1686,87 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, emit_sd(RV_REG_T2, 0, RV_REG_T1, ctx); break; + case BPF_ST | BPF_PROBE_MEM32 | BPF_B: + case BPF_ST | BPF_PROBE_MEM32 | BPF_H: + case BPF_ST | BPF_PROBE_MEM32 | BPF_W: + case BPF_ST | BPF_PROBE_MEM32 | BPF_DW: + { + int insn_len, insns_start; + + emit_add(RV_REG_T3, rd, reg_arena_vm_start, ctx); + rd = RV_REG_T3; + + /* Load imm to a register then store it */ + emit_imm(RV_REG_T1, imm, ctx); + + switch (BPF_SIZE(code)) { + case BPF_B: + if (is_12b_int(off)) { + insns_start = ctx->ninsns; + emit(rv_sb(rd, off, RV_REG_T1), ctx); + insn_len = ctx->ninsns - insns_start; + break; + } + + emit_imm(RV_REG_T2, off, ctx); + emit_add(RV_REG_T2, RV_REG_T2, rd, ctx); + insns_start = ctx->ninsns; + emit(rv_sb(RV_REG_T2, 0, RV_REG_T1), ctx); + insn_len = ctx->ninsns - insns_start; + + break; + + case BPF_H: + if (is_12b_int(off)) { + insns_start = ctx->ninsns; + emit(rv_sh(rd, off, RV_REG_T1), ctx); + insn_len = ctx->ninsns - insns_start; + break; + } + + emit_imm(RV_REG_T2, off, ctx); + emit_add(RV_REG_T2, RV_REG_T2, rd, ctx); + insns_start = ctx->ninsns; + emit(rv_sh(RV_REG_T2, 0, RV_REG_T1), ctx); + insn_len = ctx->ninsns - insns_start; + break; + case BPF_W: + if (is_12b_int(off)) { + insns_start = ctx->ninsns; + emit_sw(rd, off, RV_REG_T1, ctx); + insn_len = ctx->ninsns - insns_start; + break; + } + + emit_imm(RV_REG_T2, off, ctx); + emit_add(RV_REG_T2, RV_REG_T2, rd, ctx); + insns_start = ctx->ninsns; + emit_sw(RV_REG_T2, 0, RV_REG_T1, ctx); + insn_len = ctx->ninsns - insns_start; + break; + case BPF_DW: + if (is_12b_int(off)) { + insns_start = ctx->ninsns; + emit_sd(rd, off, RV_REG_T1, ctx); + insn_len = ctx->ninsns - insns_start; + break; + } + + emit_imm(RV_REG_T2, off, ctx); + emit_add(RV_REG_T2, RV_REG_T2, rd, ctx); + insns_start = ctx->ninsns; + emit_sd(RV_REG_T2, 0, RV_REG_T1, ctx); + insn_len = ctx->ninsns - insns_start; + break; + } + + ret = add_exception_handler(insn, ctx, rd, insn_len); + if (ret) + return ret; + + break; + } + /* STX: *(size *)(dst + off) = src */ case BPF_STX | BPF_MEM | BPF_B: if (is_12b_int(off)) { @@ -1712,6 +1813,83 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, emit_atomic(rd, rs, off, imm, BPF_SIZE(code) == BPF_DW, ctx); break; + + case BPF_STX | BPF_PROBE_MEM32 | BPF_B: + case BPF_STX | BPF_PROBE_MEM32 | BPF_H: + case BPF_STX | BPF_PROBE_MEM32 | BPF_W: + case BPF_STX | BPF_PROBE_MEM32 | BPF_DW: + { + int insn_len, insns_start; + + emit_add(RV_REG_T2, rd, reg_arena_vm_start, ctx); + rd = RV_REG_T2; + + switch (BPF_SIZE(code)) { + case BPF_B: + if (is_12b_int(off)) { + insns_start = ctx->ninsns; + emit(rv_sb(rd, off, rs), ctx); + insn_len = ctx->ninsns - insns_start; + break; + } + + emit_imm(RV_REG_T1, off, ctx); + emit_add(RV_REG_T1, RV_REG_T1, rd, ctx); + insns_start = ctx->ninsns; + emit(rv_sb(RV_REG_T1, 0, rs), ctx); + insn_len = ctx->ninsns - insns_start; + break; + case BPF_H: + if (is_12b_int(off)) { + insns_start = ctx->ninsns; + emit(rv_sh(rd, off, rs), ctx); + insn_len = ctx->ninsns - insns_start; + break; + } + + emit_imm(RV_REG_T1, off, ctx); + emit_add(RV_REG_T1, RV_REG_T1, rd, ctx); + insns_start = ctx->ninsns; + emit(rv_sh(RV_REG_T1, 0, rs), ctx); + insn_len = ctx->ninsns - insns_start; + break; + case BPF_W: + if (is_12b_int(off)) { + insns_start = ctx->ninsns; + emit_sw(rd, off, rs, ctx); + insn_len = ctx->ninsns - insns_start; + break; + } + + emit_imm(RV_REG_T1, off, ctx); + emit_add(RV_REG_T1, RV_REG_T1, rd, ctx); + insns_start = ctx->ninsns; + emit_sw(RV_REG_T1, 0, rs, ctx); + insn_len = ctx->ninsns - insns_start; + break; + case BPF_DW: + if (is_12b_int(off)) { + insns_start = ctx->ninsns; + emit_sd(rd, off, rs, ctx); + insn_len = ctx->ninsns - insns_start; + break; + } + + emit_imm(RV_REG_T1, off, ctx); + emit_add(RV_REG_T1, RV_REG_T1, rd, ctx); + insns_start = ctx->ninsns; + emit_sd(RV_REG_T1, 0, rs, ctx); + insn_len = ctx->ninsns - insns_start; + break; + } + + ret = add_exception_handler(insn, ctx, rd, insn_len); + if (ret) + return ret; + + break; + } + default: pr_err("bpf-jit: unknown opcode %02x\n", code); return -EINVAL; @@ -1743,6 +1921,8 @@ void bpf_jit_build_prologue(struct rv_jit_context *ctx, bool is_subprog) stack_adjust += 8; if (seen_reg(RV_REG_S6, ctx)) stack_adjust += 8; + if (ctx->arena_vm_start) + stack_adjust += 8; stack_adjust = round_up(stack_adjust, 16); stack_adjust += bpf_stack_adjust; @@ -1794,6 +1974,10 @@ void bpf_jit_build_prologue(struct rv_jit_context *ctx, bool is_subprog) emit_sd(RV_REG_SP, store_offset, RV_REG_S6, ctx); store_offset -= 8; } + if (ctx->arena_vm_start) { + emit_sd(RV_REG_SP, store_offset, RV_REG_S11, ctx); + store_offset -= 8; + } emit_addi(RV_REG_FP, RV_REG_SP, stack_adjust, ctx); @@ -1807,6 +1991,9 @@ void bpf_jit_build_prologue(struct rv_jit_context *ctx, bool is_subprog) emit_mv(RV_REG_TCC_SAVED, RV_REG_TCC, ctx); ctx->stack_size = stack_adjust; + + if (ctx->arena_vm_start) + emit_imm(RV_REG_S11, ctx->arena_vm_start, ctx); } void bpf_jit_build_epilogue(struct rv_jit_context *ctx) diff --git a/arch/riscv/net/bpf_jit_core.c b/arch/riscv/net/bpf_jit_core.c index 6b3acac30c06..9b6696b1290a 100644 --- a/arch/riscv/net/bpf_jit_core.c +++ b/arch/riscv/net/bpf_jit_core.c @@ -50,6 +50,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) int pass = 0, prev_ninsns = 0, i; struct rv_jit_data *jit_data; struct rv_jit_context *ctx; + u64 arena_vm_start; if (!prog->jit_requested) return orig_prog; @@ -80,6 +81,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) goto skip_init_ctx; } + ctx->arena_vm_start = bpf_arena_get_kern_vm_start(prog->aux->arena); ctx->prog = prog; ctx->offset = kcalloc(prog->len, sizeof(int), GFP_KERNEL); if (!ctx->offset) { -- 2.40.1 _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv ^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [PATCH bpf-next 1/2] bpf,riscv: Implement PROBE_MEM32 pseudo instructions 2024-03-23 15:46 ` Puranjay Mohan @ 2024-03-23 16:40 ` Pu Lehui -1 siblings, 0 replies; 16+ messages in thread From: Pu Lehui @ 2024-03-23 16:40 UTC (permalink / raw) To: Puranjay Mohan Cc: Puranjay Mohan, Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, Björn Töpel, Luke Nelson, Xi Wang, Paul Walmsley, Palmer Dabbelt, Albert Ou, bpf, linux-riscv, linux-kernel On 2024/3/23 23:46, Puranjay Mohan wrote: > Add support for [LDX | STX | ST], PROBE_MEM32, [B | H | W | DW] [snip] > > #define BPF_FIXUP_OFFSET_MASK GENMASK(26, 0) > #define BPF_FIXUP_REG_MASK GENMASK(31, 27) > +#define DONT_CLEAR 16 /* RV_REG_A6 unused in BPF */ This is a bit misleading. RV_REG_A6 is actually used in riscv64. Maybe "RV_REG_A6 unused in pt_regmap" or change to other register will be better. > > bool ex_handler_bpf(const struct exception_table_entry *ex, [snip] > > stack_adjust = round_up(stack_adjust, 16); > stack_adjust += bpf_stack_adjust; > @@ -1794,6 +1974,10 @@ void bpf_jit_build_prologue(struct rv_jit_context *ctx, bool is_subprog) > emit_sd(RV_REG_SP, store_offset, RV_REG_S6, ctx); > store_offset -= 8; > } > + if (ctx->arena_vm_start) { > + emit_sd(RV_REG_SP, store_offset, RV_REG_S11, ctx); > + store_offset -= 8; > + } I think it's fine to use s7 and keep the original dynamic stack code style. > > emit_addi(RV_REG_FP, RV_REG_SP, stack_adjust, ctx); > > @@ -1807,6 +1991,9 @@ void bpf_jit_build_prologue(struct rv_jit_context *ctx, bool is_subprog) > emit_mv(RV_REG_TCC_SAVED, RV_REG_TCC, ctx); > > ctx->stack_size = stack_adjust; > + > + if (ctx->arena_vm_start) > + emit_imm(RV_REG_S11, ctx->arena_vm_start, ctx); > } > > void bpf_jit_build_epilogue(struct rv_jit_context *ctx) > diff --git a/arch/riscv/net/bpf_jit_core.c b/arch/riscv/net/bpf_jit_core.c > index 6b3acac30c06..9b6696b1290a 100644 > --- a/arch/riscv/net/bpf_jit_core.c > +++ b/arch/riscv/net/bpf_jit_core.c > @@ -50,6 +50,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) > int pass = 0, prev_ninsns = 0, i; > struct rv_jit_data *jit_data; > struct rv_jit_context *ctx; > + u64 arena_vm_start; unused variable ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH bpf-next 1/2] bpf,riscv: Implement PROBE_MEM32 pseudo instructions @ 2024-03-23 16:40 ` Pu Lehui 0 siblings, 0 replies; 16+ messages in thread From: Pu Lehui @ 2024-03-23 16:40 UTC (permalink / raw) To: Puranjay Mohan Cc: Puranjay Mohan, Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, Björn Töpel, Luke Nelson, Xi Wang, Paul Walmsley, Palmer Dabbelt, Albert Ou, bpf, linux-riscv, linux-kernel On 2024/3/23 23:46, Puranjay Mohan wrote: > Add support for [LDX | STX | ST], PROBE_MEM32, [B | H | W | DW] [snip] > > #define BPF_FIXUP_OFFSET_MASK GENMASK(26, 0) > #define BPF_FIXUP_REG_MASK GENMASK(31, 27) > +#define DONT_CLEAR 16 /* RV_REG_A6 unused in BPF */ This is a bit misleading. RV_REG_A6 is actually used in riscv64. Maybe "RV_REG_A6 unused in pt_regmap" or change to other register will be better. > > bool ex_handler_bpf(const struct exception_table_entry *ex, [snip] > > stack_adjust = round_up(stack_adjust, 16); > stack_adjust += bpf_stack_adjust; > @@ -1794,6 +1974,10 @@ void bpf_jit_build_prologue(struct rv_jit_context *ctx, bool is_subprog) > emit_sd(RV_REG_SP, store_offset, RV_REG_S6, ctx); > store_offset -= 8; > } > + if (ctx->arena_vm_start) { > + emit_sd(RV_REG_SP, store_offset, RV_REG_S11, ctx); > + store_offset -= 8; > + } I think it's fine to use s7 and keep the original dynamic stack code style. > > emit_addi(RV_REG_FP, RV_REG_SP, stack_adjust, ctx); > > @@ -1807,6 +1991,9 @@ void bpf_jit_build_prologue(struct rv_jit_context *ctx, bool is_subprog) > emit_mv(RV_REG_TCC_SAVED, RV_REG_TCC, ctx); > > ctx->stack_size = stack_adjust; > + > + if (ctx->arena_vm_start) > + emit_imm(RV_REG_S11, ctx->arena_vm_start, ctx); > } > > void bpf_jit_build_epilogue(struct rv_jit_context *ctx) > diff --git a/arch/riscv/net/bpf_jit_core.c b/arch/riscv/net/bpf_jit_core.c > index 6b3acac30c06..9b6696b1290a 100644 > --- a/arch/riscv/net/bpf_jit_core.c > +++ b/arch/riscv/net/bpf_jit_core.c > @@ -50,6 +50,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) > int pass = 0, prev_ninsns = 0, i; > struct rv_jit_data *jit_data; > struct rv_jit_context *ctx; > + u64 arena_vm_start; unused variable _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv ^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH bpf-next 2/2] bpf,riscv: Implement bpf_addr_space_cast instruction 2024-03-23 15:46 ` Puranjay Mohan @ 2024-03-23 15:46 ` Puranjay Mohan -1 siblings, 0 replies; 16+ messages in thread From: Puranjay Mohan @ 2024-03-23 15:46 UTC (permalink / raw) To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, Björn Töpel, Luke Nelson, Xi Wang, Paul Walmsley, Palmer Dabbelt, Albert Ou, bpf, linux-riscv, linux-kernel Cc: puranjay12 LLVM generates bpf_addr_space_cast instruction while translating pointers between native (zero) address space and __attribute__((address_space(N))). The addr_space=0 is reserved as bpf_arena address space. rY = addr_space_cast(rX, 0, 1) is processed by the verifier and converted to normal 32-bit move: wX = wY rY = addr_space_cast(rX, 1, 0) has to be converted by JIT: Here I explain using symbolic language what the JIT is supposed to do: We have: src = [src_upper32][src_lower32] // 64 bit src kernel pointer uvm = [uvm_upper32][uvm_lower32] // 64 bit user_vm_start The JIT has to make the dst reg like following dst = [uvm_upper32][src_lower32] // if src_lower32 != 0 dst = [00000000000][00000000000] // if src_lower32 == 0 Signed-off-by: Puranjay Mohan <puranjay12@gmail.com> --- arch/riscv/net/bpf_jit.h | 1 + arch/riscv/net/bpf_jit_comp64.c | 15 +++++++++++++++ arch/riscv/net/bpf_jit_core.c | 1 + 3 files changed, 17 insertions(+) diff --git a/arch/riscv/net/bpf_jit.h b/arch/riscv/net/bpf_jit.h index 8a47da08dd9c..5fc374ed98ea 100644 --- a/arch/riscv/net/bpf_jit.h +++ b/arch/riscv/net/bpf_jit.h @@ -82,6 +82,7 @@ struct rv_jit_context { unsigned long flags; int stack_size; u64 arena_vm_start; + u64 user_vm_start; }; /* Convert from ninsns to bytes. */ diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c index f51b832eafb6..3c389e75cb96 100644 --- a/arch/riscv/net/bpf_jit_comp64.c +++ b/arch/riscv/net/bpf_jit_comp64.c @@ -1083,6 +1083,16 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, /* dst = src */ case BPF_ALU | BPF_MOV | BPF_X: case BPF_ALU64 | BPF_MOV | BPF_X: + if (BPF_CLASS(insn->code) == BPF_ALU64 && insn->off == BPF_ADDR_SPACE_CAST && + insn->imm == 1U << 16) { + emit_mv(RV_REG_T1, rs, ctx); + emit_zextw(RV_REG_T1, RV_REG_T1, ctx); + emit_imm(rd, (ctx->user_vm_start >> 32) << 32, ctx); + emit(rv_beq(RV_REG_T1, RV_REG_ZERO, 4), ctx); + emit_or(RV_REG_T1, rd, RV_REG_T1, ctx); + emit_mv(rd, RV_REG_T1, ctx); + break; + } if (imm == 1) { /* Special mov32 for zext */ emit_zextw(rd, rd, ctx); @@ -2010,3 +2020,8 @@ bool bpf_jit_supports_ptr_xchg(void) { return true; } + +bool bpf_jit_supports_arena(void) +{ + return true; +} diff --git a/arch/riscv/net/bpf_jit_core.c b/arch/riscv/net/bpf_jit_core.c index 9b6696b1290a..aaef1d0c7c46 100644 --- a/arch/riscv/net/bpf_jit_core.c +++ b/arch/riscv/net/bpf_jit_core.c @@ -82,6 +82,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) } ctx->arena_vm_start = bpf_arena_get_kern_vm_start(prog->aux->arena); + ctx->user_vm_start = bpf_arena_get_user_vm_start(prog->aux->arena); ctx->prog = prog; ctx->offset = kcalloc(prog->len, sizeof(int), GFP_KERNEL); if (!ctx->offset) { -- 2.40.1 ^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH bpf-next 2/2] bpf,riscv: Implement bpf_addr_space_cast instruction @ 2024-03-23 15:46 ` Puranjay Mohan 0 siblings, 0 replies; 16+ messages in thread From: Puranjay Mohan @ 2024-03-23 15:46 UTC (permalink / raw) To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, Björn Töpel, Luke Nelson, Xi Wang, Paul Walmsley, Palmer Dabbelt, Albert Ou, bpf, linux-riscv, linux-kernel Cc: puranjay12 LLVM generates bpf_addr_space_cast instruction while translating pointers between native (zero) address space and __attribute__((address_space(N))). The addr_space=0 is reserved as bpf_arena address space. rY = addr_space_cast(rX, 0, 1) is processed by the verifier and converted to normal 32-bit move: wX = wY rY = addr_space_cast(rX, 1, 0) has to be converted by JIT: Here I explain using symbolic language what the JIT is supposed to do: We have: src = [src_upper32][src_lower32] // 64 bit src kernel pointer uvm = [uvm_upper32][uvm_lower32] // 64 bit user_vm_start The JIT has to make the dst reg like following dst = [uvm_upper32][src_lower32] // if src_lower32 != 0 dst = [00000000000][00000000000] // if src_lower32 == 0 Signed-off-by: Puranjay Mohan <puranjay12@gmail.com> --- arch/riscv/net/bpf_jit.h | 1 + arch/riscv/net/bpf_jit_comp64.c | 15 +++++++++++++++ arch/riscv/net/bpf_jit_core.c | 1 + 3 files changed, 17 insertions(+) diff --git a/arch/riscv/net/bpf_jit.h b/arch/riscv/net/bpf_jit.h index 8a47da08dd9c..5fc374ed98ea 100644 --- a/arch/riscv/net/bpf_jit.h +++ b/arch/riscv/net/bpf_jit.h @@ -82,6 +82,7 @@ struct rv_jit_context { unsigned long flags; int stack_size; u64 arena_vm_start; + u64 user_vm_start; }; /* Convert from ninsns to bytes. */ diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c index f51b832eafb6..3c389e75cb96 100644 --- a/arch/riscv/net/bpf_jit_comp64.c +++ b/arch/riscv/net/bpf_jit_comp64.c @@ -1083,6 +1083,16 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, /* dst = src */ case BPF_ALU | BPF_MOV | BPF_X: case BPF_ALU64 | BPF_MOV | BPF_X: + if (BPF_CLASS(insn->code) == BPF_ALU64 && insn->off == BPF_ADDR_SPACE_CAST && + insn->imm == 1U << 16) { + emit_mv(RV_REG_T1, rs, ctx); + emit_zextw(RV_REG_T1, RV_REG_T1, ctx); + emit_imm(rd, (ctx->user_vm_start >> 32) << 32, ctx); + emit(rv_beq(RV_REG_T1, RV_REG_ZERO, 4), ctx); + emit_or(RV_REG_T1, rd, RV_REG_T1, ctx); + emit_mv(rd, RV_REG_T1, ctx); + break; + } if (imm == 1) { /* Special mov32 for zext */ emit_zextw(rd, rd, ctx); @@ -2010,3 +2020,8 @@ bool bpf_jit_supports_ptr_xchg(void) { return true; } + +bool bpf_jit_supports_arena(void) +{ + return true; +} diff --git a/arch/riscv/net/bpf_jit_core.c b/arch/riscv/net/bpf_jit_core.c index 9b6696b1290a..aaef1d0c7c46 100644 --- a/arch/riscv/net/bpf_jit_core.c +++ b/arch/riscv/net/bpf_jit_core.c @@ -82,6 +82,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) } ctx->arena_vm_start = bpf_arena_get_kern_vm_start(prog->aux->arena); + ctx->user_vm_start = bpf_arena_get_user_vm_start(prog->aux->arena); ctx->prog = prog; ctx->offset = kcalloc(prog->len, sizeof(int), GFP_KERNEL); if (!ctx->offset) { -- 2.40.1 _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv ^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [PATCH bpf-next 2/2] bpf,riscv: Implement bpf_addr_space_cast instruction 2024-03-23 15:46 ` Puranjay Mohan @ 2024-03-23 16:43 ` Pu Lehui -1 siblings, 0 replies; 16+ messages in thread From: Pu Lehui @ 2024-03-23 16:43 UTC (permalink / raw) To: Puranjay Mohan Cc: Puranjay Mohan, Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, Björn Töpel, Luke Nelson, Xi Wang, Paul Walmsley, Palmer Dabbelt, Albert Ou, bpf, linux-riscv, linux-kernel On 2024/3/23 23:46, Puranjay Mohan wrote: > LLVM generates bpf_addr_space_cast instruction while translating [snip] > > /* Convert from ninsns to bytes. */ > diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c > index f51b832eafb6..3c389e75cb96 100644 > --- a/arch/riscv/net/bpf_jit_comp64.c > +++ b/arch/riscv/net/bpf_jit_comp64.c > @@ -1083,6 +1083,16 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, > /* dst = src */ > case BPF_ALU | BPF_MOV | BPF_X: > case BPF_ALU64 | BPF_MOV | BPF_X: > + if (BPF_CLASS(insn->code) == BPF_ALU64 && insn->off == BPF_ADDR_SPACE_CAST && > + insn->imm == 1U << 16) { > + emit_mv(RV_REG_T1, rs, ctx); > + emit_zextw(RV_REG_T1, RV_REG_T1, ctx); combine mv and zextw will be better > + emit_imm(rd, (ctx->user_vm_start >> 32) << 32, ctx); > + emit(rv_beq(RV_REG_T1, RV_REG_ZERO, 4), ctx); > + emit_or(RV_REG_T1, rd, RV_REG_T1, ctx); > + emit_mv(rd, RV_REG_T1, ctx); ditto, but for or and mv ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH bpf-next 2/2] bpf,riscv: Implement bpf_addr_space_cast instruction @ 2024-03-23 16:43 ` Pu Lehui 0 siblings, 0 replies; 16+ messages in thread From: Pu Lehui @ 2024-03-23 16:43 UTC (permalink / raw) To: Puranjay Mohan Cc: Puranjay Mohan, Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, Björn Töpel, Luke Nelson, Xi Wang, Paul Walmsley, Palmer Dabbelt, Albert Ou, bpf, linux-riscv, linux-kernel On 2024/3/23 23:46, Puranjay Mohan wrote: > LLVM generates bpf_addr_space_cast instruction while translating [snip] > > /* Convert from ninsns to bytes. */ > diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c > index f51b832eafb6..3c389e75cb96 100644 > --- a/arch/riscv/net/bpf_jit_comp64.c > +++ b/arch/riscv/net/bpf_jit_comp64.c > @@ -1083,6 +1083,16 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, > /* dst = src */ > case BPF_ALU | BPF_MOV | BPF_X: > case BPF_ALU64 | BPF_MOV | BPF_X: > + if (BPF_CLASS(insn->code) == BPF_ALU64 && insn->off == BPF_ADDR_SPACE_CAST && > + insn->imm == 1U << 16) { > + emit_mv(RV_REG_T1, rs, ctx); > + emit_zextw(RV_REG_T1, RV_REG_T1, ctx); combine mv and zextw will be better > + emit_imm(rd, (ctx->user_vm_start >> 32) << 32, ctx); > + emit(rv_beq(RV_REG_T1, RV_REG_ZERO, 4), ctx); > + emit_or(RV_REG_T1, rd, RV_REG_T1, ctx); > + emit_mv(rd, RV_REG_T1, ctx); ditto, but for or and mv _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH bpf-next 2/2] bpf,riscv: Implement bpf_addr_space_cast instruction 2024-03-23 16:43 ` Pu Lehui @ 2024-03-23 16:49 ` Puranjay Mohan -1 siblings, 0 replies; 16+ messages in thread From: Puranjay Mohan @ 2024-03-23 16:49 UTC (permalink / raw) To: Pu Lehui Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, Björn Töpel, Luke Nelson, Xi Wang, Paul Walmsley, Palmer Dabbelt, Albert Ou, bpf, linux-riscv, linux-kernel Pu Lehui <pulehui@huawei.com> writes: > On 2024/3/23 23:46, Puranjay Mohan wrote: >> LLVM generates bpf_addr_space_cast instruction while translating > [snip] >> >> /* Convert from ninsns to bytes. */ >> diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c >> index f51b832eafb6..3c389e75cb96 100644 >> --- a/arch/riscv/net/bpf_jit_comp64.c >> +++ b/arch/riscv/net/bpf_jit_comp64.c >> @@ -1083,6 +1083,16 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, >> /* dst = src */ >> case BPF_ALU | BPF_MOV | BPF_X: >> case BPF_ALU64 | BPF_MOV | BPF_X: >> + if (BPF_CLASS(insn->code) == BPF_ALU64 && insn->off == BPF_ADDR_SPACE_CAST && >> + insn->imm == 1U << 16) { >> + emit_mv(RV_REG_T1, rs, ctx); > + emit_zextw(RV_REG_T1, RV_REG_T1, ctx); > combine mv and zextw will be better Do you suggest doing: emit_zextw(RV_REG_T1, rs, ctx); Will do it in next version. >> + emit_imm(rd, (ctx->user_vm_start >> 32) << 32, ctx); >> + emit(rv_beq(RV_REG_T1, RV_REG_ZERO, 4), ctx); >> + emit_or(RV_REG_T1, rd, RV_REG_T1, ctx); >> + emit_mv(rd, RV_REG_T1, ctx); > ditto, but for or and mv How would we combine or and mv? also, we have a beq above and in one case both or and mv should happen, but in other case only mv should happen. Thanks, Puranjay ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH bpf-next 2/2] bpf,riscv: Implement bpf_addr_space_cast instruction @ 2024-03-23 16:49 ` Puranjay Mohan 0 siblings, 0 replies; 16+ messages in thread From: Puranjay Mohan @ 2024-03-23 16:49 UTC (permalink / raw) To: Pu Lehui Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, Björn Töpel, Luke Nelson, Xi Wang, Paul Walmsley, Palmer Dabbelt, Albert Ou, bpf, linux-riscv, linux-kernel Pu Lehui <pulehui@huawei.com> writes: > On 2024/3/23 23:46, Puranjay Mohan wrote: >> LLVM generates bpf_addr_space_cast instruction while translating > [snip] >> >> /* Convert from ninsns to bytes. */ >> diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c >> index f51b832eafb6..3c389e75cb96 100644 >> --- a/arch/riscv/net/bpf_jit_comp64.c >> +++ b/arch/riscv/net/bpf_jit_comp64.c >> @@ -1083,6 +1083,16 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, >> /* dst = src */ >> case BPF_ALU | BPF_MOV | BPF_X: >> case BPF_ALU64 | BPF_MOV | BPF_X: >> + if (BPF_CLASS(insn->code) == BPF_ALU64 && insn->off == BPF_ADDR_SPACE_CAST && >> + insn->imm == 1U << 16) { >> + emit_mv(RV_REG_T1, rs, ctx); > + emit_zextw(RV_REG_T1, RV_REG_T1, ctx); > combine mv and zextw will be better Do you suggest doing: emit_zextw(RV_REG_T1, rs, ctx); Will do it in next version. >> + emit_imm(rd, (ctx->user_vm_start >> 32) << 32, ctx); >> + emit(rv_beq(RV_REG_T1, RV_REG_ZERO, 4), ctx); >> + emit_or(RV_REG_T1, rd, RV_REG_T1, ctx); >> + emit_mv(rd, RV_REG_T1, ctx); > ditto, but for or and mv How would we combine or and mv? also, we have a beq above and in one case both or and mv should happen, but in other case only mv should happen. Thanks, Puranjay _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH bpf-next 2/2] bpf,riscv: Implement bpf_addr_space_cast instruction 2024-03-23 16:49 ` Puranjay Mohan @ 2024-03-24 2:39 ` Pu Lehui -1 siblings, 0 replies; 16+ messages in thread From: Pu Lehui @ 2024-03-24 2:39 UTC (permalink / raw) To: Puranjay Mohan Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, Björn Töpel, Luke Nelson, Xi Wang, Paul Walmsley, Palmer Dabbelt, Albert Ou, bpf, linux-riscv, linux-kernel On 2024/3/24 0:49, Puranjay Mohan wrote: > Pu Lehui <pulehui@huawei.com> writes: > >> On 2024/3/23 23:46, Puranjay Mohan wrote: >>> LLVM generates bpf_addr_space_cast instruction while translating >> [snip] >>> >>> /* Convert from ninsns to bytes. */ >>> diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c >>> index f51b832eafb6..3c389e75cb96 100644 >>> --- a/arch/riscv/net/bpf_jit_comp64.c >>> +++ b/arch/riscv/net/bpf_jit_comp64.c >>> @@ -1083,6 +1083,16 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, >>> /* dst = src */ >>> case BPF_ALU | BPF_MOV | BPF_X: >>> case BPF_ALU64 | BPF_MOV | BPF_X: >>> + if (BPF_CLASS(insn->code) == BPF_ALU64 && insn->off == BPF_ADDR_SPACE_CAST && >>> + insn->imm == 1U << 16) { >>> + emit_mv(RV_REG_T1, rs, ctx); > + emit_zextw(RV_REG_T1, RV_REG_T1, ctx); >> combine mv and zextw will be better > > Do you suggest doing: > > emit_zextw(RV_REG_T1, rs, ctx); > > Will do it in next version. > >>> + emit_imm(rd, (ctx->user_vm_start >> 32) << 32, ctx); >>> + emit(rv_beq(RV_REG_T1, RV_REG_ZERO, 4), ctx); >>> + emit_or(RV_REG_T1, rd, RV_REG_T1, ctx); >>> + emit_mv(rd, RV_REG_T1, ctx); >> ditto, but for or and mv > > How would we combine or and mv? > also, we have a beq above and in one case both or and mv should happen, > but in other case only mv should happen. > Okay, another branch is that t1 is zero, but not rd. > Thanks, > Puranjay ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH bpf-next 2/2] bpf,riscv: Implement bpf_addr_space_cast instruction @ 2024-03-24 2:39 ` Pu Lehui 0 siblings, 0 replies; 16+ messages in thread From: Pu Lehui @ 2024-03-24 2:39 UTC (permalink / raw) To: Puranjay Mohan Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, Björn Töpel, Luke Nelson, Xi Wang, Paul Walmsley, Palmer Dabbelt, Albert Ou, bpf, linux-riscv, linux-kernel On 2024/3/24 0:49, Puranjay Mohan wrote: > Pu Lehui <pulehui@huawei.com> writes: > >> On 2024/3/23 23:46, Puranjay Mohan wrote: >>> LLVM generates bpf_addr_space_cast instruction while translating >> [snip] >>> >>> /* Convert from ninsns to bytes. */ >>> diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c >>> index f51b832eafb6..3c389e75cb96 100644 >>> --- a/arch/riscv/net/bpf_jit_comp64.c >>> +++ b/arch/riscv/net/bpf_jit_comp64.c >>> @@ -1083,6 +1083,16 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, >>> /* dst = src */ >>> case BPF_ALU | BPF_MOV | BPF_X: >>> case BPF_ALU64 | BPF_MOV | BPF_X: >>> + if (BPF_CLASS(insn->code) == BPF_ALU64 && insn->off == BPF_ADDR_SPACE_CAST && >>> + insn->imm == 1U << 16) { >>> + emit_mv(RV_REG_T1, rs, ctx); > + emit_zextw(RV_REG_T1, RV_REG_T1, ctx); >> combine mv and zextw will be better > > Do you suggest doing: > > emit_zextw(RV_REG_T1, rs, ctx); > > Will do it in next version. > >>> + emit_imm(rd, (ctx->user_vm_start >> 32) << 32, ctx); >>> + emit(rv_beq(RV_REG_T1, RV_REG_ZERO, 4), ctx); >>> + emit_or(RV_REG_T1, rd, RV_REG_T1, ctx); >>> + emit_mv(rd, RV_REG_T1, ctx); >> ditto, but for or and mv > > How would we combine or and mv? > also, we have a beq above and in one case both or and mv should happen, > but in other case only mv should happen. > Okay, another branch is that t1 is zero, but not rd. > Thanks, > Puranjay _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH bpf-next 2/2] bpf,riscv: Implement bpf_addr_space_cast instruction 2024-03-23 15:46 ` Puranjay Mohan @ 2024-03-23 18:37 ` Alexei Starovoitov -1 siblings, 0 replies; 16+ messages in thread From: Alexei Starovoitov @ 2024-03-23 18:37 UTC (permalink / raw) To: Puranjay Mohan Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, Björn Töpel, Luke Nelson, Xi Wang, Paul Walmsley, Palmer Dabbelt, Albert Ou, bpf, linux-riscv, LKML On Sat, Mar 23, 2024 at 8:47 AM Puranjay Mohan <puranjay12@gmail.com> wrote: > > LLVM generates bpf_addr_space_cast instruction while translating > pointers between native (zero) address space and > __attribute__((address_space(N))). The addr_space=0 is reserved as > bpf_arena address space. > > rY = addr_space_cast(rX, 0, 1) is processed by the verifier and > converted to normal 32-bit move: wX = wY > > rY = addr_space_cast(rX, 1, 0) has to be converted by JIT: > > Here I explain using symbolic language what the JIT is supposed to do: > We have: > src = [src_upper32][src_lower32] // 64 bit src kernel pointer > uvm = [uvm_upper32][uvm_lower32] // 64 bit user_vm_start This is a bit misleading. src_lower32 are always equal to uvm_lower32 and src_upper32 are either zero or uvm_upper32. Hence most of the time llvm doesn't generate this insn, since it knows that upper 32 bit are uvm_upper32. > The JIT has to make the dst reg like following > dst = [uvm_upper32][src_lower32] // if src_lower32 != 0 > dst = [00000000000][00000000000] // if src_lower32 == 0 > > Signed-off-by: Puranjay Mohan <puranjay12@gmail.com> > --- > arch/riscv/net/bpf_jit.h | 1 + > arch/riscv/net/bpf_jit_comp64.c | 15 +++++++++++++++ > arch/riscv/net/bpf_jit_core.c | 1 + > 3 files changed, 17 insertions(+) > > diff --git a/arch/riscv/net/bpf_jit.h b/arch/riscv/net/bpf_jit.h > index 8a47da08dd9c..5fc374ed98ea 100644 > --- a/arch/riscv/net/bpf_jit.h > +++ b/arch/riscv/net/bpf_jit.h > @@ -82,6 +82,7 @@ struct rv_jit_context { > unsigned long flags; > int stack_size; > u64 arena_vm_start; > + u64 user_vm_start; > }; > > /* Convert from ninsns to bytes. */ > diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c > index f51b832eafb6..3c389e75cb96 100644 > --- a/arch/riscv/net/bpf_jit_comp64.c > +++ b/arch/riscv/net/bpf_jit_comp64.c > @@ -1083,6 +1083,16 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, > /* dst = src */ > case BPF_ALU | BPF_MOV | BPF_X: > case BPF_ALU64 | BPF_MOV | BPF_X: > + if (BPF_CLASS(insn->code) == BPF_ALU64 && insn->off == BPF_ADDR_SPACE_CAST && > + insn->imm == 1U << 16) { Let's add a generic helper like insn_is_zext(), call it insn_is_cast_user() ? and use it in all JIT-s ? I should have added it right away when I did x86 part. Sorry. And a comment next to the helper that addr space cast 0->1 is for converting bpf arena pointers to user vma. Hence the name. Same comments for arm64 JIT arena support. pw-bot: cr ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH bpf-next 2/2] bpf,riscv: Implement bpf_addr_space_cast instruction @ 2024-03-23 18:37 ` Alexei Starovoitov 0 siblings, 0 replies; 16+ messages in thread From: Alexei Starovoitov @ 2024-03-23 18:37 UTC (permalink / raw) To: Puranjay Mohan Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, Björn Töpel, Luke Nelson, Xi Wang, Paul Walmsley, Palmer Dabbelt, Albert Ou, bpf, linux-riscv, LKML On Sat, Mar 23, 2024 at 8:47 AM Puranjay Mohan <puranjay12@gmail.com> wrote: > > LLVM generates bpf_addr_space_cast instruction while translating > pointers between native (zero) address space and > __attribute__((address_space(N))). The addr_space=0 is reserved as > bpf_arena address space. > > rY = addr_space_cast(rX, 0, 1) is processed by the verifier and > converted to normal 32-bit move: wX = wY > > rY = addr_space_cast(rX, 1, 0) has to be converted by JIT: > > Here I explain using symbolic language what the JIT is supposed to do: > We have: > src = [src_upper32][src_lower32] // 64 bit src kernel pointer > uvm = [uvm_upper32][uvm_lower32] // 64 bit user_vm_start This is a bit misleading. src_lower32 are always equal to uvm_lower32 and src_upper32 are either zero or uvm_upper32. Hence most of the time llvm doesn't generate this insn, since it knows that upper 32 bit are uvm_upper32. > The JIT has to make the dst reg like following > dst = [uvm_upper32][src_lower32] // if src_lower32 != 0 > dst = [00000000000][00000000000] // if src_lower32 == 0 > > Signed-off-by: Puranjay Mohan <puranjay12@gmail.com> > --- > arch/riscv/net/bpf_jit.h | 1 + > arch/riscv/net/bpf_jit_comp64.c | 15 +++++++++++++++ > arch/riscv/net/bpf_jit_core.c | 1 + > 3 files changed, 17 insertions(+) > > diff --git a/arch/riscv/net/bpf_jit.h b/arch/riscv/net/bpf_jit.h > index 8a47da08dd9c..5fc374ed98ea 100644 > --- a/arch/riscv/net/bpf_jit.h > +++ b/arch/riscv/net/bpf_jit.h > @@ -82,6 +82,7 @@ struct rv_jit_context { > unsigned long flags; > int stack_size; > u64 arena_vm_start; > + u64 user_vm_start; > }; > > /* Convert from ninsns to bytes. */ > diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c > index f51b832eafb6..3c389e75cb96 100644 > --- a/arch/riscv/net/bpf_jit_comp64.c > +++ b/arch/riscv/net/bpf_jit_comp64.c > @@ -1083,6 +1083,16 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, > /* dst = src */ > case BPF_ALU | BPF_MOV | BPF_X: > case BPF_ALU64 | BPF_MOV | BPF_X: > + if (BPF_CLASS(insn->code) == BPF_ALU64 && insn->off == BPF_ADDR_SPACE_CAST && > + insn->imm == 1U << 16) { Let's add a generic helper like insn_is_zext(), call it insn_is_cast_user() ? and use it in all JIT-s ? I should have added it right away when I did x86 part. Sorry. And a comment next to the helper that addr space cast 0->1 is for converting bpf arena pointers to user vma. Hence the name. Same comments for arm64 JIT arena support. pw-bot: cr _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv ^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2024-03-24 2:40 UTC | newest] Thread overview: 16+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2024-03-23 15:46 [PATCH bpf-next 0/2] bpf,riscv: Add support for BPF Arena Puranjay Mohan 2024-03-23 15:46 ` Puranjay Mohan 2024-03-23 15:46 ` [PATCH bpf-next 1/2] bpf,riscv: Implement PROBE_MEM32 pseudo instructions Puranjay Mohan 2024-03-23 15:46 ` Puranjay Mohan 2024-03-23 16:40 ` Pu Lehui 2024-03-23 16:40 ` Pu Lehui 2024-03-23 15:46 ` [PATCH bpf-next 2/2] bpf,riscv: Implement bpf_addr_space_cast instruction Puranjay Mohan 2024-03-23 15:46 ` Puranjay Mohan 2024-03-23 16:43 ` Pu Lehui 2024-03-23 16:43 ` Pu Lehui 2024-03-23 16:49 ` Puranjay Mohan 2024-03-23 16:49 ` Puranjay Mohan 2024-03-24 2:39 ` Pu Lehui 2024-03-24 2:39 ` Pu Lehui 2024-03-23 18:37 ` Alexei Starovoitov 2024-03-23 18:37 ` Alexei Starovoitov
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.