* [PATCH bpf-next v4 1/2] powerpc64/bpf: Support internal-only MOV instruction to resolve per-CPU addrs
2025-12-10 6:50 [PATCH bpf-next v4 0/2] powerpc64/bpf: Inline helper in powerpc JIT Saket Kumar Bhaskar
@ 2025-12-10 6:50 ` Saket Kumar Bhaskar
2025-12-10 6:50 ` [PATCH bpf-next v4 2/2] powerpc64/bpf: Inline bpf_get_smp_processor_id() and bpf_get_current_task/_btf() Saket Kumar Bhaskar
1 sibling, 0 replies; 3+ messages in thread
From: Saket Kumar Bhaskar @ 2025-12-10 6:50 UTC (permalink / raw)
To: bpf, linuxppc-dev, linux-kernel
Cc: hbathini, sachinpb, venkat88, andrii, eddyz87, ast, daniel,
martin.lau, song, yonghong.song, john.fastabend, kpsingh, sdf,
haoluo, jolsa, christophe.leroy, naveen, maddy, mpe, npiggin
With the introduction of commit 7bdbf7446305 ("bpf: add special
internal-only MOV instruction to resolve per-CPU addrs"),
a new BPF instruction BPF_MOV64_PERCPU_REG has been added to
resolve absolute addresses of per-CPU data from their per-CPU
offsets. This update requires enabling support for this
instruction in the powerpc JIT compiler.
As of commit 7a0268fa1a36 ("[PATCH] powerpc/64: per cpu data
optimisations"), the per-CPU data offset for the CPU is stored in
the paca.
To support this BPF instruction in the powerpc JIT, the following
powerpc instructions are emitted:
if (IS_ENABLED(CONFIG_SMP))
ld tmp1_reg, 48(13) //Load per-CPU data offset from paca(r13) in tmp1_reg.
add dst_reg, src_reg, tmp1_reg //Add the per cpu offset to the dst.
else if (src_reg != dst_reg)
mr dst_reg, src_reg //Move src_reg to dst_reg, if src_reg != dst_reg
To evaluate the performance improvements introduced by this change,
the benchmark described in [1] was employed.
Before Change:
glob-arr-inc : 41.580 ± 0.034M/s
arr-inc : 39.592 ± 0.055M/s
hash-inc : 25.873 ± 0.012M/s
After Change:
glob-arr-inc : 42.024 ± 0.049M/s
arr-inc : 55.447 ± 0.031M/s
hash-inc : 26.565 ± 0.014M/s
[1] https://github.com/anakryiko/linux/commit/8dec900975ef
Reviewed-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Saket Kumar Bhaskar <skb99@linux.ibm.com>
---
arch/powerpc/net/bpf_jit_comp.c | 5 +++++
arch/powerpc/net/bpf_jit_comp64.c | 10 ++++++++++
2 files changed, 15 insertions(+)
diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c
index 5e976730b2f5..d53e9cd7563f 100644
--- a/arch/powerpc/net/bpf_jit_comp.c
+++ b/arch/powerpc/net/bpf_jit_comp.c
@@ -466,6 +466,11 @@ bool bpf_jit_supports_insn(struct bpf_insn *insn, bool in_arena)
return true;
}
+bool bpf_jit_supports_percpu_insn(void)
+{
+ return IS_ENABLED(CONFIG_PPC64);
+}
+
void *arch_alloc_bpf_trampoline(unsigned int size)
{
return bpf_prog_pack_alloc(size, bpf_jit_fill_ill_insns);
diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c
index 1fe37128c876..37723ee9344e 100644
--- a/arch/powerpc/net/bpf_jit_comp64.c
+++ b/arch/powerpc/net/bpf_jit_comp64.c
@@ -918,6 +918,16 @@ int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, u32 *fimage, struct code
case BPF_ALU | BPF_MOV | BPF_X: /* (u32) dst = src */
case BPF_ALU64 | BPF_MOV | BPF_X: /* dst = src */
+ if (insn_is_mov_percpu_addr(&insn[i])) {
+ if (IS_ENABLED(CONFIG_SMP)) {
+ EMIT(PPC_RAW_LD(tmp1_reg, _R13, offsetof(struct paca_struct, data_offset)));
+ EMIT(PPC_RAW_ADD(dst_reg, src_reg, tmp1_reg));
+ } else if (src_reg != dst_reg) {
+ EMIT(PPC_RAW_MR(dst_reg, src_reg));
+ }
+ break;
+ }
+
if (insn_is_cast_user(&insn[i])) {
EMIT(PPC_RAW_RLDICL_DOT(tmp1_reg, src_reg, 0, 32));
PPC_LI64(dst_reg, (ctx->user_vm_start & 0xffffffff00000000UL));
--
2.51.0
^ permalink raw reply related [flat|nested] 3+ messages in thread* [PATCH bpf-next v4 2/2] powerpc64/bpf: Inline bpf_get_smp_processor_id() and bpf_get_current_task/_btf()
2025-12-10 6:50 [PATCH bpf-next v4 0/2] powerpc64/bpf: Inline helper in powerpc JIT Saket Kumar Bhaskar
2025-12-10 6:50 ` [PATCH bpf-next v4 1/2] powerpc64/bpf: Support internal-only MOV instruction to resolve per-CPU addrs Saket Kumar Bhaskar
@ 2025-12-10 6:50 ` Saket Kumar Bhaskar
1 sibling, 0 replies; 3+ messages in thread
From: Saket Kumar Bhaskar @ 2025-12-10 6:50 UTC (permalink / raw)
To: bpf, linuxppc-dev, linux-kernel
Cc: hbathini, sachinpb, venkat88, andrii, eddyz87, ast, daniel,
martin.lau, song, yonghong.song, john.fastabend, kpsingh, sdf,
haoluo, jolsa, christophe.leroy, naveen, maddy, mpe, npiggin
Inline the calls to bpf_get_smp_processor_id() and bpf_get_current_task/_btf()
in the powerpc bpf jit.
powerpc saves the Logical processor number (paca_index) and pointer
to current task (__current) in paca.
Here is how the powerpc JITed assembly changes after this commit:
Before:
cpu = bpf_get_smp_processor_id();
addis 12, 2, -517
addi 12, 12, -29456
mtctr 12
bctrl
mr 8, 3
After:
cpu = bpf_get_smp_processor_id();
lhz 8, 8(13)
To evaluate the performance improvements introduced by this change,
the benchmark described in [1] was employed.
+---------------+-------------------+-------------------+--------------+
| Name | Before | After | % change |
|---------------+-------------------+-------------------+--------------|
| glob-arr-inc | 40.701 ± 0.008M/s | 55.207 ± 0.021M/s | + 35.64% |
| arr-inc | 39.401 ± 0.007M/s | 56.275 ± 0.023M/s | + 42.42% |
| hash-inc | 24.944 ± 0.004M/s | 26.212 ± 0.003M/s | + 5.08% |
+---------------+-------------------+-------------------+--------------+
[1] https://github.com/anakryiko/linux/commit/8dec900975ef
Reviewed-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Saket Kumar Bhaskar <skb99@linux.ibm.com>
---
arch/powerpc/net/bpf_jit_comp.c | 12 ++++++++++++
arch/powerpc/net/bpf_jit_comp64.c | 11 +++++++++++
2 files changed, 23 insertions(+)
diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c
index d53e9cd7563f..b243ee205885 100644
--- a/arch/powerpc/net/bpf_jit_comp.c
+++ b/arch/powerpc/net/bpf_jit_comp.c
@@ -471,6 +471,18 @@ bool bpf_jit_supports_percpu_insn(void)
return IS_ENABLED(CONFIG_PPC64);
}
+bool bpf_jit_inlines_helper_call(s32 imm)
+{
+ switch (imm) {
+ case BPF_FUNC_get_smp_processor_id:
+ case BPF_FUNC_get_current_task:
+ case BPF_FUNC_get_current_task_btf:
+ return true;
+ default:
+ return false;
+ }
+}
+
void *arch_alloc_bpf_trampoline(unsigned int size)
{
return bpf_prog_pack_alloc(size, bpf_jit_fill_ill_insns);
diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c
index 37723ee9344e..6c827e7aa691 100644
--- a/arch/powerpc/net/bpf_jit_comp64.c
+++ b/arch/powerpc/net/bpf_jit_comp64.c
@@ -1400,6 +1400,17 @@ int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, u32 *fimage, struct code
case BPF_JMP | BPF_CALL:
ctx->seen |= SEEN_FUNC;
+ if (src_reg == bpf_to_ppc(BPF_REG_0)) {
+ if (imm == BPF_FUNC_get_smp_processor_id) {
+ EMIT(PPC_RAW_LHZ(src_reg, _R13, offsetof(struct paca_struct, paca_index)));
+ break;
+ } else if (imm == BPF_FUNC_get_current_task ||
+ imm == BPF_FUNC_get_current_task_btf) {
+ EMIT(PPC_RAW_LD(src_reg, _R13, offsetof(struct paca_struct, __current)));
+ break;
+ }
+ }
+
ret = bpf_jit_get_func_addr(fp, &insn[i], extra_pass,
&func_addr, &func_addr_fixed);
if (ret < 0)
--
2.51.0
^ permalink raw reply related [flat|nested] 3+ messages in thread