* [RFC PATCH bpf-next v2 1/2] bpf: Support private stack for bpf progs
@ 2024-07-11 16:42 Yonghong Song
2024-07-11 16:42 ` [RFC PATCH bpf-next v2 2/2] [no_merge] selftests/bpf: Benchmark runtime performance with private stack Yonghong Song
0 siblings, 1 reply; 6+ messages in thread
From: Yonghong Song @ 2024-07-11 16:42 UTC (permalink / raw)
To: bpf
Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann, kernel-team,
Martin KaFai Lau
The main motivation for private stack comes from nested
scheduler in sched-ext from Tejun. The basic idea is that
- each cgroup will its own associated bpf program,
- bpf program with parent cgroup will call bpf programs
in immediate child cgroups.
Let us say we have the following cgroup hierarchy:
root_cg (prog0):
cg1 (prog1):
cg11 (prog11):
cg111 (prog111)
cg112 (prog112)
cg12 (prog12):
cg121 (prog121)
cg122 (prog122)
cg2 (prog2):
cg21 (prog21)
cg22 (prog22)
cg23 (prog23)
In the above example, prog0 will call a kfunc which will
call prog1 and prog2 to get sched info for cg1 and cg2 and
then the information is summarized and sent back to prog0.
Similarly, prog11 and prog12 will be invoked in the kfunc
and the result will be summarized and sent back to prog1, etc.
Currently, for each thread, the x86 kernel allocate 8KB stack.
The each bpf program (including its subprograms) has maximum
512B stack size to avoid potential stack overflow.
And nested bpf programs increase the risk of stack overflow.
To avoid potential stack overflow caused by bpf programs,
this patch implemented a private stack so bpf program stack
space is allocated dynamically when the program is jited.
Such private stack is applied to tracing programs like
kprobe/uprobe, perf_event, tracepoint, raw tracepoint and
tracing.
But more than one instance of the same bpf program may
run in the system. To make things simple, percpu private
stack is allocated for each program, so if the same program
is running on different cpus concurrently, we won't have
any issue. Note that the kernel already have logic to prevent
the recursion for the same bpf program on the same cpu
(kprobe, fentry, etc.).
The patch implemented a percpu private stack based approach
for x86 arch.
- The stack size will be 0 and any stack access is from
jit-time allocated percpu storage.
- In the beginning of jit, r9 is used to save percpu
private stack pointer.
- Each rbp in the bpf asm insn is replaced by r9.
- For each call, push r9 before the call and pop r9
after the call to preserve r9 value.
Compared to previous RFC patch [1], this patch added
some conditions to enable private stack, e.g., verifier
calculated stack size, prog type, etc. The new patch
also added a performance test to compare private stack
vs. no private stack.
The following are some code example to illustrate the idea
for selftest cgroup_skb_sk_lookup:
the existing code the private-stack approach code
endbr64 endbr64
nop DWORD PTR [rax+rax*1+0x0] nop DWORD PTR [rax+rax*1+0x0]
xchg ax,ax xchg ax,ax
push rbp push rbp
mov rbp,rsp mov rbp,rsp
endbr64 endbr64
sub rsp,0x68
push rbx push rbx
... ...
... mov r9d,0x8c1c860
... add r9,QWORD PTR gs:0x21a00
... ...
mov rdx,rbp mov rdx, r9
add rdx,0xffffffffffffffb4 rdx,0xffffffffffffffb4
... ...
mov ecx,0x28 mov ecx,0x28
push r9
call 0xffffffffe305e474 call 0xffffffffe305e524
pop r9
mov rdi,rax mov rdi,rax
... ...
movzx rdi,BYTE PTR [rbp-0x46] movzx rdi,BYTE PTR [r9-0x46]
... ...
So the number of insns is increased by 1 + num_of_calls * 2.
Here the number of calls are those calls in the final jited binary.
Comparing function call itself, the push/pop overhead should be
minimum in most common cases.
Question:
Note that I didn't add struct_ops bpf prgram in the list. We probably
should for the use case I mentioned in the above for nested scheduler.
But for tcp congestion control, there is no need to use private stack
since there is no nesting there. I guess I can add struct_ops for
sched-ext only once it is merged.
[1] https://lore.kernel.org/bpf/707970c5-6bba-450a-be08-adf24d8b9276@linux.dev/T/
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
arch/x86/net/bpf_jit_comp.c | 63 ++++++++++++++++++++++++++++++++++---
include/linux/bpf.h | 2 ++
kernel/bpf/core.c | 20 ++++++++++++
kernel/bpf/syscall.c | 1 +
4 files changed, 82 insertions(+), 4 deletions(-)
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index d25d81c8ecc0..60f5d86fb6aa 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -1309,6 +1309,22 @@ static void emit_shiftx(u8 **pprog, u32 dst_reg, u8 src_reg, bool is64, u8 op)
*pprog = prog;
}
+static void emit_private_frame_ptr(u8 **pprog, void *private_frame_ptr)
+{
+ u8 *prog = *pprog;
+
+ /* movabs r9, private_frame_ptr */
+ emit_mov_imm64(&prog, X86_REG_R9, (long) private_frame_ptr >> 32,
+ (u32) (long) private_frame_ptr);
+
+ /* add <r9>, gs:[<off>] */
+ EMIT2(0x65, 0x4c);
+ EMIT3(0x03, 0x0c, 0x25);
+ EMIT((u32)(unsigned long)&this_cpu_off, 4);
+
+ *pprog = prog;
+}
+
#define INSN_SZ_DIFF (((addrs[i] - addrs[i - 1]) - (prog - temp)))
/* mov rax, qword ptr [rbp - rounded_stack_depth - 8] */
@@ -1324,18 +1340,25 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
int insn_cnt = bpf_prog->len;
bool seen_exit = false;
u8 temp[BPF_MAX_INSN_SIZE + BPF_INSN_SAFETY];
+ u32 stack_depth = bpf_prog->aux->stack_depth;
+ void __percpu *private_frame_ptr = NULL;
u64 arena_vm_start, user_vm_start;
int i, excnt = 0;
int ilen, proglen = 0;
u8 *prog = temp;
int err;
+ if (bpf_prog->private_stack_ptr) {
+ private_frame_ptr = bpf_prog->private_stack_ptr + round_up(stack_depth, 8);
+ stack_depth = 0;
+ }
+
arena_vm_start = bpf_arena_get_kern_vm_start(bpf_prog->aux->arena);
user_vm_start = bpf_arena_get_user_vm_start(bpf_prog->aux->arena);
detect_reg_usage(insn, insn_cnt, callee_regs_used);
- emit_prologue(&prog, bpf_prog->aux->stack_depth,
+ emit_prologue(&prog, stack_depth,
bpf_prog_was_classic(bpf_prog), tail_call_reachable,
bpf_is_subprog(bpf_prog), bpf_prog->aux->exception_cb);
/* Exception callback will clobber callee regs for its own use, and
@@ -1357,6 +1380,9 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
emit_mov_imm64(&prog, X86_REG_R12,
arena_vm_start >> 32, (u32) arena_vm_start);
+ if (private_frame_ptr)
+ emit_private_frame_ptr(&prog, private_frame_ptr);
+
ilen = prog - temp;
if (rw_image)
memcpy(rw_image + proglen, temp, ilen);
@@ -1376,6 +1402,14 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
u8 *func;
int nops;
+ if (private_frame_ptr) {
+ if (src_reg == BPF_REG_FP)
+ src_reg = X86_REG_R9;
+
+ if (dst_reg == BPF_REG_FP)
+ dst_reg = X86_REG_R9;
+ }
+
switch (insn->code) {
/* ALU */
case BPF_ALU | BPF_ADD | BPF_X:
@@ -2007,6 +2041,7 @@ st: if (is_imm8(insn->off))
emit_mov_reg(&prog, is64, real_src_reg, BPF_REG_0);
/* Restore R0 after clobbering RAX */
emit_mov_reg(&prog, true, BPF_REG_0, BPF_REG_AX);
+
break;
}
@@ -2031,14 +2066,20 @@ st: if (is_imm8(insn->off))
func = (u8 *) __bpf_call_base + imm32;
if (tail_call_reachable) {
- RESTORE_TAIL_CALL_CNT(bpf_prog->aux->stack_depth);
+ RESTORE_TAIL_CALL_CNT(stack_depth);
ip += 7;
}
if (!imm32)
return -EINVAL;
+ if (private_frame_ptr) {
+ EMIT2(0x41, 0x51); /* push r9 */
+ ip += 2;
+ }
ip += x86_call_depth_emit_accounting(&prog, func, ip);
if (emit_call(&prog, func, ip))
return -EINVAL;
+ if (private_frame_ptr)
+ EMIT2(0x41, 0x59); /* pop r9 */
break;
}
@@ -2048,13 +2089,13 @@ st: if (is_imm8(insn->off))
&bpf_prog->aux->poke_tab[imm32 - 1],
&prog, image + addrs[i - 1],
callee_regs_used,
- bpf_prog->aux->stack_depth,
+ stack_depth,
ctx);
else
emit_bpf_tail_call_indirect(bpf_prog,
&prog,
callee_regs_used,
- bpf_prog->aux->stack_depth,
+ stack_depth,
image + addrs[i - 1],
ctx);
break;
@@ -3218,6 +3259,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
struct bpf_binary_header *rw_header = NULL;
struct bpf_binary_header *header = NULL;
+ void __percpu *private_stack_ptr = NULL;
struct bpf_prog *tmp, *orig_prog = prog;
struct x64_jit_data *jit_data;
int proglen, oldproglen = 0;
@@ -3284,6 +3326,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.cleanup_addr = proglen;
skip_init_addrs:
+ if (bpf_enable_private_stack(prog) && !prog->private_stack_ptr) {
+ private_stack_ptr = __alloc_percpu_gfp(prog->aux->stack_depth, 8, GFP_KERNEL);
+ if (!private_stack_ptr) {
+ prog = orig_prog;
+ goto out_addrs;
+ }
+ prog->private_stack_ptr = private_stack_ptr;
+ }
+
/*
* JITed image shrinks with every pass and the loop iterates
* until the image stops shrinking. Very large BPF programs
@@ -3309,6 +3360,10 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
prog->jited = 0;
prog->jited_len = 0;
}
+ if (private_stack_ptr) {
+ free_percpu(private_stack_ptr);
+ prog->private_stack_ptr = NULL;
+ }
goto out_addrs;
}
if (image) {
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 4f1d4a97b9d1..19a3f5355363 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -1563,6 +1563,7 @@ struct bpf_prog {
const struct bpf_insn *insn);
struct bpf_prog_aux *aux; /* Auxiliary fields */
struct sock_fprog_kern *orig_prog; /* Original BPF program */
+ void __percpu *private_stack_ptr;
/* Instructions for interpreter */
union {
DECLARE_FLEX_ARRAY(struct sock_filter, insns);
@@ -1819,6 +1820,7 @@ static inline void bpf_module_put(const void *data, struct module *owner)
module_put(owner);
}
int bpf_struct_ops_link_create(union bpf_attr *attr);
+bool bpf_enable_private_stack(struct bpf_prog *prog);
#ifdef CONFIG_NET
/* Define it here to avoid the use of forward declaration */
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 7ee62e38faf0..f69eb0c5fe03 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -2813,6 +2813,26 @@ void bpf_prog_free(struct bpf_prog *fp)
}
EXPORT_SYMBOL_GPL(bpf_prog_free);
+bool bpf_enable_private_stack(struct bpf_prog *prog)
+{
+ if (prog->aux->stack_depth <= 64)
+ return false;
+
+ switch (prog->aux->prog->type) {
+ case BPF_PROG_TYPE_KPROBE:
+ case BPF_PROG_TYPE_TRACEPOINT:
+ case BPF_PROG_TYPE_PERF_EVENT:
+ case BPF_PROG_TYPE_RAW_TRACEPOINT:
+ return true;
+ case BPF_PROG_TYPE_TRACING:
+ if (prog->expected_attach_type != BPF_TRACE_ITER)
+ return true;
+ fallthrough;
+ default:
+ return false;
+ }
+}
+
/* RNG for unprivileged user space with separated state from prandom_u32(). */
static DEFINE_PER_CPU(struct rnd_state, bpf_user_rnd_state);
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 869265852d51..89162ddb4747 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -2244,6 +2244,7 @@ static void __bpf_prog_put_rcu(struct rcu_head *rcu)
kvfree(aux->func_info);
kfree(aux->func_info_aux);
+ free_percpu(aux->prog->private_stack_ptr);
free_uid(aux->user);
security_bpf_prog_free(aux->prog);
bpf_prog_free(aux->prog);
--
2.43.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [RFC PATCH bpf-next v2 2/2] [no_merge] selftests/bpf: Benchmark runtime performance with private stack
2024-07-11 16:42 [RFC PATCH bpf-next v2 1/2] bpf: Support private stack for bpf progs Yonghong Song
@ 2024-07-11 16:42 ` Yonghong Song
2024-07-12 20:16 ` Alexei Starovoitov
0 siblings, 1 reply; 6+ messages in thread
From: Yonghong Song @ 2024-07-11 16:42 UTC (permalink / raw)
To: bpf
Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann, kernel-team,
Martin KaFai Lau
This patch intends to show some benchmark results comparing a bpf
program with vs. without private stack. The patch is not intended
to land since it hacks existing kernel interface in order to
do proper comparison.
The jited code without private stack:
0: f3 0f 1e fa endbr64
4: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
9: 66 90 xchg ax,ax
b: 55 push rbp
c: 48 89 e5 mov rbp,rsp
f: f3 0f 1e fa endbr64
13: 48 81 ec 60 00 00 00 sub rsp,0x60
1a: 48 bf 00 20 1a 00 00 movabs rdi,0xffffc900001a2000
21: c9 ff ff
24: 48 8b 77 00 mov rsi,QWORD PTR [rdi+0x0]
28: 48 83 c6 01 add rsi,0x1
2c: 48 89 77 00 mov QWORD PTR [rdi+0x0],rsi
30: 31 ff xor edi,edi
32: 48 89 7d f8 mov QWORD PTR [rbp-0x8],rdi
36: be 05 00 00 00 mov esi,0x5
3b: 89 75 f8 mov DWORD PTR [rbp-0x8],esi
3e: 48 89 7d f0 mov QWORD PTR [rbp-0x10],rdi
42: 48 89 7d e8 mov QWORD PTR [rbp-0x18],rdi
46: 48 89 7d e0 mov QWORD PTR [rbp-0x20],rdi
4a: 48 89 7d d8 mov QWORD PTR [rbp-0x28],rdi
4e: 48 89 7d d0 mov QWORD PTR [rbp-0x30],rdi
52: 48 89 7d c0 mov QWORD PTR [rbp-0x40],rdi
56: 48 89 7d c8 mov QWORD PTR [rbp-0x38],rdi
5a: 48 89 7d b8 mov QWORD PTR [rbp-0x48],rdi
5e: 48 89 7d b0 mov QWORD PTR [rbp-0x50],rdi
62: 48 89 7d a8 mov QWORD PTR [rbp-0x58],rdi
66: 48 89 7d a0 mov QWORD PTR [rbp-0x60],rdi
6a: bf 0a 00 00 00 mov edi,0xa
6f: 89 7d c0 mov DWORD PTR [rbp-0x40],edi
72: 48 89 ee mov rsi,rbp
75: 48 83 c6 d0 add rsi,0xffffffffffffffd0
79: 48 bf 00 f8 4b 0a 81 movabs rdi,0xffff88810a4bf800
80: 88 ff ff
83: e8 e0 1d 5f e1 call 0xffffffffe15f1e68
88: 48 85 c0 test rax,rax
8b: 74 04 je 0x91
8d: 48 83 c0 60 add rax,0x60
91: 48 85 c0 test rax,rax
94: 75 1f jne 0xb5
96: 48 89 ee mov rsi,rbp
99: 48 83 c6 d0 add rsi,0xffffffffffffffd0
9d: 48 89 ea mov rdx,rbp
a0: 48 83 c2 a0 add rdx,0xffffffffffffffa0
a4: 48 bf 00 f8 4b 0a 81 movabs rdi,0xffff88810a4bf800
ab: 88 ff ff
ae: 31 c9 xor ecx,ecx
b0: e8 73 d7 5e e1 call 0xffffffffe15ed828
b5: 48 89 ee mov rsi,rbp
b8: 48 83 c6 d0 add rsi,0xffffffffffffffd0
bc: 48 bf 00 f8 4b 0a 81 movabs rdi,0xffff88810a4bf800
c3: 88 ff ff
c6: e8 0d e0 5e e1 call 0xffffffffe15ee0d8
cb: 31 c0 xor eax,eax
cd: c9 leave
ce: c3 ret
The jited code with private stack:
0: f3 0f 1e fa endbr64
4: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
9: 66 90 xchg ax,ax
b: 55 push rbp
c: 48 89 e5 mov rbp,rsp
f: f3 0f 1e fa endbr64
13: 49 b9 40 af c1 08 7e movabs r9,0x607e08c1af40
1a: 60 00 00
1d: 65 4c 03 0c 25 00 1a add r9,QWORD PTR gs:0x21a00
24: 02 00
26: 48 bf 00 60 68 00 00 movabs rdi,0xffffc90000686000
2d: c9 ff ff
30: 48 8b 77 00 mov rsi,QWORD PTR [rdi+0x0]
34: 48 83 c6 01 add rsi,0x1
38: 48 89 77 00 mov QWORD PTR [rdi+0x0],rsi
3c: 31 ff xor edi,edi
3e: 49 89 79 f8 mov QWORD PTR [r9-0x8],rdi
42: be 05 00 00 00 mov esi,0x5
47: 41 89 71 f8 mov DWORD PTR [r9-0x8],esi
4b: 49 89 79 f0 mov QWORD PTR [r9-0x10],rdi
4f: 49 89 79 e8 mov QWORD PTR [r9-0x18],rdi
53: 49 89 79 e0 mov QWORD PTR [r9-0x20],rdi
57: 49 89 79 d8 mov QWORD PTR [r9-0x28],rdi
5b: 49 89 79 d0 mov QWORD PTR [r9-0x30],rdi
5f: 49 89 79 c0 mov QWORD PTR [r9-0x40],rdi
63: 49 89 79 c8 mov QWORD PTR [r9-0x38],rdi
67: 49 89 79 b8 mov QWORD PTR [r9-0x48],rdi
6b: 49 89 79 b0 mov QWORD PTR [r9-0x50],rdi
6f: 49 89 79 a8 mov QWORD PTR [r9-0x58],rdi
73: 49 89 79 a0 mov QWORD PTR [r9-0x60],rdi
77: bf 0a 00 00 00 mov edi,0xa
7c: 41 89 79 c0 mov DWORD PTR [r9-0x40],edi
80: 4c 89 ce mov rsi,r9
83: 48 83 c6 d0 add rsi,0xffffffffffffffd0
87: 48 bf 00 e0 22 0c 81 movabs rdi,0xffff88810c22e000
8e: 88 ff ff
91: 41 51 push r9
93: e8 10 1d 5f e1 call 0xffffffffe15f1da8
98: 41 59 pop r9
9a: 48 85 c0 test rax,rax
9d: 74 04 je 0xa3
9f: 48 83 c0 60 add rax,0x60
a3: 48 85 c0 test rax,rax
a6: 75 23 jne 0xcb
a8: 4c 89 ce mov rsi,r9
ab: 48 83 c6 d0 add rsi,0xffffffffffffffd0
af: 4c 89 ca mov rdx,r9
b2: 48 83 c2 a0 add rdx,0xffffffffffffffa0
b6: 48 bf 00 e0 22 0c 81 movabs rdi,0xffff88810c22e000
bd: 88 ff ff
c0: 31 c9 xor ecx,ecx
c2: 41 51 push r9
c4: e8 9f d6 5e e1 call 0xffffffffe15ed768
c9: 41 59 pop r9
cb: 4c 89 ce mov rsi,r9
ce: 48 83 c6 d0 add rsi,0xffffffffffffffd0
d2: 48 bf 00 e0 22 0c 81 movabs rdi,0xffff88810c22e000
d9: 88 ff ff
dc: 41 51 push r9
de: e8 35 df 5e e1 call 0xffffffffe15ee018
e3: 41 59 pop r9
e5: 31 c0 xor eax,eax
e7: c9 leave
e8: c3 ret
It is clear that the main overhead is the push/pop r9 for
three calls.
Five runs of the benchmarks:
[root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
no-private-stack: 0.662 ± 0.019M/s (drops 0.000 ± 0.000M/s)
private-stack: 0.673 ± 0.017M/s (drops 0.000 ± 0.000M/s)
[root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
no-private-stack: 0.684 ± 0.005M/s (drops 0.000 ± 0.000M/s)
private-stack: 0.676 ± 0.008M/s (drops 0.000 ± 0.000M/s)
[root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
no-private-stack: 0.673 ± 0.017M/s (drops 0.000 ± 0.000M/s)
private-stack: 0.683 ± 0.006M/s (drops 0.000 ± 0.000M/s)
[root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
no-private-stack: 0.680 ± 0.011M/s (drops 0.000 ± 0.000M/s)
private-stack: 0.626 ± 0.050M/s (drops 0.000 ± 0.000M/s)
[root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
no-private-stack: 0.686 ± 0.007M/s (drops 0.000 ± 0.000M/s)
private-stack: 0.683 ± 0.003M/s (drops 0.000 ± 0.000M/s)
The performance is very similar between private-stack and no-private-stack.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
include/linux/bpf.h | 3 +-
include/uapi/linux/bpf.h | 3 +
kernel/bpf/core.c | 2 +-
kernel/bpf/syscall.c | 4 +-
tools/include/uapi/linux/bpf.h | 3 +
tools/testing/selftests/bpf/Makefile | 2 +
tools/testing/selftests/bpf/bench.c | 6 +
.../bpf/benchs/bench_private_stack.c | 142 ++++++++++++++++++
.../bpf/benchs/run_bench_private_stack.sh | 9 ++
.../selftests/bpf/progs/private_stack.c | 40 +++++
10 files changed, 211 insertions(+), 3 deletions(-)
create mode 100644 tools/testing/selftests/bpf/benchs/bench_private_stack.c
create mode 100755 tools/testing/selftests/bpf/benchs/run_bench_private_stack.sh
create mode 100644 tools/testing/selftests/bpf/progs/private_stack.c
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 19a3f5355363..2f8708465c19 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -1551,7 +1551,8 @@ struct bpf_prog {
call_get_stack:1, /* Do we call bpf_get_stack() or bpf_get_stackid() */
call_get_func_ip:1, /* Do we call get_func_ip() */
tstamp_type_access:1, /* Accessed __sk_buff->tstamp_type */
- sleepable:1; /* BPF program is sleepable */
+ sleepable:1, /* BPF program is sleepable */
+ disable_private_stack:1; /* Disable private stack */
enum bpf_prog_type type; /* Type of BPF program */
enum bpf_attach_type expected_attach_type; /* For some prog types */
u32 len; /* Number of filter blocks */
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 35bcf52dbc65..98af8ea8a4d6 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -1409,6 +1409,9 @@ enum {
/* Do not translate kernel bpf_arena pointers to user pointers */
BPF_F_NO_USER_CONV = (1U << 18),
+
+/* Disable private stack */
+ BPF_F_DISABLE_PRIVATE_STACK = (1U << 19),
};
/* Flags for BPF_PROG_QUERY. */
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index f69eb0c5fe03..297e76a8f463 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -2815,7 +2815,7 @@ EXPORT_SYMBOL_GPL(bpf_prog_free);
bool bpf_enable_private_stack(struct bpf_prog *prog)
{
- if (prog->aux->stack_depth <= 64)
+ if (prog->disable_private_stack || prog->aux->stack_depth <= 64)
return false;
switch (prog->aux->prog->type) {
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 89162ddb4747..bb2b632c9c2c 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -2715,7 +2715,8 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr, u32 uattr_size)
BPF_F_XDP_HAS_FRAGS |
BPF_F_XDP_DEV_BOUND_ONLY |
BPF_F_TEST_REG_INVARIANTS |
- BPF_F_TOKEN_FD))
+ BPF_F_TOKEN_FD |
+ BPF_F_DISABLE_PRIVATE_STACK))
return -EINVAL;
bpf_prog_load_fixup_attach_type(attr);
@@ -2828,6 +2829,7 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr, u32 uattr_size)
prog->expected_attach_type = attr->expected_attach_type;
prog->sleepable = !!(attr->prog_flags & BPF_F_SLEEPABLE);
+ prog->disable_private_stack = !!(attr->prog_flags & BPF_F_DISABLE_PRIVATE_STACK);
prog->aux->attach_btf = attach_btf;
prog->aux->attach_btf_id = attr->attach_btf_id;
prog->aux->dst_prog = dst_prog;
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 35bcf52dbc65..98af8ea8a4d6 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -1409,6 +1409,9 @@ enum {
/* Do not translate kernel bpf_arena pointers to user pointers */
BPF_F_NO_USER_CONV = (1U << 18),
+
+/* Disable private stack */
+ BPF_F_DISABLE_PRIVATE_STACK = (1U << 19),
};
/* Flags for BPF_PROG_QUERY. */
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index dd49c1d23a60..44a6a43da71c 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -733,6 +733,7 @@ $(OUTPUT)/bench_local_storage_create.o: $(OUTPUT)/bench_local_storage_create.ske
$(OUTPUT)/bench_bpf_hashmap_lookup.o: $(OUTPUT)/bpf_hashmap_lookup.skel.h
$(OUTPUT)/bench_htab_mem.o: $(OUTPUT)/htab_mem_bench.skel.h
$(OUTPUT)/bench_bpf_crypto.o: $(OUTPUT)/crypto_bench.skel.h
+$(OUTPUT)/bench_private_stack.o: $(OUTPUT)/private_stack.skel.h
$(OUTPUT)/bench.o: bench.h testing_helpers.h $(BPFOBJ)
$(OUTPUT)/bench: LDLIBS += -lm
$(OUTPUT)/bench: $(OUTPUT)/bench.o \
@@ -753,6 +754,7 @@ $(OUTPUT)/bench: $(OUTPUT)/bench.o \
$(OUTPUT)/bench_local_storage_create.o \
$(OUTPUT)/bench_htab_mem.o \
$(OUTPUT)/bench_bpf_crypto.o \
+ $(OUTPUT)/bench_private_stack.o \
#
$(call msg,BINARY,,$@)
$(Q)$(CC) $(CFLAGS) $(LDFLAGS) $(filter %.a %.o,$^) $(LDLIBS) -o $@
diff --git a/tools/testing/selftests/bpf/bench.c b/tools/testing/selftests/bpf/bench.c
index 627b74ae041b..4f4867cd80f9 100644
--- a/tools/testing/selftests/bpf/bench.c
+++ b/tools/testing/selftests/bpf/bench.c
@@ -282,6 +282,7 @@ extern struct argp bench_local_storage_create_argp;
extern struct argp bench_htab_mem_argp;
extern struct argp bench_trigger_batch_argp;
extern struct argp bench_crypto_argp;
+extern struct argp bench_private_stack_argp;
static const struct argp_child bench_parsers[] = {
{ &bench_ringbufs_argp, 0, "Ring buffers benchmark", 0 },
@@ -296,6 +297,7 @@ static const struct argp_child bench_parsers[] = {
{ &bench_htab_mem_argp, 0, "hash map memory benchmark", 0 },
{ &bench_trigger_batch_argp, 0, "BPF triggering benchmark", 0 },
{ &bench_crypto_argp, 0, "bpf crypto benchmark", 0 },
+ { &bench_private_stack_argp, 0, "bpf private stack benchmark", 0 },
{},
};
@@ -542,6 +544,8 @@ extern const struct bench bench_local_storage_create;
extern const struct bench bench_htab_mem;
extern const struct bench bench_crypto_encrypt;
extern const struct bench bench_crypto_decrypt;
+extern const struct bench bench_no_private_stack;
+extern const struct bench bench_private_stack;
static const struct bench *benchs[] = {
&bench_count_global,
@@ -596,6 +600,8 @@ static const struct bench *benchs[] = {
&bench_htab_mem,
&bench_crypto_encrypt,
&bench_crypto_decrypt,
+ &bench_no_private_stack,
+ &bench_private_stack,
};
static void find_benchmark(void)
diff --git a/tools/testing/selftests/bpf/benchs/bench_private_stack.c b/tools/testing/selftests/bpf/benchs/bench_private_stack.c
new file mode 100644
index 000000000000..3d8aa695823e
--- /dev/null
+++ b/tools/testing/selftests/bpf/benchs/bench_private_stack.c
@@ -0,0 +1,142 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */
+
+#include <argp.h>
+#include "bench.h"
+#include "private_stack.skel.h"
+
+static struct ctx {
+ struct private_stack *skel;
+} ctx;
+
+static struct {
+ bool disable_private_stack;
+} args = {
+ .disable_private_stack = 0,
+};
+
+enum {
+ ARG_DISABLE_PRIVATE_STACK = 3000,
+};
+
+static const struct argp_option opts[] = {
+ { "disable-private-stack", ARG_DISABLE_PRIVATE_STACK, "DISABLE_PRIVATE_STACK",
+ 0, "Disable private stack" },
+ {},
+};
+
+static error_t private_stack_parse_arg(int key, char *arg, struct argp_state *state)
+{
+ long ret;
+
+ switch (key) {
+ case ARG_DISABLE_PRIVATE_STACK:
+ ret = strtoul(arg, NULL, 10);
+ if (ret != 1)
+ argp_usage(state);
+ args.disable_private_stack = 1;
+ break;
+ default:
+ return ARGP_ERR_UNKNOWN;
+ }
+
+ return 0;
+}
+
+const struct argp bench_private_stack_argp = {
+ .options = opts,
+ .parser = private_stack_parse_arg,
+};
+
+static void private_stack_validate(void)
+{
+ if (env.consumer_cnt != 0) {
+ fprintf(stderr,
+ "The private stack benchmarks do not support consumer\n");
+ exit(1);
+ }
+}
+
+static void common_setup(bool disable_private_stack)
+{
+ struct private_stack *skel;
+ struct bpf_link *link;
+ __u32 old_flags;
+ int err;
+
+ skel = private_stack__open();
+ if(!skel) {
+ fprintf(stderr, "failed to open skeleton\n");
+ exit(1);
+ }
+ ctx.skel = skel;
+
+ if (disable_private_stack) {
+ old_flags = bpf_program__flags(skel->progs.stack0);
+ bpf_program__set_flags(skel->progs.stack0, old_flags | BPF_F_DISABLE_PRIVATE_STACK);
+ }
+
+ err = private_stack__load(skel);
+ if (err) {
+ fprintf(stderr, "failed to load program\n");
+ exit(1);
+ }
+
+ link = bpf_program__attach(skel->progs.stack0);
+ if (!link) {
+ fprintf(stderr, "failed to attach program\n");
+ exit(1);
+ }
+}
+
+static void no_private_stack_setup(void)
+{
+ common_setup(true);
+}
+
+static void private_stack_setup(void)
+{
+ common_setup(false);
+}
+
+static void private_stack_measure(struct bench_res *res)
+{
+ struct private_stack *skel = ctx.skel;
+ unsigned long total_hits = 0;
+ static unsigned long last_hits;
+
+ total_hits = skel->bss->hits;
+ res->hits = total_hits - last_hits;
+ res->drops = 0;
+ res->false_hits = 0;
+ last_hits = total_hits;
+}
+
+static void *private_stack_producer(void *unused)
+{
+ while (true)
+ syscall(__NR_getpgid);
+ return NULL;
+}
+
+const struct bench bench_no_private_stack = {
+ .name = "no-private-stack",
+ .argp = &bench_private_stack_argp,
+ .validate = private_stack_validate,
+ .setup = no_private_stack_setup,
+ .producer_thread = private_stack_producer,
+ .measure = private_stack_measure,
+ .report_progress = hits_drops_report_progress,
+ .report_final = hits_drops_report_final,
+};
+
+const struct bench bench_private_stack = {
+ .name = "private-stack",
+ .argp = &bench_private_stack_argp,
+ .validate = private_stack_validate,
+ .setup = private_stack_setup,
+ .producer_thread = private_stack_producer,
+ .measure = private_stack_measure,
+ .report_progress = hits_drops_report_progress,
+ .report_final = hits_drops_report_final,
+};
diff --git a/tools/testing/selftests/bpf/benchs/run_bench_private_stack.sh b/tools/testing/selftests/bpf/benchs/run_bench_private_stack.sh
new file mode 100755
index 000000000000..e032353f7fa6
--- /dev/null
+++ b/tools/testing/selftests/bpf/benchs/run_bench_private_stack.sh
@@ -0,0 +1,9 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+source ./benchs/run_common.sh
+
+set -eufo pipefail
+
+summarize "no-private-stack: " "$($RUN_BENCH --disable-private-stack 1 no-private-stack)"
+summarize "private-stack: " "$($RUN_BENCH private-stack)"
diff --git a/tools/testing/selftests/bpf/progs/private_stack.c b/tools/testing/selftests/bpf/progs/private_stack.c
new file mode 100644
index 000000000000..3b062e5b27e9
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/private_stack.c
@@ -0,0 +1,40 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */
+#include <linux/types.h>
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+
+char _license[] SEC("license") = "GPL";
+
+struct data_t {
+ unsigned int d[12];
+};
+
+struct {
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __uint(max_entries, 10);
+ __type(key, struct data_t);
+ __type(value, struct data_t);
+} htab SEC(".maps");
+
+unsigned long hits = 0;
+
+SEC("tp/syscalls/sys_enter_getpgid")
+int stack0(void *ctx)
+{
+ struct data_t key = {}, value = {};
+ struct data_t *pvalue;
+
+ hits++;
+ key.d[10] = 5;
+ value.d[8] = 10;
+
+ pvalue = bpf_map_lookup_elem(&htab, &key);
+ if (!pvalue)
+ bpf_map_update_elem(&htab, &key, &value, 0);
+ bpf_map_delete_elem(&htab, &key);
+
+ return 0;
+}
+
--
2.43.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [RFC PATCH bpf-next v2 2/2] [no_merge] selftests/bpf: Benchmark runtime performance with private stack
2024-07-11 16:42 ` [RFC PATCH bpf-next v2 2/2] [no_merge] selftests/bpf: Benchmark runtime performance with private stack Yonghong Song
@ 2024-07-12 20:16 ` Alexei Starovoitov
2024-07-12 20:48 ` Yonghong Song
0 siblings, 1 reply; 6+ messages in thread
From: Alexei Starovoitov @ 2024-07-12 20:16 UTC (permalink / raw)
To: Yonghong Song
Cc: bpf, Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Kernel Team, Martin KaFai Lau
On Thu, Jul 11, 2024 at 9:42 AM Yonghong Song <yonghong.song@linux.dev> wrote:
>
>
> It is clear that the main overhead is the push/pop r9 for
> three calls.
>
> Five runs of the benchmarks:
>
> [root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
> no-private-stack: 0.662 ± 0.019M/s (drops 0.000 ± 0.000M/s)
> private-stack: 0.673 ± 0.017M/s (drops 0.000 ± 0.000M/s)
> [root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
> no-private-stack: 0.684 ± 0.005M/s (drops 0.000 ± 0.000M/s)
> private-stack: 0.676 ± 0.008M/s (drops 0.000 ± 0.000M/s)
> [root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
> no-private-stack: 0.673 ± 0.017M/s (drops 0.000 ± 0.000M/s)
> private-stack: 0.683 ± 0.006M/s (drops 0.000 ± 0.000M/s)
> [root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
> no-private-stack: 0.680 ± 0.011M/s (drops 0.000 ± 0.000M/s)
> private-stack: 0.626 ± 0.050M/s (drops 0.000 ± 0.000M/s)
> [root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
> no-private-stack: 0.686 ± 0.007M/s (drops 0.000 ± 0.000M/s)
> private-stack: 0.683 ± 0.003M/s (drops 0.000 ± 0.000M/s)
>
> The performance is very similar between private-stack and no-private-stack.
I'm not so sure.
What is the "perf report" before/after?
Are you sure that bench spends enough time inside the program itself?
By the look of it it seems that most of the time will be in hashmap
and syscall overhead.
You need that batch's one that uses for loop and attached to a helper.
See commit 7df4e597ea2c ("selftests/bpf: add batched, mostly in-kernel
BPF triggering benchmarks")
I think the next version doesn't need RFC tag. patch 1 lgtm.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [RFC PATCH bpf-next v2 2/2] [no_merge] selftests/bpf: Benchmark runtime performance with private stack
2024-07-12 20:16 ` Alexei Starovoitov
@ 2024-07-12 20:48 ` Yonghong Song
2024-07-12 21:47 ` Andrii Nakryiko
0 siblings, 1 reply; 6+ messages in thread
From: Yonghong Song @ 2024-07-12 20:48 UTC (permalink / raw)
To: Alexei Starovoitov
Cc: bpf, Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
Kernel Team, Martin KaFai Lau
On 7/12/24 1:16 PM, Alexei Starovoitov wrote:
> On Thu, Jul 11, 2024 at 9:42 AM Yonghong Song <yonghong.song@linux.dev> wrote:
>>
>> It is clear that the main overhead is the push/pop r9 for
>> three calls.
>>
>> Five runs of the benchmarks:
>>
>> [root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
>> no-private-stack: 0.662 ± 0.019M/s (drops 0.000 ± 0.000M/s)
>> private-stack: 0.673 ± 0.017M/s (drops 0.000 ± 0.000M/s)
>> [root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
>> no-private-stack: 0.684 ± 0.005M/s (drops 0.000 ± 0.000M/s)
>> private-stack: 0.676 ± 0.008M/s (drops 0.000 ± 0.000M/s)
>> [root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
>> no-private-stack: 0.673 ± 0.017M/s (drops 0.000 ± 0.000M/s)
>> private-stack: 0.683 ± 0.006M/s (drops 0.000 ± 0.000M/s)
>> [root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
>> no-private-stack: 0.680 ± 0.011M/s (drops 0.000 ± 0.000M/s)
>> private-stack: 0.626 ± 0.050M/s (drops 0.000 ± 0.000M/s)
>> [root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
>> no-private-stack: 0.686 ± 0.007M/s (drops 0.000 ± 0.000M/s)
>> private-stack: 0.683 ± 0.003M/s (drops 0.000 ± 0.000M/s)
>>
>> The performance is very similar between private-stack and no-private-stack.
> I'm not so sure.
> What is the "perf report" before/after?
> Are you sure that bench spends enough time inside the program itself?
> By the look of it it seems that most of the time will be in hashmap
> and syscall overhead.
>
> You need that batch's one that uses for loop and attached to a helper.
> See commit 7df4e597ea2c ("selftests/bpf: add batched, mostly in-kernel
> BPF triggering benchmarks")
Okay, I see. The current approach is one trigger, one prog run where
each prog run exercise 3 syscalls. I should add a loop to the bpf
program to make bpf program spends majority of time. Will do this
in the next revision, plus running 'perf report'.
>
> I think the next version doesn't need RFC tag. patch 1 lgtm.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [RFC PATCH bpf-next v2 2/2] [no_merge] selftests/bpf: Benchmark runtime performance with private stack
2024-07-12 20:48 ` Yonghong Song
@ 2024-07-12 21:47 ` Andrii Nakryiko
2024-07-12 23:42 ` Yonghong Song
0 siblings, 1 reply; 6+ messages in thread
From: Andrii Nakryiko @ 2024-07-12 21:47 UTC (permalink / raw)
To: Yonghong Song
Cc: Alexei Starovoitov, bpf, Alexei Starovoitov, Andrii Nakryiko,
Daniel Borkmann, Kernel Team, Martin KaFai Lau
On Fri, Jul 12, 2024 at 1:48 PM Yonghong Song <yonghong.song@linux.dev> wrote:
>
>
> On 7/12/24 1:16 PM, Alexei Starovoitov wrote:
> > On Thu, Jul 11, 2024 at 9:42 AM Yonghong Song <yonghong.song@linux.dev> wrote:
> >>
> >> It is clear that the main overhead is the push/pop r9 for
> >> three calls.
> >>
> >> Five runs of the benchmarks:
> >>
> >> [root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
> >> no-private-stack: 0.662 ± 0.019M/s (drops 0.000 ± 0.000M/s)
> >> private-stack: 0.673 ± 0.017M/s (drops 0.000 ± 0.000M/s)
> >> [root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
> >> no-private-stack: 0.684 ± 0.005M/s (drops 0.000 ± 0.000M/s)
> >> private-stack: 0.676 ± 0.008M/s (drops 0.000 ± 0.000M/s)
> >> [root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
> >> no-private-stack: 0.673 ± 0.017M/s (drops 0.000 ± 0.000M/s)
> >> private-stack: 0.683 ± 0.006M/s (drops 0.000 ± 0.000M/s)
> >> [root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
> >> no-private-stack: 0.680 ± 0.011M/s (drops 0.000 ± 0.000M/s)
> >> private-stack: 0.626 ± 0.050M/s (drops 0.000 ± 0.000M/s)
> >> [root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
> >> no-private-stack: 0.686 ± 0.007M/s (drops 0.000 ± 0.000M/s)
> >> private-stack: 0.683 ± 0.003M/s (drops 0.000 ± 0.000M/s)
> >>
> >> The performance is very similar between private-stack and no-private-stack.
> > I'm not so sure.
> > What is the "perf report" before/after?
> > Are you sure that bench spends enough time inside the program itself?
> > By the look of it it seems that most of the time will be in hashmap
> > and syscall overhead.
> >
> > You need that batch's one that uses for loop and attached to a helper.
> > See commit 7df4e597ea2c ("selftests/bpf: add batched, mostly in-kernel
> > BPF triggering benchmarks")
>
> Okay, I see. The current approach is one trigger, one prog run where
> each prog run exercise 3 syscalls. I should add a loop to the bpf
> program to make bpf program spends majority of time. Will do this
> in the next revision, plus running 'perf report'.
please also benchmark on real hardware, VM will not give reliable results
>
> >
> > I think the next version doesn't need RFC tag. patch 1 lgtm.
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [RFC PATCH bpf-next v2 2/2] [no_merge] selftests/bpf: Benchmark runtime performance with private stack
2024-07-12 21:47 ` Andrii Nakryiko
@ 2024-07-12 23:42 ` Yonghong Song
0 siblings, 0 replies; 6+ messages in thread
From: Yonghong Song @ 2024-07-12 23:42 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: Alexei Starovoitov, bpf, Alexei Starovoitov, Andrii Nakryiko,
Daniel Borkmann, Kernel Team, Martin KaFai Lau
On 7/12/24 2:47 PM, Andrii Nakryiko wrote:
> On Fri, Jul 12, 2024 at 1:48 PM Yonghong Song <yonghong.song@linux.dev> wrote:
>>
>> On 7/12/24 1:16 PM, Alexei Starovoitov wrote:
>>> On Thu, Jul 11, 2024 at 9:42 AM Yonghong Song <yonghong.song@linux.dev> wrote:
>>>> It is clear that the main overhead is the push/pop r9 for
>>>> three calls.
>>>>
>>>> Five runs of the benchmarks:
>>>>
>>>> [root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
>>>> no-private-stack: 0.662 ± 0.019M/s (drops 0.000 ± 0.000M/s)
>>>> private-stack: 0.673 ± 0.017M/s (drops 0.000 ± 0.000M/s)
>>>> [root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
>>>> no-private-stack: 0.684 ± 0.005M/s (drops 0.000 ± 0.000M/s)
>>>> private-stack: 0.676 ± 0.008M/s (drops 0.000 ± 0.000M/s)
>>>> [root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
>>>> no-private-stack: 0.673 ± 0.017M/s (drops 0.000 ± 0.000M/s)
>>>> private-stack: 0.683 ± 0.006M/s (drops 0.000 ± 0.000M/s)
>>>> [root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
>>>> no-private-stack: 0.680 ± 0.011M/s (drops 0.000 ± 0.000M/s)
>>>> private-stack: 0.626 ± 0.050M/s (drops 0.000 ± 0.000M/s)
>>>> [root@arch-fb-vm1 bpf]# ./benchs/run_bench_private_stack.sh
>>>> no-private-stack: 0.686 ± 0.007M/s (drops 0.000 ± 0.000M/s)
>>>> private-stack: 0.683 ± 0.003M/s (drops 0.000 ± 0.000M/s)
>>>>
>>>> The performance is very similar between private-stack and no-private-stack.
>>> I'm not so sure.
>>> What is the "perf report" before/after?
>>> Are you sure that bench spends enough time inside the program itself?
>>> By the look of it it seems that most of the time will be in hashmap
>>> and syscall overhead.
>>>
>>> You need that batch's one that uses for loop and attached to a helper.
>>> See commit 7df4e597ea2c ("selftests/bpf: add batched, mostly in-kernel
>>> BPF triggering benchmarks")
>> Okay, I see. The current approach is one trigger, one prog run where
>> each prog run exercise 3 syscalls. I should add a loop to the bpf
>> program to make bpf program spends majority of time. Will do this
>> in the next revision, plus running 'perf report'.
> please also benchmark on real hardware, VM will not give reliable results
Sure. Will do.
>
>>> I think the next version doesn't need RFC tag. patch 1 lgtm.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2024-07-12 23:42 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-07-11 16:42 [RFC PATCH bpf-next v2 1/2] bpf: Support private stack for bpf progs Yonghong Song
2024-07-11 16:42 ` [RFC PATCH bpf-next v2 2/2] [no_merge] selftests/bpf: Benchmark runtime performance with private stack Yonghong Song
2024-07-12 20:16 ` Alexei Starovoitov
2024-07-12 20:48 ` Yonghong Song
2024-07-12 21:47 ` Andrii Nakryiko
2024-07-12 23:42 ` Yonghong Song
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox