* [PATCH bpf-next v4 0/2] bpf, x86: inline bpf_get_current_task() for x86_64 @ 2026-01-12 10:45 Menglong Dong 2026-01-12 10:45 ` [PATCH bpf-next v4 1/2] " Menglong Dong 2026-01-12 10:45 ` [PATCH bpf-next v4 2/2] selftests/bpf: test the jited inline of bpf_get_current_task Menglong Dong 0 siblings, 2 replies; 7+ messages in thread From: Menglong Dong @ 2026-01-12 10:45 UTC (permalink / raw) To: ast, eddyz87 Cc: daniel, john.fastabend, andrii, martin.lau, song, yonghong.song, kpsingh, sdf, haoluo, jolsa, bpf, linux-kernel Inline bpf_get_current_task() and bpf_get_current_task_btf() for x86_64 to obtain better performance, and add the testcase for it. I'd prefer to the approach in V2. The code is still architecture specific, and now we implement it in the verifier.c, which is a little weird for me. What's more, it need 3 instructions with BPF_MOV64_PERCPU_REG(): mov rax, ¤t_task addq rax, gs:[this_cpu_off] movq rax, rax[0] which is 1 instruction in V2: movq rax, gs:[current_task] Maybe we can implement a BPF_LDX_MEM_PERCPU() instead: #define BPF_LDX_MEM_PERCPU(dst, size, variable) and use it with: BPF_LDX_MEM_PERCPU(BPF_REG_0, BPF_DW, current_task) which will generate the instruction: movq rax, gs:[current_task] Changes since v3: * handle the !CONFIG_SMP case * ignore the !CONFIG_SMP case in the testcase, as we enable CONFIG_SMP for x86_64 in the selftests Changes since v2: * implement it in the verifier with BPF_MOV64_PERCPU_REG() instead of in x86_64 JIT (Alexei). Changes since v1: * add the testcase * remove the usage of const_current_task Menglong Dong (2): bpf, x86: inline bpf_get_current_task() for x86_64 selftests/bpf: test the jited inline of bpf_get_current_task kernel/bpf/verifier.c | 29 +++++++++++++++ .../selftests/bpf/prog_tests/verifier.c | 2 ++ .../selftests/bpf/progs/verifier_jit_inline.c | 35 +++++++++++++++++++ 3 files changed, 66 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/verifier_jit_inline.c -- 2.52.0 ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH bpf-next v4 1/2] bpf, x86: inline bpf_get_current_task() for x86_64 2026-01-12 10:45 [PATCH bpf-next v4 0/2] bpf, x86: inline bpf_get_current_task() for x86_64 Menglong Dong @ 2026-01-12 10:45 ` Menglong Dong 2026-01-13 17:50 ` Alexei Starovoitov 2026-01-12 10:45 ` [PATCH bpf-next v4 2/2] selftests/bpf: test the jited inline of bpf_get_current_task Menglong Dong 1 sibling, 1 reply; 7+ messages in thread From: Menglong Dong @ 2026-01-12 10:45 UTC (permalink / raw) To: ast, eddyz87 Cc: daniel, john.fastabend, andrii, martin.lau, song, yonghong.song, kpsingh, sdf, haoluo, jolsa, bpf, linux-kernel Inline bpf_get_current_task() and bpf_get_current_task_btf() for x86_64 to obtain better performance. In !CONFIG_SMP case, the percpu variable is just a normal variable, and we can read the current_task directly. Signed-off-by: Menglong Dong <dongml2@chinatelecom.cn> --- v4: - handle the !CONFIG_SMP case v3: - implement it in the verifier with BPF_MOV64_PERCPU_REG() instead of in x86_64 JIT. --- kernel/bpf/verifier.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 3d44c5d06623..12e99171afd8 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -17688,6 +17688,8 @@ static bool verifier_inlines_helper_call(struct bpf_verifier_env *env, s32 imm) switch (imm) { #ifdef CONFIG_X86_64 case BPF_FUNC_get_smp_processor_id: + case BPF_FUNC_get_current_task_btf: + case BPF_FUNC_get_current_task: return env->prog->jit_requested && bpf_jit_supports_percpu_insn(); #endif default: @@ -23273,6 +23275,33 @@ static int do_misc_fixups(struct bpf_verifier_env *env) insn = new_prog->insnsi + i + delta; goto next_insn; } + + /* Implement bpf_get_current_task() and bpf_get_current_task_btf() inline. */ + if ((insn->imm == BPF_FUNC_get_current_task || insn->imm == BPF_FUNC_get_current_task_btf) && + verifier_inlines_helper_call(env, insn->imm)) { +#ifdef CONFIG_SMP + insn_buf[0] = BPF_MOV64_IMM(BPF_REG_0, (u32)(unsigned long)¤t_task); + insn_buf[1] = BPF_MOV64_PERCPU_REG(BPF_REG_0, BPF_REG_0); + insn_buf[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 0); +#else + struct bpf_insn ld_current_addr[2] = { + BPF_LD_IMM64(BPF_REG_0, (unsigned long)¤t_task) + }; + insn_buf[0] = ld_current_addr[0]; + insn_buf[1] = ld_current_addr[1]; + insn_buf[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 0); +#endif + cnt = 3; + + new_prog = bpf_patch_insn_data(env, i + delta, insn_buf, cnt); + if (!new_prog) + return -ENOMEM; + + delta += cnt - 1; + env->prog = prog = new_prog; + insn = new_prog->insnsi + i + delta; + goto next_insn; + } #endif /* Implement bpf_get_func_arg inline. */ if (prog_type == BPF_PROG_TYPE_TRACING && -- 2.52.0 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH bpf-next v4 1/2] bpf, x86: inline bpf_get_current_task() for x86_64 2026-01-12 10:45 ` [PATCH bpf-next v4 1/2] " Menglong Dong @ 2026-01-13 17:50 ` Alexei Starovoitov 2026-01-14 1:19 ` Menglong Dong 0 siblings, 1 reply; 7+ messages in thread From: Alexei Starovoitov @ 2026-01-13 17:50 UTC (permalink / raw) To: Menglong Dong Cc: Alexei Starovoitov, Eduard, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf, LKML On Mon, Jan 12, 2026 at 2:45 AM Menglong Dong <menglong8.dong@gmail.com> wrote: > > Inline bpf_get_current_task() and bpf_get_current_task_btf() for x86_64 > to obtain better performance. > > In !CONFIG_SMP case, the percpu variable is just a normal variable, and > we can read the current_task directly. > > Signed-off-by: Menglong Dong <dongml2@chinatelecom.cn> > --- > v4: > - handle the !CONFIG_SMP case > > v3: > - implement it in the verifier with BPF_MOV64_PERCPU_REG() instead of in > x86_64 JIT. > --- > kernel/bpf/verifier.c | 29 +++++++++++++++++++++++++++++ > 1 file changed, 29 insertions(+) > > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c > index 3d44c5d06623..12e99171afd8 100644 > --- a/kernel/bpf/verifier.c > +++ b/kernel/bpf/verifier.c > @@ -17688,6 +17688,8 @@ static bool verifier_inlines_helper_call(struct bpf_verifier_env *env, s32 imm) > switch (imm) { > #ifdef CONFIG_X86_64 > case BPF_FUNC_get_smp_processor_id: > + case BPF_FUNC_get_current_task_btf: > + case BPF_FUNC_get_current_task: > return env->prog->jit_requested && bpf_jit_supports_percpu_insn(); > #endif > default: > @@ -23273,6 +23275,33 @@ static int do_misc_fixups(struct bpf_verifier_env *env) > insn = new_prog->insnsi + i + delta; > goto next_insn; > } > + > + /* Implement bpf_get_current_task() and bpf_get_current_task_btf() inline. */ > + if ((insn->imm == BPF_FUNC_get_current_task || insn->imm == BPF_FUNC_get_current_task_btf) && > + verifier_inlines_helper_call(env, insn->imm)) { Though verifier_inlines_helper_call() gates this with CONFIG_X86_64, I think we still need explicit: #if defined(CONFIG_X86_64) && !defined(CONFIG_UML) just like we did for BPF_FUNC_get_smp_processor_id. Please check. I suspect UML will break without it. > +#ifdef CONFIG_SMP > + insn_buf[0] = BPF_MOV64_IMM(BPF_REG_0, (u32)(unsigned long)¤t_task); > + insn_buf[1] = BPF_MOV64_PERCPU_REG(BPF_REG_0, BPF_REG_0); > + insn_buf[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 0); > +#else > + struct bpf_insn ld_current_addr[2] = { > + BPF_LD_IMM64(BPF_REG_0, (unsigned long)¤t_task) > + }; > + insn_buf[0] = ld_current_addr[0]; > + insn_buf[1] = ld_current_addr[1]; > + insn_buf[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 0); > +#endif I wouldn't bother with !SMP. If we need to add On Mon, Jan 12, 2026 at 2:45 AM Menglong Dong <menglong8.dong@gmail.com> wrote: > > Inline bpf_get_current_task() and bpf_get_current_task_btf() for x86_64 > to obtain better performance. > > In !CONFIG_SMP case, the percpu variable is just a normal variable, and > we can read the current_task directly. > > Signed-off-by: Menglong Dong <dongml2@chinatelecom.cn> > --- > v4: > - handle the !CONFIG_SMP case > > v3: > - implement it in the verifier with BPF_MOV64_PERCPU_REG() instead of in > x86_64 JIT. > --- > kernel/bpf/verifier.c | 29 +++++++++++++++++++++++++++++ > 1 file changed, 29 insertions(+) > > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c > index 3d44c5d06623..12e99171afd8 100644 > --- a/kernel/bpf/verifier.c > +++ b/kernel/bpf/verifier.c > @@ -17688,6 +17688,8 @@ static bool verifier_inlines_helper_call(struct bpf_verifier_env *env, s32 imm) > switch (imm) { > #ifdef CONFIG_X86_64 > case BPF_FUNC_get_smp_processor_id: > + case BPF_FUNC_get_current_task_btf: > + case BPF_FUNC_get_current_task: > return env->prog->jit_requested && bpf_jit_supports_percpu_insn(); > #endif > default: > @@ -23273,6 +23275,33 @@ static int do_misc_fixups(struct bpf_verifier_env *env) > insn = new_prog->insnsi + i + delta; > goto next_insn; > } > + > + /* Implement bpf_get_current_task() and bpf_get_current_task_btf() inline. */ > + if ((insn->imm == BPF_FUNC_get_current_task || insn->imm == BPF_FUNC_get_current_task_btf) && > + verifier_inlines_helper_call(env, insn->imm)) { Though verifier_inlines_helper_call() gates this with CONFIG_X86_64, I think we still need explicit: #if defined(CONFIG_X86_64) && !defined(CONFIG_UML) just like we did for BPF_FUNC_get_smp_processor_id. Please check. I suspect UML will break without it. > +#ifdef CONFIG_SMP > + insn_buf[0] = BPF_MOV64_IMM(BPF_REG_0, (u32)(unsigned long)¤t_task); > + insn_buf[1] = BPF_MOV64_PERCPU_REG(BPF_REG_0, BPF_REG_0); > + insn_buf[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 0); > +#else > + struct bpf_insn ld_current_addr[2] = { > + BPF_LD_IMM64(BPF_REG_0, (unsigned long)¤t_task) > + }; > + insn_buf[0] = ld_current_addr[0]; > + insn_buf[1] = ld_current_addr[1]; > + insn_buf[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 0); > +#endif I wouldn't bother with !SMP. If we need to add On Mon, Jan 12, 2026 at 2:45 AM Menglong Dong <menglong8.dong@gmail.com> wrote: > > Inline bpf_get_current_task() and bpf_get_current_task_btf() for x86_64 > to obtain better performance. > > In !CONFIG_SMP case, the percpu variable is just a normal variable, and > we can read the current_task directly. > > Signed-off-by: Menglong Dong <dongml2@chinatelecom.cn> > --- > v4: > - handle the !CONFIG_SMP case > > v3: > - implement it in the verifier with BPF_MOV64_PERCPU_REG() instead of in > x86_64 JIT. > --- > kernel/bpf/verifier.c | 29 +++++++++++++++++++++++++++++ > 1 file changed, 29 insertions(+) > > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c > index 3d44c5d06623..12e99171afd8 100644 > --- a/kernel/bpf/verifier.c > +++ b/kernel/bpf/verifier.c > @@ -17688,6 +17688,8 @@ static bool verifier_inlines_helper_call(struct bpf_verifier_env *env, s32 imm) > switch (imm) { > #ifdef CONFIG_X86_64 > case BPF_FUNC_get_smp_processor_id: > + case BPF_FUNC_get_current_task_btf: > + case BPF_FUNC_get_current_task: > return env->prog->jit_requested && bpf_jit_supports_percpu_insn(); > #endif > default: > @@ -23273,6 +23275,33 @@ static int do_misc_fixups(struct bpf_verifier_env *env) > insn = new_prog->insnsi + i + delta; > goto next_insn; > } > + > + /* Implement bpf_get_current_task() and bpf_get_current_task_btf() inline. */ > + if ((insn->imm == BPF_FUNC_get_current_task || insn->imm == BPF_FUNC_get_current_task_btf) && > + verifier_inlines_helper_call(env, insn->imm)) { Though verifier_inlines_helper_call() gates this with CONFIG_X86_64, I think we still need explicit: #if defined(CONFIG_X86_64) && !defined(CONFIG_UML) just like we did for BPF_FUNC_get_smp_processor_id. Please check. I suspect UML will break without it. > +#ifdef CONFIG_SMP > + insn_buf[0] = BPF_MOV64_IMM(BPF_REG_0, (u32)(unsigned long)¤t_task); > + insn_buf[1] = BPF_MOV64_PERCPU_REG(BPF_REG_0, BPF_REG_0); > + insn_buf[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 0); > +#else > + struct bpf_insn ld_current_addr[2] = { > + BPF_LD_IMM64(BPF_REG_0, (unsigned long)¤t_task) > + }; > + insn_buf[0] = ld_current_addr[0]; > + insn_buf[1] = ld_current_addr[1]; > + insn_buf[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 0); > +#endif I wouldn't bother with !SMP. If we need to add defined(CONFIG_X86_64) && !defined(CONFIG_UML) I would add && defined(CONFIG_SMP) to it. pw-bot: cr ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH bpf-next v4 1/2] bpf, x86: inline bpf_get_current_task() for x86_64 2026-01-13 17:50 ` Alexei Starovoitov @ 2026-01-14 1:19 ` Menglong Dong 2026-01-14 1:24 ` Alexei Starovoitov 0 siblings, 1 reply; 7+ messages in thread From: Menglong Dong @ 2026-01-14 1:19 UTC (permalink / raw) To: Menglong Dong, Alexei Starovoitov Cc: Alexei Starovoitov, Eduard, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf, LKML On 2026/1/14 01:50 Alexei Starovoitov <alexei.starovoitov@gmail.com> write: > On Mon, Jan 12, 2026 at 2:45 AM Menglong Dong <menglong8.dong@gmail.com> wrote: > > > > Inline bpf_get_current_task() and bpf_get_current_task_btf() for x86_64 > > to obtain better performance. > > > > In !CONFIG_SMP case, the percpu variable is just a normal variable, and > > we can read the current_task directly. > > > > Signed-off-by: Menglong Dong <dongml2@chinatelecom.cn> > > --- > > v4: > > - handle the !CONFIG_SMP case > > > > v3: > > - implement it in the verifier with BPF_MOV64_PERCPU_REG() instead of in > > x86_64 JIT. > > --- > > kernel/bpf/verifier.c | 29 +++++++++++++++++++++++++++++ > > 1 file changed, 29 insertions(+) > > > > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c > > index 3d44c5d06623..12e99171afd8 100644 > > --- a/kernel/bpf/verifier.c > > +++ b/kernel/bpf/verifier.c > > @@ -17688,6 +17688,8 @@ static bool verifier_inlines_helper_call(struct bpf_verifier_env *env, s32 imm) > > switch (imm) { > > #ifdef CONFIG_X86_64 > > case BPF_FUNC_get_smp_processor_id: > > + case BPF_FUNC_get_current_task_btf: > > + case BPF_FUNC_get_current_task: > > return env->prog->jit_requested && bpf_jit_supports_percpu_insn(); > > #endif > > default: > > @@ -23273,6 +23275,33 @@ static int do_misc_fixups(struct bpf_verifier_env *env) > > insn = new_prog->insnsi + i + delta; > > goto next_insn; > > } > > + > > + /* Implement bpf_get_current_task() and bpf_get_current_task_btf() inline. */ > > + if ((insn->imm == BPF_FUNC_get_current_task || insn->imm == BPF_FUNC_get_current_task_btf) && > > + verifier_inlines_helper_call(env, insn->imm)) { > > Though verifier_inlines_helper_call() gates this with CONFIG_X86_64, > I think we still need explicit: > #if defined(CONFIG_X86_64) && !defined(CONFIG_UML) > > just like we did for BPF_FUNC_get_smp_processor_id. > Please check. I suspect UML will break without it. Do you mean that we need to use #if defined(CONFIG_X86_64) && !defined(CONFIG_UML) here? The whole code is already within it. You can have a look up: #if defined(CONFIG_X86_64) && !defined(CONFIG_UML) /* Implement bpf_get_smp_processor_id() inline. */ if (insn->imm == BPF_FUNC_get_smp_processor_id && verifier_inlines_helper_call(env, insn->imm)) { [......] /* Implement bpf_get_current_task() and bpf_get_current_task_btf() inline. */ if ((insn->imm == BPF_FUNC_get_current_task || insn->imm == BPF_FUNC_get_current_task_btf) && verifier_inlines_helper_call(env, insn->imm)) { [......] #endif > > > +#ifdef CONFIG_SMP > > + insn_buf[0] = BPF_MOV64_IMM(BPF_REG_0, (u32)(unsigned long)¤t_task); > > + insn_buf[1] = BPF_MOV64_PERCPU_REG(BPF_REG_0, BPF_REG_0); > > + insn_buf[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 0); > > +#else > > + struct bpf_insn ld_current_addr[2] = { > > + BPF_LD_IMM64(BPF_REG_0, (unsigned long)¤t_task) > > + }; > > + insn_buf[0] = ld_current_addr[0]; > > + insn_buf[1] = ld_current_addr[1]; > > + insn_buf[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 0); > > +#endif > > I wouldn't bother with !SMP. > If we need to add defined(CONFIG_X86_64) && !defined(CONFIG_UML) > I would add && defined(CONFIG_SMP) to it. OK, let's skip the !SMP case to make the code more clear. Thanks! Menglong Dong > > pw-bot: cr > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH bpf-next v4 1/2] bpf, x86: inline bpf_get_current_task() for x86_64 2026-01-14 1:19 ` Menglong Dong @ 2026-01-14 1:24 ` Alexei Starovoitov 2026-01-14 1:33 ` Menglong Dong 0 siblings, 1 reply; 7+ messages in thread From: Alexei Starovoitov @ 2026-01-14 1:24 UTC (permalink / raw) To: Menglong Dong Cc: Menglong Dong, Alexei Starovoitov, Eduard, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf, LKML On Tue, Jan 13, 2026 at 5:19 PM Menglong Dong <menglong.dong@linux.dev> wrote: > > On 2026/1/14 01:50 Alexei Starovoitov <alexei.starovoitov@gmail.com> write: > > On Mon, Jan 12, 2026 at 2:45 AM Menglong Dong <menglong8.dong@gmail.com> wrote: > > > > > > Inline bpf_get_current_task() and bpf_get_current_task_btf() for x86_64 > > > to obtain better performance. > > > > > > In !CONFIG_SMP case, the percpu variable is just a normal variable, and > > > we can read the current_task directly. > > > > > > Signed-off-by: Menglong Dong <dongml2@chinatelecom.cn> > > > --- > > > v4: > > > - handle the !CONFIG_SMP case > > > > > > v3: > > > - implement it in the verifier with BPF_MOV64_PERCPU_REG() instead of in > > > x86_64 JIT. > > > --- > > > kernel/bpf/verifier.c | 29 +++++++++++++++++++++++++++++ > > > 1 file changed, 29 insertions(+) > > > > > > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c > > > index 3d44c5d06623..12e99171afd8 100644 > > > --- a/kernel/bpf/verifier.c > > > +++ b/kernel/bpf/verifier.c > > > @@ -17688,6 +17688,8 @@ static bool verifier_inlines_helper_call(struct bpf_verifier_env *env, s32 imm) > > > switch (imm) { > > > #ifdef CONFIG_X86_64 > > > case BPF_FUNC_get_smp_processor_id: > > > + case BPF_FUNC_get_current_task_btf: > > > + case BPF_FUNC_get_current_task: > > > return env->prog->jit_requested && bpf_jit_supports_percpu_insn(); > > > #endif > > > default: > > > @@ -23273,6 +23275,33 @@ static int do_misc_fixups(struct bpf_verifier_env *env) > > > insn = new_prog->insnsi + i + delta; > > > goto next_insn; > > > } > > > + > > > + /* Implement bpf_get_current_task() and bpf_get_current_task_btf() inline. */ > > > + if ((insn->imm == BPF_FUNC_get_current_task || insn->imm == BPF_FUNC_get_current_task_btf) && > > > + verifier_inlines_helper_call(env, insn->imm)) { > > > > Though verifier_inlines_helper_call() gates this with CONFIG_X86_64, > > I think we still need explicit: > > #if defined(CONFIG_X86_64) && !defined(CONFIG_UML) > > > > just like we did for BPF_FUNC_get_smp_processor_id. > > Please check. I suspect UML will break without it. > > Do you mean that we need to use > #if defined(CONFIG_X86_64) && !defined(CONFIG_UML) > here? > > The whole code is already within it. You can have a look up: > > #if defined(CONFIG_X86_64) && !defined(CONFIG_UML) > /* Implement bpf_get_smp_processor_id() inline. */ > if (insn->imm == BPF_FUNC_get_smp_processor_id && > verifier_inlines_helper_call(env, insn->imm)) { > [......] > /* Implement bpf_get_current_task() and bpf_get_current_task_btf() inline. */ > if ((insn->imm == BPF_FUNC_get_current_task || insn->imm == BPF_FUNC_get_current_task_btf) && > verifier_inlines_helper_call(env, insn->imm)) { > [......] > #endif oh. I see. I misread it as '+#endif' (with a +) and assumed it's part of new code. > > > > > > +#ifdef CONFIG_SMP > > > + insn_buf[0] = BPF_MOV64_IMM(BPF_REG_0, (u32)(unsigned long)¤t_task); > > > + insn_buf[1] = BPF_MOV64_PERCPU_REG(BPF_REG_0, BPF_REG_0); > > > + insn_buf[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 0); > > > +#else > > > + struct bpf_insn ld_current_addr[2] = { > > > + BPF_LD_IMM64(BPF_REG_0, (unsigned long)¤t_task) > > > + }; > > > + insn_buf[0] = ld_current_addr[0]; > > > + insn_buf[1] = ld_current_addr[1]; > > > + insn_buf[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 0); > > > +#endif > > > > I wouldn't bother with !SMP. > > If we need to add defined(CONFIG_X86_64) && !defined(CONFIG_UML) > > I would add && defined(CONFIG_SMP) to it. > > OK, let's skip the !SMP case to make the code more clear. Similar thoughts about your other patch where you introduce decl_tag to deal with different configs. For bpf CI we don't need to do such things. The kernel has to be configured with selftest/bpf/config. So doing extra work in test_progs to recognize !SMP looks like overkill. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH bpf-next v4 1/2] bpf, x86: inline bpf_get_current_task() for x86_64 2026-01-14 1:24 ` Alexei Starovoitov @ 2026-01-14 1:33 ` Menglong Dong 0 siblings, 0 replies; 7+ messages in thread From: Menglong Dong @ 2026-01-14 1:33 UTC (permalink / raw) To: Alexei Starovoitov Cc: Menglong Dong, Alexei Starovoitov, Eduard, Daniel Borkmann, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf, LKML On 2026/1/14 09:24 Alexei Starovoitov <alexei.starovoitov@gmail.com> write: > On Tue, Jan 13, 2026 at 5:19 PM Menglong Dong <menglong.dong@linux.dev> wrote: > > > > On 2026/1/14 01:50 Alexei Starovoitov <alexei.starovoitov@gmail.com> write: > > > On Mon, Jan 12, 2026 at 2:45 AM Menglong Dong <menglong8.dong@gmail.com> wrote: > > > > > > > > Inline bpf_get_current_task() and bpf_get_current_task_btf() for x86_64 > > > > to obtain better performance. > > > > > > > > In !CONFIG_SMP case, the percpu variable is just a normal variable, and > > > > we can read the current_task directly. > > > > > > > > Signed-off-by: Menglong Dong <dongml2@chinatelecom.cn> > > > > --- > > > > v4: > > > > - handle the !CONFIG_SMP case > > > > > > > > v3: > > > > - implement it in the verifier with BPF_MOV64_PERCPU_REG() instead of in > > > > x86_64 JIT. > > > > --- > > > > kernel/bpf/verifier.c | 29 +++++++++++++++++++++++++++++ > > > > 1 file changed, 29 insertions(+) > > > > > > > > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c > > > > index 3d44c5d06623..12e99171afd8 100644 > > > > --- a/kernel/bpf/verifier.c > > > > +++ b/kernel/bpf/verifier.c > > > > @@ -17688,6 +17688,8 @@ static bool verifier_inlines_helper_call(struct bpf_verifier_env *env, s32 imm) > > > > switch (imm) { > > > > #ifdef CONFIG_X86_64 > > > > case BPF_FUNC_get_smp_processor_id: > > > > + case BPF_FUNC_get_current_task_btf: > > > > + case BPF_FUNC_get_current_task: > > > > return env->prog->jit_requested && bpf_jit_supports_percpu_insn(); > > > > #endif > > > > default: > > > > @@ -23273,6 +23275,33 @@ static int do_misc_fixups(struct bpf_verifier_env *env) > > > > insn = new_prog->insnsi + i + delta; > > > > goto next_insn; > > > > } > > > > + > > > > + /* Implement bpf_get_current_task() and bpf_get_current_task_btf() inline. */ > > > > + if ((insn->imm == BPF_FUNC_get_current_task || insn->imm == BPF_FUNC_get_current_task_btf) && > > > > + verifier_inlines_helper_call(env, insn->imm)) { > > > > > > Though verifier_inlines_helper_call() gates this with CONFIG_X86_64, > > > I think we still need explicit: > > > #if defined(CONFIG_X86_64) && !defined(CONFIG_UML) > > > > > > just like we did for BPF_FUNC_get_smp_processor_id. > > > Please check. I suspect UML will break without it. > > > > Do you mean that we need to use > > #if defined(CONFIG_X86_64) && !defined(CONFIG_UML) > > here? > > > > The whole code is already within it. You can have a look up: > > > > #if defined(CONFIG_X86_64) && !defined(CONFIG_UML) > > /* Implement bpf_get_smp_processor_id() inline. */ > > if (insn->imm == BPF_FUNC_get_smp_processor_id && > > verifier_inlines_helper_call(env, insn->imm)) { > > [......] > > /* Implement bpf_get_current_task() and bpf_get_current_task_btf() inline. */ > > if ((insn->imm == BPF_FUNC_get_current_task || insn->imm == BPF_FUNC_get_current_task_btf) && > > verifier_inlines_helper_call(env, insn->imm)) { > > [......] > > #endif > > oh. I see. I misread it as '+#endif' (with a +) and assumed > it's part of new code. > > > > > > > > > > +#ifdef CONFIG_SMP > > > > + insn_buf[0] = BPF_MOV64_IMM(BPF_REG_0, (u32)(unsigned long)¤t_task); > > > > + insn_buf[1] = BPF_MOV64_PERCPU_REG(BPF_REG_0, BPF_REG_0); > > > > + insn_buf[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 0); > > > > +#else > > > > + struct bpf_insn ld_current_addr[2] = { > > > > + BPF_LD_IMM64(BPF_REG_0, (unsigned long)¤t_task) > > > > + }; > > > > + insn_buf[0] = ld_current_addr[0]; > > > > + insn_buf[1] = ld_current_addr[1]; > > > > + insn_buf[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 0); > > > > +#endif > > > > > > I wouldn't bother with !SMP. > > > If we need to add defined(CONFIG_X86_64) && !defined(CONFIG_UML) > > > I would add && defined(CONFIG_SMP) to it. > > > > OK, let's skip the !SMP case to make the code more clear. > > Similar thoughts about your other patch where you introduce > decl_tag to deal with different configs. > For bpf CI we don't need to do such things. > The kernel has to be configured with selftest/bpf/config. > So doing extra work in test_progs to recognize !SMP looks like overkill. You are right, and that's why I removed that patch in this version after I realized this point. Thanks! Menglong Dong ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH bpf-next v4 2/2] selftests/bpf: test the jited inline of bpf_get_current_task 2026-01-12 10:45 [PATCH bpf-next v4 0/2] bpf, x86: inline bpf_get_current_task() for x86_64 Menglong Dong 2026-01-12 10:45 ` [PATCH bpf-next v4 1/2] " Menglong Dong @ 2026-01-12 10:45 ` Menglong Dong 1 sibling, 0 replies; 7+ messages in thread From: Menglong Dong @ 2026-01-12 10:45 UTC (permalink / raw) To: ast, eddyz87 Cc: daniel, john.fastabend, andrii, martin.lau, song, yonghong.song, kpsingh, sdf, haoluo, jolsa, bpf, linux-kernel Add the testcase for the jited inline of bpf_get_current_task(). Signed-off-by: Menglong Dong <dongml2@chinatelecom.cn> --- .../selftests/bpf/prog_tests/verifier.c | 2 ++ .../selftests/bpf/progs/verifier_jit_inline.c | 35 +++++++++++++++++++ 2 files changed, 37 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/verifier_jit_inline.c diff --git a/tools/testing/selftests/bpf/prog_tests/verifier.c b/tools/testing/selftests/bpf/prog_tests/verifier.c index 5829ffd70f8f..47eb78c808c0 100644 --- a/tools/testing/selftests/bpf/prog_tests/verifier.c +++ b/tools/testing/selftests/bpf/prog_tests/verifier.c @@ -110,6 +110,7 @@ #include "verifier_xdp_direct_packet_access.skel.h" #include "verifier_bits_iter.skel.h" #include "verifier_lsm.skel.h" +#include "verifier_jit_inline.skel.h" #include "irq.skel.h" #define MAX_ENTRIES 11 @@ -251,6 +252,7 @@ void test_verifier_bits_iter(void) { RUN(verifier_bits_iter); } void test_verifier_lsm(void) { RUN(verifier_lsm); } void test_irq(void) { RUN(irq); } void test_verifier_mtu(void) { RUN(verifier_mtu); } +void test_verifier_jit_inline(void) { RUN(verifier_jit_inline); } static int init_test_val_map(struct bpf_object *obj, char *map_name) { diff --git a/tools/testing/selftests/bpf/progs/verifier_jit_inline.c b/tools/testing/selftests/bpf/progs/verifier_jit_inline.c new file mode 100644 index 000000000000..0938ca1dac87 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/verifier_jit_inline.c @@ -0,0 +1,35 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <vmlinux.h> +#include <bpf/bpf_helpers.h> +#include "bpf_misc.h" + +#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64) + +SEC("fentry/bpf_fentry_test1") +__description("Jit inline, bpf_get_current_task") +__success __retval(0) +__arch_x86_64 +__jited(" addq %gs:{{.*}}, %rax") +__arch_arm64 +__jited(" mrs x7, SP_EL0") +int inline_bpf_get_current_task(void) +{ + bpf_get_current_task(); + + return 0; +} + +#else + +SEC("kprobe") +__description("Jit inline is not supported, use a dummy test") +__success +int dummy_test(void) +{ + return 0; +} + +#endif + +char _license[] SEC("license") = "GPL"; -- 2.52.0 ^ permalink raw reply related [flat|nested] 7+ messages in thread
end of thread, other threads:[~2026-01-14 1:33 UTC | newest] Thread overview: 7+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-01-12 10:45 [PATCH bpf-next v4 0/2] bpf, x86: inline bpf_get_current_task() for x86_64 Menglong Dong 2026-01-12 10:45 ` [PATCH bpf-next v4 1/2] " Menglong Dong 2026-01-13 17:50 ` Alexei Starovoitov 2026-01-14 1:19 ` Menglong Dong 2026-01-14 1:24 ` Alexei Starovoitov 2026-01-14 1:33 ` Menglong Dong 2026-01-12 10:45 ` [PATCH bpf-next v4 2/2] selftests/bpf: test the jited inline of bpf_get_current_task Menglong Dong
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox