From: Menglong Dong <menglong8.dong@gmail.com>
To: ast@kernel.org, paulmck@kernel.org
Cc: frederic@kernel.org, neeraj.upadhyay@kernel.org,
joelagnelf@nvidia.com, josh@joshtriplett.org,
boqun.feng@gmail.com, urezki@gmail.com, rostedt@goodmis.org,
mathieu.desnoyers@efficios.com, jiangshanlai@gmail.com,
qiang.zhang@linux.dev, daniel@iogearbox.net,
john.fastabend@gmail.com, andrii@kernel.org,
martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org,
yonghong.song@linux.dev, kpsingh@kernel.org, sdf@fomichev.me,
haoluo@google.com, jolsa@kernel.org, rcu@vger.kernel.org,
linux-kernel@vger.kernel.org, bpf@vger.kernel.org
Subject: [PATCH bpf-next v2 7/7] bpf: use rcu_read_lock_dont_migrate() for trampoline.c
Date: Tue, 19 Aug 2025 17:34:24 +0800 [thread overview]
Message-ID: <20250819093424.1011645-8-dongml2@chinatelecom.cn> (raw)
In-Reply-To: <20250819093424.1011645-1-dongml2@chinatelecom.cn>
Use rcu_read_lock_dont_migrate() and rcu_read_unlock_migrate() in
trampoline.c to obtain better performance when PREEMPT_RCU is not enabled.
Signed-off-by: Menglong Dong <dongml2@chinatelecom.cn>
---
v2:
- use rcu_read_lock_dont_migrate() instead of rcu_migrate_disable()
---
kernel/bpf/trampoline.c | 18 ++++++------------
1 file changed, 6 insertions(+), 12 deletions(-)
diff --git a/kernel/bpf/trampoline.c b/kernel/bpf/trampoline.c
index 0e364614c3a2..5949095e51c3 100644
--- a/kernel/bpf/trampoline.c
+++ b/kernel/bpf/trampoline.c
@@ -899,8 +899,7 @@ static __always_inline u64 notrace bpf_prog_start_time(void)
static u64 notrace __bpf_prog_enter_recur(struct bpf_prog *prog, struct bpf_tramp_run_ctx *run_ctx)
__acquires(RCU)
{
- rcu_read_lock();
- migrate_disable();
+ rcu_read_lock_dont_migrate();
run_ctx->saved_run_ctx = bpf_set_run_ctx(&run_ctx->run_ctx);
@@ -949,8 +948,7 @@ static void notrace __bpf_prog_exit_recur(struct bpf_prog *prog, u64 start,
update_prog_stats(prog, start);
this_cpu_dec(*(prog->active));
- migrate_enable();
- rcu_read_unlock();
+ rcu_read_unlock_migrate();
}
static u64 notrace __bpf_prog_enter_lsm_cgroup(struct bpf_prog *prog,
@@ -960,8 +958,7 @@ static u64 notrace __bpf_prog_enter_lsm_cgroup(struct bpf_prog *prog,
/* Runtime stats are exported via actual BPF_LSM_CGROUP
* programs, not the shims.
*/
- rcu_read_lock();
- migrate_disable();
+ rcu_read_lock_dont_migrate();
run_ctx->saved_run_ctx = bpf_set_run_ctx(&run_ctx->run_ctx);
@@ -974,8 +971,7 @@ static void notrace __bpf_prog_exit_lsm_cgroup(struct bpf_prog *prog, u64 start,
{
bpf_reset_run_ctx(run_ctx->saved_run_ctx);
- migrate_enable();
- rcu_read_unlock();
+ rcu_read_unlock_migrate();
}
u64 notrace __bpf_prog_enter_sleepable_recur(struct bpf_prog *prog,
@@ -1033,8 +1029,7 @@ static u64 notrace __bpf_prog_enter(struct bpf_prog *prog,
struct bpf_tramp_run_ctx *run_ctx)
__acquires(RCU)
{
- rcu_read_lock();
- migrate_disable();
+ rcu_read_lock_dont_migrate();
run_ctx->saved_run_ctx = bpf_set_run_ctx(&run_ctx->run_ctx);
@@ -1048,8 +1043,7 @@ static void notrace __bpf_prog_exit(struct bpf_prog *prog, u64 start,
bpf_reset_run_ctx(run_ctx->saved_run_ctx);
update_prog_stats(prog, start);
- migrate_enable();
- rcu_read_unlock();
+ rcu_read_unlock_migrate();
}
void notrace __bpf_tramp_enter(struct bpf_tramp_image *tr)
--
2.50.1
prev parent reply other threads:[~2025-08-19 9:35 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-08-19 9:34 [PATCH bpf-next v2 0/7] bpf: introduce and use rcu_read_lock_dont_migrate Menglong Dong
2025-08-19 9:34 ` [PATCH bpf-next v2 1/7] rcu: add rcu_read_lock_dont_migrate() Menglong Dong
2025-08-19 14:57 ` Paul E. McKenney
2025-08-20 1:04 ` Menglong Dong
2025-08-19 9:34 ` [PATCH bpf-next v2 2/7] bpf: use rcu_read_lock_dont_migrate() for bpf_cgrp_storage_free() Menglong Dong
2025-08-19 9:34 ` [PATCH bpf-next v2 3/7] bpf: use rcu_read_lock_dont_migrate() for bpf_inode_storage_free() Menglong Dong
2025-08-19 9:34 ` [PATCH bpf-next v2 4/7] bpf: use rcu_read_lock_dont_migrate() for bpf_iter_run_prog() Menglong Dong
2025-08-19 9:34 ` [PATCH bpf-next v2 5/7] bpf: use rcu_read_lock_dont_migrate() for bpf_task_storage_free() Menglong Dong
2025-08-19 9:34 ` [PATCH bpf-next v2 6/7] bpf: use rcu_read_lock_dont_migrate() for bpf_prog_run_array_cg() Menglong Dong
2025-08-19 9:34 ` Menglong Dong [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250819093424.1011645-8-dongml2@chinatelecom.cn \
--to=menglong8.dong@gmail.com \
--cc=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=boqun.feng@gmail.com \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=eddyz87@gmail.com \
--cc=frederic@kernel.org \
--cc=haoluo@google.com \
--cc=jiangshanlai@gmail.com \
--cc=joelagnelf@nvidia.com \
--cc=john.fastabend@gmail.com \
--cc=jolsa@kernel.org \
--cc=josh@joshtriplett.org \
--cc=kpsingh@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=martin.lau@linux.dev \
--cc=mathieu.desnoyers@efficios.com \
--cc=neeraj.upadhyay@kernel.org \
--cc=paulmck@kernel.org \
--cc=qiang.zhang@linux.dev \
--cc=rcu@vger.kernel.org \
--cc=rostedt@goodmis.org \
--cc=sdf@fomichev.me \
--cc=song@kernel.org \
--cc=urezki@gmail.com \
--cc=yonghong.song@linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).