From: Boqun Feng <boqun.feng@gmail.com>
To: Joel Fernandes <joelagnelf@nvidia.com>
Cc: linux-kernel@vger.kernel.org,
"Paul E. McKenney" <paulmck@kernel.org>,
Frederic Weisbecker <frederic@kernel.org>,
Neeraj Upadhyay <neeraj.upadhyay@kernel.org>,
Joel Fernandes <joel@joelfernandes.org>,
Josh Triplett <josh@joshtriplett.org>,
Uladzislau Rezki <urezki@gmail.com>,
Steven Rostedt <rostedt@goodmis.org>,
Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
Lai Jiangshan <jiangshanlai@gmail.com>,
Zqiang <qiang.zhang1211@gmail.com>,
Xiongfeng Wang <wangxiongfeng2@huawei.com>,
rcu@vger.kernel.org, bpf@vger.kernel.org
Subject: Re: [PATCH 2/2] rcu: Fix lockup when RCU reader used while IRQ exiting
Date: Wed, 11 Jun 2025 09:05:06 -0700 [thread overview]
Message-ID: <aEmpMohtFsVf4Uh_@tardis.local> (raw)
In-Reply-To: <20250609180125.2988129-2-joelagnelf@nvidia.com>
On Mon, Jun 09, 2025 at 02:01:24PM -0400, Joel Fernandes wrote:
> During rcu_read_unlock_special(), if this happens during irq_exit(), we
> can lockup if an IPI is issued. This is because the IPI itself triggers
> the irq_exit() path causing a recursive lock up.
>
> This is precisely what Xiongfeng found when invoking a BPF program on
> the trace_tick_stop() tracepoint As shown in the trace below. Fix by
> using context-tracking to tell us if we're still in an IRQ.
> context-tracking keeps track of the IRQ until after the tracepoint, so
> it cures the issues.
>
> irq_exit()
> __irq_exit_rcu()
> /* in_hardirq() returns false after this */
> preempt_count_sub(HARDIRQ_OFFSET)
> tick_irq_exit()
@Frederic, while we are at it, what's the purpose of in_hardirq() in
tick_irq_exit()? For nested interrupt detection?
Regards,
Boqun
> tick_nohz_irq_exit()
> tick_nohz_stop_sched_tick()
> trace_tick_stop() /* a bpf prog is hooked on this trace point */
> __bpf_trace_tick_stop()
> bpf_trace_run2()
> rcu_read_unlock_special()
> /* will send a IPI to itself */
> irq_work_queue_on(&rdp->defer_qs_iw, rdp->cpu);
>
> A simple reproducer can also be obtained by doing the following in
> tick_irq_exit(). It will hang on boot without the patch:
>
> static inline void tick_irq_exit(void)
> {
> + rcu_read_lock();
> + WRITE_ONCE(current->rcu_read_unlock_special.b.need_qs, true);
> + rcu_read_unlock();
> +
>
> While at it, add some comments to this code.
>
> Reported-by: Xiongfeng Wang <wangxiongfeng2@huawei.com>
> Closes: https://lore.kernel.org/all/9acd5f9f-6732-7701-6880-4b51190aa070@huawei.com/
> Tested-by: Xiongfeng Wang <wangxiongfeng2@huawei.com>
> Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
[...]
next prev parent reply other threads:[~2025-06-11 16:05 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-06-09 18:01 [PATCH 1/2] context_tracking: Provide helper to determine if we're in IRQ Joel Fernandes
2025-06-09 18:01 ` [PATCH 2/2] rcu: Fix lockup when RCU reader used while IRQ exiting Joel Fernandes
2025-06-09 19:49 ` Boqun Feng
2025-06-09 23:26 ` Frederic Weisbecker
2025-06-10 0:49 ` Boqun Feng
2025-06-10 12:23 ` Frederic Weisbecker
2025-06-10 15:47 ` Joel Fernandes
2025-06-12 3:06 ` Xiongfeng Wang
2025-06-12 11:37 ` Frederic Weisbecker
2025-06-11 16:05 ` Boqun Feng [this message]
2025-06-11 16:16 ` Paul E. McKenney
2025-06-11 16:21 ` Boqun Feng
2025-06-09 18:05 ` [PATCH 1/2] context_tracking: Provide helper to determine if we're in IRQ Joel Fernandes
2025-06-11 16:25 ` Frederic Weisbecker
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=aEmpMohtFsVf4Uh_@tardis.local \
--to=boqun.feng@gmail.com \
--cc=bpf@vger.kernel.org \
--cc=frederic@kernel.org \
--cc=jiangshanlai@gmail.com \
--cc=joel@joelfernandes.org \
--cc=joelagnelf@nvidia.com \
--cc=josh@joshtriplett.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=neeraj.upadhyay@kernel.org \
--cc=paulmck@kernel.org \
--cc=qiang.zhang1211@gmail.com \
--cc=rcu@vger.kernel.org \
--cc=rostedt@goodmis.org \
--cc=urezki@gmail.com \
--cc=wangxiongfeng2@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox