From: Vasily Gorbik <gor@linux.ibm.com>
To: "Paul E. McKenney" <paulmck@kernel.org>
Cc: Boqun Feng <boqun@kernel.org>,
Frederic Weisbecker <frederic@kernel.org>,
Neeraj Upadhyay <neeraj.upadhyay@kernel.org>,
Joel Fernandes <joelagnelf@nvidia.com>,
Uladzislau Rezki <urezki@gmail.com>,
rcu@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-s390@vger.kernel.org, Samir M <samir@linux.ibm.com>,
Srikar Dronamraju <srikar@linux.ibm.com>
Subject: Re: BUG: workqueue lockup - SRCU schedules work on not-online CPUs during size transition
Date: Wed, 29 Apr 2026 19:50:31 +0200 [thread overview]
Message-ID: <tte9o87@ub.hpns> (raw)
In-Reply-To: <ed1fa6cd-7343-4ca3-8b9d-d699ca496f83@paulmck-laptop>
On Tue, Apr 14, 2026 at 12:24:12PM -0700, Paul E. McKenney wrote:
> On Thu, Apr 09, 2026 at 09:03:26PM -0700, Paul E. McKenney wrote:
> Please see below for the full patch, including refraining from queueing
> workqueue handlers on not-yet-online CPUs and diverting SRCU callbacks
> from not-yet-fully-online CPUs to the boot CPU's callback queue.
...
> commit ce533a60b2ef29a9b516cc717e77c6b679bc09c0
> Author: Paul E. McKenney <paulmck@kernel.org>
> Date: Thu Apr 9 11:16:02 2026 -0700
>
> srcu: Don't queue workqueue handlers to never-online CPUs
>
> While an srcu_struct structure is in the midst of switching from CPU-0
> to all-CPUs state, it can attempt to invoke callbacks for CPUs that
> have never been online. Worse yet, it can attempt in invoke callbacks
> for CPUs that never will be online due to not being present in the
> cpu_possible_mask. This can cause hangs on s390, which is not set up to
> deal with workqueue handlers being scheduled on such CPUs. This commit
> therefore causes Tree SRCU to refrain from queueing workqueue handlers
> on CPUs that have not yet (and might never) come online.
>
> Because callbacks are not invoked on CPUs that have not been
> online, it is an error to invoke call_srcu(), synchronize_srcu(), or
> synchronize_srcu_expedited() on a CPU that is not yet fully online.
> However, it turns out to be less code to redirect the callbacks
> from too-early invocations of call_srcu() than to warn about such
> invocations. This commit therefore also redirects callbacks queued on
> not-yet-fully-online CPUs to the boot CPU.
>
> Reported-by: Vasily Gorbik <gor@linux.ibm.com>
> Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
> Tested-by: Vasily Gorbik <gor@linux.ibm.com>
> Cc: Tejun Heo <tj@kernel.org>
I retested it on s390 and on x86 KVM with --smp 16,maxcpus=255, all
looks good to me.
FWIW, again:
Tested-by: Vasily Gorbik <gor@linux.ibm.com>
Would you mind adding Cc: stable so it gets picked up for v7.0?
61bbcfb50514 ("srcu: Push srcu_node allocation to GP when
non-preemptible") is what made it reproducible for us.
Thank you!
next prev parent reply other threads:[~2026-04-29 17:50 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-04-09 13:08 BUG: workqueue lockup - SRCU schedules work on not-online CPUs during size transition Vasily Gorbik
2026-04-09 17:22 ` Paul E. McKenney
2026-04-09 19:15 ` Vasily Gorbik
2026-04-09 20:10 ` Paul E. McKenney
2026-04-10 4:03 ` Paul E. McKenney
2026-04-14 19:24 ` Paul E. McKenney
2026-04-29 17:50 ` Vasily Gorbik [this message]
2026-04-29 18:05 ` Paul E. McKenney
2026-04-29 18:23 ` Vasily Gorbik
2026-04-09 17:26 ` Boqun Feng
2026-04-09 17:40 ` Boqun Feng
2026-04-09 17:47 ` Tejun Heo
2026-04-09 17:48 ` Tejun Heo
2026-04-09 18:04 ` Paul E. McKenney
2026-04-09 18:09 ` Tejun Heo
2026-04-09 18:15 ` Paul E. McKenney
2026-04-09 18:10 ` Boqun Feng
2026-04-09 18:27 ` Paul E. McKenney
2026-04-10 18:53 ` Tejun Heo
2026-04-10 19:17 ` Paul E. McKenney
2026-04-10 19:29 ` Tejun Heo
2026-04-29 15:00 ` Srikar Dronamraju
2026-04-29 17:08 ` Vasily Gorbik
2026-04-29 17:18 ` Paul E. McKenney
2026-04-29 17:44 ` Shrikanth Hegde
2026-04-29 18:01 ` Paul E. McKenney
2026-04-29 18:17 ` Samir M
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=tte9o87@ub.hpns \
--to=gor@linux.ibm.com \
--cc=boqun@kernel.org \
--cc=frederic@kernel.org \
--cc=joelagnelf@nvidia.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-s390@vger.kernel.org \
--cc=neeraj.upadhyay@kernel.org \
--cc=paulmck@kernel.org \
--cc=rcu@vger.kernel.org \
--cc=samir@linux.ibm.com \
--cc=srikar@linux.ibm.com \
--cc=urezki@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox