* [PATCH 1/2] firmware: arm_scmi: Fix deadlock between reboot notifier and forceful work
[not found] <20261116084211.3066460-1-cristian.marussi@arm.com>
@ 2026-11-16 8:42 ` Cristian Marussi
2026-11-16 18:25 ` Sudeep Holla
0 siblings, 1 reply; 3+ messages in thread
From: Cristian Marussi @ 2026-11-16 8:42 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, arm-scmi
Cc: sudeep.holla, philip.radford, Chengyu Liu, stable,
Cristian Marussi
From: Chengyu Liu <lui2012qwq@gmail.com>
scmi_reboot_notifier() calls cancel_delayed_work_sync() while holding
sc->state_mtx, but the work it waits on - scmi_forceful_work_func() -
takes the same mutex. If a forceful transition work is in flight when
the reboot notifier fires, the notifier holds the lock and waits for
the callback while the callback waits for the lock: deadlock, and the
reboot wedges.
Moving the cancel outside the lock is not sufficient on its own, as the
callback also calls unregister_reboot_notifier() unconditionally, which
blocks on the reboot chain rwsem (down_write vs. the in-progress
chain's down_read) while the notifier waits for the callback - the
deadlock would just move to a different lock. Fix both sides:
- Notifier: compute the decision under state_mtx, drop the lock, then
cancel. A callback that slips in between sees state ==
SCMI_SYSPOWER_REBOOTING (not IN_PROGRESS), skips the forceful
request and finishes, so the cancel cannot miss it. (The callback's
own system_state early-return does not cover a work item that
started before the reboot began: kernel_restart_prepare() runs the
notifier chain before setting system_state.)
- Callback: only unregister the reboot notifier when actually forcing
the transition - that is the only case that needs it, because the
forceful kernel_restart()/kernel_power_off() would otherwise
re-enter scmi_reboot_notifier and self-deadlock on state_mtx. When
the state is not IN_PROGRESS there is nothing to force, and skipping
the unregister keeps the callback from blocking on the reboot chain
rwsem while the notifier waits for it.
Found by static analysis.
Cc: stable@vger.kernel.org
Fixes: 2c4b97fee94a ("firmware: arm_scmi: Add SCMI System Power Control driver")
Signed-off-by: Chengyu Liu <lui2012qwq@gmail.com>
[Cristian: drop lock before triggering forceful reboot (needs_forceful)]
Tested-by: Cristian Marussi <cristian.marussi@arm.com>
Signed-off-by: Cristian Marussi <cristian.marussi@arm.com>
---
.../firmware/arm_scmi/scmi_power_control.c | 26 ++++++++++++++-----
1 file changed, 19 insertions(+), 7 deletions(-)
diff --git a/drivers/firmware/arm_scmi/scmi_power_control.c b/drivers/firmware/arm_scmi/scmi_power_control.c
index e791a206768c..e0680e33180e 100644
--- a/drivers/firmware/arm_scmi/scmi_power_control.c
+++ b/drivers/firmware/arm_scmi/scmi_power_control.c
@@ -121,6 +121,7 @@ static int scmi_reboot_notifier(struct notifier_block *nb,
unsigned long reason, void *__unused)
{
struct scmi_syspower_conf *sc = reboot_nb_to_sconf(nb);
+ bool cancel_work;
mutex_lock(&sc->state_mtx);
switch (reason) {
@@ -138,11 +139,18 @@ static int scmi_reboot_notifier(struct notifier_block *nb,
break;
}
- if (sc->state == SCMI_SYSPOWER_REBOOTING) {
+ cancel_work = sc->state == SCMI_SYSPOWER_REBOOTING;
+ mutex_unlock(&sc->state_mtx);
+
+ if (cancel_work) {
dev_dbg(sc->dev, "Reboot in progress...cancel delayed work.\n");
+ /*
+ * cancel_delayed_work_sync() waits for a running
+ * scmi_forceful_work_func(), which takes state_mtx - it must
+ * not be called with the lock held.
+ */
cancel_delayed_work_sync(&sc->forceful_work);
}
- mutex_unlock(&sc->state_mtx);
return NOTIFY_OK;
}
@@ -181,6 +189,7 @@ static void scmi_forceful_work_func(struct work_struct *work)
{
struct scmi_syspower_conf *sc;
struct delayed_work *dwork;
+ bool needs_forceful;
if (system_state > SYSTEM_RUNNING)
return;
@@ -188,13 +197,16 @@ static void scmi_forceful_work_func(struct work_struct *work)
dwork = to_delayed_work(work);
sc = dwork_to_sconf(dwork);
- dev_dbg(sc->dev, "Graceful request timed out...forcing !\n");
mutex_lock(&sc->state_mtx);
- /* avoid deadlock by unregistering reboot notifier first */
- unregister_reboot_notifier(&sc->reboot_nb);
- if (sc->state == SCMI_SYSPOWER_IN_PROGRESS)
- scmi_request_forceful_transition(sc);
+ needs_forceful = sc->state == SCMI_SYSPOWER_IN_PROGRESS;
mutex_unlock(&sc->state_mtx);
+
+ if (needs_forceful) {
+ dev_dbg(sc->dev, "Graceful request timed out...forcing !\n");
+ /* avoid deadlock by unregistering reboot notifier first */
+ unregister_reboot_notifier(&sc->reboot_nb);
+ scmi_request_forceful_transition(sc);
+ }
}
/**
--
2.54.0
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH 1/2] firmware: arm_scmi: Fix deadlock between reboot notifier and forceful work
2026-11-16 8:42 ` [PATCH 1/2] firmware: arm_scmi: Fix deadlock between reboot notifier and forceful work Cristian Marussi
@ 2026-11-16 18:25 ` Sudeep Holla
2026-11-17 11:25 ` Cristian Marussi
0 siblings, 1 reply; 3+ messages in thread
From: Sudeep Holla @ 2026-11-16 18:25 UTC (permalink / raw)
To: Cristian Marussi
Cc: linux-kernel, linux-arm-kernel, arm-scmi, philip.radford,
Sudeep Holla, Chengyu Liu, stable
On Mon, Nov 16, 2026 at 08:42:10AM +0000, Cristian Marussi wrote:
> From: Chengyu Liu <lui2012qwq@gmail.com>
>
> scmi_reboot_notifier() calls cancel_delayed_work_sync() while holding
> sc->state_mtx, but the work it waits on - scmi_forceful_work_func() -
> takes the same mutex. If a forceful transition work is in flight when
> the reboot notifier fires, the notifier holds the lock and waits for
> the callback while the callback waits for the lock: deadlock, and the
> reboot wedges.
>
> Moving the cancel outside the lock is not sufficient on its own, as the
> callback also calls unregister_reboot_notifier() unconditionally, which
> blocks on the reboot chain rwsem (down_write vs. the in-progress
> chain's down_read) while the notifier waits for the callback - the
> deadlock would just move to a different lock. Fix both sides:
>
> - Notifier: compute the decision under state_mtx, drop the lock, then
> cancel. A callback that slips in between sees state ==
> SCMI_SYSPOWER_REBOOTING (not IN_PROGRESS), skips the forceful
> request and finishes, so the cancel cannot miss it. (The callback's
> own system_state early-return does not cover a work item that
> started before the reboot began: kernel_restart_prepare() runs the
> notifier chain before setting system_state.)
>
> - Callback: only unregister the reboot notifier when actually forcing
> the transition - that is the only case that needs it, because the
> forceful kernel_restart()/kernel_power_off() would otherwise
> re-enter scmi_reboot_notifier and self-deadlock on state_mtx. When
> the state is not IN_PROGRESS there is nothing to force, and skipping
> the unregister keeps the callback from blocking on the reboot chain
> rwsem while the notifier waits for it.
>
> Found by static analysis.
>
> Cc: stable@vger.kernel.org
> Fixes: 2c4b97fee94a ("firmware: arm_scmi: Add SCMI System Power Control driver")
> Signed-off-by: Chengyu Liu <lui2012qwq@gmail.com>
> [Cristian: drop lock before triggering forceful reboot (needs_forceful)]
> Tested-by: Cristian Marussi <cristian.marussi@arm.com>
> Signed-off-by: Cristian Marussi <cristian.marussi@arm.com>
> ---
> .../firmware/arm_scmi/scmi_power_control.c | 26 ++++++++++++++-----
> 1 file changed, 19 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/firmware/arm_scmi/scmi_power_control.c b/drivers/firmware/arm_scmi/scmi_power_control.c
> index e791a206768c..e0680e33180e 100644
> --- a/drivers/firmware/arm_scmi/scmi_power_control.c
> +++ b/drivers/firmware/arm_scmi/scmi_power_control.c
> @@ -121,6 +121,7 @@ static int scmi_reboot_notifier(struct notifier_block *nb,
> unsigned long reason, void *__unused)
> {
> struct scmi_syspower_conf *sc = reboot_nb_to_sconf(nb);
> + bool cancel_work;
>
> mutex_lock(&sc->state_mtx);
> switch (reason) {
> @@ -138,11 +139,18 @@ static int scmi_reboot_notifier(struct notifier_block *nb,
> break;
> }
>
> - if (sc->state == SCMI_SYSPOWER_REBOOTING) {
> + cancel_work = sc->state == SCMI_SYSPOWER_REBOOTING;
> + mutex_unlock(&sc->state_mtx);
> +
> + if (cancel_work) {
> dev_dbg(sc->dev, "Reboot in progress...cancel delayed work.\n");
> + /*
> + * cancel_delayed_work_sync() waits for a running
> + * scmi_forceful_work_func(), which takes state_mtx - it must
> + * not be called with the lock held.
> + */
> cancel_delayed_work_sync(&sc->forceful_work);
Sashiko still complains about the deadlock.
How about changing this to non-sync cancel_delayed_work() so that it doesn't
wait and deadlock like it complains ?
> }
> - mutex_unlock(&sc->state_mtx);
>
> return NOTIFY_OK;
> }
> @@ -181,6 +189,7 @@ static void scmi_forceful_work_func(struct work_struct *work)
> {
> struct scmi_syspower_conf *sc;
> struct delayed_work *dwork;
> + bool needs_forceful;
>
> if (system_state > SYSTEM_RUNNING)
> return;
> @@ -188,13 +197,16 @@ static void scmi_forceful_work_func(struct work_struct *work)
> dwork = to_delayed_work(work);
> sc = dwork_to_sconf(dwork);
>
> - dev_dbg(sc->dev, "Graceful request timed out...forcing !\n");
> mutex_lock(&sc->state_mtx);
> - /* avoid deadlock by unregistering reboot notifier first */
> - unregister_reboot_notifier(&sc->reboot_nb);
> - if (sc->state == SCMI_SYSPOWER_IN_PROGRESS)
> - scmi_request_forceful_transition(sc);
> + needs_forceful = sc->state == SCMI_SYSPOWER_IN_PROGRESS;
> mutex_unlock(&sc->state_mtx);
> +
> + if (needs_forceful) {
> + dev_dbg(sc->dev, "Graceful request timed out...forcing !\n");
> + /* avoid deadlock by unregistering reboot notifier first */
> + unregister_reboot_notifier(&sc->reboot_nb);
And re-evaluate needs_forceful again here after un-registration before
calling below forced transition ?
--
Regards,
Sudeep
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH 1/2] firmware: arm_scmi: Fix deadlock between reboot notifier and forceful work
2026-11-16 18:25 ` Sudeep Holla
@ 2026-11-17 11:25 ` Cristian Marussi
0 siblings, 0 replies; 3+ messages in thread
From: Cristian Marussi @ 2026-11-17 11:25 UTC (permalink / raw)
To: Sudeep Holla
Cc: Cristian Marussi, linux-kernel, linux-arm-kernel, arm-scmi,
philip.radford, Chengyu Liu, stable
On Mon, Nov 16, 2026 at 06:25:52PM +0000, Sudeep Holla wrote:
> On Mon, Nov 16, 2026 at 08:42:10AM +0000, Cristian Marussi wrote:
> > From: Chengyu Liu <lui2012qwq@gmail.com>
Hi Sudeep,
> >
> > scmi_reboot_notifier() calls cancel_delayed_work_sync() while holding
> > sc->state_mtx, but the work it waits on - scmi_forceful_work_func() -
> > takes the same mutex. If a forceful transition work is in flight when
> > the reboot notifier fires, the notifier holds the lock and waits for
> > the callback while the callback waits for the lock: deadlock, and the
> > reboot wedges.
> >
> > Moving the cancel outside the lock is not sufficient on its own, as the
> > callback also calls unregister_reboot_notifier() unconditionally, which
> > blocks on the reboot chain rwsem (down_write vs. the in-progress
> > chain's down_read) while the notifier waits for the callback - the
> > deadlock would just move to a different lock. Fix both sides:
> >
> > - Notifier: compute the decision under state_mtx, drop the lock, then
> > cancel. A callback that slips in between sees state ==
> > SCMI_SYSPOWER_REBOOTING (not IN_PROGRESS), skips the forceful
> > request and finishes, so the cancel cannot miss it. (The callback's
> > own system_state early-return does not cover a work item that
> > started before the reboot began: kernel_restart_prepare() runs the
> > notifier chain before setting system_state.)
> >
> > - Callback: only unregister the reboot notifier when actually forcing
> > the transition - that is the only case that needs it, because the
> > forceful kernel_restart()/kernel_power_off() would otherwise
> > re-enter scmi_reboot_notifier and self-deadlock on state_mtx. When
> > the state is not IN_PROGRESS there is nothing to force, and skipping
> > the unregister keeps the callback from blocking on the reboot chain
> > rwsem while the notifier waits for it.
> >
> > Found by static analysis.
> >
> > Cc: stable@vger.kernel.org
> > Fixes: 2c4b97fee94a ("firmware: arm_scmi: Add SCMI System Power Control driver")
> > Signed-off-by: Chengyu Liu <lui2012qwq@gmail.com>
> > [Cristian: drop lock before triggering forceful reboot (needs_forceful)]
> > Tested-by: Cristian Marussi <cristian.marussi@arm.com>
> > Signed-off-by: Cristian Marussi <cristian.marussi@arm.com>
> > ---
> > .../firmware/arm_scmi/scmi_power_control.c | 26 ++++++++++++++-----
> > 1 file changed, 19 insertions(+), 7 deletions(-)
> >
> > diff --git a/drivers/firmware/arm_scmi/scmi_power_control.c b/drivers/firmware/arm_scmi/scmi_power_control.c
> > index e791a206768c..e0680e33180e 100644
> > --- a/drivers/firmware/arm_scmi/scmi_power_control.c
> > +++ b/drivers/firmware/arm_scmi/scmi_power_control.c
> > @@ -121,6 +121,7 @@ static int scmi_reboot_notifier(struct notifier_block *nb,
> > unsigned long reason, void *__unused)
> > {
> > struct scmi_syspower_conf *sc = reboot_nb_to_sconf(nb);
> > + bool cancel_work;
> >
> > mutex_lock(&sc->state_mtx);
> > switch (reason) {
> > @@ -138,11 +139,18 @@ static int scmi_reboot_notifier(struct notifier_block *nb,
> > break;
> > }
> >
> > - if (sc->state == SCMI_SYSPOWER_REBOOTING) {
> > + cancel_work = sc->state == SCMI_SYSPOWER_REBOOTING;
> > + mutex_unlock(&sc->state_mtx);
> > +
> > + if (cancel_work) {
> > dev_dbg(sc->dev, "Reboot in progress...cancel delayed work.\n");
> > + /*
> > + * cancel_delayed_work_sync() waits for a running
> > + * scmi_forceful_work_func(), which takes state_mtx - it must
> > + * not be called with the lock held.
> > + */
> > cancel_delayed_work_sync(&sc->forceful_work);
>
> Sashiko still complains about the deadlock.
>
Yes I know.
> How about changing this to non-sync cancel_delayed_work() so that it doesn't
> wait and deadlock like it complains ?
>
Indeed I think this is just the solution...I mean dropping this patch as
a whole and keeping the original wider locking section BUT do not cancel
synchronously, but simply with cancel_delayed_work(&sc->forceful_work);
> > }
> > - mutex_unlock(&sc->state_mtx);
> >
> > return NOTIFY_OK;
> > }
> > @@ -181,6 +189,7 @@ static void scmi_forceful_work_func(struct work_struct *work)
> > {
> > struct scmi_syspower_conf *sc;
> > struct delayed_work *dwork;
> > + bool needs_forceful;
> >
> > if (system_state > SYSTEM_RUNNING)
> > return;
> > @@ -188,13 +197,16 @@ static void scmi_forceful_work_func(struct work_struct *work)
> > dwork = to_delayed_work(work);
> > sc = dwork_to_sconf(dwork);
> >
> > - dev_dbg(sc->dev, "Graceful request timed out...forcing !\n");
> > mutex_lock(&sc->state_mtx);
> > - /* avoid deadlock by unregistering reboot notifier first */
> > - unregister_reboot_notifier(&sc->reboot_nb);
> > - if (sc->state == SCMI_SYSPOWER_IN_PROGRESS)
> > - scmi_request_forceful_transition(sc);
> > + needs_forceful = sc->state == SCMI_SYSPOWER_IN_PROGRESS;
> > mutex_unlock(&sc->state_mtx);
> > +
> > + if (needs_forceful) {
> > + dev_dbg(sc->dev, "Graceful request timed out...forcing !\n");
> > + /* avoid deadlock by unregistering reboot notifier first */
> > + unregister_reboot_notifier(&sc->reboot_nb);
>
> And re-evaluate needs_forceful again here after un-registration before
> calling below forced transition ?
>
If all of this is locked as it was originally there is no need, the two
threads should just sync properly...
Let's see what Sashiko thinks..
Thanks,
Cristian
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2026-11-17 11:25 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <20261116084211.3066460-1-cristian.marussi@arm.com>
2026-11-16 8:42 ` [PATCH 1/2] firmware: arm_scmi: Fix deadlock between reboot notifier and forceful work Cristian Marussi
2026-11-16 18:25 ` Sudeep Holla
2026-11-17 11:25 ` Cristian Marussi
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox