* [PATCH v3] accel/ivpu: Defer recovery until probe completes
@ 2026-11-10 12:15 Karol Wachowski
2026-11-12 17:35 ` Jeff Hugo
2026-11-15 15:24 ` Toast
0 siblings, 2 replies; 4+ messages in thread
From: Karol Wachowski @ 2026-11-10 12:15 UTC (permalink / raw)
To: Daksh Sahni, dri-devel
Cc: Andrzej Kacprowski, Oded Gabbay, Jeff Hugo, dawid.osuchowski,
Karol Wachowski, stable
An error IRQ that fires after ivpu_boot() enables interrupts queues
recovery_work during probe. If probe then fails, the device is torn
down while the work runs, and the work faults on the unmapped BARs:
BUG: unable to handle page fault for address: ffffcfc6680200f4
Workqueue: events_unbound ivpu_pm_recovery_work [intel_vpu]
RIP: 0010:ivpu_hw_ip_ipc_rx_addr_get+0xc/0xb0 [intel_vpu]
Call Trace:
ivpu_wait_for_ready+0x155/0x220 [intel_vpu]
ivpu_boot+0x9e/0x270 [intel_vpu]
ivpu_resume+0xa4/0xd0 [intel_vpu]
ivpu_pm_reset_complete+0x2c/0x90 [intel_vpu]
ivpu_pm_recovery_work+0xf6/0x140 [intel_vpu]
If probe succeeds, the recovery runs concurrently with the rest of
probe, and its pm_runtime_disable() discards the initial autosuspend
request, leaving the NPU active until first use. Any recovery that
starts while the NPU is idle loses the pending autosuspend the same way.
Keep recovery_work disabled from ivpu_pm_init() until ivpu_pm_enable().
A trigger during probe still runs diagnostics and masks interrupts, and
ivpu_pm_enable() runs the deferred recovery before the initial
autosuspend request. A trigger racing with ivpu_pm_enable() can cause
a second, harmless recovery. Request autosuspend at the end of
ivpu_pm_reset_complete() so the NPU suspends again after every reset.
Fixes: 852be13f3bd3 ("accel/ivpu: Add PM support")
Cc: stable@vger.kernel.org
Reported-by: Daksh Sahni <swizzguy76@gmail.com>
Closes: https://lore.kernel.org/r/20261107174215.15818-1-swizzguy76@gmail.com
Signed-off-by: Karol Wachowski <karol.wachowski@linux.intel.com>
---
Notes:
v3:
- Request autosuspend at the end of ivpu_pm_reset_complete().
pm_runtime_disable() in ivpu_pm_reset_begin() cancels a pending
autosuspend request and pm_runtime_enable() does not re-issue it, so
a recovery racing with the initial autosuspend request in
ivpu_pm_enable() left the NPU active until first use (Sashiko).
A trigger racing with ivpu_pm_enable() can still queue recovery
twice. The second run is a full reset that leaves the device in a
consistent state, and autosuspend is now re-requested after each
reset.
- Replace pm_runtime_mark_last_busy() with pm_request_autosuspend() in
ivpu_pm_reset_complete(); it marks the device busy itself.
- Shorten the commit message.
- v2: https://lore.kernel.org/r/20261110085003.3527014-1-karol.wachowski@linux.intel.com
v2:
- Rework of Daksh's "accel/ivpu: Disable recovery work on probe failure":
https://lore.kernel.org/r/20261107174215.15818-1-swizzguy76@gmail.com
- Keep recovery_work disabled from ivpu_pm_init() until ivpu_pm_enable()
instead of disabling it on the probe error path, which is too late as
recovery may already run concurrently with ivpu_boot() (Sashiko)
- Run a recovery requested during probe from ivpu_pm_enable() and wait
for it before the initial autosuspend request, so the NPU is reset
before use and the autosuspend is not lost when probe succeeds
v1:
- Link: https://lore.kernel.org/r/20261107174215.15818-1-swizzguy76@gmail.com
drivers/accel/ivpu/ivpu_pm.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/drivers/accel/ivpu/ivpu_pm.c b/drivers/accel/ivpu/ivpu_pm.c
index aa813262253e..e4b79c946ca0 100644
--- a/drivers/accel/ivpu/ivpu_pm.c
+++ b/drivers/accel/ivpu/ivpu_pm.c
@@ -150,8 +150,8 @@ static void ivpu_pm_reset_complete(struct ivpu_device *vdev)
up_write(&vdev->pm->reset_lock);
atomic_set(&vdev->pm->reset_pending, 0);
- pm_runtime_mark_last_busy(vdev->drm.dev);
pm_runtime_enable(vdev->drm.dev);
+ pm_request_autosuspend(vdev->drm.dev);
}
static void ivpu_pm_recovery_work(struct work_struct *work)
@@ -415,6 +415,7 @@ void ivpu_pm_init(struct ivpu_device *vdev)
atomic_set(&pm->engine_reset_counter, 0);
INIT_WORK(&pm->recovery_work, ivpu_pm_recovery_work);
+ disable_work(&pm->recovery_work);
INIT_DELAYED_WORK(&pm->job_timeout_work, ivpu_job_timeout_work);
if (ivpu_disable_recovery)
@@ -439,6 +440,12 @@ void ivpu_pm_enable(struct ivpu_device *vdev)
{
struct device *dev = vdev->drm.dev;
+ enable_work(&vdev->pm->recovery_work);
+ if (atomic_read(&vdev->pm->reset_pending)) {
+ queue_work(system_dfl_wq, &vdev->pm->recovery_work);
+ flush_work(&vdev->pm->recovery_work);
+ }
+
pm_runtime_allow(dev);
pm_runtime_put_autosuspend(dev);
}
base-commit: 54d3ff1b5ac08d0e28029b93a1719ddd9ff670a4
--
2.43.0
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH v3] accel/ivpu: Defer recovery until probe completes
2026-11-10 12:15 [PATCH v3] accel/ivpu: Defer recovery until probe completes Karol Wachowski
@ 2026-11-12 17:35 ` Jeff Hugo
2026-11-15 15:24 ` Toast
1 sibling, 0 replies; 4+ messages in thread
From: Jeff Hugo @ 2026-11-12 17:35 UTC (permalink / raw)
To: Karol Wachowski, Daksh Sahni, dri-devel
Cc: Andrzej Kacprowski, Oded Gabbay, dawid.osuchowski, stable
On 11/10/2026 5:15 AM, Karol Wachowski wrote:
> An error IRQ that fires after ivpu_boot() enables interrupts queues
> recovery_work during probe. If probe then fails, the device is torn
> down while the work runs, and the work faults on the unmapped BARs:
>
> BUG: unable to handle page fault for address: ffffcfc6680200f4
> Workqueue: events_unbound ivpu_pm_recovery_work [intel_vpu]
> RIP: 0010:ivpu_hw_ip_ipc_rx_addr_get+0xc/0xb0 [intel_vpu]
> Call Trace:
> ivpu_wait_for_ready+0x155/0x220 [intel_vpu]
> ivpu_boot+0x9e/0x270 [intel_vpu]
> ivpu_resume+0xa4/0xd0 [intel_vpu]
> ivpu_pm_reset_complete+0x2c/0x90 [intel_vpu]
> ivpu_pm_recovery_work+0xf6/0x140 [intel_vpu]
>
> If probe succeeds, the recovery runs concurrently with the rest of
> probe, and its pm_runtime_disable() discards the initial autosuspend
> request, leaving the NPU active until first use. Any recovery that
> starts while the NPU is idle loses the pending autosuspend the same way.
>
> Keep recovery_work disabled from ivpu_pm_init() until ivpu_pm_enable().
> A trigger during probe still runs diagnostics and masks interrupts, and
> ivpu_pm_enable() runs the deferred recovery before the initial
> autosuspend request. A trigger racing with ivpu_pm_enable() can cause
> a second, harmless recovery. Request autosuspend at the end of
> ivpu_pm_reset_complete() so the NPU suspends again after every reset.
>
> Fixes: 852be13f3bd3 ("accel/ivpu: Add PM support")
> Cc: stable@vger.kernel.org
> Reported-by: Daksh Sahni <swizzguy76@gmail.com>
> Closes: https://lore.kernel.org/r/20261107174215.15818-1-swizzguy76@gmail.com
> Signed-off-by: Karol Wachowski <karol.wachowski@linux.intel.com>
Reviewed-by: Jeff Hugo <jeff.hugo@oss.qualcomm.com>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH v3] accel/ivpu: Defer recovery until probe completes
2026-11-10 12:15 [PATCH v3] accel/ivpu: Defer recovery until probe completes Karol Wachowski
2026-11-12 17:35 ` Jeff Hugo
@ 2026-11-15 15:24 ` Toast
2026-11-16 7:09 ` Karol Wachowski
1 sibling, 1 reply; 4+ messages in thread
From: Toast @ 2026-11-15 15:24 UTC (permalink / raw)
To: Karol Wachowski
Cc: dri-devel, Andrzej Kacprowski, Oded Gabbay, Jeff Hugo,
dawid.osuchowski, stable
Tested v3 on Meteor Lake (Core Ultra 5 125H, vpu_37xx firmware 20260219)
on top of v7.4-rc1 (d48723711bae):
- 4 kexec runs with the NPU runtime-suspended (one from a 7.0 kernel,
three from 7.4): every run hit ATS_ERR during probe, recovery_work was
not queued, probe failed with -ETIMEDOUT, and there was no oops
- FLR plus rebind brings the NPU back, and it autosuspends afterwards
- debugfs force_recovery after a successful probe recovers the NPU, and
it returns to runtime-suspended
None of the runs had probe succeed after the error IRQ, so the deferred
recovery in ivpu_pm_enable() isn't covered by this.
One question: v2 had "Cc: stable@vger.kernel.org # v6.10+" and v3 dropped
the annotation. disable_work()/enable_work() only exist since v6.10 and
the Fixes: commit is in v6.3, so should the "# v6.10+" come back?
Tested-by: Daksh Sahni <swizzguy76@gmail.com>
On Tue, Nov 10, 2026 at 5:45 PM Karol Wachowski
<karol.wachowski@linux.intel.com> wrote:
>
> An error IRQ that fires after ivpu_boot() enables interrupts queues
> recovery_work during probe. If probe then fails, the device is torn
> down while the work runs, and the work faults on the unmapped BARs:
>
> BUG: unable to handle page fault for address: ffffcfc6680200f4
> Workqueue: events_unbound ivpu_pm_recovery_work [intel_vpu]
> RIP: 0010:ivpu_hw_ip_ipc_rx_addr_get+0xc/0xb0 [intel_vpu]
> Call Trace:
> ivpu_wait_for_ready+0x155/0x220 [intel_vpu]
> ivpu_boot+0x9e/0x270 [intel_vpu]
> ivpu_resume+0xa4/0xd0 [intel_vpu]
> ivpu_pm_reset_complete+0x2c/0x90 [intel_vpu]
> ivpu_pm_recovery_work+0xf6/0x140 [intel_vpu]
>
> If probe succeeds, the recovery runs concurrently with the rest of
> probe, and its pm_runtime_disable() discards the initial autosuspend
> request, leaving the NPU active until first use. Any recovery that
> starts while the NPU is idle loses the pending autosuspend the same way.
>
> Keep recovery_work disabled from ivpu_pm_init() until ivpu_pm_enable().
> A trigger during probe still runs diagnostics and masks interrupts, and
> ivpu_pm_enable() runs the deferred recovery before the initial
> autosuspend request. A trigger racing with ivpu_pm_enable() can cause
> a second, harmless recovery. Request autosuspend at the end of
> ivpu_pm_reset_complete() so the NPU suspends again after every reset.
>
> Fixes: 852be13f3bd3 ("accel/ivpu: Add PM support")
> Cc: stable@vger.kernel.org
> Reported-by: Daksh Sahni <swizzguy76@gmail.com>
> Closes: https://lore.kernel.org/r/20261107174215.15818-1-swizzguy76@gmail.com
> Signed-off-by: Karol Wachowski <karol.wachowski@linux.intel.com>
> ---
>
> Notes:
> v3:
> - Request autosuspend at the end of ivpu_pm_reset_complete().
> pm_runtime_disable() in ivpu_pm_reset_begin() cancels a pending
> autosuspend request and pm_runtime_enable() does not re-issue it, so
> a recovery racing with the initial autosuspend request in
> ivpu_pm_enable() left the NPU active until first use (Sashiko).
> A trigger racing with ivpu_pm_enable() can still queue recovery
> twice. The second run is a full reset that leaves the device in a
> consistent state, and autosuspend is now re-requested after each
> reset.
> - Replace pm_runtime_mark_last_busy() with pm_request_autosuspend() in
> ivpu_pm_reset_complete(); it marks the device busy itself.
> - Shorten the commit message.
> - v2: https://lore.kernel.org/r/20261110085003.3527014-1-karol.wachowski@linux.intel.com
>
> v2:
> - Rework of Daksh's "accel/ivpu: Disable recovery work on probe failure":
> https://lore.kernel.org/r/20261107174215.15818-1-swizzguy76@gmail.com
> - Keep recovery_work disabled from ivpu_pm_init() until ivpu_pm_enable()
> instead of disabling it on the probe error path, which is too late as
> recovery may already run concurrently with ivpu_boot() (Sashiko)
> - Run a recovery requested during probe from ivpu_pm_enable() and wait
> for it before the initial autosuspend request, so the NPU is reset
> before use and the autosuspend is not lost when probe succeeds
>
> v1:
> - Link: https://lore.kernel.org/r/20261107174215.15818-1-swizzguy76@gmail.com
>
> drivers/accel/ivpu/ivpu_pm.c | 9 ++++++++-
> 1 file changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/accel/ivpu/ivpu_pm.c b/drivers/accel/ivpu/ivpu_pm.c
> index aa813262253e..e4b79c946ca0 100644
> --- a/drivers/accel/ivpu/ivpu_pm.c
> +++ b/drivers/accel/ivpu/ivpu_pm.c
> @@ -150,8 +150,8 @@ static void ivpu_pm_reset_complete(struct ivpu_device *vdev)
> up_write(&vdev->pm->reset_lock);
> atomic_set(&vdev->pm->reset_pending, 0);
>
> - pm_runtime_mark_last_busy(vdev->drm.dev);
> pm_runtime_enable(vdev->drm.dev);
> + pm_request_autosuspend(vdev->drm.dev);
> }
>
> static void ivpu_pm_recovery_work(struct work_struct *work)
> @@ -415,6 +415,7 @@ void ivpu_pm_init(struct ivpu_device *vdev)
> atomic_set(&pm->engine_reset_counter, 0);
>
> INIT_WORK(&pm->recovery_work, ivpu_pm_recovery_work);
> + disable_work(&pm->recovery_work);
> INIT_DELAYED_WORK(&pm->job_timeout_work, ivpu_job_timeout_work);
>
> if (ivpu_disable_recovery)
> @@ -439,6 +440,12 @@ void ivpu_pm_enable(struct ivpu_device *vdev)
> {
> struct device *dev = vdev->drm.dev;
>
> + enable_work(&vdev->pm->recovery_work);
> + if (atomic_read(&vdev->pm->reset_pending)) {
> + queue_work(system_dfl_wq, &vdev->pm->recovery_work);
> + flush_work(&vdev->pm->recovery_work);
> + }
> +
> pm_runtime_allow(dev);
> pm_runtime_put_autosuspend(dev);
> }
>
> base-commit: 54d3ff1b5ac08d0e28029b93a1719ddd9ff670a4
> --
> 2.43.0
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH v3] accel/ivpu: Defer recovery until probe completes
2026-11-15 15:24 ` Toast
@ 2026-11-16 7:09 ` Karol Wachowski
0 siblings, 0 replies; 4+ messages in thread
From: Karol Wachowski @ 2026-11-16 7:09 UTC (permalink / raw)
To: Toast
Cc: dri-devel, Andrzej Kacprowski, Oded Gabbay, Jeff Hugo,
dawid.osuchowski, stable
On 15-Nov-26 16:24, Toast wrote:
> Tested v3 on Meteor Lake (Core Ultra 5 125H, vpu_37xx firmware 20260219)
> on top of v7.4-rc1 (d48723711bae):
>
> - 4 kexec runs with the NPU runtime-suspended (one from a 7.0 kernel,
> three from 7.4): every run hit ATS_ERR during probe, recovery_work was
> not queued, probe failed with -ETIMEDOUT, and there was no oops
> - FLR plus rebind brings the NPU back, and it autosuspends afterwards
> - debugfs force_recovery after a successful probe recovers the NPU, and
> it returns to runtime-suspended
>
> None of the runs had probe succeed after the error IRQ, so the deferred
> recovery in ivpu_pm_enable() isn't covered by this.
>
> One question: v2 had"Cc: stable@vger.kernel.org # v6.10+" and v3 dropped
> the annotation. disable_work()/enable_work() only exist since v6.10 and
> the Fixes: commit is in v6.3, so should the "# v6.10+" come back?
Thanks for testing, and good catch. I added "# v6.10+" to the stable tag
when applying.
Applied to drm-misc-fixes.
>
> Tested-by: Daksh Sahni<swizzguy76@gmail.com>
>
> On Tue, Nov 10, 2026 at 5:45 PM Karol Wachowski
> <karol.wachowski@linux.intel.com> wrote:
>> An error IRQ that fires after ivpu_boot() enables interrupts queues
>> recovery_work during probe. If probe then fails, the device is torn
>> down while the work runs, and the work faults on the unmapped BARs:
>>
>> BUG: unable to handle page fault for address: ffffcfc6680200f4
>> Workqueue: events_unbound ivpu_pm_recovery_work [intel_vpu]
>> RIP: 0010:ivpu_hw_ip_ipc_rx_addr_get+0xc/0xb0 [intel_vpu]
>> Call Trace:
>> ivpu_wait_for_ready+0x155/0x220 [intel_vpu]
>> ivpu_boot+0x9e/0x270 [intel_vpu]
>> ivpu_resume+0xa4/0xd0 [intel_vpu]
>> ivpu_pm_reset_complete+0x2c/0x90 [intel_vpu]
>> ivpu_pm_recovery_work+0xf6/0x140 [intel_vpu]
>>
>> If probe succeeds, the recovery runs concurrently with the rest of
>> probe, and its pm_runtime_disable() discards the initial autosuspend
>> request, leaving the NPU active until first use. Any recovery that
>> starts while the NPU is idle loses the pending autosuspend the same way.
>>
>> Keep recovery_work disabled from ivpu_pm_init() until ivpu_pm_enable().
>> A trigger during probe still runs diagnostics and masks interrupts, and
>> ivpu_pm_enable() runs the deferred recovery before the initial
>> autosuspend request. A trigger racing with ivpu_pm_enable() can cause
>> a second, harmless recovery. Request autosuspend at the end of
>> ivpu_pm_reset_complete() so the NPU suspends again after every reset.
>>
>> Fixes: 852be13f3bd3 ("accel/ivpu: Add PM support")
>> Cc:stable@vger.kernel.org
>> Reported-by: Daksh Sahni<swizzguy76@gmail.com>
>> Closes:https://lore.kernel.org/r/20261107174215.15818-1-swizzguy76@gmail.com
>> Signed-off-by: Karol Wachowski<karol.wachowski@linux.intel.com>
>> ---
>>
>> Notes:
>> v3:
>> - Request autosuspend at the end of ivpu_pm_reset_complete().
>> pm_runtime_disable() in ivpu_pm_reset_begin() cancels a pending
>> autosuspend request and pm_runtime_enable() does not re-issue it, so
>> a recovery racing with the initial autosuspend request in
>> ivpu_pm_enable() left the NPU active until first use (Sashiko).
>> A trigger racing with ivpu_pm_enable() can still queue recovery
>> twice. The second run is a full reset that leaves the device in a
>> consistent state, and autosuspend is now re-requested after each
>> reset.
>> - Replace pm_runtime_mark_last_busy() with pm_request_autosuspend() in
>> ivpu_pm_reset_complete(); it marks the device busy itself.
>> - Shorten the commit message.
>> - v2:https://lore.kernel.org/r/20261110085003.3527014-1-
>> karol.wachowski@linux.intel.com
>>
>> v2:
>> - Rework of Daksh's "accel/ivpu: Disable recovery work on probe failure":
>> https://lore.kernel.org/r/20261107174215.15818-1-swizzguy76@gmail.com
>> - Keep recovery_work disabled from ivpu_pm_init() until ivpu_pm_enable()
>> instead of disabling it on the probe error path, which is too late as
>> recovery may already run concurrently with ivpu_boot() (Sashiko)
>> - Run a recovery requested during probe from ivpu_pm_enable() and wait
>> for it before the initial autosuspend request, so the NPU is reset
>> before use and the autosuspend is not lost when probe succeeds
>>
>> v1:
>> - Link:https://lore.kernel.org/r/20261107174215.15818-1-swizzguy76@gmail.com
>>
>> drivers/accel/ivpu/ivpu_pm.c | 9 ++++++++-
>> 1 file changed, 8 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/accel/ivpu/ivpu_pm.c b/drivers/accel/ivpu/ivpu_pm.c
>> index aa813262253e..e4b79c946ca0 100644
>> --- a/drivers/accel/ivpu/ivpu_pm.c
>> +++ b/drivers/accel/ivpu/ivpu_pm.c
>> @@ -150,8 +150,8 @@ static void ivpu_pm_reset_complete(struct ivpu_device *vdev)
>> up_write(&vdev->pm->reset_lock);
>> atomic_set(&vdev->pm->reset_pending, 0);
>>
>> - pm_runtime_mark_last_busy(vdev->drm.dev);
>> pm_runtime_enable(vdev->drm.dev);
>> + pm_request_autosuspend(vdev->drm.dev);
>> }
>>
>> static void ivpu_pm_recovery_work(struct work_struct *work)
>> @@ -415,6 +415,7 @@ void ivpu_pm_init(struct ivpu_device *vdev)
>> atomic_set(&pm->engine_reset_counter, 0);
>>
>> INIT_WORK(&pm->recovery_work, ivpu_pm_recovery_work);
>> + disable_work(&pm->recovery_work);
>> INIT_DELAYED_WORK(&pm->job_timeout_work, ivpu_job_timeout_work);
>>
>> if (ivpu_disable_recovery)
>> @@ -439,6 +440,12 @@ void ivpu_pm_enable(struct ivpu_device *vdev)
>> {
>> struct device *dev = vdev->drm.dev;
>>
>> + enable_work(&vdev->pm->recovery_work);
>> + if (atomic_read(&vdev->pm->reset_pending)) {
>> + queue_work(system_dfl_wq, &vdev->pm->recovery_work);
>> + flush_work(&vdev->pm->recovery_work);
>> + }
>> +
>> pm_runtime_allow(dev);
>> pm_runtime_put_autosuspend(dev);
>> }
>>
>> base-commit: 54d3ff1b5ac08d0e28029b93a1719ddd9ff670a4
>> --
>> 2.43.0
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2026-11-16 7:10 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-10 12:15 [PATCH v3] accel/ivpu: Defer recovery until probe completes Karol Wachowski
2026-11-12 17:35 ` Jeff Hugo
2026-11-15 15:24 ` Toast
2026-11-16 7:09 ` Karol Wachowski
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox