[PATCH v2] accel/ivpu: Defer recovery until probe completes

[PATCH v2] accel/ivpu: Defer recovery until probe completes

[Karol Wachowski]

The Buttress, MMU and WDT IRQ handlers call ivpu_pm_trigger_recovery()
once ivpu_boot() enables interrupts. If such an IRQ fires during probe
and ivpu_boot() then fails, the probe error path tears the device down
while recovery_work is still queued or running. devres unmaps the
register BARs and the work faults on its first register access:

  BUG: unable to handle page fault for address: ffffcfc6680200f4
  Workqueue: events_unbound ivpu_pm_recovery_work [intel_vpu]
  RIP: 0010:ivpu_hw_ip_ipc_rx_addr_get+0xc/0xb0 [intel_vpu]
  Call Trace:
   ivpu_wait_for_ready+0x155/0x220 [intel_vpu]
   ivpu_boot+0x9e/0x270 [intel_vpu]
   ivpu_resume+0xa4/0xd0 [intel_vpu]
   ivpu_pm_reset_complete+0x2c/0x90 [intel_vpu]
   ivpu_pm_recovery_work+0xf6/0x140 [intel_vpu]

If ivpu_boot() succeeds instead, recovery runs concurrently with the
rest of probe and can disable runtime PM while ivpu_pm_enable() issues
the initial autosuspend request, leaving the NPU active until first use.

Disable recovery_work in ivpu_pm_init() and enable it in
ivpu_pm_enable() before runtime PM is allowed. A trigger during probe
still records reset_pending, runs failure diagnostics and masks NPU
interrupts, but cannot queue the work. If ivpu_boot() fails, probe
unwinds with no work pending. If probe succeeds, ivpu_pm_enable() runs
the deferred recovery and waits for it to finish before issuing the
initial autosuspend request, so the NPU is reset before it is used and
the request is not lost. reset_pending is checked after enable_work(),
so a trigger racing with ivpu_pm_enable() cannot be lost either.

Fixes: 852be13f3bd3 ("accel/ivpu: Add PM support")
Cc: stable@vger.kernel.org # v6.10+
Reported-by: Daksh Sahni <swizzguy76@gmail.com>
Closes: https://lore.kernel.org/r/20261107174215.15818-1-swizzguy76@gmail.com
Signed-off-by: Karol Wachowski <karol.wachowski@linux.intel.com>
---

Notes:
    v1:
    - Link: https://lore.kernel.org/r/20261107174215.15818-1-swizzguy76@gmail.com
    v2:
    - Rework of Daksh's "accel/ivpu: Disable recovery work on probe failure":
      https://lore.kernel.org/r/20261107174215.15818-1-swizzguy76@gmail.com
    - Keep recovery_work disabled from ivpu_pm_init() until ivpu_pm_enable()
      instead of disabling it on the probe error path, which is too late as
      recovery may already run concurrently with ivpu_boot() (Sashiko)
    - Run a recovery requested during probe from ivpu_pm_enable() and wait
      for it before the initial autosuspend request, so the NPU is reset
      before use and the autosuspend is not lost when probe succeeds
    - Add stable prerequisite 69a79ada8eb0

 drivers/accel/ivpu/ivpu_pm.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/drivers/accel/ivpu/ivpu_pm.c b/drivers/accel/ivpu/ivpu_pm.c
index aa813262253e..7aee728b19ef 100644
--- a/drivers/accel/ivpu/ivpu_pm.c
+++ b/drivers/accel/ivpu/ivpu_pm.c
@@ -415,6 +415,7 @@ void ivpu_pm_init(struct ivpu_device *vdev)
 	atomic_set(&pm->engine_reset_counter, 0);
 
 	INIT_WORK(&pm->recovery_work, ivpu_pm_recovery_work);
+	disable_work(&pm->recovery_work);
 	INIT_DELAYED_WORK(&pm->job_timeout_work, ivpu_job_timeout_work);
 
 	if (ivpu_disable_recovery)
@@ -439,6 +440,12 @@ void ivpu_pm_enable(struct ivpu_device *vdev)
 {
 	struct device *dev = vdev->drm.dev;
 
+	enable_work(&vdev->pm->recovery_work);
+	if (atomic_read(&vdev->pm->reset_pending)) {
+		queue_work(system_dfl_wq, &vdev->pm->recovery_work);
+		flush_work(&vdev->pm->recovery_work);
+	}
+
 	pm_runtime_allow(dev);
 	pm_runtime_put_autosuspend(dev);
 }

base-commit: 54d3ff1b5ac08d0e28029b93a1719ddd9ff670a4
-- 
2.43.0

Re: [PATCH v2] accel/ivpu: Defer recovery until probe completes

[Toast]

Tested on Meteor Lake (Core Ultra 5 125H, vpu_37xx firmware 20260219)
on top of v7.4-rc1:

- kexec from a 7.0 kernel and from 7.4 with the NPU runtime-suspended:
  both runs hit ATS_ERR during probe, recovery_work was not queued,
  probe failed once with -ETIMEDOUT, and there was no oops
- FLR plus rebind brings the NPU back
- debugfs force_recovery after a successful probe recovers the NPU

I couldn't hit an error IRQ after ivpu_boot() succeeds, so the deferred
recovery in ivpu_pm_enable() isn't covered by this.

Tested-by: Daksh Sahni <swizzguy76@gmail.com>

Re: [PATCH v2] accel/ivpu: Defer recovery until probe completes

[Karol Wachowski]

On 13-Nov-26 9:49, Toast wrote:
> Tested on Meteor Lake (Core Ultra 5 125H, vpu_37xx firmware 20260219)
> on top of v7.4-rc1:
> 
> - kexec from a 7.0 kernel and from 7.4 with the NPU runtime-suspended:
>    both runs hit ATS_ERR during probe, recovery_work was not queued,
>    probe failed once with -ETIMEDOUT, and there was no oops
> - FLR plus rebind brings the NPU back
> - debugfs force_recovery after a successful probe recovers the NPU
> 
> I couldn't hit an error IRQ after ivpu_boot() succeeds, so the deferred
> recovery in ivpu_pm_enable() isn't covered by this.
> 
> Tested-by: Daksh Sahni <swizzguy76@gmail.com>

Hi Daksh,

Thanks a lot for testing!

Just a heads-up: I already sent v3 a few days ago:
https://lore.kernel.org/all/20261110121534.3566028-1-karol.wachowski@linux.intel.com/

which includes a small adjustment in ivpu_pm_reset_complete() so 
autosuspend isn't lost after a reset (reported by Sashiko).

Because of this change, I haven't carried over your Tested-by tag 
automatically. If you happen to have a spare moment to give v3 a quick 
spin, that would be great, otherwise no worries I've already have your 
Reported-by tag.

Thanks,
Karol