[PATCH 0/2] drm/xe/guc: handle exec queue teardown after hot-unplug

[Nitin Gote <nitin.r.gote@intel.com>]

This series moves the hot-unplug handling for GuC exec queue teardown into
the GuC message submision layer.

The issue was seen with the new core_hotunplug "with-load" IGT subtests,
where an exec queue can be destroyed from a deferred fd close after the
device has already been hot-unplugged:

  guc_exec_queue_add_msg()
  guc_exec_queue_destroy()
  xe_exec_queue_destroy()
  xe_file_close()

The warning is:

  Missing outer runtime PM protection

After some debugging, this does not look like a missing runtime PM
reference or a zero usage count. At the warning point I saw:
usage_count=3, status=RPM_SUSPENDED and disable_depth=1

During hot-unplug, the PCI remove path disables runtime PM. The old
struct device can still remain alive because the old drm_device is still
referenced by an open fd. When that fd is finally closed,
guc_exec_queue_destroy() runs on the old device where runtime PM is
already disabled. In that state pm_runtime_get_if_in_use() returns
-EINVAL even with a non-zero usage count, so
xe_pm_runtime_get_noresume() cannot observe the existing PM state and
warns.

Instead of suppressing the warning in xe_pm_runtime_get_noresume(), this
series handles the unplug case where the GuC exec queue message is queued
and processed.

Patch 1 reworks the GuC exec queue message teardown handling.
Patch 2 removes EXEC_QUEUE_FLAG_PERMANENT, which is no longer needed.

Observed with new igt tests:

  igt@core_hotunplug@hotreplug-with-load
  igt@core_hotunplug@hotunplug-rescan-with-load
  
Link: https://patchwork.freedesktop.org/patch/725773/?series=166744&rev=4

v2:
 - Drop the drm_dev_is_unplugged() bypass from guc_exec_queue_destroy()
   and instead exclude hot-unplug from the WARN in
   xe_pm_runtime_get_noresume().

v3:
 - Rework the fix in the message-submission layer instead of touching the
   WARN; gate PM/HW work on drm_dev_enter() and route all queues through
   the CLEANUP message. (Matthew Brost)
 - Prove the root cause (runtime PM disabled on unplug, not a zero
   refcount) and record it in the commit message. (Matt B, Raag Jadav)
 - Also remove EXEC_QUEUE_FLAG_PERMANENT flag in
   separate patch (Matthew Brost)


Nitin Gote (2):
  drm/xe/guc: rework exec queue teardown PM/unplug handling
  drm/xe: remove EXEC_QUEUE_FLAG_PERMANENT

 drivers/gpu/drm/xe/xe_exec_queue.c       |  3 -
 drivers/gpu/drm/xe/xe_exec_queue_types.h | 14 ++--
 drivers/gpu/drm/xe/xe_gsc.c              |  3 +-
 drivers/gpu/drm/xe/xe_guc_submit.c       | 98 ++++++++++++------------
 drivers/gpu/drm/xe/xe_migrate.c          |  2 -
 drivers/gpu/drm/xe/xe_pm.c               |  5 ++
 drivers/gpu/drm/xe/xe_pxp_submit.c       |  5 +-
 drivers/gpu/drm/xe/xe_sriov_vf_ccs.c     |  1 -
 drivers/gpu/drm/xe/xe_vm.c               |  1 -
 9 files changed, 65 insertions(+), 67 deletions(-)

-- 
2.50.1