[PATCH V2 00/31] Rework GPU TLB invalidation

[Alex Deucher <alexander.deucher@amd.com>]

GMC 9-12 use KIQ or MES for TLB invalidations to avoid using MMIO which would
require disallowing GFXOFF.  KIQ and MES are management queues however and if
they hang, they cannot be recovered by a queue reset since they are the
mechanisms which handle queue resets.  Since using KIQ or MES will exit GFXOFF
anyway, explicitly disallow it on the MMIO path and use that. Next, switch to
using SDMA for TLB invalidations.  SDMA 4.4.x and newer have special packets
specifically for this purpose.  If SDMA hangs while doing the invalidation for
some reason, it's easier to reset the SDMA queue than KIQ or MES.  Finally,
most of the TLB invalidation code between GMC 9 through 12 was identical, so
move it to common GMC helpers and remove the IP specific code.  If the SMDA
and MMIO pathes prove to be stable, the KIQ pathes can be removed in the future
to further simplify things. SDMA 4.x could also be updated to support PASID
invalidation via SDMA using either the new packet (SDMA 4.4.x) or via
REG_WRITE/REG_WAIT packets (SDMA 4.0.x).

Code is available on this branch as well:
https://gitlab.freedesktop.org/agd5f/linux/-/commits/tlb_inv_rework?ref_type=heads

V2:
- Add missing hub callbacks in gmc9 hubs

Alex Deucher (31):
  drm/amdgpu/gmc9: disallow gfxoff around TLB flushes
  drm/amdgpu/gmc10: disallow gfxoff around TLB flushes
  drm/amdgpu/gmc11: disallow gfxoff around TLB flushes
  drm/amdgpu/gmc12: disallow gfxoff around TLB flushes
  drm/amdgpu/gmc9: set vmhub funcs for gfxhub and mmhub
  drm/amdgpu: add a gmc flag for using MMIO for TLB flush
  drm/amdgpu/gmc9: use MMIO for TLB flushes
  drm/amdgpu/gmc10: use MMIO for TLB flushes
  drm/amdgpu/gmc11: use MMIO for TLB flushes
  drm/amdgpu/gmc12: use MMIO for TLB flushes
  drm/amdgpu: add a buffer funcs callback for TLB invalidation
  drm/amdgpu/sdma5.0: add tlb invalidation buffer func callback
  drm/amdgpu/sdma5.2: add tlb invalidation buffer func callback
  drm/amdgpu/sdma6: add tlb invalidation buffer func callback
  drm/amdgpu/sdma7: add tlb invalidation buffer func callback
  drm/amdgpu: simplify amdgpu_gmc_flush_gpu_tlb()
  drm/amdgpu: add tlb invalidation method enum
  drm/amdgpu: plumb tlb inv method in amdgpu_gmc_flush_gpu_tlb_gart()
  drm/amdgpu: uplevel reset check in amdgpu_gmc_flush_gpu_tlb_gart()
  drm/amdgpu/gmc: add new callback to lookup vmid to pasid mapping
  drm/amdgpu: add a gmc callback for the inv semaphore
  drm/amdgpu/gmc: rework pasid flushing
  drm/amdgpu/gmc9: use SDMA for gart TLB invalidation
  drm/amdgpu/gmc10: use SDMA for gart TLB invalidation
  drm/amdgpu/gmc11: use SDMA for gart TLB invalidation
  drm/amdgpu/gmc12: use SDMA for gart TLB invalidation
  drm/amdgpu/gmc10: use SDMA for pasid TLB invalidation
  drm/amdgpu/gmc11: use SDMA for pasid TLB invalidation
  drm/amdgpu/gmc12: use MES or SDMA for pasid TLB invalidation
  drm/amdgpu/gmc12: drop MES tlb inv in gmc callbacks
  drm/amdgpu/gmc: add helpers for various tlb inv functions

 drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c |   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c  | 486 +++++++++++++++++++----
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h  |  33 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_sdma.h |  18 +
 drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c   |   2 +
 drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c |  33 ++
 drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c |  33 ++
 drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c   | 202 +---------
 drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c   | 207 +---------
 drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c   | 250 ++----------
 drivers/gpu/drm/amd/amdgpu/gmc_v12_1.c   | 225 +----------
 drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c    | 135 ++-----
 drivers/gpu/drm/amd/amdgpu/mes_v12_0.c   |   4 +
 drivers/gpu/drm/amd/amdgpu/mes_v12_1.c   |   4 +
 drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c  |  33 ++
 drivers/gpu/drm/amd/amdgpu/mmhub_v1_7.c  |  32 ++
 drivers/gpu/drm/amd/amdgpu/mmhub_v1_8.c  |  32 ++
 drivers/gpu/drm/amd/amdgpu/mmhub_v9_4.c  |  32 ++
 drivers/gpu/drm/amd/amdgpu/sdma_v5_0.c   |  49 +++
 drivers/gpu/drm/amd/amdgpu/sdma_v5_2.c   |  49 +++
 drivers/gpu/drm/amd/amdgpu/sdma_v6_0.c   |  49 +++
 drivers/gpu/drm/amd/amdgpu/sdma_v7_0.c   |  48 +++
 22 files changed, 943 insertions(+), 1015 deletions(-)

-- 
2.55.0