[PATCH v2 0/5] drm/gpuvm: two pass locking for exec

[Matthew Brost <matthew.brost@intel.com>]

Two processes share a set of buffers, and each has buffers of its own
which the other never sees. Say process A has mapped

  S        the shared buffers, also mapped by B
  P        buffers private to A

and B has mapped S plus private buffers of its own. The overlap is
exactly S, and the work each process wants to do on its own buffers is
independent of the other.

A submits. Its exec locks the dma-resv of everything it has mapped, S
and P both, then finds something in P has been evicted and migrates it
back in. B submits, and blocks on S for as long as that migration takes,
even though the migration is of a buffer belonging to A which B has
never seen.

So the stall does not come from the overlapping set. The buffers in S
are resident, and neither exec has anything to do to them beyond
attaching a fence. They are held only because an exec locks everything
it has mapped in one go, and they stay held until the slowest unrelated
thing in that transaction is done.

Which buffers get evicted is a separate matter, and one which already
has answers: eviction heuristics which leave shared buffers alone, or
one process' allocations outranking another's. This is what is left once
those work.

Where this tends to show up is compositors and presentation, which is
also where userspace has worked hardest to avoid it. Wayland explicit
sync exists so that a compositor is not latched onto its clients'
rendering, waiting on fences it never asked for. The locking above
reintroduces that coupling anyway, in the kernel, and does it under
memory pressure, which is where a missed frame is least welcome and the
cause is hardest to see.

The fix is to stop coupling "lock the VM" to "validate it". Instead of
locking everything and then validating, lock the private buffers and the
evicted external ones, validate those, and only then lock the rest, all
within the same drm_exec transaction.

Patch 1 lets a driver split the locking of an exec that way, patches 2
and 3 use it in Xe and Panthor, whose panthor_vm_bo_validate() swaps
pages back in under those same shared locks. It is opt-in, and drivers
which do not ask for it are unaffected.

Patches 4 and 5 do the same for MSM. Its VM_BIND submit path locks every
BO mapped in the VM and then validates the evicted ones, which means
getting their pages and mapping them again, with the shared BOs locked
throughout. Patch 4 switches VM_BIND VMs to DRM_GPUVM_RESV_PROTECTED,
which two pass locking requires, and patch 5 splits the submit's locking
into the two passes. Kernel managed VMs and the legacy submit path are
left alone.

The MSM part should resolve exactly the problem Anna Maniscalco
presented at XDC, render jobs stalling the compositor on shared buffers,
which is currently worked around with a downstream hack in MSM for
SteamOS [1][2] (the talk starts at 5:30).

[1] https://indico.freedesktop.org/event/12/contributions/627/
[2] https://www.youtube.com/watch?v=j5W5ErEMnvM&t=330s

Matt

Cc: Abhinav Kumar <abhinav.kumar@linux.dev>
Cc: Alice Ryhl <aliceryhl@google.com>
Cc: Anna Maniscalco <anna.maniscalco2000@gmail.com>
Cc: Boris Brezillon <boris.brezillon@collabora.com>
Cc: Danilo Krummrich <dakr@kernel.org>
Cc: David Airlie <airlied@gmail.com>
Cc: Dmitry Baryshkov <lumag@kernel.org>
Cc: Jessica Zhang <jesszhan0024@gmail.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liviu Dudau <liviu.dudau@arm.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Marijn Suijten <marijn.suijten@somainline.org>
Cc: Maxime Ripard <mripard@kernel.org>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Rob Clark <robin.clark@oss.qualcomm.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Sean Paul <sean@poorly.run>
Cc: Shuah Khan <skhan@linuxfoundation.org>
Cc: Simona Vetter <simona@ffwll.ch>
Cc: Steven Price <steven.price@arm.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Assisted-by: LLM

Matthew Brost (5):
  drm/gpuvm: allow locking external objects in two passes
  drm/xe: lock the resident BOs of an exec last
  drm/panthor: lock the resident BOs of a submit last
  drm/msm: use DRM_GPUVM_RESV_PROTECTED for VM_BIND VMs
  drm/msm: lock the resident BOs of a VM_BIND submit last

 Documentation/gpu/drm-mm.rst          |   6 +
 drivers/gpu/drm/drm_gpuvm.c           | 480 +++++++++++++++++++++++++-
 drivers/gpu/drm/msm/msm_gem_submit.c  |  82 ++++-
 drivers/gpu/drm/msm/msm_gem_vma.c     |  15 +-
 drivers/gpu/drm/panthor/panthor_mmu.c |  53 ++-
 drivers/gpu/drm/xe/xe_exec.c          |  23 +-
 drivers/gpu/drm/xe/xe_vm.c            |  43 ++-
 drivers/gpu/drm/xe/xe_vm.h            |   3 +-
 include/drm/drm_gpuvm.h               | 133 ++++++-
 9 files changed, 784 insertions(+), 54 deletions(-)

-- 
2.34.1