Re: [PATCH v2 00/33] drm/ttm, drm/xe: Minimize dma-resv hold times and defragment sub-optimally backed BOs

[Matthew Brost <matthew.brost@intel.com>]

On Sat, Jul 11, 2026 at 12:33:20PM +0200, Christian König wrote:
> On 7/10/26 23:54, Matthew Brost wrote:
> > The overarching goal of this series is to keep user-facing IOCTLs snappy
> > by holding the BO dma-resv lock for the shortest possible time, and to
> > push the expensive, best-effort work - page (re)allocation, DMA mapping
> > and placement fixups - into the background or out of the locked critical
> > section entirely.
> > 
> > Two related mechanisms fall out of that goal:
> > 
> >  - Out-of-lock backing. Page allocation and DMA mapping are the dominant
> >    cost of gem_create and of BO moves, and doing them under the dma-resv
> >    lock serializes otherwise-independent clients.
> 
> Saying that doesn't make sense.
> 
> The DMA-resv lock is either per BO or per VM and so the clients are not even remotely independent of that lock.
> 

s/clients/threads:

Same VM:

Thread A - allocate memory under VM's dma-resv, takes a very long time
Thread B - tries to submit, blocks on Thread A

> >    This series preallocates
> >    the backing pages (and, where applicable, builds the DMA/IOVA mapping)
> >    outside the lock, then transfers the result into the BO under a brief
> >    lock hold. The user gets a fast IOCTL return; the heavy lifting happens
> >    without contending the reservation.
> 
> Ok, that sounds like a really bad idea.
> 
> Why in the world would you do this? The dma_resv lock is exactly what is used to prevent something like that.
> 

Allocating pages from the core MM subsystem does not really have anything to do
with dma_resv. Moving pages attached to a BO, however, is related to dma_resv,
and that is where TTM comes into play.

The TL;DR is that a defragmenter does not work well when page allocations are
performed while holding the dma_resv lock. During the v1 implementation, I
quickly noticed that the dma_resv lock could be held for up to 100 ms in the
worst case, and more typically for 5–10 ms. When the frame budget at 60 Hz is
only 16 ms, this is obviously a problem, and it becomes even more problematic at
higher refresh rates.

After quite a bit of work, the common-case lock hold time is now below 100 µs,
while the worst case is under 1 ms. At that point, a defragmenter becomes much
more viable.

I'm applying the same defragmentation logic to IOCTLs. Why hold the lock any
longer than necessary and introduce additional pipeline stalls?

> > 
> >  - Page defragmentation. A BO allocated under memory pressure keeps its
> >    scattered, sub-optimally-ordered backing for its entire lifetime,
> >    costing TLB efficiency forever. TTM grows the plumbing to track
> >    order-failure and to re-back a populated BO in place at the beneficial
> >    order, and Xe wires up a background delayed worker that promotes such
> >    BOs on the GPU once memory is available again - again, off the hot
> >    path and without stalling the submitting thread.
> 
> That could be quite beneficial, but so far falling back to low order allocation was only seen as last resort to avoid OOM.
> 

Right now, the situation is effectively "once placed badly, forever placed
badly."

Matt

> Regards,
> Christian.
> 
> > 
> > Since v1 [1] the series has grown considerably. The bulk of the new
> > material is a direct result of profiling: once the defragmenter was in
> > place, the remaining dma-resv hold times and the per-BO
> > allocation/mapping costs showed up clearly, which motivated the
> > out-of-lock preallocation, the IOVA-based mapping path, and the amdgpu
> > counterpart. Rather than land the defragmenter alone, v2 folds in these
> > optimizations since they share the same infrastructure and the same
> > "hold dma-resv briefly, fix up in the background" architecture.
> > 
> > The series is organized in sections rather than described patch by patch:
> > 
> >  - Patches 1-10 (drm/ttm): core TTM preparation - order-failure tracking,
> >    the defragmentation move, reclaim backoff, and out-of-lock page
> >    preallocation plumbing.
> >  - Patches 11-14: other dependent drm/gpusvm and drm/xe patches this
> >    series builds on (DMA-mapping accounting, per-order DMA stats, async
> >    L2 flush, and a VM-teardown ordering fix).
> >  - Patches 15-23 (drm/xe): the page defragmenter itself - BO tracking,
> >    the on-GPU defrag copy, xe_bo_move() handling, and the background
> >    worker with its stats and configuration.
> >  - Patches 24-25 (drm/xe): out-of-lock system BO backing preallocation
> >    in gem_create, moving page allocation out of the dma-resv lock.
> >  - Patches 26-32 (drm/xe): IOVA-based DMA mapping optimizations, building
> >    and finalizing the mapping outside the lock.
> >  - Patch 33 (drm/amdgpu): the equivalent out-of-lock system BO
> >    preallocation for amdgpu, exercising the shared TTM plumbing.
> > 
> > Testing
> > =======
> > 
> >  - 3D benchmarks on Ubuntu and on Android, with memory intentionally
> >    fragmented by a separate program at launch (plus beneficial-order
> >    error injection). BOs are initially backed at a sub-optimal order and
> >    scores start lower; the background defrag worker then promotes the
> >    backing to the beneficial order and scores climb back in line with the
> >    unfragmented baseline.
> >  - IGT:
> >    https://patchwork.freedesktop.org/patch/739052/?series=170046&rev=2
> > 
> > Matt
> > 
> > [1] https://patchwork.freedesktop.org/series/169053/
> > 
> > Cc: amd-gfx@lists.freedesktop.org
> > Cc: Alex Deucher <alexander.deucher@amd.com>
> > Cc: Carlos Santa <carlos.santa@intel.com>
> > Cc: Ryan Neph <ryanneph@google.com>
> > Cc: Christian Koenig <christian.koenig@amd.com>
> > Cc: Huang Rui <ray.huang@amd.com>
> > Cc: Matthew Auld <matthew.auld@intel.com>
> > Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
> > Cc: Maxime Ripard <mripard@kernel.org>
> > Cc: Thomas Zimmermann <tzimmermann@suse.de>
> > Cc: David Airlie <airlied@gmail.com>
> > Cc: Simona Vetter <simona@ffwll.ch>
> > Cc: dri-devel@lists.freedesktop.org
> > Cc: linux-kernel@vger.kernel.org
> > Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
> > 
> > 
> > 
> > Matthew Brost (31):
> >   drm/ttm/pool: Allow backing off reclaim at the beneficial order
> >   drm/ttm/pool: Add ttm_pool_page_order_nodma() helper
> >   drm/ttm: Record sub-optimal page order allocations in ttm_tt
> >   drm/ttm: Introduce ttm_pool_alloc_iter for __ttm_pool_alloc()
> >   drm/ttm: Support defragmentation moves
> >   drm/ttm: Add fault injection for beneficial-order allocation failures
> >   drm/ttm: Harvest beneficial-order pages on defragmentation moves
> >   drm/ttm: Bound page (re)allocation per defragmentation move
> >   drm/ttm: Preallocate beneficial-order defrag pages outside the lock
> >   drm/ttm: Add full out-of-lock preallocation for ttm_pool_alloc()
> >   drm/xe: Flush L2 asynchronously in xe_bo_trigger_rebind()
> >   drm/xe: Destroy page tables after unlinking all VMAs on VM close
> >   drm/xe: Track BOs backed at a sub-optimal page order
> >   drm/xe: Back off beneficial-order reclaim under defrag pressure
> >   drm/xe: Add xe_migrate_copy_defrag() for on-GPU defrag copies
> >   drm/xe: Handle defrag moves in xe_bo_move()
> >   drm/xe: Skip self-copies for borrowed pages on defrag moves
> >   drm/xe: Add a page defragmentation worker
> >   drm/xe: Add defrag GT stats
> >   drm/xe: Add Kconfig.profile options for BO defrag configuration
> >   drm/xe: Defrag using out-of-lock page preallocation
> >   drm/xe: Add defrag profiling tracepoints
> >   drm/xe: Preallocate system BO backing outside the dma-resv lock
> >   drm/xe: Add tracepoint for xe_gem_create_ioctl
> >   drm/xe: Add IOVA-based xe_res_cursor variant
> >   drm/xe: Use IOVA-based DMA mapping for eligible tt BOs
> >   drm/xe: Add per-device dependency scheduler for IOVA defrag finalize
> >   drm/xe: Add packed copy-step IOVA mapping for defrag
> >   drm/xe: Blit src-natural to dst-packed for defrag-IOVA copies
> >   drm/xe: Finalize defrag-IOVA moves with post-copy job
> >   drm/amdgpu: Preallocate system BO pages outside the reservation lock
> > 
> > Thomas Hellström (2):
> >   drm/gpusvm: Add a DMA-mapping accounting callback
> >   drm/xe: Add debugfs stats for DMA-mapped pages per order
> > 
> >  .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  |    4 +-
> >  drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c   |    2 +-
> >  drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c       |   45 +-
> >  drivers/gpu/drm/amd/amdgpu/amdgpu_gem.h       |    5 +-
> >  drivers/gpu/drm/amd/amdgpu/amdgpu_object.c    |    3 +-
> >  drivers/gpu/drm/amd/amdgpu/amdgpu_object.h    |    4 +
> >  drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |   39 +-
> >  drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h       |    3 +
> >  drivers/gpu/drm/drm_gpusvm.c                  |   17 +-
> >  drivers/gpu/drm/ttm/ttm_bo.c                  |  101 +-
> >  drivers/gpu/drm/ttm/ttm_bo_util.c             |   21 +-
> >  drivers/gpu/drm/ttm/ttm_pool.c                |  931 +++++++++-
> >  drivers/gpu/drm/ttm/ttm_tt.c                  |   67 +
> >  drivers/gpu/drm/xe/Kconfig.profile            |   40 +
> >  drivers/gpu/drm/xe/tests/xe_bo.c              |    8 +-
> >  drivers/gpu/drm/xe/tests/xe_dma_buf.c         |    2 +-
> >  drivers/gpu/drm/xe/tests/xe_migrate.c         |   12 +-
> >  drivers/gpu/drm/xe/xe_bo.c                    | 1632 ++++++++++++++++-
> >  drivers/gpu/drm/xe/xe_bo.h                    |   15 +-
> >  drivers/gpu/drm/xe/xe_bo_types.h              |    6 +
> >  drivers/gpu/drm/xe/xe_debugfs.c               |   26 +
> >  drivers/gpu/drm/xe/xe_device.c                |   35 +
> >  drivers/gpu/drm/xe/xe_device_types.h          |   57 +
> >  drivers/gpu/drm/xe/xe_dma_buf.c               |    2 +-
> >  drivers/gpu/drm/xe/xe_ggtt.c                  |    2 +-
> >  drivers/gpu/drm/xe/xe_gt_stats.c              |    5 +
> >  drivers/gpu/drm/xe/xe_gt_stats_types.h        |   17 +
> >  drivers/gpu/drm/xe/xe_migrate.c               |  531 +++++-
> >  drivers/gpu/drm/xe/xe_migrate.h               |   17 +
> >  drivers/gpu/drm/xe/xe_pt.c                    |    2 +-
> >  drivers/gpu/drm/xe/xe_res_cursor.h            |   56 +-
> >  drivers/gpu/drm/xe/xe_svm.c                   |   36 +-
> >  drivers/gpu/drm/xe/xe_svm.h                   |    3 +-
> >  drivers/gpu/drm/xe/xe_trace_bo.h              |   90 +
> >  drivers/gpu/drm/xe/xe_userptr.c               |   55 +
> >  drivers/gpu/drm/xe/xe_userptr.h               |    1 +
> >  drivers/gpu/drm/xe/xe_vm.c                    |  149 +-
> >  drivers/gpu/drm/xe/xe_vm.h                    |    5 +
> >  include/drm/drm_gpusvm.h                      |   19 +
> >  include/drm/ttm/ttm_bo.h                      |   64 +
> >  include/drm/ttm/ttm_pool.h                    |   46 +
> >  include/drm/ttm/ttm_tt.h                      |   43 +-
> >  42 files changed, 4029 insertions(+), 189 deletions(-)
> > 
>