[RFC PATCH v3 00/11] Use new dma-map IOVA alloc, link, and sync API in GPU SVM and DRM pagemap

[Matthew Brost <matthew.brost@intel.com>]

The dma-map IOVA alloc, link, and sync APIs perform significantly better
than dma-map / dma-unmap, as they avoid costly IOMMU synchronizations.
This difference is especially noticeable when mapping a 2MB region in
4KB pages.

Use dma-map IOVA alloc, link, and sync APIs for GPU SVM and DRM page,
which mappings between the CPU and GPU.

Initial results are promising.

Baseline CPU time during 2M / 64K fault with a migration:
Average migrate 2M cpu time (us, percentage): 552.36049107142857142857, .71943789893868318799
Average migrate 64K cpu time (us, percentage): 24.97767857142857142857, .34789908128526791960

After patch 4 in this series CPU time during 2M / 64K fault with a migration:
Average migrate 2M cpu time (us, percentage): 224.81808035714285714286, .51412827364772602557
Average migrate 64K cpu time (us, percentage): 14.65625000000000000000, .25659463050529524405

After patch 5 in this series CPU time during 2M / 64K fault with a migration:
Average migrate 2M cpu time (us, percentage): 158.52901785714285714286, .42740093699505625280
Average migrate 64K cpu time (us, percentage): 14.00000000000000000000, .23276613905846987437

- Patches 2-4 implement dma-map IOVA API for system pages

- Patch 5 optimizes this a bit but questionable if we need if this
  needed if 2M pages are enabled

- Patches 6-11 implement a IOVA API for multi-GPU PoC. I'm told drivers
  shouldn't call dma-map IOVA API for multi-device but including it as
  reference for a design we'd like to get to with high-speed fabrics in
  mind.

Still an RFC as patches 6-11 may not be able to merged in there current
form, the series can be split if needed to merge eariler patches.

Matt

v2:
 - Include missing basline patch for CI
v3:
 - Fix memory corruption
 - PoC IOVA alloc for multi-GPU

Francois Dugast (1):
  drm/pagemap: Add helper to access zone_device_data

Matthew Brost (10):
  drm/gpusvm: Use dma-map IOVA alloc, link, and sync API in GPU SVM
  drm/pagemap: Split drm_pagemap_migrate_map_pages into device / system
  drm/pagemap: Use dma-map IOVA alloc, link, and sync API for DRM
    pagemap
  drm/pagemap: Reduce number of IOVA link calls
  drm/pagemap: Add IOVA interface to DRM pagemap
  drm/xe: Stub out DRM pagemap IOVA alloc implementation
  drm/pagemap: Use device-to-device IOVA alloc, link, and sync API for
    DRM pagemap
  drm/xe: Drop BO dma-resv lock during SVM migrate-to-device
  drm/xe: Implement DRM pagemap IOVA vfuncs
  drm/gpusvm: Use device-to-device IOVA alloc, link, and sync API in GPU
    SVM

 drivers/gpu/drm/drm_gpusvm.c  | 196 ++++++++++++++++---
 drivers/gpu/drm/drm_pagemap.c | 351 +++++++++++++++++++++++++++-------
 drivers/gpu/drm/xe/xe_svm.c   | 143 +++++++++++++-
 include/drm/drm_gpusvm.h      |   5 +
 include/drm/drm_pagemap.h     | 101 ++++++++++
 5 files changed, 693 insertions(+), 103 deletions(-)

-- 
2.34.1