[PATCH v5 0/8] drm/amdgpu: SVM VRAM migration via drm_pagemap (unified XNACK on/off)

[Junhua Shen <Junhua.Shen@amd.com>]

This series adds VRAM migration support to amdgpu's SVM (Shared Virtual
Memory) implementation, using the drm_pagemap framework for ZONE_DEVICE
page management and SDMA for data migration.

This v5 is the unified version supporting both XNACK-on (GPU fault-driven)
and XNACK-off (ioctl-driven / restore-based) migration paths, built on top
of the drm_gpusvm-based amdgpu SVM core [1] and xnack-off restore
infrastructure [2]. Previous v1/v2/v3 were XNACK-off only; v4 was XNACK-on
only; this v5 merges both paths into a single unified series.

The implementation follows the Xe driver's approach for TTM eviction,
using synchronous bo_move to migrate device-private pages back to system
RAM when TTM needs to evict SVM BOs.

Key design points:
- GPU VRAM registered as ZONE_DEVICE via devm_memremap_pages(),
  wrapped in struct amdgpu_pagemap with drm_pagemap state
- SDMA-based data transfer through GART aperture window for both
  copy_to_devmem and copy_to_ram callbacks
- amdgpu_bo_svm: lightweight BO subtype with drm_pagemap_devmem for
  ZONE_DEVICE page ownership tracking
- Synchronous TTM eviction via drm_pagemap_evict_to_ram() in bo_move,
  following the Xe pattern (no eviction fences needed)
- Migration policy driven by SVM range attributes (preferred location,
  prefetch hints) for both XNACK-on fault path and XNACK-off restore path
- XNACK-off integration: migration triggered in restore worker and
  attr-change boundary realign paths

Limitations:
- Single GPU only; multi-GPU migration is not addressed
- No VRAM-to-VRAM (peer GPU) migration

Open issue:
- Unnecessary TTM system memory allocation during eviction: when TTM
  evicts an SVM BO, it allocates a destination system memory resource
  (TTM_PL_SYSTEM) before calling bo_move, then frees it afterwards.
  This allocation is unnecessary because the actual data migration is
  done via drm_pagemap_evict_to_ram() → migrate_device_* which
  migrates device-private pages directly to regular system pages,
  bypassing the TTM-allocated resource entirely. The current TTM
  framework does not support num_placement=0 to skip this redundant
  allocation; this needs further discussion.

Dependencies:

This series applies on top of:
[1] amdgpu drm_gpusvm SVM core (xnack-on):
    https://lore.kernel.org/amd-gfx/20260529054928.596825-1-honglei1.huang@amd.com/
[2] amdgpu drm_gpusvm SVM xnack-off restore:
    https://lore.kernel.org/amd-gfx/20260529054928.596825-1-honglei1.huang@amd.com/

Changes since v4:
- Unified XNACK-on and XNACK-off migration into a single series
  (v4 was XNACK-on only)
- Added patch 6/8: refactor SVM attr devmem_possible and prefer_vram
  API for cleaner integration with both paths
- Added patch 8/8: integrate VRAM migration into XNACK-off SVM restore
  and attr-change boundary realign paths
- Split the integration patch into two: fault/prefetch path (7/8) and
  restore/realign path (8/8) for better reviewability
- Rebased on latest SVM core with xnack-off restore support

Changes since v3:
- Rebased on drm_gpusvm-based amdgpu SVM core [1], switching from
  XNACK-off ioctl-driven to XNACK-on GPU fault-driven migration
- Introduced amdgpu_bo_svm subtype with drm_pagemap_devmem embedding
  and two-layer reference counting (GEM refcount + TTM kref)
- Added synchronous TTM eviction via drm_pagemap_evict_to_ram() in
  amdgpu_bo_move(), following the Xe driver pattern
- Added amdgpu_bo_is_amdgpu_bo() check for SVM BOs in TTM path
- Cleaned up container_of macros to follow amdgpu conventions
  (to_amdgpu_bo_svm as #define, devmem_to_amdgpu_bo_svm as inline)

Changes since v2:
- Moved amdgpu_pagemap entirely to amdgpu side, eliminating all KFD
  modifications
- Split commits for better reviewability: separated infrastructure
  from SDMA callbacks, decision layer from integration
- Merged ZONE_DEVICE registration hook into the integration patch

Changes since v1:
- Dropped the eviction fence patch (was 4/6) after Christian König
  pointed out it violates the dma_fence contract
- Refactored migration integration: extracted migration logic into
  new files amdgpu_svm_range_migrate.{c,h}
- Introduced enum amdgpu_svm_migrate_mode (PREFERRED, TO_VRAM,
  TO_SYSMEM, NONE) to make migration intent explicit, replacing
  the _ex functions used in v1

Previous versions:
v1 (XNACK-off): https://lore.kernel.org/amd-gfx/20260410113146.146212-1-Junhua.Shen@amd.com/
v2 (XNACK-off): https://lore.kernel.org/amd-gfx/20260413103031.181953-1-Junhua.Shen@amd.com/
v3 (XNACK-off): https://lore.kernel.org/amd-gfx/20260427100522.7014-1-Junhua.Shen@amd.com/
v4 (XNACK-on):  https://lore.kernel.org/amd-gfx/20260513085725.163786-1-Junhua.Shen@amd.com/

Test results:
  Tested on gfx943 (MI300X) and gfx906 (MI60) with both XNACK on and off:
  - KFD test: 95%+ passed (both modes).
  - ROCR test: 98%+ passed (both modes).

Patch overview:
  1/8  Core VRAM migration infrastructure (ZONE_DEVICE registration,
       amdgpu_pagemap, amdgpu_bo_svm subtype, drm_pagemap_ops)
  2/8  SDMA migration callbacks (copy_to_devmem, copy_to_ram,
       populate_devmem_pfn via GART aperture window)
  3/8  Synchronous TTM eviction for SVM BOs (amdgpu_svm_bo_evict
       in bo_move path, amdgpu_bo_is_amdgpu_bo check)
  4/8  Hook up ZONE_DEVICE registration in device init and GPU reset
  5/8  SVM range migration helpers (range-level migrate_to_vram /
       migrate_to_sysmem decision layer)
  6/8  Refactor SVM attr devmem_possible and prefer_vram API for
       unified xnack-on/off usage
  7/8  Wire up VRAM migration into SVM fault and prefetch paths
       (XNACK-on)
  8/8  Wire up VRAM migration into SVM restore and attr-change
       realign paths (XNACK-off)

Junhua Shen (8):
  drm/amdgpu: add VRAM migration infrastructure for drm_pagemap
  drm/amdgpu: implement drm_pagemap SDMA migration callbacks
  drm/amdgpu: implement synchronous TTM eviction for SVM BOs
  drm/amdgpu: hook up ZONE_DEVICE registration in device init and reset
  drm/amdgpu: add SVM range migration helpers for drm_pagemap
  drm/amdgpu: refactor SVM attr devmem_possible and prefer_vram API
  drm/amdgpu: integrate VRAM migration into SVM fault and prefetch paths
  drm/amdgpu: integrate VRAM migration into SVM restore and realign paths

 drivers/gpu/drm/amd/amdgpu/Makefile                |   8 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu.h                |   8 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c         |   2 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c        | 831 +++++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h        | 102 +++
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c         |   4 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c          |   2 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_svm.c            |  12 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_svm.h            |   4 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_svm_attr.c       |  18 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_svm_attr.h       |   5 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_svm_fault.c      |  20 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range.c      |  32 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range.h      |   1 +
 .../gpu/drm/amd/amdgpu/amdgpu_svm_range_migrate.c  | 115 +++
 .../gpu/drm/amd/amdgpu/amdgpu_svm_range_migrate.h  |  35 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c            |  20 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_userptr.c        |  85 ++-
 18 files changed, 1244 insertions(+), 60 deletions(-)
 create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c
 create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h
 create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range_migrate.c
 create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range_migrate.h

-- 
2.34.1