AMD-GFX Archive on lore.kernel.org
 help / color / mirror / Atom feed
* [RFC PATCH 0/3] drm/amdgpu: make the reserved SPM VMID per-partition
@ 2026-11-17 16:59 James Zhu
  2026-11-17 16:59 ` [RFC PATCH 1/3] drm/amdgpu: expose the XCC-masked setup_vm_pt_regs as a gfxhub callback James Zhu
                   ` (4 more replies)
  0 siblings, 5 replies; 17+ messages in thread
From: James Zhu @ 2026-11-17 16:59 UTC (permalink / raw)
  To: amd-gfx; +Cc: Christian König, Sunil Khatri, James Zhu

The reserved (SPM) VMID handling was designed on single-XCC hardware,
where one GFXHUB is all there is. On multi-XCC parts with partitioning
every XCC has an own GFXHUB and every partition is exposed as an own
render node, and the current code is incoherent in three ways at once:

1. AMDGPU_VM_OP_RESERVE_VMID records the reservation on
   AMDGPU_GFXHUB(0) only. The VMID grab path checks
   vm->reserved_vmid[ring->vm_hub], and the rings of XCC i use
   AMDGPU_GFXHUB(i) - so for any job running on XCC 1..n the
   reservation silently does nothing. A debugger relying on a stable
   VMID gets one on exactly one XCC of the partition it runs on, or
   none at all.

2. The physical programming goes the other way: the setup_vm_pt_regs
   gfxhub callback broadcasts the SPM VMID's page table base to every
   XCC of the device, including XCCs of partitions the caller cannot
   even submit to.

3. The exclusion is device-global: because every reservation contends
   for hub 0's id_mgr, one partition's render node blocks every other
   partition from reserving, which breaks the isolation the render
   nodes are supposed to provide.

None of this is visible on single-XCC hardware, where "hub 0", "all
hubs" and "the caller's hubs" happen to be the same thing. gfx 9.4.3
partition modes are where the three diverge.

This series makes the per-hub infrastructure, which already exists in
the data structures (vm->reserved_vmid[] and id_mgr[] are per-vmhub
arrays), real end to end:

Patch 1 exposes the XCC-masked setup_vm_pt_regs variant, which gfxhub
v1.2 and v12.1 already implement internally, as a gfxhub callback.

Patch 2 makes the hub argument of the binding path mean what it says:
amdgpu_gmc_bind_vmid() programs and flushes only the given hub,
amdgpu_vmid_free_spm() unbinds the hub it frees, amdgpu_vm_spm_update()
binds every hub the VM holds the reservation on, and the ioctl reserves
on every GFXHUB instance. This fixes 1. and 2. while keeping the
device-wide scope.

Patch 3 then derives the hub set from the caller's partition
(fpriv->xcp_id), fixing 3.: each partition reserves, binds and
conflicts only on its own XCCs. Unreserve keeps operating on every hub,
freeing whatever the VM actually holds.

A note on the record granularity: the reservation could instead be
recorded on the partition's first GFXHUB only and treated as a pure
ownership token, closest to the original single-hub design. But the
record plays three roles at once: the ownership token, the set of hubs
whose page table base has to be bound (each XCC translates through its
own hub, so the binding must cover every XCC of the partition either
way), and the per-hub check of the VMID grab and CS paths
(vm->reserved_vmid[ring->vm_hub]). Recording on every hub of the
partition keeps the three consistent by construction, lets
amdgpu_vm_spm_update() derive the binding set from the existing
structures without a new field, and gives jobs on every XCC of the
partition the same reserved-VMID semantics instead of only on the
first one. The VMID number itself is the same everywhere either way:
it is always adev->vm_manager.spm_vmid.

No UAPI change. On devices without partitioning, or on the whole-device
node, the mask degenerates to every XCC and nothing changes; on
single-XCC parts the series is a no-op.

Compile-tested on amd-staging-drm-next with CONFIG_DRM_AMDGPU=m
(amdgpu_vm.o, amdgpu_ids.o, amdgpu_gmc.o, gfxhub_v1_2.o,
gfxhub_v12_1.o build warning-free); checkpatch clean.

James Zhu (3):
  drm/amdgpu: expose the XCC-masked setup_vm_pt_regs as a gfxhub
    callback
  drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance
  drm/amdgpu: scope AMDGPU_VM_OP_RESERVE_VMID to the caller's partition

 drivers/gpu/drm/amd/amdgpu/amdgpu_gfxhub.h |  2 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c    |  4 ++
 drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c    | 49 ++++++++++++++++++++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ids.h    |  4 ++
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c     | 50 ++++++++++++++++------
 drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c  |  1 +
 drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c   |  1 +
 7 files changed, 96 insertions(+), 15 deletions(-)

-- 
2.34.1


^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2026-11-18 16:25 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-17 16:59 [RFC PATCH 0/3] drm/amdgpu: make the reserved SPM VMID per-partition James Zhu
2026-11-17 16:59 ` [RFC PATCH 1/3] drm/amdgpu: expose the XCC-masked setup_vm_pt_regs as a gfxhub callback James Zhu
2026-11-18  9:36   ` Christian König
2026-11-17 16:59 ` [RFC PATCH 2/3] drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance James Zhu
2026-11-17 17:21   ` sashiko-bot
2026-11-18  9:38   ` Christian König
2026-11-17 16:59 ` [RFC PATCH 3/3] drm/amdgpu: scope AMDGPU_VM_OP_RESERVE_VMID to the caller's partition James Zhu
2026-11-17 17:18   ` sashiko-bot
2026-11-17 18:19 ` [RFC PATCH v2 0/3] drm/amdgpu: make the reserved SPM VMID per-partition James Zhu
2026-11-17 18:19   ` [RFC PATCH v2 1/3] drm/amdgpu: expose the XCC-masked setup_vm_pt_regs as a gfxhub callback James Zhu
2026-11-17 18:19   ` [RFC PATCH v2 2/3] drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance James Zhu
2026-11-17 18:47     ` sashiko-bot
2026-11-17 18:19   ` [RFC PATCH v2 3/3] drm/amdgpu: scope AMDGPU_VM_OP_RESERVE_VMID to the caller's partition James Zhu
2026-11-18 16:24 ` [RFC PATCH v3 0/3] drm/amdgpu: make the reserved SPM VMID per-partition James Zhu
2026-11-18 16:24   ` [RFC PATCH v3 1/3] drm/amdgpu: pass an XCC mask to the setup_vm_pt_regs gfxhub callback James Zhu
2026-11-18 16:24   ` [RFC PATCH v3 2/3] drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance James Zhu
2026-11-18 16:24   ` [RFC PATCH v3 3/3] drm/amdgpu: scope AMDGPU_VM_OP_RESERVE_VMID to the caller's partition James Zhu

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox