* [RFC PATCH 0/3] drm/amdgpu: make the reserved SPM VMID per-partition
@ 2026-11-17 16:59 James Zhu
2026-11-17 16:59 ` [RFC PATCH 1/3] drm/amdgpu: expose the XCC-masked setup_vm_pt_regs as a gfxhub callback James Zhu
` (3 more replies)
0 siblings, 4 replies; 11+ messages in thread
From: James Zhu @ 2026-11-17 16:59 UTC (permalink / raw)
To: amd-gfx; +Cc: Christian König, Sunil Khatri, James Zhu
The reserved (SPM) VMID handling was designed on single-XCC hardware,
where one GFXHUB is all there is. On multi-XCC parts with partitioning
every XCC has an own GFXHUB and every partition is exposed as an own
render node, and the current code is incoherent in three ways at once:
1. AMDGPU_VM_OP_RESERVE_VMID records the reservation on
AMDGPU_GFXHUB(0) only. The VMID grab path checks
vm->reserved_vmid[ring->vm_hub], and the rings of XCC i use
AMDGPU_GFXHUB(i) - so for any job running on XCC 1..n the
reservation silently does nothing. A debugger relying on a stable
VMID gets one on exactly one XCC of the partition it runs on, or
none at all.
2. The physical programming goes the other way: the setup_vm_pt_regs
gfxhub callback broadcasts the SPM VMID's page table base to every
XCC of the device, including XCCs of partitions the caller cannot
even submit to.
3. The exclusion is device-global: because every reservation contends
for hub 0's id_mgr, one partition's render node blocks every other
partition from reserving, which breaks the isolation the render
nodes are supposed to provide.
None of this is visible on single-XCC hardware, where "hub 0", "all
hubs" and "the caller's hubs" happen to be the same thing. gfx 9.4.3
partition modes are where the three diverge.
This series makes the per-hub infrastructure, which already exists in
the data structures (vm->reserved_vmid[] and id_mgr[] are per-vmhub
arrays), real end to end:
Patch 1 exposes the XCC-masked setup_vm_pt_regs variant, which gfxhub
v1.2 and v12.1 already implement internally, as a gfxhub callback.
Patch 2 makes the hub argument of the binding path mean what it says:
amdgpu_gmc_bind_vmid() programs and flushes only the given hub,
amdgpu_vmid_free_spm() unbinds the hub it frees, amdgpu_vm_spm_update()
binds every hub the VM holds the reservation on, and the ioctl reserves
on every GFXHUB instance. This fixes 1. and 2. while keeping the
device-wide scope.
Patch 3 then derives the hub set from the caller's partition
(fpriv->xcp_id), fixing 3.: each partition reserves, binds and
conflicts only on its own XCCs. Unreserve keeps operating on every hub,
freeing whatever the VM actually holds.
A note on the record granularity: the reservation could instead be
recorded on the partition's first GFXHUB only and treated as a pure
ownership token, closest to the original single-hub design. But the
record plays three roles at once: the ownership token, the set of hubs
whose page table base has to be bound (each XCC translates through its
own hub, so the binding must cover every XCC of the partition either
way), and the per-hub check of the VMID grab and CS paths
(vm->reserved_vmid[ring->vm_hub]). Recording on every hub of the
partition keeps the three consistent by construction, lets
amdgpu_vm_spm_update() derive the binding set from the existing
structures without a new field, and gives jobs on every XCC of the
partition the same reserved-VMID semantics instead of only on the
first one. The VMID number itself is the same everywhere either way:
it is always adev->vm_manager.spm_vmid.
No UAPI change. On devices without partitioning, or on the whole-device
node, the mask degenerates to every XCC and nothing changes; on
single-XCC parts the series is a no-op.
Compile-tested on amd-staging-drm-next with CONFIG_DRM_AMDGPU=m
(amdgpu_vm.o, amdgpu_ids.o, amdgpu_gmc.o, gfxhub_v1_2.o,
gfxhub_v12_1.o build warning-free); checkpatch clean.
James Zhu (3):
drm/amdgpu: expose the XCC-masked setup_vm_pt_regs as a gfxhub
callback
drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance
drm/amdgpu: scope AMDGPU_VM_OP_RESERVE_VMID to the caller's partition
drivers/gpu/drm/amd/amdgpu/amdgpu_gfxhub.h | 2 +
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 4 ++
drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c | 49 ++++++++++++++++++++-
drivers/gpu/drm/amd/amdgpu/amdgpu_ids.h | 4 ++
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 50 ++++++++++++++++------
drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c | 1 +
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c | 1 +
7 files changed, 96 insertions(+), 15 deletions(-)
--
2.34.1
^ permalink raw reply [flat|nested] 11+ messages in thread* [RFC PATCH 1/3] drm/amdgpu: expose the XCC-masked setup_vm_pt_regs as a gfxhub callback 2026-11-17 16:59 [RFC PATCH 0/3] drm/amdgpu: make the reserved SPM VMID per-partition James Zhu @ 2026-11-17 16:59 ` James Zhu 2026-11-17 16:59 ` [RFC PATCH 2/3] drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance James Zhu ` (2 subsequent siblings) 3 siblings, 0 replies; 11+ messages in thread From: James Zhu @ 2026-11-17 16:59 UTC (permalink / raw) To: amd-gfx; +Cc: Christian König, Sunil Khatri, James Zhu On GFX IPs with multiple XCCs each XCC has an own GFXHUB, but the setup_vm_pt_regs gfxhub callback programs a VMID's page table base on every XCC. Callers which bind a VMID on a single VMHUB, like the SPM VMID binding, have no way to reach one GFXHUB through the callback interface. gfxhub v1.2 and v12.1 already implement an XCC-masked variant internally. Expose it as a new setup_vm_pt_regs_xcc callback. GFXHUBs of IPs with a single XCC don't implement it, for them the existing callback already is per-hub. Signed-off-by: James Zhu <James.Zhu@amd.com> Assisted-by: Claude:claude-fable-5 --- drivers/gpu/drm/amd/amdgpu/amdgpu_gfxhub.h | 2 ++ drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c | 1 + drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c | 1 + 3 files changed, 4 insertions(+) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfxhub.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfxhub.h index c7b44aeb671b..19b961cb12f6 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfxhub.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfxhub.h @@ -28,6 +28,8 @@ struct amdgpu_gfxhub_funcs { u64 (*get_mc_fb_offset)(struct amdgpu_device *adev); void (*setup_vm_pt_regs)(struct amdgpu_device *adev, uint32_t vmid, uint64_t page_table_base); + void (*setup_vm_pt_regs_xcc)(struct amdgpu_device *adev, uint32_t vmid, + uint64_t page_table_base, uint32_t xcc_mask); int (*gart_enable)(struct amdgpu_device *adev); void (*gart_disable)(struct amdgpu_device *adev); diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c index b13967a1c8fb..7bc17721f463 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c @@ -809,6 +809,7 @@ const struct amdgpu_gfxhub_funcs gfxhub_v12_1_funcs = { .get_fb_location = gfxhub_v12_1_get_fb_location, .get_mc_fb_offset = gfxhub_v12_1_get_mc_fb_offset, .setup_vm_pt_regs = gfxhub_v12_1_setup_vm_pt_regs, + .setup_vm_pt_regs_xcc = gfxhub_v12_1_xcc_setup_vm_pt_regs, .gart_enable = gfxhub_v12_1_gart_enable, .gart_disable = gfxhub_v12_1_gart_disable, .set_fault_enable_default = gfxhub_v12_1_set_fault_enable_default, diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c index e89813590539..a56c072daf15 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c @@ -634,6 +634,7 @@ static int gfxhub_v1_2_get_xgmi_info(struct amdgpu_device *adev) const struct amdgpu_gfxhub_funcs gfxhub_v1_2_funcs = { .get_mc_fb_offset = gfxhub_v1_2_get_mc_fb_offset, .setup_vm_pt_regs = gfxhub_v1_2_setup_vm_pt_regs, + .setup_vm_pt_regs_xcc = gfxhub_v1_2_xcc_setup_vm_pt_regs, .gart_enable = gfxhub_v1_2_gart_enable, .gart_disable = gfxhub_v1_2_gart_disable, .set_fault_enable_default = gfxhub_v1_2_set_fault_enable_default, -- 2.34.1 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [RFC PATCH 2/3] drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance 2026-11-17 16:59 [RFC PATCH 0/3] drm/amdgpu: make the reserved SPM VMID per-partition James Zhu 2026-11-17 16:59 ` [RFC PATCH 1/3] drm/amdgpu: expose the XCC-masked setup_vm_pt_regs as a gfxhub callback James Zhu @ 2026-11-17 16:59 ` James Zhu 2026-11-17 17:21 ` sashiko-bot 2026-11-17 16:59 ` [RFC PATCH 3/3] drm/amdgpu: scope AMDGPU_VM_OP_RESERVE_VMID to the caller's partition James Zhu 2026-11-17 18:19 ` [RFC PATCH v2 0/3] drm/amdgpu: make the reserved SPM VMID per-partition James Zhu 3 siblings, 1 reply; 11+ messages in thread From: James Zhu @ 2026-11-17 16:59 UTC (permalink / raw) To: amd-gfx; +Cc: Christian König, Sunil Khatri, James Zhu The SPM VMID reservation made through AMDGPU_VM_OP_RESERVE_VMID is recorded on AMDGPU_GFXHUB(0) only, while gfxhub v1.2's setup_vm_pt_regs broadcasts the page table base to every XCC. On multi-XCC parts the two disagree, in both directions: - the reservation only takes effect for jobs running on XCC 0, since the VMID grab path checks vm->reserved_vmid[ring->vm_hub] and the rings of XCC i use AMDGPU_GFXHUB(i); - the page table base is physically programmed on all XCCs anyway, and conflicting reservations are only detected on hub 0's id_mgr. Make the hub argument real end to end: amdgpu_gmc_bind_vmid() programs only the given hub, through the XCC-masked gfxhub callback where one exists, and amdgpu_vmid_free_spm() unbinds the hub it frees instead of hub 0. amdgpu_vm_spm_update() binds every GFXHUB on which @vm holds the reservation. The VM ioctl reserves the VMID on every GFXHUB instance through the new amdgpu_vmid_alloc_spm_gfxhubs() helper, so the reservation now takes effect, is bound and is flushed on every XCC the VM's jobs can run on, and a conflicting reservation shows up on whichever hub two VMs share. Signed-off-by: James Zhu <James.Zhu@amd.com> Assisted-by: Claude:claude-fable-5 --- drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 4 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c | 49 ++++++++++++++++++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_ids.h | 4 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 40 +++++++++++++------- 4 files changed, 82 insertions(+), 15 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c index 08d3ff61edcc..704a30ad388a 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c @@ -797,6 +797,7 @@ void amdgpu_gmc_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid, * * For VMIDs that are bound directly to a VM's page table outside the * normal per-job VM flush path, e.g. a dedicated out-of-pool VMID. + * Only @vmhub is programmed and flushed, other hubs keep their binding. */ void amdgpu_gmc_bind_vmid(struct amdgpu_device *adev, unsigned int vmhub, unsigned int vmid, uint64_t pd_addr) @@ -805,6 +806,9 @@ void amdgpu_gmc_bind_vmid(struct amdgpu_device *adev, unsigned int vmhub, if (is_mmhub) adev->mmhub.funcs->setup_vm_pt_regs(adev, vmid, pd_addr); + else if (adev->gfxhub.funcs->setup_vm_pt_regs_xcc) + adev->gfxhub.funcs->setup_vm_pt_regs_xcc(adev, vmid, pd_addr, + BIT(vmhub - AMDGPU_GFXHUB_START)); else adev->gfxhub.funcs->setup_vm_pt_regs(adev, vmid, pd_addr); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c index 9c879ac68352..41f3dba530e3 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c @@ -651,12 +651,59 @@ void amdgpu_vmid_free_spm(struct amdgpu_device *adev, struct amdgpu_vm *vm, id_mgr->reserved_vmid = false; if (adev->vm_manager.spm_vmid && adev->gfxhub.funcs) - amdgpu_gmc_bind_vmid(adev, AMDGPU_GFXHUB(0), + amdgpu_gmc_bind_vmid(adev, vmhub, adev->vm_manager.spm_vmid, 0); } mutex_unlock(&id_mgr->lock); } +/* + * amdgpu_vmid_alloc_spm_gfxhubs - reserve the SPM VMID on a set of GFXHUBs + * @adev: amdgpu device structure + * @vm: the VM to reserve the ID for + * @xcc_mask: XCC instances on whose GFXHUBs to reserve + * + * Reserve the SPM VMID on the GFXHUB of every XCC in @xcc_mask, so that + * the reservation covers every hub the VM's jobs can run on. + * + * Returns: 0 for success, the first per-hub error otherwise. On an error + * the reservation is released on every hub in @xcc_mask again. + */ +int amdgpu_vmid_alloc_spm_gfxhubs(struct amdgpu_device *adev, + struct amdgpu_vm *vm, uint32_t xcc_mask) +{ + int i, r; + + for_each_inst(i, xcc_mask) { + r = amdgpu_vmid_alloc_spm(adev, vm, AMDGPU_GFXHUB(i)); + if (r) { + amdgpu_vmid_free_spm_gfxhubs(adev, vm, xcc_mask); + return r; + } + } + + return 0; +} + +/* + * amdgpu_vmid_free_spm_gfxhubs - undo amdgpu_vmid_alloc_spm_gfxhubs + * @adev: amdgpu device structure + * @vm: the VM with the reserved ID + * @xcc_mask: XCC instances on whose GFXHUBs to free the reservation + * + * Free the SPM VMID on every GFXHUB in @xcc_mask @vm holds it on, hubs + * without a reservation are skipped. The same wait requirement as for + * amdgpu_vmid_free_spm() applies. + */ +void amdgpu_vmid_free_spm_gfxhubs(struct amdgpu_device *adev, + struct amdgpu_vm *vm, uint32_t xcc_mask) +{ + int i; + + for_each_inst(i, xcc_mask) + amdgpu_vmid_free_spm(adev, vm, AMDGPU_GFXHUB(i)); +} + /** * amdgpu_vmid_reset - reset VMID to zero * diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.h index c134000eecfd..d2b225d0aa38 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.h @@ -129,6 +129,10 @@ int amdgpu_vmid_alloc_spm(struct amdgpu_device *adev, struct amdgpu_vm *vm, unsigned vmhub); void amdgpu_vmid_free_spm(struct amdgpu_device *adev, struct amdgpu_vm *vm, unsigned vmhub); +int amdgpu_vmid_alloc_spm_gfxhubs(struct amdgpu_device *adev, + struct amdgpu_vm *vm, uint32_t xcc_mask); +void amdgpu_vmid_free_spm_gfxhubs(struct amdgpu_device *adev, + struct amdgpu_vm *vm, uint32_t xcc_mask); int amdgpu_vmid_grab(struct amdgpu_vm *vm, struct amdgpu_ring *ring, struct amdgpu_job *job, struct dma_fence **fence); void amdgpu_vmid_reset(struct amdgpu_device *adev, unsigned vmhub, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index 6b4dc0276e27..bd9aa35933dc 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -589,30 +589,36 @@ uint64_t amdgpu_vm_generation(struct amdgpu_device *adev, struct amdgpu_vm *vm) * The SPM VMID is fixed (adev->vm_manager.spm_vmid), so its page-table-base * register must track whichever VM currently holds the reservation instead * of being set per-job like a normal VMID. Bind it to @vm's real root PD - * address if @vm holds the reservation; otherwise do nothing, since @vm is - * not the one that could legitimately change the current binding. + * address on every GFXHUB @vm holds the reservation on; hubs reserved by + * other VMs or not reserved at all are left alone, since @vm is not the + * one that could legitimately change their binding. * * @vm's root PD must be locked by the caller. */ void amdgpu_vm_spm_update(struct amdgpu_device *adev, struct amdgpu_vm *vm) { - struct amdgpu_kq_vmid_mgr *id_mgr = - &adev->vm_manager.id_mgr[AMDGPU_GFXHUB(0)]; uint64_t pd_addr; + int i; if (!adev->vm_manager.spm_vmid || !adev->gfxhub.funcs) return; - mutex_lock(&id_mgr->lock); - if (vm->reserved_vmid[AMDGPU_GFXHUB(0)] != - &id_mgr->ids[adev->vm_manager.spm_vmid] || !amdgpu_vm_ready(vm)) { - mutex_unlock(&id_mgr->lock); + if (!amdgpu_vm_ready(vm)) return; - } pd_addr = amdgpu_gmc_pd_addr(vm->root.bo); - amdgpu_gmc_bind_vmid(adev, AMDGPU_GFXHUB(0), adev->vm_manager.spm_vmid, pd_addr); - mutex_unlock(&id_mgr->lock); + for_each_inst(i, adev->gfx.xcc_mask) { + struct amdgpu_kq_vmid_mgr *id_mgr = + &adev->vm_manager.id_mgr[AMDGPU_GFXHUB(i)]; + + mutex_lock(&id_mgr->lock); + if (vm->reserved_vmid[AMDGPU_GFXHUB(i)] == + &id_mgr->ids[adev->vm_manager.spm_vmid]) + amdgpu_gmc_bind_vmid(adev, AMDGPU_GFXHUB(i), + adev->vm_manager.spm_vmid, + pd_addr); + mutex_unlock(&id_mgr->lock); + } } /** @@ -3018,7 +3024,8 @@ int amdgpu_vm_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) int r; /* We only have requirement to reserve vmid from gfxhub */ - r = amdgpu_vmid_alloc_spm(adev, vm, AMDGPU_GFXHUB(0)); + r = amdgpu_vmid_alloc_spm_gfxhubs(adev, vm, + adev->gfx.xcc_mask); if (!r && !amdgpu_bo_reserve(vm->root.bo, true)) { amdgpu_vm_spm_update(adev, vm); amdgpu_bo_unreserve(vm->root.bo); @@ -3027,14 +3034,19 @@ int amdgpu_vm_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) } case AMDGPU_VM_OP_UNRESERVE_VMID: { struct amdgpu_bo *root = vm->root.bo; + bool reserved = false; + int i; amdgpu_bo_reserve(root, true); + for_each_inst(i, adev->gfx.xcc_mask) + reserved |= amdgpu_vmid_uses_reserved(vm, + AMDGPU_GFXHUB(i)); /* Wait for the jobs which might still use the reserved VMID */ - if (vm->reserved_vmid[AMDGPU_GFXHUB(0)]) + if (reserved) dma_resv_wait_timeout(root->tbo.base.resv, DMA_RESV_USAGE_READ, false, MAX_SCHEDULE_TIMEOUT); - amdgpu_vmid_free_spm(adev, vm, AMDGPU_GFXHUB(0)); + amdgpu_vmid_free_spm_gfxhubs(adev, vm, adev->gfx.xcc_mask); amdgpu_bo_unreserve(root); break; } -- 2.34.1 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [RFC PATCH 2/3] drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance 2026-11-17 16:59 ` [RFC PATCH 2/3] drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance James Zhu @ 2026-11-17 17:21 ` sashiko-bot 0 siblings, 0 replies; 11+ messages in thread From: sashiko-bot @ 2026-11-17 17:21 UTC (permalink / raw) To: James Zhu, amd-gfx Thank you for your contribution! Sashiko AI review found 3 potential issues to consider: - [High] drm/amdgpu: physical xcc_mask used as logical index in amdgpu_vm_spm_update() - [High] drm/amdgpu: amdgpu_vm_fini() only checks GFXHUB(0) before waiting on jobs - [Medium] drm/amdgpu: unlocked allocation and unsafe rollback in amdgpu_vmid_alloc_spm_gfxhubs() -- commit 0df4e49e9ba90dc0c43a497872e17c100a8d4894 Author: James Zhu <James.Zhu@amd.com> drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance This commit updates SPM VMID reservation, binding, and unbinding to operate per GFXHUB instance across XCCs rather than only on AMDGPU_GFXHUB(0). > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c > index 9c879ac683520..41f3dba530e38 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c [ ... ] > +int amdgpu_vmid_alloc_spm_gfxhubs(struct amdgpu_device *adev, > + struct amdgpu_vm *vm, uint32_t xcc_mask) > +{ > + int i, r; > + > + for_each_inst(i, xcc_mask) { > + r = amdgpu_vmid_alloc_spm(adev, vm, AMDGPU_GFXHUB(i)); > + if (r) { > + amdgpu_vmid_free_spm_gfxhubs(adev, vm, xcc_mask); > + return r; > + } > + } > + > + return 0; > +} [Severity: Medium] If amdgpu_vmid_alloc_spm() fails on a later hub, can rolling back with amdgpu_vmid_free_spm_gfxhubs(adev, vm, xcc_mask) unbind an active SPM VMID to 0 while GPU jobs are still using it? amdgpu_vmid_alloc_spm() returns 0 when vm->reserved_vmid[vmhub] is already set, and AMDGPU_VM_OP_RESERVE_VMID calls amdgpu_vmid_alloc_spm_gfxhubs() before reserving vm->root.bo. If vm already held a reservation on an earlier hub, or a concurrent job submission grabs a newly published vm->reserved_vmid[AMDGPU_GFXHUB(i)] before a subsequent hub fails with -ENOENT, the rollback frees and unbinds those earlier hubs to 0 without waiting via dma_resv_wait_timeout(). > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c > index 94b1cfe05e3e0..e1691145da903 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c > @@ -589,30 +589,36 @@ uint64_t amdgpu_vm_generation(struct amdgpu_device *adev, struct amdgpu_vm *vm) > * The SPM VMID is fixed (adev->vm_manager.spm_vmid), so its page-table-base > * register must track whichever VM currently holds the reservation instead > * of being set per-job like a normal VMID. Bind it to @vm's real root PD > - * address if @vm holds the reservation; otherwise do nothing, since @vm is > - * not the one that could legitimately change the current binding. > + * address on every GFXHUB @vm holds the reservation on; hubs reserved by > + * other VMs or not reserved at all are left alone, since @vm is not the > + * one that could legitimately change their binding. > * > * @vm's root PD must be locked by the caller. > */ > void amdgpu_vm_spm_update(struct amdgpu_device *adev, struct amdgpu_vm *vm) > { > - struct amdgpu_kq_vmid_mgr *id_mgr = > - &adev->vm_manager.id_mgr[AMDGPU_GFXHUB(0)]; > uint64_t pd_addr; > + int i; > > if (!adev->vm_manager.spm_vmid || !adev->gfxhub.funcs) > return; > > - mutex_lock(&id_mgr->lock); > - if (vm->reserved_vmid[AMDGPU_GFXHUB(0)] != > - &id_mgr->ids[adev->vm_manager.spm_vmid] || !amdgpu_vm_ready(vm)) { > - mutex_unlock(&id_mgr->lock); > + if (!amdgpu_vm_ready(vm)) > return; > - } > > pd_addr = amdgpu_gmc_pd_addr(vm->root.bo); > - amdgpu_gmc_bind_vmid(adev, AMDGPU_GFXHUB(0), adev->vm_manager.spm_vmid, pd_addr); > - mutex_unlock(&id_mgr->lock); > + for_each_inst(i, adev->gfx.xcc_mask) { > + struct amdgpu_kq_vmid_mgr *id_mgr = > + &adev->vm_manager.id_mgr[AMDGPU_GFXHUB(i)]; > + > + mutex_lock(&id_mgr->lock); > + if (vm->reserved_vmid[AMDGPU_GFXHUB(i)] == > + &id_mgr->ids[adev->vm_manager.spm_vmid]) > + amdgpu_gmc_bind_vmid(adev, AMDGPU_GFXHUB(i), > + adev->vm_manager.spm_vmid, > + pd_addr); > + mutex_unlock(&id_mgr->lock); > + } > } [Severity: High] Can iterating over adev->gfx.xcc_mask here and in amdgpu_vm_ioctl() index adev->reg_offset[GC_HWIP] out of bounds when lower-numbered physical XCCs are harvested? adev->gfx.xcc_mask is a bitmask of physical XCC instances, whereas AMDGPU_GFXHUB(i), adev->vmhub[], ring->vm_hub, and setup_vm_pt_regs_xcc() all use contiguous logical XCC indices 0 .. NUM_XCC(adev->gfx.xcc_mask) - 1 and translate them via GET_INST(GC, i): drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c:gfxhub_v1_2_xcc_setup_vm_pt_regs() { for_each_inst(i, xcc_mask) { hub = &adev->vmhub[AMDGPU_GFXHUB(i)]; WREG32_SOC15_OFFSET(GC, GET_INST(GC, i), regVM_CONTEXT0_PAGE_TABLE_BASE_ADDR_LO32, hub->ctx_addr_distance * vmid, lower_32_bits(page_table_base)); ... } When a lower physical XCC is harvested, amdgpu_populate_ip_map() sets adev->ip_map.dev_inst[GC_HWIP][i] to -1 for i >= NUM_XCC(adev->gfx.xcc_mask). Passing a physical bit position i >= NUM_XCC(adev->gfx.xcc_mask) through amdgpu_gmc_bind_vmid() causes GET_INST(GC, i) to return 0xffffffff, reading adev->reg_offset[GC_HWIP][0xffffffff] out of bounds and flushing an uninitialized adev->vmhub[AMDGPU_GFXHUB(i)], while skipping the logical GFXHUBs at the harvested bit positions. Using GENMASK(NUM_XCC(adev->gfx.xcc_mask) - 1, 0) avoids mixing physical and logical XCC masks. [ ... ] > @@ -3027,14 +3034,19 @@ int amdgpu_vm_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) > } > case AMDGPU_VM_OP_UNRESERVE_VMID: { > struct amdgpu_bo *root = vm->root.bo; > + bool reserved = false; > + int i; > > amdgpu_bo_reserve(root, true); > + for_each_inst(i, adev->gfx.xcc_mask) > + reserved |= amdgpu_vmid_uses_reserved(vm, > + AMDGPU_GFXHUB(i)); > /* Wait for the jobs which might still use the reserved VMID */ > - if (vm->reserved_vmid[AMDGPU_GFXHUB(0)]) > + if (reserved) > dma_resv_wait_timeout(root->tbo.base.resv, > DMA_RESV_USAGE_READ, false, > MAX_SCHEDULE_TIMEOUT); > - amdgpu_vmid_free_spm(adev, vm, AMDGPU_GFXHUB(0)); > + amdgpu_vmid_free_spm_gfxhubs(adev, vm, adev->gfx.xcc_mask); > amdgpu_bo_unreserve(root); > break; > } [Severity: High] Should amdgpu_vm_fini() also check all GFXHUBs before waiting on root->tbo.base.resv, matching this change in AMDGPU_VM_OP_UNRESERVE_VMID? drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c:amdgpu_vm_fini() { ... /* Wait for the jobs which might still use the reserved VMID */ if (vm->reserved_vmid[AMDGPU_GFXHUB(0)]) dma_resv_wait_timeout(root->tbo.base.resv, DMA_RESV_USAGE_READ, false, MAX_SCHEDULE_TIMEOUT); ... for (i = 0; i < AMDGPU_MAX_VMHUBS; i++) { amdgpu_vmid_free_spm(adev, vm, i); } ... } If a VM holds an SPM reservation on a non-zero GFXHUB while vm->reserved_vmid[AMDGPU_GFXHUB(0)] is NULL (for example when reserved on a subset of GFXHUBs via amdgpu_vmid_alloc_spm_gfxhubs()), closing the DRM file skips dma_resv_wait_timeout(), frees the root PD in amdgpu_vm_pt_free_root(), and unbinds the SPM VMID to 0 while jobs using that VMID are still running. -- Sashiko AI review · https://sashiko.dev/#/patchset/20261117165952.3558762-1-James.Zhu@amd.com?part=2 ^ permalink raw reply [flat|nested] 11+ messages in thread
* [RFC PATCH 3/3] drm/amdgpu: scope AMDGPU_VM_OP_RESERVE_VMID to the caller's partition 2026-11-17 16:59 [RFC PATCH 0/3] drm/amdgpu: make the reserved SPM VMID per-partition James Zhu 2026-11-17 16:59 ` [RFC PATCH 1/3] drm/amdgpu: expose the XCC-masked setup_vm_pt_regs as a gfxhub callback James Zhu 2026-11-17 16:59 ` [RFC PATCH 2/3] drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance James Zhu @ 2026-11-17 16:59 ` James Zhu 2026-11-17 17:18 ` sashiko-bot 2026-11-17 18:19 ` [RFC PATCH v2 0/3] drm/amdgpu: make the reserved SPM VMID per-partition James Zhu 3 siblings, 1 reply; 11+ messages in thread From: James Zhu @ 2026-11-17 16:59 UTC (permalink / raw) To: amd-gfx; +Cc: Christian König, Sunil Khatri, James Zhu On partitioned devices every partition is exposed as an own render node and fpriv->xcp_id records which partition a drm file belongs to. But AMDGPU_VM_OP_RESERVE_VMID reserves the VMID on every GFXHUB of the device: a reservation made on one partition's render node blocks every other partition from reserving, and rebinds the VMID's page table base on XCCs the caller can't even submit to. Derive the GFXHUB set from the caller's partition instead. The reservation then only covers the XCCs of the caller's partition, and each partition can hold the reserved VMID on its own hubs independently, matching the isolation the render nodes promise. Unreserve keeps operating on every hub, freeing whatever the VM actually holds. Without partitioning, or on the whole-device node, the mask stays adev->gfx.xcc_mask and nothing changes. Signed-off-by: James Zhu <James.Zhu@amd.com> Assisted-by: Claude:claude-fable-5 --- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index bd9aa35933dc..d37af0dbd691 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -3021,11 +3021,21 @@ int amdgpu_vm_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) switch (args->in.op) { case AMDGPU_VM_OP_RESERVE_VMID: { + uint32_t xcc_mask = adev->gfx.xcc_mask; int r; + /* + * Scope the reservation to the hardware partition behind the + * caller's render node, so other partitions can hold the + * reserved VMID on their own XCCs independently. + */ + if (adev->xcp_mgr && fpriv->xcp_id != AMDGPU_XCP_NO_PARTITION) + amdgpu_xcp_get_inst_details( + &adev->xcp_mgr->xcp[fpriv->xcp_id], + AMDGPU_XCP_GFX, &xcc_mask); + /* We only have requirement to reserve vmid from gfxhub */ - r = amdgpu_vmid_alloc_spm_gfxhubs(adev, vm, - adev->gfx.xcc_mask); + r = amdgpu_vmid_alloc_spm_gfxhubs(adev, vm, xcc_mask); if (!r && !amdgpu_bo_reserve(vm->root.bo, true)) { amdgpu_vm_spm_update(adev, vm); amdgpu_bo_unreserve(vm->root.bo); -- 2.34.1 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [RFC PATCH 3/3] drm/amdgpu: scope AMDGPU_VM_OP_RESERVE_VMID to the caller's partition 2026-11-17 16:59 ` [RFC PATCH 3/3] drm/amdgpu: scope AMDGPU_VM_OP_RESERVE_VMID to the caller's partition James Zhu @ 2026-11-17 17:18 ` sashiko-bot 0 siblings, 0 replies; 11+ messages in thread From: sashiko-bot @ 2026-11-17 17:18 UTC (permalink / raw) To: James Zhu, amd-gfx Thank you for your contribution! Sashiko AI review found 2 potential issues to consider: - [High] drm/amdgpu: skipped wait in amdgpu_vm_fini() for partition SPM VMID - [Medium] drm/amdgpu: unchecked amdgpu_xcp_get_inst_details() in amdgpu_vm_ioctl() -- commit a5f43653ba23f302ebd963bef5053a4d46508cd1 Author: James Zhu <James.Zhu@amd.com> drm/amdgpu: scope AMDGPU_VM_OP_RESERVE_VMID to the caller's partition This commit scopes AMDGPU_VM_OP_RESERVE_VMID to the GFXHUB instances of the caller's compute partition when fpriv->xcp_id is set. > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c > index e1691145da903..8098fbb542c82 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c > @@ -3021,11 +3021,21 @@ int amdgpu_vm_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) > > switch (args->in.op) { > case AMDGPU_VM_OP_RESERVE_VMID: { > + uint32_t xcc_mask = adev->gfx.xcc_mask; > int r; > > + /* > + * Scope the reservation to the hardware partition behind the > + * caller's render node, so other partitions can hold the > + * reserved VMID on their own XCCs independently. > + */ > + if (adev->xcp_mgr && fpriv->xcp_id != AMDGPU_XCP_NO_PARTITION) > + amdgpu_xcp_get_inst_details( > + &adev->xcp_mgr->xcp[fpriv->xcp_id], > + AMDGPU_XCP_GFX, &xcc_mask); [Severity: Medium] Should the return value of amdgpu_xcp_get_inst_details() be checked here? If a runtime compute partition switch reduces the number of partitions while a DRM file opened on a higher-numbered partition remains open, xcp[fpriv->xcp_id].valid becomes false and amdgpu_xcp_get_inst_details() returns -EINVAL without updating xcc_mask. The reservation then silently falls back to adev->gfx.xcc_mask and reserves the SPM VMID across every GFXHUB on the device. > + > /* We only have requirement to reserve vmid from gfxhub */ > - r = amdgpu_vmid_alloc_spm_gfxhubs(adev, vm, > - adev->gfx.xcc_mask); > + r = amdgpu_vmid_alloc_spm_gfxhubs(adev, vm, xcc_mask); [Severity: High] When a process on a non-zero partition (fpriv->xcp_id > 0) closes its DRM file without calling AMDGPU_VM_OP_UNRESERVE_VMID, does amdgpu_vm_fini() skip waiting for in-flight jobs before unbinding and freeing the reserved SPM VMID? For fpriv->xcp_id > 0, xcc_mask only includes that partition's XCCs, so vm->reserved_vmid[AMDGPU_GFXHUB(0)] stays NULL. Unlike AMDGPU_VM_OP_UNRESERVE_VMID, amdgpu_vm_fini() only checks GFXHUB(0) before waiting and then frees the SPM VMID on all hubs: drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c:amdgpu_vm_fini() { ... /* Wait for the jobs which might still use the reserved VMID */ if (vm->reserved_vmid[AMDGPU_GFXHUB(0)]) dma_resv_wait_timeout(root->tbo.base.resv, DMA_RESV_USAGE_READ, false, MAX_SCHEDULE_TIMEOUT); ... for (i = 0; i < AMDGPU_MAX_VMHUBS; i++) { amdgpu_vmid_free_spm(adev, vm, i); } ... } amdgpu_vmid_free_spm() then unbinds spm_vmid to address 0 and clears id_mgr->reserved_vmid while GPU jobs on that partition are still running under spm_vmid, causing in-flight work to page-fault and letting another process rebind the VMID concurrently. Checking all GFXHUBs in adev->gfx.xcc_mask in amdgpu_vm_fini(), matching AMDGPU_VM_OP_UNRESERVE_VMID, would avoid this regression. -- Sashiko AI review · https://sashiko.dev/#/patchset/20261117165952.3558762-1-James.Zhu@amd.com?part=3 ^ permalink raw reply [flat|nested] 11+ messages in thread
* [RFC PATCH v2 0/3] drm/amdgpu: make the reserved SPM VMID per-partition 2026-11-17 16:59 [RFC PATCH 0/3] drm/amdgpu: make the reserved SPM VMID per-partition James Zhu ` (2 preceding siblings ...) 2026-11-17 16:59 ` [RFC PATCH 3/3] drm/amdgpu: scope AMDGPU_VM_OP_RESERVE_VMID to the caller's partition James Zhu @ 2026-11-17 18:19 ` James Zhu 2026-11-17 18:19 ` [RFC PATCH v2 1/3] drm/amdgpu: expose the XCC-masked setup_vm_pt_regs as a gfxhub callback James Zhu ` (2 more replies) 3 siblings, 3 replies; 11+ messages in thread From: James Zhu @ 2026-11-17 18:19 UTC (permalink / raw) To: amd-gfx; +Cc: Christian König, Sunil Khatri, James Zhu The reserved (SPM) VMID handling was designed on single-XCC hardware, where one GFXHUB is all there is. On multi-XCC parts with partitioning every XCC has an own GFXHUB and every partition is exposed as an own render node, and the current code is incoherent in three ways at once: 1. AMDGPU_VM_OP_RESERVE_VMID records the reservation on AMDGPU_GFXHUB(0) only. The VMID grab path checks vm->reserved_vmid[ring->vm_hub], and the rings of XCC i use AMDGPU_GFXHUB(i) - so for any job running on XCC 1..n the reservation silently does nothing. A debugger relying on a stable VMID gets one on exactly one XCC of the partition it runs on, or none at all. 2. The physical programming goes the other way: the setup_vm_pt_regs gfxhub callback broadcasts the SPM VMID's page table base to every XCC of the device, including XCCs of partitions the caller cannot even submit to. 3. The exclusion is device-global: because every reservation contends for hub 0's id_mgr, one partition's render node blocks every other partition from reserving, which breaks the isolation the render nodes are supposed to provide. None of this is visible on single-XCC hardware, where "hub 0", "all hubs" and "the caller's hubs" happen to be the same thing. gfx 9.4.3 partition modes are where the three diverge. This series makes the per-hub infrastructure, which already exists in the data structures (vm->reserved_vmid[] and id_mgr[] are per-vmhub arrays), real end to end: Patch 1 exposes the XCC-masked setup_vm_pt_regs variant, which gfxhub v1.2 and v12.1 already implement internally, as a gfxhub callback. Patch 2 makes the hub argument of the binding path mean what it says: amdgpu_gmc_bind_vmid() programs and flushes only the given hub, amdgpu_vmid_free_spm() unbinds the hub it frees, amdgpu_vm_spm_update() binds every hub the VM holds the reservation on, and the ioctl reserves on every GFXHUB instance. This fixes 1. and 2. while keeping the device-wide scope. Patch 3 then derives the hub set from the caller's partition (fpriv->xcp_id), fixing 3.: each partition reserves, binds and conflicts only on its own XCCs. Unreserve keeps operating on every hub, freeing whatever the VM actually holds. A note on the record granularity: the reservation could instead be recorded on the partition's first GFXHUB only and treated as a pure ownership token, closest to the original single-hub design. But the record plays three roles at once: the ownership token, the set of hubs whose page table base has to be bound (each XCC translates through its own hub, so the binding must cover every XCC of the partition either way), and the per-hub check of the VMID grab and CS paths (vm->reserved_vmid[ring->vm_hub]). Recording on every hub of the partition keeps the three consistent by construction, lets amdgpu_vm_spm_update() derive the binding set from the existing structures without a new field, and gives jobs on every XCC of the partition the same reserved-VMID semantics instead of only on the first one. The VMID number itself is the same everywhere either way: it is always adev->vm_manager.spm_vmid. No UAPI change. On devices without partitioning, or on the whole-device node, the mask degenerates to every XCC and nothing changes; on single-XCC parts the series is a no-op. Compile-tested on amd-staging-drm-next with CONFIG_DRM_AMDGPU=m; checkpatch clean. v2: address the Sashiko AI review findings, all of which checked out: - Iterate the GFXHUBs with a logical XCC mask (GENMASK(NUM_XCC(adev->gfx.xcc_mask) - 1, 0)) instead of the physical adev->gfx.xcc_mask, which can be sparse when an XCC is harvested and would send out-of-range logical indices into GET_INST() (amdgpu_vm_spm_update(), both ioctl ops). The partition path already used the logical xcp inst_mask. - amdgpu_vm_fini() now checks every VMHUB, not just GFXHUB(0), before waiting for jobs that may still use a reserved VMID; with patch 3 a partition > 0 holds no reservation on hub 0 and the wait was skipped. - amdgpu_vmid_alloc_spm_gfxhubs() rolls back only the hubs it newly reserved, and waits on the root PD reservation first, honoring amdgpu_vmid_free_spm()'s wait requirement; previously the rollback could free pre-held hubs and unbind a VMID a just-submitted job uses. - The return of amdgpu_xcp_get_inst_details() is checked; a stale fpriv->xcp_id now fails the ioctl instead of silently reserving the whole device. James Zhu (3): drm/amdgpu: expose the XCC-masked setup_vm_pt_regs as a gfxhub callback drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance drm/amdgpu: scope AMDGPU_VM_OP_RESERVE_VMID to the caller's partition drivers/gpu/drm/amd/amdgpu/amdgpu_gfxhub.h | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 4 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c | 68 ++++++++++++++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_ids.h | 4 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 70 ++++++++++++++++------ drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c | 1 + drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c | 1 + 7 files changed, 132 insertions(+), 18 deletions(-) -- 2.34.1 ^ permalink raw reply [flat|nested] 11+ messages in thread
* [RFC PATCH v2 1/3] drm/amdgpu: expose the XCC-masked setup_vm_pt_regs as a gfxhub callback 2026-11-17 18:19 ` [RFC PATCH v2 0/3] drm/amdgpu: make the reserved SPM VMID per-partition James Zhu @ 2026-11-17 18:19 ` James Zhu 2026-11-17 18:19 ` [RFC PATCH v2 2/3] drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance James Zhu 2026-11-17 18:19 ` [RFC PATCH v2 3/3] drm/amdgpu: scope AMDGPU_VM_OP_RESERVE_VMID to the caller's partition James Zhu 2 siblings, 0 replies; 11+ messages in thread From: James Zhu @ 2026-11-17 18:19 UTC (permalink / raw) To: amd-gfx; +Cc: Christian König, Sunil Khatri, James Zhu On GFX IPs with multiple XCCs each XCC has an own GFXHUB, but the setup_vm_pt_regs gfxhub callback programs a VMID's page table base on every XCC. Callers which bind a VMID on a single VMHUB, like the SPM VMID binding, have no way to reach one GFXHUB through the callback interface. gfxhub v1.2 and v12.1 already implement an XCC-masked variant internally. Expose it as a new setup_vm_pt_regs_xcc callback. GFXHUBs of IPs with a single XCC don't implement it, for them the existing callback already is per-hub. Signed-off-by: James Zhu <James.Zhu@amd.com> Assisted-by: Claude:claude-fable-5 --- drivers/gpu/drm/amd/amdgpu/amdgpu_gfxhub.h | 2 ++ drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c | 1 + drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c | 1 + 3 files changed, 4 insertions(+) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfxhub.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfxhub.h index c7b44aeb671b..19b961cb12f6 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfxhub.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfxhub.h @@ -28,6 +28,8 @@ struct amdgpu_gfxhub_funcs { u64 (*get_mc_fb_offset)(struct amdgpu_device *adev); void (*setup_vm_pt_regs)(struct amdgpu_device *adev, uint32_t vmid, uint64_t page_table_base); + void (*setup_vm_pt_regs_xcc)(struct amdgpu_device *adev, uint32_t vmid, + uint64_t page_table_base, uint32_t xcc_mask); int (*gart_enable)(struct amdgpu_device *adev); void (*gart_disable)(struct amdgpu_device *adev); diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c index b13967a1c8fb..7bc17721f463 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c @@ -809,6 +809,7 @@ const struct amdgpu_gfxhub_funcs gfxhub_v12_1_funcs = { .get_fb_location = gfxhub_v12_1_get_fb_location, .get_mc_fb_offset = gfxhub_v12_1_get_mc_fb_offset, .setup_vm_pt_regs = gfxhub_v12_1_setup_vm_pt_regs, + .setup_vm_pt_regs_xcc = gfxhub_v12_1_xcc_setup_vm_pt_regs, .gart_enable = gfxhub_v12_1_gart_enable, .gart_disable = gfxhub_v12_1_gart_disable, .set_fault_enable_default = gfxhub_v12_1_set_fault_enable_default, diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c index e89813590539..a56c072daf15 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c @@ -634,6 +634,7 @@ static int gfxhub_v1_2_get_xgmi_info(struct amdgpu_device *adev) const struct amdgpu_gfxhub_funcs gfxhub_v1_2_funcs = { .get_mc_fb_offset = gfxhub_v1_2_get_mc_fb_offset, .setup_vm_pt_regs = gfxhub_v1_2_setup_vm_pt_regs, + .setup_vm_pt_regs_xcc = gfxhub_v1_2_xcc_setup_vm_pt_regs, .gart_enable = gfxhub_v1_2_gart_enable, .gart_disable = gfxhub_v1_2_gart_disable, .set_fault_enable_default = gfxhub_v1_2_set_fault_enable_default, -- 2.34.1 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [RFC PATCH v2 2/3] drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance 2026-11-17 18:19 ` [RFC PATCH v2 0/3] drm/amdgpu: make the reserved SPM VMID per-partition James Zhu 2026-11-17 18:19 ` [RFC PATCH v2 1/3] drm/amdgpu: expose the XCC-masked setup_vm_pt_regs as a gfxhub callback James Zhu @ 2026-11-17 18:19 ` James Zhu 2026-11-17 18:47 ` sashiko-bot 2026-11-17 18:19 ` [RFC PATCH v2 3/3] drm/amdgpu: scope AMDGPU_VM_OP_RESERVE_VMID to the caller's partition James Zhu 2 siblings, 1 reply; 11+ messages in thread From: James Zhu @ 2026-11-17 18:19 UTC (permalink / raw) To: amd-gfx; +Cc: Christian König, Sunil Khatri, James Zhu The SPM VMID reservation made through AMDGPU_VM_OP_RESERVE_VMID is recorded on AMDGPU_GFXHUB(0) only, while gfxhub v1.2's setup_vm_pt_regs broadcasts the page table base to every XCC. On multi-XCC parts the two disagree, in both directions: - the reservation only takes effect for jobs running on XCC 0, since the VMID grab path checks vm->reserved_vmid[ring->vm_hub] and the rings of XCC i use AMDGPU_GFXHUB(i); - the page table base is physically programmed on all XCCs anyway, and conflicting reservations are only detected on hub 0's id_mgr. Make the hub argument real end to end: amdgpu_gmc_bind_vmid() programs only the given hub, through the XCC-masked gfxhub callback where one exists, and amdgpu_vmid_free_spm() unbinds the hub it frees instead of hub 0. amdgpu_vm_spm_update() binds every GFXHUB on which @vm holds the reservation. The VM ioctl reserves the VMID on every GFXHUB instance through the new amdgpu_vmid_alloc_spm_gfxhubs() helper, so the reservation now takes effect, is bound and is flushed on every XCC the VM's jobs can run on, and a conflicting reservation shows up on whichever hub two VMs share. Signed-off-by: James Zhu <James.Zhu@amd.com> Assisted-by: Claude:claude-fable-5 --- drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 4 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c | 68 ++++++++++++++++++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_ids.h | 4 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 56 +++++++++++++------- 4 files changed, 114 insertions(+), 18 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c index 08d3ff61edcc..704a30ad388a 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c @@ -797,6 +797,7 @@ void amdgpu_gmc_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid, * * For VMIDs that are bound directly to a VM's page table outside the * normal per-job VM flush path, e.g. a dedicated out-of-pool VMID. + * Only @vmhub is programmed and flushed, other hubs keep their binding. */ void amdgpu_gmc_bind_vmid(struct amdgpu_device *adev, unsigned int vmhub, unsigned int vmid, uint64_t pd_addr) @@ -805,6 +806,9 @@ void amdgpu_gmc_bind_vmid(struct amdgpu_device *adev, unsigned int vmhub, if (is_mmhub) adev->mmhub.funcs->setup_vm_pt_regs(adev, vmid, pd_addr); + else if (adev->gfxhub.funcs->setup_vm_pt_regs_xcc) + adev->gfxhub.funcs->setup_vm_pt_regs_xcc(adev, vmid, pd_addr, + BIT(vmhub - AMDGPU_GFXHUB_START)); else adev->gfxhub.funcs->setup_vm_pt_regs(adev, vmid, pd_addr); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c index 9c879ac68352..a4403e5cc344 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c @@ -651,12 +651,78 @@ void amdgpu_vmid_free_spm(struct amdgpu_device *adev, struct amdgpu_vm *vm, id_mgr->reserved_vmid = false; if (adev->vm_manager.spm_vmid && adev->gfxhub.funcs) - amdgpu_gmc_bind_vmid(adev, AMDGPU_GFXHUB(0), + amdgpu_gmc_bind_vmid(adev, vmhub, adev->vm_manager.spm_vmid, 0); } mutex_unlock(&id_mgr->lock); } +/* + * amdgpu_vmid_alloc_spm_gfxhubs - reserve the SPM VMID on a set of GFXHUBs + * @adev: amdgpu device structure + * @vm: the VM to reserve the ID for + * @xcc_mask: XCC instances on whose GFXHUBs to reserve + * + * Reserve the SPM VMID on the GFXHUB of every XCC in @xcc_mask, so that + * the reservation covers every hub the VM's jobs can run on. + * + * Returns: 0 for success, the first per-hub error otherwise. On an error + * only the hubs this call reserved are released again, after waiting for + * the jobs which might have grabbed the reserved VMID in the meantime; + * hubs @vm already held before the call stay reserved. + */ +int amdgpu_vmid_alloc_spm_gfxhubs(struct amdgpu_device *adev, + struct amdgpu_vm *vm, uint32_t xcc_mask) +{ + uint32_t allocated = 0; + int i, r; + + for_each_inst(i, xcc_mask) { + bool held = amdgpu_vmid_uses_reserved(vm, AMDGPU_GFXHUB(i)); + + r = amdgpu_vmid_alloc_spm(adev, vm, AMDGPU_GFXHUB(i)); + if (r) + goto error_free; + if (!held) + allocated |= BIT(i); + } + + return 0; + +error_free: + /* + * A job may already have grabbed a freshly reserved VMID, give it + * the wait amdgpu_vmid_free_spm() demands before the rollback + * unbinds the VMID again. + */ + if (allocated) { + dma_resv_wait_timeout(vm->root.bo->tbo.base.resv, + DMA_RESV_USAGE_READ, false, + MAX_SCHEDULE_TIMEOUT); + amdgpu_vmid_free_spm_gfxhubs(adev, vm, allocated); + } + return r; +} + +/* + * amdgpu_vmid_free_spm_gfxhubs - undo amdgpu_vmid_alloc_spm_gfxhubs + * @adev: amdgpu device structure + * @vm: the VM with the reserved ID + * @xcc_mask: XCC instances on whose GFXHUBs to free the reservation + * + * Free the SPM VMID on every GFXHUB in @xcc_mask @vm holds it on, hubs + * without a reservation are skipped. The same wait requirement as for + * amdgpu_vmid_free_spm() applies. + */ +void amdgpu_vmid_free_spm_gfxhubs(struct amdgpu_device *adev, + struct amdgpu_vm *vm, uint32_t xcc_mask) +{ + int i; + + for_each_inst(i, xcc_mask) + amdgpu_vmid_free_spm(adev, vm, AMDGPU_GFXHUB(i)); +} + /** * amdgpu_vmid_reset - reset VMID to zero * diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.h index c134000eecfd..d2b225d0aa38 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.h @@ -129,6 +129,10 @@ int amdgpu_vmid_alloc_spm(struct amdgpu_device *adev, struct amdgpu_vm *vm, unsigned vmhub); void amdgpu_vmid_free_spm(struct amdgpu_device *adev, struct amdgpu_vm *vm, unsigned vmhub); +int amdgpu_vmid_alloc_spm_gfxhubs(struct amdgpu_device *adev, + struct amdgpu_vm *vm, uint32_t xcc_mask); +void amdgpu_vmid_free_spm_gfxhubs(struct amdgpu_device *adev, + struct amdgpu_vm *vm, uint32_t xcc_mask); int amdgpu_vmid_grab(struct amdgpu_vm *vm, struct amdgpu_ring *ring, struct amdgpu_job *job, struct dma_fence **fence); void amdgpu_vmid_reset(struct amdgpu_device *adev, unsigned vmhub, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index 6b4dc0276e27..a9c23759d058 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -589,30 +589,38 @@ uint64_t amdgpu_vm_generation(struct amdgpu_device *adev, struct amdgpu_vm *vm) * The SPM VMID is fixed (adev->vm_manager.spm_vmid), so its page-table-base * register must track whichever VM currently holds the reservation instead * of being set per-job like a normal VMID. Bind it to @vm's real root PD - * address if @vm holds the reservation; otherwise do nothing, since @vm is - * not the one that could legitimately change the current binding. + * address on every GFXHUB @vm holds the reservation on; hubs reserved by + * other VMs or not reserved at all are left alone, since @vm is not the + * one that could legitimately change their binding. * * @vm's root PD must be locked by the caller. */ void amdgpu_vm_spm_update(struct amdgpu_device *adev, struct amdgpu_vm *vm) { - struct amdgpu_kq_vmid_mgr *id_mgr = - &adev->vm_manager.id_mgr[AMDGPU_GFXHUB(0)]; + /* the GFXHUBs are indexed by logical XCC, adev->gfx.xcc_mask is physical */ + uint32_t gfxhub_mask = GENMASK(NUM_XCC(adev->gfx.xcc_mask) - 1, 0); uint64_t pd_addr; + int i; if (!adev->vm_manager.spm_vmid || !adev->gfxhub.funcs) return; - mutex_lock(&id_mgr->lock); - if (vm->reserved_vmid[AMDGPU_GFXHUB(0)] != - &id_mgr->ids[adev->vm_manager.spm_vmid] || !amdgpu_vm_ready(vm)) { - mutex_unlock(&id_mgr->lock); + if (!amdgpu_vm_ready(vm)) return; - } pd_addr = amdgpu_gmc_pd_addr(vm->root.bo); - amdgpu_gmc_bind_vmid(adev, AMDGPU_GFXHUB(0), adev->vm_manager.spm_vmid, pd_addr); - mutex_unlock(&id_mgr->lock); + for_each_inst(i, gfxhub_mask) { + struct amdgpu_kq_vmid_mgr *id_mgr = + &adev->vm_manager.id_mgr[AMDGPU_GFXHUB(i)]; + + mutex_lock(&id_mgr->lock); + if (vm->reserved_vmid[AMDGPU_GFXHUB(i)] == + &id_mgr->ids[adev->vm_manager.spm_vmid]) + amdgpu_gmc_bind_vmid(adev, AMDGPU_GFXHUB(i), + adev->vm_manager.spm_vmid, + pd_addr); + mutex_unlock(&id_mgr->lock); + } } /** @@ -2871,9 +2879,14 @@ void amdgpu_vm_fini(struct amdgpu_device *adev, struct amdgpu_vm *vm) root = amdgpu_bo_ref(vm->root.bo); amdgpu_bo_reserve(root, true); /* Wait for the jobs which might still use the reserved VMID */ - if (vm->reserved_vmid[AMDGPU_GFXHUB(0)]) - dma_resv_wait_timeout(root->tbo.base.resv, DMA_RESV_USAGE_READ, - false, MAX_SCHEDULE_TIMEOUT); + for (i = 0; i < AMDGPU_MAX_VMHUBS; i++) { + if (vm->reserved_vmid[i]) { + dma_resv_wait_timeout(root->tbo.base.resv, + DMA_RESV_USAGE_READ, + false, MAX_SCHEDULE_TIMEOUT); + break; + } + } dma_fence_wait(vm->last_unlocked, false); dma_fence_put(vm->last_unlocked); dma_fence_wait(vm->last_seq_flush, false); @@ -3015,10 +3028,12 @@ int amdgpu_vm_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) switch (args->in.op) { case AMDGPU_VM_OP_RESERVE_VMID: { + /* logical XCCs, the GFXHUB index space; xcc_mask is physical */ + uint32_t gfxhub_mask = GENMASK(NUM_XCC(adev->gfx.xcc_mask) - 1, 0); int r; /* We only have requirement to reserve vmid from gfxhub */ - r = amdgpu_vmid_alloc_spm(adev, vm, AMDGPU_GFXHUB(0)); + r = amdgpu_vmid_alloc_spm_gfxhubs(adev, vm, gfxhub_mask); if (!r && !amdgpu_bo_reserve(vm->root.bo, true)) { amdgpu_vm_spm_update(adev, vm); amdgpu_bo_unreserve(vm->root.bo); @@ -3026,15 +3041,22 @@ int amdgpu_vm_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) return r; } case AMDGPU_VM_OP_UNRESERVE_VMID: { + /* logical XCCs, the GFXHUB index space; xcc_mask is physical */ + uint32_t gfxhub_mask = GENMASK(NUM_XCC(adev->gfx.xcc_mask) - 1, 0); struct amdgpu_bo *root = vm->root.bo; + bool reserved = false; + int i; amdgpu_bo_reserve(root, true); + for_each_inst(i, gfxhub_mask) + reserved |= amdgpu_vmid_uses_reserved(vm, + AMDGPU_GFXHUB(i)); /* Wait for the jobs which might still use the reserved VMID */ - if (vm->reserved_vmid[AMDGPU_GFXHUB(0)]) + if (reserved) dma_resv_wait_timeout(root->tbo.base.resv, DMA_RESV_USAGE_READ, false, MAX_SCHEDULE_TIMEOUT); - amdgpu_vmid_free_spm(adev, vm, AMDGPU_GFXHUB(0)); + amdgpu_vmid_free_spm_gfxhubs(adev, vm, gfxhub_mask); amdgpu_bo_unreserve(root); break; } -- 2.34.1 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [RFC PATCH v2 2/3] drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance 2026-11-17 18:19 ` [RFC PATCH v2 2/3] drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance James Zhu @ 2026-11-17 18:47 ` sashiko-bot 0 siblings, 0 replies; 11+ messages in thread From: sashiko-bot @ 2026-11-17 18:47 UTC (permalink / raw) To: James Zhu, amd-gfx Thank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Medium] drm/amdgpu: missing root PD lock in amdgpu_vmid_alloc_spm_gfxhubs() -- commit 4216e97db26fd6fc889673c452d2434887bc13af Author: James Zhu <James.Zhu@amd.com> drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance This commit reserves, binds, and unbinds the SPM VMID on every GFXHUB instance rather than only on AMDGPU_GFXHUB(0). > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c > index 9c879ac683520..a4403e5cc3443 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c > @@ -651,12 +651,78 @@ void amdgpu_vmid_free_spm(struct amdgpu_device *adev, struct amdgpu_vm *vm, > id_mgr->reserved_vmid = false; > > if (adev->vm_manager.spm_vmid && adev->gfxhub.funcs) > - amdgpu_gmc_bind_vmid(adev, AMDGPU_GFXHUB(0), > + amdgpu_gmc_bind_vmid(adev, vmhub, > adev->vm_manager.spm_vmid, 0); > } > mutex_unlock(&id_mgr->lock); > } > > +/* > + * amdgpu_vmid_alloc_spm_gfxhubs - reserve the SPM VMID on a set of GFXHUBs > + * @adev: amdgpu device structure > + * @vm: the VM to reserve the ID for > + * @xcc_mask: XCC instances on whose GFXHUBs to reserve > + * > + * Reserve the SPM VMID on the GFXHUB of every XCC in @xcc_mask, so that > + * the reservation covers every hub the VM's jobs can run on. > + * > + * Returns: 0 for success, the first per-hub error otherwise. On an error > + * only the hubs this call reserved are released again, after waiting for > + * the jobs which might have grabbed the reserved VMID in the meantime; > + * hubs @vm already held before the call stay reserved. > + */ > +int amdgpu_vmid_alloc_spm_gfxhubs(struct amdgpu_device *adev, > + struct amdgpu_vm *vm, uint32_t xcc_mask) > +{ > + uint32_t allocated = 0; > + int i, r; > + > + for_each_inst(i, xcc_mask) { > + bool held = amdgpu_vmid_uses_reserved(vm, AMDGPU_GFXHUB(i)); > + > + r = amdgpu_vmid_alloc_spm(adev, vm, AMDGPU_GFXHUB(i)); > + if (r) > + goto error_free; > + if (!held) > + allocated |= BIT(i); > + } > + > + return 0; > + > +error_free: > + /* > + * A job may already have grabbed a freshly reserved VMID, give it > + * the wait amdgpu_vmid_free_spm() demands before the rollback > + * unbinds the VMID again. > + */ > + if (allocated) { > + dma_resv_wait_timeout(vm->root.bo->tbo.base.resv, > + DMA_RESV_USAGE_READ, false, > + MAX_SCHEDULE_TIMEOUT); > + amdgpu_vmid_free_spm_gfxhubs(adev, vm, allocated); > + } > + return r; > +} [ ... ] > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c > index 94b1cfe05e3e0..f64020162a4bf 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c [ ... ] > @@ -3015,10 +3028,12 @@ int amdgpu_vm_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) > > switch (args->in.op) { > case AMDGPU_VM_OP_RESERVE_VMID: { > + /* logical XCCs, the GFXHUB index space; xcc_mask is physical */ > + uint32_t gfxhub_mask = GENMASK(NUM_XCC(adev->gfx.xcc_mask) - 1, 0); > int r; > > /* We only have requirement to reserve vmid from gfxhub */ > - r = amdgpu_vmid_alloc_spm(adev, vm, AMDGPU_GFXHUB(0)); > + r = amdgpu_vmid_alloc_spm_gfxhubs(adev, vm, gfxhub_mask); > if (!r && !amdgpu_bo_reserve(vm->root.bo, true)) { > amdgpu_vm_spm_update(adev, vm); > amdgpu_bo_unreserve(vm->root.bo); > } > return r; > } [Severity: Medium] Can the error rollback in amdgpu_vmid_alloc_spm_gfxhubs() unbind a newly reserved SPM VMID while a concurrent CS job is still using it? Unlike AMDGPU_VM_OP_UNRESERVE_VMID, AMDGPU_VM_OP_RESERVE_VMID calls amdgpu_vmid_alloc_spm_gfxhubs() without holding the vm->root.bo reservation lock: CPU0 CPU1 amdgpu_vm_ioctl(RESERVE_VMID) amdgpu_vmid_alloc_spm_gfxhubs() amdgpu_vmid_alloc_spm(hub 0) vm->reserved_vmid[hub 0] = id amdgpu_vmid_alloc_spm(hub 1) -> -ENOENT dma_resv_wait_timeout(root) amdgpu_cs_ioctl() /* reserves root, adds fence */ amdgpu_job_prepare_job() amdgpu_vmid_grab() /* grabs reserved VMID on hub 0 */ amdgpu_vmid_free_spm(hub 0) amdgpu_gmc_bind_vmid(..., 0) <-- unbinds VMID under active job Because vm->root.bo is not reserved across the wait and free, a command submission on the same VM can attach a new fence and grab the reserved VMID on hub 0 after dma_resv_wait_timeout() returns and right before amdgpu_vmid_free_spm() unbinds its page-table-base register to 0, causing GPU page faults for the in-flight job. Running without vm->root.bo reserved also leaves held = amdgpu_vmid_uses_reserved() unsynchronized: amdgpu_vmid_alloc_spm() returns 0 when vm->reserved_vmid[vmhub] is already set, so two concurrent AMDGPU_VM_OP_RESERVE_VMID calls on the same VM can both observe held == false on hub 0 and set allocated |= BIT(0), causing a failure on a later hub in one thread to free the reservation held by the other. Similarly, concurrent RESERVE_VMID and UNRESERVE_VMID calls can interleave across hubs, and if amdgpu_bo_reserve(vm->root.bo, true) fails after amdgpu_vmid_alloc_spm_gfxhubs() succeeds, amdgpu_vm_spm_update() is skipped while returning 0. Reserving vm->root.bo in AMDGPU_VM_OP_RESERVE_VMID before calling amdgpu_vmid_alloc_spm_gfxhubs() and holding it across amdgpu_vm_spm_update() (matching AMDGPU_VM_OP_UNRESERVE_VMID) would serialize per-VM reservation state and prevent new CS jobs during rollback. -- Sashiko AI review · https://sashiko.dev/#/patchset/20261117181941.3968996-1-James.Zhu@amd.com?part=2 ^ permalink raw reply [flat|nested] 11+ messages in thread
* [RFC PATCH v2 3/3] drm/amdgpu: scope AMDGPU_VM_OP_RESERVE_VMID to the caller's partition 2026-11-17 18:19 ` [RFC PATCH v2 0/3] drm/amdgpu: make the reserved SPM VMID per-partition James Zhu 2026-11-17 18:19 ` [RFC PATCH v2 1/3] drm/amdgpu: expose the XCC-masked setup_vm_pt_regs as a gfxhub callback James Zhu 2026-11-17 18:19 ` [RFC PATCH v2 2/3] drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance James Zhu @ 2026-11-17 18:19 ` James Zhu 2 siblings, 0 replies; 11+ messages in thread From: James Zhu @ 2026-11-17 18:19 UTC (permalink / raw) To: amd-gfx; +Cc: Christian König, Sunil Khatri, James Zhu On partitioned devices every partition is exposed as an own render node and fpriv->xcp_id records which partition a drm file belongs to. But AMDGPU_VM_OP_RESERVE_VMID reserves the VMID on every GFXHUB of the device: a reservation made on one partition's render node blocks every other partition from reserving, and rebinds the VMID's page table base on XCCs the caller can't even submit to. Derive the GFXHUB set from the caller's partition instead. The reservation then only covers the XCCs of the caller's partition, and each partition can hold the reserved VMID on its own hubs independently, matching the isolation the render nodes promise. Unreserve keeps operating on every hub, freeing whatever the VM actually holds. Without partitioning, or on the whole-device node, the mask stays adev->gfx.xcc_mask and nothing changes. Signed-off-by: James Zhu <James.Zhu@amd.com> Assisted-by: Claude:claude-fable-5 --- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index a9c23759d058..403807cb615d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -3032,6 +3032,20 @@ int amdgpu_vm_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) uint32_t gfxhub_mask = GENMASK(NUM_XCC(adev->gfx.xcc_mask) - 1, 0); int r; + /* + * Scope the reservation to the hardware partition behind the + * caller's render node, so other partitions can hold the + * reserved VMID on their own XCCs independently. The + * partition's inst_mask is logical like the GFXHUB indices. + */ + if (adev->xcp_mgr && fpriv->xcp_id != AMDGPU_XCP_NO_PARTITION) { + r = amdgpu_xcp_get_inst_details( + &adev->xcp_mgr->xcp[fpriv->xcp_id], + AMDGPU_XCP_GFX, &gfxhub_mask); + if (r) + return r; + } + /* We only have requirement to reserve vmid from gfxhub */ r = amdgpu_vmid_alloc_spm_gfxhubs(adev, vm, gfxhub_mask); if (!r && !amdgpu_bo_reserve(vm->root.bo, true)) { -- 2.34.1 ^ permalink raw reply related [flat|nested] 11+ messages in thread
end of thread, other threads:[~2026-11-17 18:48 UTC | newest] Thread overview: 11+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-11-17 16:59 [RFC PATCH 0/3] drm/amdgpu: make the reserved SPM VMID per-partition James Zhu 2026-11-17 16:59 ` [RFC PATCH 1/3] drm/amdgpu: expose the XCC-masked setup_vm_pt_regs as a gfxhub callback James Zhu 2026-11-17 16:59 ` [RFC PATCH 2/3] drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance James Zhu 2026-11-17 17:21 ` sashiko-bot 2026-11-17 16:59 ` [RFC PATCH 3/3] drm/amdgpu: scope AMDGPU_VM_OP_RESERVE_VMID to the caller's partition James Zhu 2026-11-17 17:18 ` sashiko-bot 2026-11-17 18:19 ` [RFC PATCH v2 0/3] drm/amdgpu: make the reserved SPM VMID per-partition James Zhu 2026-11-17 18:19 ` [RFC PATCH v2 1/3] drm/amdgpu: expose the XCC-masked setup_vm_pt_regs as a gfxhub callback James Zhu 2026-11-17 18:19 ` [RFC PATCH v2 2/3] drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance James Zhu 2026-11-17 18:47 ` sashiko-bot 2026-11-17 18:19 ` [RFC PATCH v2 3/3] drm/amdgpu: scope AMDGPU_VM_OP_RESERVE_VMID to the caller's partition James Zhu
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.