AMD-GFX Archive on lore.kernel.org
 help / color / mirror / Atom feed
From: sashiko-bot@kernel.org
To: "James Zhu" <James.Zhu@amd.com>, amd-gfx@lists.freedesktop.org
Subject: Re: [RFC PATCH v2 2/3] drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance
Date: Tue, 17 Nov 2026 18:47:50 +0000	[thread overview]
Message-ID: <sashiko-outbox-219386@kernel.org> (raw)
In-Reply-To: <20261117181941.3968996-3-James.Zhu@amd.com>

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] drm/amdgpu: missing root PD lock in
  amdgpu_vmid_alloc_spm_gfxhubs()
--

commit 4216e97db26fd6fc889673c452d2434887bc13af
Author: James Zhu <James.Zhu@amd.com>

drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance

This commit reserves, binds, and unbinds the SPM VMID on every GFXHUB
instance rather than only on AMDGPU_GFXHUB(0).

> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c
> index 9c879ac683520..a4403e5cc3443 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c
> @@ -651,12 +651,78 @@ void amdgpu_vmid_free_spm(struct amdgpu_device *adev, struct amdgpu_vm *vm,
>  		id_mgr->reserved_vmid = false;
>  
>  		if (adev->vm_manager.spm_vmid && adev->gfxhub.funcs)
> -			amdgpu_gmc_bind_vmid(adev, AMDGPU_GFXHUB(0),
> +			amdgpu_gmc_bind_vmid(adev, vmhub,
>  					     adev->vm_manager.spm_vmid, 0);
>  	}
>  	mutex_unlock(&id_mgr->lock);
>  }
>  
> +/*
> + * amdgpu_vmid_alloc_spm_gfxhubs - reserve the SPM VMID on a set of GFXHUBs
> + * @adev: amdgpu device structure
> + * @vm: the VM to reserve the ID for
> + * @xcc_mask: XCC instances on whose GFXHUBs to reserve
> + *
> + * Reserve the SPM VMID on the GFXHUB of every XCC in @xcc_mask, so that
> + * the reservation covers every hub the VM's jobs can run on.
> + *
> + * Returns: 0 for success, the first per-hub error otherwise. On an error
> + * only the hubs this call reserved are released again, after waiting for
> + * the jobs which might have grabbed the reserved VMID in the meantime;
> + * hubs @vm already held before the call stay reserved.
> + */
> +int amdgpu_vmid_alloc_spm_gfxhubs(struct amdgpu_device *adev,
> +				  struct amdgpu_vm *vm, uint32_t xcc_mask)
> +{
> +	uint32_t allocated = 0;
> +	int i, r;
> +
> +	for_each_inst(i, xcc_mask) {
> +		bool held = amdgpu_vmid_uses_reserved(vm, AMDGPU_GFXHUB(i));
> +
> +		r = amdgpu_vmid_alloc_spm(adev, vm, AMDGPU_GFXHUB(i));
> +		if (r)
> +			goto error_free;
> +		if (!held)
> +			allocated |= BIT(i);
> +	}
> +
> +	return 0;
> +
> +error_free:
> +	/*
> +	 * A job may already have grabbed a freshly reserved VMID, give it
> +	 * the wait amdgpu_vmid_free_spm() demands before the rollback
> +	 * unbinds the VMID again.
> +	 */
> +	if (allocated) {
> +		dma_resv_wait_timeout(vm->root.bo->tbo.base.resv,
> +				      DMA_RESV_USAGE_READ, false,
> +				      MAX_SCHEDULE_TIMEOUT);
> +		amdgpu_vmid_free_spm_gfxhubs(adev, vm, allocated);
> +	}
> +	return r;
> +}
[ ... ]
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> index 94b1cfe05e3e0..f64020162a4bf 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
[ ... ]
> @@ -3015,10 +3028,12 @@ int amdgpu_vm_ioctl(struct drm_device *dev, void *data, struct drm_file *filp)
>  
>  	switch (args->in.op) {
>  	case AMDGPU_VM_OP_RESERVE_VMID: {
> +		/* logical XCCs, the GFXHUB index space; xcc_mask is physical */
> +		uint32_t gfxhub_mask = GENMASK(NUM_XCC(adev->gfx.xcc_mask) - 1, 0);
>  		int r;
>  
>  		/* We only have requirement to reserve vmid from gfxhub */
> -		r = amdgpu_vmid_alloc_spm(adev, vm, AMDGPU_GFXHUB(0));
> +		r = amdgpu_vmid_alloc_spm_gfxhubs(adev, vm, gfxhub_mask);
>  		if (!r && !amdgpu_bo_reserve(vm->root.bo, true)) {
>  			amdgpu_vm_spm_update(adev, vm);
>  			amdgpu_bo_unreserve(vm->root.bo);
>  		}
>  		return r;
>  	}

[Severity: Medium]
Can the error rollback in amdgpu_vmid_alloc_spm_gfxhubs() unbind a newly
reserved SPM VMID while a concurrent CS job is still using it?

Unlike AMDGPU_VM_OP_UNRESERVE_VMID, AMDGPU_VM_OP_RESERVE_VMID calls
amdgpu_vmid_alloc_spm_gfxhubs() without holding the vm->root.bo
reservation lock:

CPU0                              CPU1
amdgpu_vm_ioctl(RESERVE_VMID)
  amdgpu_vmid_alloc_spm_gfxhubs()
    amdgpu_vmid_alloc_spm(hub 0)
      vm->reserved_vmid[hub 0] = id
    amdgpu_vmid_alloc_spm(hub 1)
      -> -ENOENT
    dma_resv_wait_timeout(root)
                                  amdgpu_cs_ioctl()
                                    /* reserves root, adds fence */
                                  amdgpu_job_prepare_job()
                                    amdgpu_vmid_grab()
                                      /* grabs reserved VMID on hub 0 */
    amdgpu_vmid_free_spm(hub 0)
      amdgpu_gmc_bind_vmid(..., 0)  <-- unbinds VMID under active job

Because vm->root.bo is not reserved across the wait and free, a command
submission on the same VM can attach a new fence and grab the reserved
VMID on hub 0 after dma_resv_wait_timeout() returns and right before
amdgpu_vmid_free_spm() unbinds its page-table-base register to 0, causing
GPU page faults for the in-flight job.

Running without vm->root.bo reserved also leaves held =
amdgpu_vmid_uses_reserved() unsynchronized: amdgpu_vmid_alloc_spm()
returns 0 when vm->reserved_vmid[vmhub] is already set, so two concurrent
AMDGPU_VM_OP_RESERVE_VMID calls on the same VM can both observe held ==
false on hub 0 and set allocated |= BIT(0), causing a failure on a later
hub in one thread to free the reservation held by the other.  Similarly,
concurrent RESERVE_VMID and UNRESERVE_VMID calls can interleave across
hubs, and if amdgpu_bo_reserve(vm->root.bo, true) fails after
amdgpu_vmid_alloc_spm_gfxhubs() succeeds, amdgpu_vm_spm_update() is
skipped while returning 0.

Reserving vm->root.bo in AMDGPU_VM_OP_RESERVE_VMID before calling
amdgpu_vmid_alloc_spm_gfxhubs() and holding it across
amdgpu_vm_spm_update() (matching AMDGPU_VM_OP_UNRESERVE_VMID) would
serialize per-VM reservation state and prevent new CS jobs during rollback.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261117181941.3968996-1-James.Zhu@amd.com?part=2

  reply	other threads:[~2026-11-17 18:48 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-17 16:59 [RFC PATCH 0/3] drm/amdgpu: make the reserved SPM VMID per-partition James Zhu
2026-11-17 16:59 ` [RFC PATCH 1/3] drm/amdgpu: expose the XCC-masked setup_vm_pt_regs as a gfxhub callback James Zhu
2026-11-18  9:36   ` Christian König
2026-11-17 16:59 ` [RFC PATCH 2/3] drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance James Zhu
2026-11-17 17:21   ` sashiko-bot
2026-11-18  9:38   ` Christian König
2026-11-17 16:59 ` [RFC PATCH 3/3] drm/amdgpu: scope AMDGPU_VM_OP_RESERVE_VMID to the caller's partition James Zhu
2026-11-17 17:18   ` sashiko-bot
2026-11-17 18:19 ` [RFC PATCH v2 0/3] drm/amdgpu: make the reserved SPM VMID per-partition James Zhu
2026-11-17 18:19   ` [RFC PATCH v2 1/3] drm/amdgpu: expose the XCC-masked setup_vm_pt_regs as a gfxhub callback James Zhu
2026-11-17 18:19   ` [RFC PATCH v2 2/3] drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance James Zhu
2026-11-17 18:47     ` sashiko-bot [this message]
2026-11-17 18:19   ` [RFC PATCH v2 3/3] drm/amdgpu: scope AMDGPU_VM_OP_RESERVE_VMID to the caller's partition James Zhu
2026-11-18 16:24 ` [RFC PATCH v3 0/3] drm/amdgpu: make the reserved SPM VMID per-partition James Zhu
2026-11-18 16:24   ` [RFC PATCH v3 1/3] drm/amdgpu: pass an XCC mask to the setup_vm_pt_regs gfxhub callback James Zhu
2026-11-18 16:24   ` [RFC PATCH v3 2/3] drm/amdgpu: reserve and bind the SPM VMID on every GFXHUB instance James Zhu
2026-11-18 16:24   ` [RFC PATCH v3 3/3] drm/amdgpu: scope AMDGPU_VM_OP_RESERVE_VMID to the caller's partition James Zhu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=sashiko-outbox-219386@kernel.org \
    --to=sashiko-bot@kernel.org \
    --cc=James.Zhu@amd.com \
    --cc=amd-gfx@lists.freedesktop.org \
    --cc=sashiko-reviews@lists.linux.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox