All of lore.kernel.org
 help / color / mirror / Atom feed
From: sashiko-bot@kernel.org
To: "Alex Deucher" <alexander.deucher@amd.com>,
	amd-gfx@lists.freedesktop.org
Subject: Re: [PATCH 10/11] drm/amdgpu/userq: enable support for creating gangs
Date: Wed, 18 Nov 2026 22:58:24 +0000	[thread overview]
Message-ID: <sashiko-outbox-222136@kernel.org> (raw)
In-Reply-To: <20261118223712.2518580-10-alexander.deucher@amd.com>

Thank you for your contribution! Sashiko AI review found 3 potential
issues to consider:
- [High] drm/amdgpu/userq: lock order inversion in
  amdgpu_userq_create_gang()
- [High] drm/amdgpu/userq: race between amdgpu_userq_destroy() and
  post_reset()
- [Medium] drm/amdgpu/userq: missing INVALID_VA check in
  amdgpu_userq_create_gang()
--

commit 8106c51d4a0229062b3c666f8f9004011c7d6501
Author: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/userq: enable support for creating gangs

This commit adds the AMDGPU_USERQ_OP_CREATE_GANG ioctl operation to link
primary and secondary usermode queues into a gang and cleans up the gang
links when a queue is destroyed.

> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
> index 3afd17af28548..388385380d8bc 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
> @@ -566,6 +566,7 @@ amdgpu_userq_destroy(struct amdgpu_userq_mgr *uq_mgr, struct amdgpu_usermode_que
>  {
>  	struct amdgpu_device *adev = uq_mgr->adev;
>  	const struct amdgpu_userq_funcs *uq_funcs = adev->userq_funcs[queue->queue_type];
> +	struct amdgpu_usermode_queue *other_gang_queue;
>  	int r = 0;
>  
>  	trace_amdgpu_userq_destroy_start(queue);
> @@ -585,6 +586,17 @@ amdgpu_userq_destroy(struct amdgpu_userq_mgr *uq_mgr, struct amdgpu_usermode_que
>  	debugfs_remove_recursive(queue->debugfs_queue);
>  #endif
>  	r = amdgpu_userq_unmap_helper(queue);
> +	/* clean up gangs */
> +	if (queue->gang_primary) {
> +		other_gang_queue = queue->gang_primary;
> +		other_gang_queue->gang_secondary = NULL;
> +		queue->gang_primary = NULL;
> +	}
> +	if (queue->gang_secondary) {
> +		other_gang_queue = queue->gang_secondary;
> +		other_gang_queue->gang_primary = NULL;
> +		queue->gang_secondary = NULL;
> +	}
>  	atomic_dec(&uq_mgr->userq_count[queue->queue_type]);
>  	amdgpu_userq_fence_driver_free(queue);
>  	queue->fence_drv = NULL;

[Severity: High]
Can amdgpu_userq_destroy() race with amdgpu_userq_post_reset() on the
partner queue's gang_secondary pointer and cause a use-after-free?

amdgpu_userq_destroy() holds uq_mgr->userq_mutex, while
amdgpu_userq_detach_doorbell() and amdgpu_userq_unmap_helper() each take
and release adev->reset_domain->sem individually.  During a GPU reset,
amdgpu_userq_post_reset() iterates adev->userq_doorbell_xa under
down_write(&adev->reset_domain->sem) without taking uq_mgr->userq_mutex:

CPU0                                  CPU1 (GPU reset)
amdgpu_userq_destroy(secondary)
  mutex_lock(&uq_mgr->userq_mutex)
  amdgpu_userq_detach_doorbell()
  amdgpu_userq_unmap_helper()
    /* releases reset_domain->sem */
                                      down_write(&adev->reset_domain->sem)
                                      amdgpu_userq_post_reset()
                                        amdgpu_userq_set_gang_helper(primary)
                                          sec = primary->gang_secondary
  primary->gang_secondary = NULL
  mutex_unlock(&uq_mgr->userq_mutex)
  uq_funcs->mqd_destroy(secondary)
  kfree(secondary)
                                          if (sec->state == ...)
                                            <-- use-after-free

In addition, if the reset begins between amdgpu_userq_detach_doorbell()
and amdgpu_userq_unmap_helper(), secondary is already erased from
adev->userq_doorbell_xa while still in AMDGPU_USERQ_STATE_MAPPED, and
amdgpu_userq_destroy() blocks on reset_domain->sem inside
amdgpu_userq_unmap_helper() before clearing primary->gang_secondary.
amdgpu_userq_post_reset() then remaps primary and calls
userq_funcs->set_gang(primary, secondary) on a secondary queue that was
never remapped after the reset and is about to be destroyed.

> @@ -838,6 +850,106 @@ amdgpu_userq_create(struct drm_file *filp, union drm_amdgpu_userq *args)
>  	return r;
>  }
>  
> +static int
> +amdgpu_userq_create_gang(struct drm_file *filp, union drm_amdgpu_userq *args)
> +{
> +	struct amdgpu_usermode_queue *primary_queue, *secondary_queue;
> +	struct amdgpu_fpriv *fpriv = filp->driver_priv;
> +	struct amdgpu_userq_mgr *uq_mgr = &fpriv->userq_mgr;
> +	const struct amdgpu_userq_funcs *userq_funcs;
> +	struct amdgpu_device *adev = uq_mgr->adev;
> +	int primary_queue_id = args->in_cg.primary_queue_id;
> +	int secondary_queue_id = args->in_cg.secondary_queue_id;
> +	int r;
> +
> +	/* The reset changes the queue state and the MES is unusable meanwhile */
> +	guard(rwsem_read)(&adev->reset_domain->sem);
> +
> +	mutex_lock(&uq_mgr->userq_mutex);

[Severity: High]
Does taking adev->reset_domain->sem before uq_mgr->userq_mutex invert the
lock hierarchy and risk an ABBA deadlock?

amdgpu_lockdep_init() and the rest of amdgpu_userq.c establish
uq_mgr->userq_mutex outer to adev->reset_domain->sem:

CPU0                              CPU1
----                              ----
lock(&uq_mgr->userq_mutex)
                                  down_read(&adev->reset_domain->sem)
down_read(&adev->reset_domain->sem)
                                  lock(&uq_mgr->userq_mutex)

 *** DEADLOCK ***

CPU0: amdgpu_userq_destroy() -> amdgpu_userq_unmap_helper()
CPU1: amdgpu_userq_create_gang()
(when a GPU reset writer is queued on adev->reset_domain->sem)

Also, because guard(rwsem_read)(&adev->reset_domain->sem) has function
scope, it is still held at the out label when amdgpu_userq_put() runs.
If a concurrent AMDGPU_USERQ_OP_FREE dropped the queue's base reference,
amdgpu_userq_put() enters amdgpu_userq_destroy() while holding
reset_domain->sem, waiting on cancel_delayed_work_sync() under the reset
semaphore and taking reset_domain->sem recursively in
amdgpu_userq_detach_doorbell() and amdgpu_userq_unmap_helper().

[ ... ]

> +	if (primary_queue->state == AMDGPU_USERQ_STATE_HUNG) {
> +		drm_file_err(uq_mgr->file, "Primary queue is hung\n");
> +		r = -EINVAL;
> +		goto out;
> +	}
> +
> +	if ((secondary_queue->queue_type != AMDGPU_HW_IP_GFX) &&
> +	    (secondary_queue->queue_type != AMDGPU_HW_IP_COMPUTE) &&
> +	    (secondary_queue->queue_type != AMDGPU_HW_IP_DMA)) {
> +		drm_file_err(uq_mgr->file, "Invalid gang secondary queue type\n");
> +		r = -EINVAL;
> +		goto out;
> +	}
> +
> +	if (secondary_queue->gang_primary || secondary_queue->gang_secondary) {
> +		drm_file_err(uq_mgr->file, "Secondary queue already in a gang\n");
> +		r = -EINVAL;
> +		goto out;
> +	}
> +
> +	if (secondary_queue->state == AMDGPU_USERQ_STATE_HUNG) {
> +		drm_file_err(uq_mgr->file, "Secondary queue is hung\n");
> +		r = -EINVAL;
> +		goto out;
> +	}
> +
> +	userq_funcs = adev->userq_funcs[primary_queue->queue_type];
> +	if ((primary_queue->state == AMDGPU_USERQ_STATE_MAPPED ||
> +	     primary_queue->state == AMDGPU_USERQ_STATE_PREEMPTED) &&
> +	    (secondary_queue->state == AMDGPU_USERQ_STATE_MAPPED ||
> +	     secondary_queue->state == AMDGPU_USERQ_STATE_PREEMPTED))
> +		r = userq_funcs->set_gang(primary_queue, secondary_queue);
> +	else
> +		/* if the queues are currently unmapped, the gang state will get
> +		 * updated when they get mapped.
> +		 */
> +		r = 0;

[Severity: Medium]
Should amdgpu_userq_create_gang() also reject queues in
AMDGPU_USERQ_STATE_INVALID_VA?

If a queue was marked AMDGPU_USERQ_STATE_INVALID_VA in
amdgpu_userq_restore_all(), amdgpu_userq_map_helper() will never remap it
(it only maps queues in AMDGPU_USERQ_STATE_UNMAPPED).  Here it passes the
HUNG checks and falls into the else branch with r = 0, linking a dead
queue into the partner queue's gang pointer without ever programming the
gang in MES.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261118223712.2518580-1-alexander.deucher@amd.com?part=10

  reply	other threads:[~2026-11-18 22:58 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-18 22:37 [PATCH 01/11] drm/amdgpu/userq: Add INFO IOCTL query for doorbell ranges Alex Deucher
2026-11-18 22:37 ` [PATCH 02/11] drm/amdgpu/mes11: add API for gang submit Alex Deucher
2026-11-18 22:37 ` [PATCH 03/11] drm/amdgpu/mes: add callback " Alex Deucher
2026-11-18 22:37 ` [PATCH 04/11] drm/amdgpu/mes11: add support for setting " Alex Deucher
2026-11-18 22:37 ` [PATCH 05/11] drm/amdgpu/mes12: " Alex Deucher
2026-11-18 22:37 ` [PATCH 06/11] drm/amdgpu/userq: add callback for gang setup Alex Deucher
2026-11-18 22:37 ` [PATCH 07/11] drm/amdgpu/userq/mes: implement set gang submit Alex Deucher
2026-11-18 22:44   ` sashiko-bot
2026-11-18 22:37 ` [PATCH 08/11] drm/amdgpu/userq: add handling for set_gang Alex Deucher
2026-11-18 22:54   ` sashiko-bot
2026-11-18 22:37 ` [PATCH 09/11] drm/amdgpu: add UAPI to create user queue gangs Alex Deucher
2026-11-20  5:15   ` Mohan Marimuthu, Yogesh
2026-11-20 19:28     ` Alex Deucher
2026-11-18 22:37 ` [PATCH 10/11] drm/amdgpu/userq: enable support for creating gangs Alex Deucher
2026-11-18 22:58   ` sashiko-bot [this message]
2026-11-18 22:37 ` [PATCH 11/11] drm/amdgpu: bump driver version for userq updates Alex Deucher
2026-11-18 23:04 ` [PATCH 01/11] drm/amdgpu/userq: Add INFO IOCTL query for doorbell ranges sashiko-bot

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=sashiko-outbox-222136@kernel.org \
    --to=sashiko-bot@kernel.org \
    --cc=alexander.deucher@amd.com \
    --cc=amd-gfx@lists.freedesktop.org \
    --cc=sashiko-reviews@lists.linux.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.