From: sashiko-bot@kernel.org
To: "Alex Deucher" <alexander.deucher@amd.com>,
amd-gfx@lists.freedesktop.org
Subject: Re: [PATCH 10/11] drm/amdgpu/userq: enable support for creating gangs
Date: Wed, 18 Nov 2026 22:58:24 +0000 [thread overview]
Message-ID: <sashiko-outbox-222136@kernel.org> (raw)
In-Reply-To: <20261118223712.2518580-10-alexander.deucher@amd.com>
Thank you for your contribution! Sashiko AI review found 3 potential
issues to consider:
- [High] drm/amdgpu/userq: lock order inversion in
amdgpu_userq_create_gang()
- [High] drm/amdgpu/userq: race between amdgpu_userq_destroy() and
post_reset()
- [Medium] drm/amdgpu/userq: missing INVALID_VA check in
amdgpu_userq_create_gang()
--
commit 8106c51d4a0229062b3c666f8f9004011c7d6501
Author: Alex Deucher <alexander.deucher@amd.com>
drm/amdgpu/userq: enable support for creating gangs
This commit adds the AMDGPU_USERQ_OP_CREATE_GANG ioctl operation to link
primary and secondary usermode queues into a gang and cleans up the gang
links when a queue is destroyed.
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
> index 3afd17af28548..388385380d8bc 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
> @@ -566,6 +566,7 @@ amdgpu_userq_destroy(struct amdgpu_userq_mgr *uq_mgr, struct amdgpu_usermode_que
> {
> struct amdgpu_device *adev = uq_mgr->adev;
> const struct amdgpu_userq_funcs *uq_funcs = adev->userq_funcs[queue->queue_type];
> + struct amdgpu_usermode_queue *other_gang_queue;
> int r = 0;
>
> trace_amdgpu_userq_destroy_start(queue);
> @@ -585,6 +586,17 @@ amdgpu_userq_destroy(struct amdgpu_userq_mgr *uq_mgr, struct amdgpu_usermode_que
> debugfs_remove_recursive(queue->debugfs_queue);
> #endif
> r = amdgpu_userq_unmap_helper(queue);
> + /* clean up gangs */
> + if (queue->gang_primary) {
> + other_gang_queue = queue->gang_primary;
> + other_gang_queue->gang_secondary = NULL;
> + queue->gang_primary = NULL;
> + }
> + if (queue->gang_secondary) {
> + other_gang_queue = queue->gang_secondary;
> + other_gang_queue->gang_primary = NULL;
> + queue->gang_secondary = NULL;
> + }
> atomic_dec(&uq_mgr->userq_count[queue->queue_type]);
> amdgpu_userq_fence_driver_free(queue);
> queue->fence_drv = NULL;
[Severity: High]
Can amdgpu_userq_destroy() race with amdgpu_userq_post_reset() on the
partner queue's gang_secondary pointer and cause a use-after-free?
amdgpu_userq_destroy() holds uq_mgr->userq_mutex, while
amdgpu_userq_detach_doorbell() and amdgpu_userq_unmap_helper() each take
and release adev->reset_domain->sem individually. During a GPU reset,
amdgpu_userq_post_reset() iterates adev->userq_doorbell_xa under
down_write(&adev->reset_domain->sem) without taking uq_mgr->userq_mutex:
CPU0 CPU1 (GPU reset)
amdgpu_userq_destroy(secondary)
mutex_lock(&uq_mgr->userq_mutex)
amdgpu_userq_detach_doorbell()
amdgpu_userq_unmap_helper()
/* releases reset_domain->sem */
down_write(&adev->reset_domain->sem)
amdgpu_userq_post_reset()
amdgpu_userq_set_gang_helper(primary)
sec = primary->gang_secondary
primary->gang_secondary = NULL
mutex_unlock(&uq_mgr->userq_mutex)
uq_funcs->mqd_destroy(secondary)
kfree(secondary)
if (sec->state == ...)
<-- use-after-free
In addition, if the reset begins between amdgpu_userq_detach_doorbell()
and amdgpu_userq_unmap_helper(), secondary is already erased from
adev->userq_doorbell_xa while still in AMDGPU_USERQ_STATE_MAPPED, and
amdgpu_userq_destroy() blocks on reset_domain->sem inside
amdgpu_userq_unmap_helper() before clearing primary->gang_secondary.
amdgpu_userq_post_reset() then remaps primary and calls
userq_funcs->set_gang(primary, secondary) on a secondary queue that was
never remapped after the reset and is about to be destroyed.
> @@ -838,6 +850,106 @@ amdgpu_userq_create(struct drm_file *filp, union drm_amdgpu_userq *args)
> return r;
> }
>
> +static int
> +amdgpu_userq_create_gang(struct drm_file *filp, union drm_amdgpu_userq *args)
> +{
> + struct amdgpu_usermode_queue *primary_queue, *secondary_queue;
> + struct amdgpu_fpriv *fpriv = filp->driver_priv;
> + struct amdgpu_userq_mgr *uq_mgr = &fpriv->userq_mgr;
> + const struct amdgpu_userq_funcs *userq_funcs;
> + struct amdgpu_device *adev = uq_mgr->adev;
> + int primary_queue_id = args->in_cg.primary_queue_id;
> + int secondary_queue_id = args->in_cg.secondary_queue_id;
> + int r;
> +
> + /* The reset changes the queue state and the MES is unusable meanwhile */
> + guard(rwsem_read)(&adev->reset_domain->sem);
> +
> + mutex_lock(&uq_mgr->userq_mutex);
[Severity: High]
Does taking adev->reset_domain->sem before uq_mgr->userq_mutex invert the
lock hierarchy and risk an ABBA deadlock?
amdgpu_lockdep_init() and the rest of amdgpu_userq.c establish
uq_mgr->userq_mutex outer to adev->reset_domain->sem:
CPU0 CPU1
---- ----
lock(&uq_mgr->userq_mutex)
down_read(&adev->reset_domain->sem)
down_read(&adev->reset_domain->sem)
lock(&uq_mgr->userq_mutex)
*** DEADLOCK ***
CPU0: amdgpu_userq_destroy() -> amdgpu_userq_unmap_helper()
CPU1: amdgpu_userq_create_gang()
(when a GPU reset writer is queued on adev->reset_domain->sem)
Also, because guard(rwsem_read)(&adev->reset_domain->sem) has function
scope, it is still held at the out label when amdgpu_userq_put() runs.
If a concurrent AMDGPU_USERQ_OP_FREE dropped the queue's base reference,
amdgpu_userq_put() enters amdgpu_userq_destroy() while holding
reset_domain->sem, waiting on cancel_delayed_work_sync() under the reset
semaphore and taking reset_domain->sem recursively in
amdgpu_userq_detach_doorbell() and amdgpu_userq_unmap_helper().
[ ... ]
> + if (primary_queue->state == AMDGPU_USERQ_STATE_HUNG) {
> + drm_file_err(uq_mgr->file, "Primary queue is hung\n");
> + r = -EINVAL;
> + goto out;
> + }
> +
> + if ((secondary_queue->queue_type != AMDGPU_HW_IP_GFX) &&
> + (secondary_queue->queue_type != AMDGPU_HW_IP_COMPUTE) &&
> + (secondary_queue->queue_type != AMDGPU_HW_IP_DMA)) {
> + drm_file_err(uq_mgr->file, "Invalid gang secondary queue type\n");
> + r = -EINVAL;
> + goto out;
> + }
> +
> + if (secondary_queue->gang_primary || secondary_queue->gang_secondary) {
> + drm_file_err(uq_mgr->file, "Secondary queue already in a gang\n");
> + r = -EINVAL;
> + goto out;
> + }
> +
> + if (secondary_queue->state == AMDGPU_USERQ_STATE_HUNG) {
> + drm_file_err(uq_mgr->file, "Secondary queue is hung\n");
> + r = -EINVAL;
> + goto out;
> + }
> +
> + userq_funcs = adev->userq_funcs[primary_queue->queue_type];
> + if ((primary_queue->state == AMDGPU_USERQ_STATE_MAPPED ||
> + primary_queue->state == AMDGPU_USERQ_STATE_PREEMPTED) &&
> + (secondary_queue->state == AMDGPU_USERQ_STATE_MAPPED ||
> + secondary_queue->state == AMDGPU_USERQ_STATE_PREEMPTED))
> + r = userq_funcs->set_gang(primary_queue, secondary_queue);
> + else
> + /* if the queues are currently unmapped, the gang state will get
> + * updated when they get mapped.
> + */
> + r = 0;
[Severity: Medium]
Should amdgpu_userq_create_gang() also reject queues in
AMDGPU_USERQ_STATE_INVALID_VA?
If a queue was marked AMDGPU_USERQ_STATE_INVALID_VA in
amdgpu_userq_restore_all(), amdgpu_userq_map_helper() will never remap it
(it only maps queues in AMDGPU_USERQ_STATE_UNMAPPED). Here it passes the
HUNG checks and falls into the else branch with r = 0, linking a dead
queue into the partner queue's gang pointer without ever programming the
gang in MES.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261118223712.2518580-1-alexander.deucher@amd.com?part=10
next prev parent reply other threads:[~2026-11-18 22:58 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-18 22:37 [PATCH 01/11] drm/amdgpu/userq: Add INFO IOCTL query for doorbell ranges Alex Deucher
2026-11-18 22:37 ` [PATCH 02/11] drm/amdgpu/mes11: add API for gang submit Alex Deucher
2026-11-18 22:37 ` [PATCH 03/11] drm/amdgpu/mes: add callback " Alex Deucher
2026-11-18 22:37 ` [PATCH 04/11] drm/amdgpu/mes11: add support for setting " Alex Deucher
2026-11-18 22:37 ` [PATCH 05/11] drm/amdgpu/mes12: " Alex Deucher
2026-11-18 22:37 ` [PATCH 06/11] drm/amdgpu/userq: add callback for gang setup Alex Deucher
2026-11-18 22:37 ` [PATCH 07/11] drm/amdgpu/userq/mes: implement set gang submit Alex Deucher
2026-11-18 22:44 ` sashiko-bot
2026-11-18 22:37 ` [PATCH 08/11] drm/amdgpu/userq: add handling for set_gang Alex Deucher
2026-11-18 22:54 ` sashiko-bot
2026-11-18 22:37 ` [PATCH 09/11] drm/amdgpu: add UAPI to create user queue gangs Alex Deucher
2026-11-18 22:37 ` [PATCH 10/11] drm/amdgpu/userq: enable support for creating gangs Alex Deucher
2026-11-18 22:58 ` sashiko-bot [this message]
2026-11-18 22:37 ` [PATCH 11/11] drm/amdgpu: bump driver version for userq updates Alex Deucher
2026-11-18 23:04 ` [PATCH 01/11] drm/amdgpu/userq: Add INFO IOCTL query for doorbell ranges sashiko-bot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=sashiko-outbox-222136@kernel.org \
--to=sashiko-bot@kernel.org \
--cc=alexander.deucher@amd.com \
--cc=amd-gfx@lists.freedesktop.org \
--cc=sashiko-reviews@lists.linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox