From: sashiko-bot@kernel.org
To: "Alex Deucher" <alexander.deucher@amd.com>,
amd-gfx@lists.freedesktop.org
Subject: Re: [PATCH 08/11] drm/amdgpu/userq: add handling for set_gang
Date: Wed, 18 Nov 2026 22:54:06 +0000 [thread overview]
Message-ID: <sashiko-outbox-222131@kernel.org> (raw)
In-Reply-To: <20261118223712.2518580-8-alexander.deucher@amd.com>
Thank you for your contribution! Sashiko AI review found 2 potential
issues to consider:
- [Medium] drm/amdgpu/userq: missing reset_domain lock in
amdgpu_userq_set_gang_helper()
- [Medium] drm/amdgpu/userq: invalid set_gang call in
amdgpu_userq_post_reset()
--
commit c119a0781eef4bb00ce5e6c3043c15d765972cac
Author: Alex Deucher <alexander.deucher@amd.com>
drm/amdgpu/userq: add handling for set_gang
This commit adds gang primary/secondary pointers to struct
amdgpu_usermode_queue and introduces amdgpu_userq_set_gang_helper() to
re-establish queue gangs during user queue restore, resume, and post-reset
recovery.
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
> index ed6f3b67e8720..3afd17af28548 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
> @@ -303,6 +303,27 @@ static bool amdgpu_userq_buffer_vas_mapped(struct amdgpu_usermode_queue *queue)
> return true;
> }
>
> +static int amdgpu_userq_set_gang_helper(struct amdgpu_usermode_queue *queue)
> +{
> + struct amdgpu_usermode_queue *secondary_queue = queue->gang_secondary;
> + struct amdgpu_userq_mgr *uq_mgr = queue->userq_mgr;
> + struct amdgpu_device *adev = uq_mgr->adev;
> + const struct amdgpu_userq_funcs *userq_funcs =
> + adev->userq_funcs[queue->queue_type];
> + int r;
> +
> + if (secondary_queue &&
> + (queue->state == AMDGPU_USERQ_STATE_MAPPED ||
> + queue->state == AMDGPU_USERQ_STATE_PREEMPTED) &&
> + (secondary_queue->state == AMDGPU_USERQ_STATE_MAPPED ||
> + secondary_queue->state == AMDGPU_USERQ_STATE_PREEMPTED)) {
> + r = userq_funcs->set_gang(queue, secondary_queue);
> + if (r)
> + return r;
> + }
> + return 0;
> +}
> +
> static int amdgpu_userq_preempt_helper(struct amdgpu_usermode_queue *queue)
> {
> struct amdgpu_userq_mgr *uq_mgr = queue->userq_mgr;
> @@ -968,6 +989,11 @@ amdgpu_userq_restore_all(struct amdgpu_userq_mgr *uq_mgr)
> if (r)
> ret = r;
> }
> + xa_for_each(&uq_mgr->userq_xa, queue_id, queue) {
> + r = amdgpu_userq_set_gang_helper(queue);
> + if (r)
> + ret = r;
> + }
> mutex_unlock(&uq_mgr->userq_mutex);
>
> if (ret)
> @@ -1446,6 +1472,13 @@ int amdgpu_userq_resume(struct amdgpu_device *adev)
> if (r)
> return r;
> }
> + xa_for_each(&adev->userq_doorbell_xa, queue_id, queue) {
> + uqm = queue->userq_mgr;
> + guard(mutex)(&uqm->userq_mutex);
> + r = amdgpu_userq_set_gang_helper(queue);
> + if (r)
> + return r;
> + }
>
> return 0;
> }
[Severity: Medium]
Should callers on the non-reset paths hold a read lock on
adev->reset_domain->sem across amdgpu_userq_set_gang_helper()?
In amdgpu_userq_restore_all() and amdgpu_userq_resume(), the first loop calls
amdgpu_userq_map_helper(), which takes and releases adev->reset_domain->sem
per queue:
drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c:amdgpu_userq_map_helper() {
...
/* The reset changes the queue state and the MES is unusable meanwhile */
guard(rwsem_read)(&adev->reset_domain->sem);
...
}
When the second loop calls amdgpu_userq_set_gang_helper(),
adev->reset_domain->sem is no longer held. If a GPU reset starts between the
two loops, amdgpu_userq_pre_reset() and amdgpu_userq_post_reset() mutate
queue->state, secondary_queue->state, and gang_ctx_array_index under the
write side of adev->reset_domain->sem without taking uq_mgr->userq_mutex:
kworker (uq_mgr->resume_work) GPU reset thread
amdgpu_userq_restore_all()
mutex_lock(&uq_mgr->userq_mutex)
amdgpu_userq_map_helper(queue)
down_read(&reset_domain->sem)
...
up_read(&reset_domain->sem)
down_write(&reset_domain->sem)
amdgpu_userq_pre_reset()
userq_funcs->unmap(queue)
queue->state = AMDGPU_USERQ_STATE_HUNG
amdgpu_userq_set_gang_helper()
userq_funcs->set_gang() <-- submits to MES during reset
Calling userq_funcs->set_gang() without adev->reset_domain->sem held also
triggers lockdep_assert_held(&adev->reset_domain->sem) in
amdgpu_device_skip_hw_access() when ringing the MES doorbell. Note that
amdgpu_userq_post_reset() also calls amdgpu_userq_set_gang_helper() while
already holding adev->reset_domain->sem for write, so taking the read lock
inside amdgpu_userq_set_gang_helper() itself would deadlock the reset path.
> @@ -1591,6 +1624,13 @@ int amdgpu_userq_post_reset(struct amdgpu_device *adev, bool vram_lost)
> queue->state = AMDGPU_USERQ_STATE_MAPPED;
> }
> }
> + xa_for_each(&adev->userq_doorbell_xa, queue_id, queue) {
> + r = amdgpu_userq_set_gang_helper(queue);
> + if (r) {
> + dev_err(adev->dev, "Failed to gang queue %ld\n", queue_id);
> + ret = r;
> + }
> + }
>
> return ret;
> }
[Severity: Medium]
Can amdgpu_userq_post_reset() call userq_funcs->set_gang() on queues that were
not re-added to MES after the reset?
During a GPU reset, amdgpu_userq_pre_reset() only transitions
AMDGPU_USERQ_STATE_MAPPED queues to AMDGPU_USERQ_STATE_HUNG, leaving any
preempted queues in AMDGPU_USERQ_STATE_PREEMPTED while MES hardware state is
wiped. In amdgpu_userq_post_reset(), the first loop only remaps queues whose
state is AMDGPU_USERQ_STATE_HUNG when !vram_lost.
Because the second loop runs unconditionally (even when vram_lost is true)
and amdgpu_userq_set_gang_helper() accepts queues in
AMDGPU_USERQ_STATE_PREEMPTED, it will issue MES SET_GANG_SUBMIT commands for
preempted queues that have not been mapped in MES after the reset, causing
MES command failures and failing amdgpu_userq_post_reset().
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261118223712.2518580-1-alexander.deucher@amd.com?part=8
next prev parent reply other threads:[~2026-11-18 22:54 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-18 22:37 [PATCH 01/11] drm/amdgpu/userq: Add INFO IOCTL query for doorbell ranges Alex Deucher
2026-11-18 22:37 ` [PATCH 02/11] drm/amdgpu/mes11: add API for gang submit Alex Deucher
2026-11-18 22:37 ` [PATCH 03/11] drm/amdgpu/mes: add callback " Alex Deucher
2026-11-18 22:37 ` [PATCH 04/11] drm/amdgpu/mes11: add support for setting " Alex Deucher
2026-11-18 22:37 ` [PATCH 05/11] drm/amdgpu/mes12: " Alex Deucher
2026-11-18 22:37 ` [PATCH 06/11] drm/amdgpu/userq: add callback for gang setup Alex Deucher
2026-11-18 22:37 ` [PATCH 07/11] drm/amdgpu/userq/mes: implement set gang submit Alex Deucher
2026-11-18 22:44 ` sashiko-bot
2026-11-18 22:37 ` [PATCH 08/11] drm/amdgpu/userq: add handling for set_gang Alex Deucher
2026-11-18 22:54 ` sashiko-bot [this message]
2026-11-18 22:37 ` [PATCH 09/11] drm/amdgpu: add UAPI to create user queue gangs Alex Deucher
2026-11-18 22:37 ` [PATCH 10/11] drm/amdgpu/userq: enable support for creating gangs Alex Deucher
2026-11-18 22:58 ` sashiko-bot
2026-11-18 22:37 ` [PATCH 11/11] drm/amdgpu: bump driver version for userq updates Alex Deucher
2026-11-18 23:04 ` [PATCH 01/11] drm/amdgpu/userq: Add INFO IOCTL query for doorbell ranges sashiko-bot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=sashiko-outbox-222131@kernel.org \
--to=sashiko-bot@kernel.org \
--cc=alexander.deucher@amd.com \
--cc=amd-gfx@lists.freedesktop.org \
--cc=sashiko-reviews@lists.linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox