AMD-GFX Archive on lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v2] drm/amdkfd: Move gpu_recovery location
@ 2026-11-16  7:50 Amber Lin
  2026-11-16  8:13 ` sashiko-bot
  0 siblings, 1 reply; 2+ messages in thread
From: Amber Lin @ 2026-11-16  7:50 UTC (permalink / raw)
  To: amd-gfx, alexander.deucher; +Cc: jesse.zhang, Amber Lin

amdgpu_gfx_reset_mes_compute combines two features into one:
1) hung queue/pipe reset - suspend all/queue reset/resume all
2) bad queue removal - suspend_all/remove_queue/resume_all
GPU recovery aims on 1)resetting hung queues/pipes. When gpu_recovery is
disabled, we need to keep 2) going. Therefore move the gpu_recovery
check from KFD's reset_queues_mes into amdgpu_gfx_reset_mes_compute
where queue reset is done.

Signed-off-by: Amber Lin <amber.lin@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c               | 10 +++++++---
 drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c |  9 +++------
 2 files changed, 10 insertions(+), 9 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
index c5a2dcff5058..c30fecc653b4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
@@ -2594,14 +2594,17 @@ int amdgpu_gfx_reset_mes_compute(struct amdgpu_device *adev,
 	 * reset detect will return the array of bad queue doorbells
 	 */
 	r = amdgpu_mes_suspend(adev, xcc_id);
+	dev_dbg(adev->dev, "Suspend all returns %d. xcc_id %d\n", r, xcc_id);
+
+	if (!amdgpu_gpu_recovery)
+		goto after_reset;
+
 	/* if suspend all success, it should no hang queue */
-	if (!r) {
-		dev_dbg(adev->dev, "Suspend all succeeds. xcc_id %d\n", xcc_id);
+	if (!r)
 		/* always reset the KCQ/userq since we need to signal the fence
 		 * and we could be stuck in a loop which is preemptable.
 		 */
 		goto fence_reset;
-	}
 	r = amdgpu_mes_detect_and_reset_hung_queues(adev, AMDGPU_RING_TYPE_COMPUTE,
 						    true, &num_hung, adev->gfx.mec.mes_hung_db_array, xcc_id);
 	if (r) {
@@ -2662,6 +2665,7 @@ int amdgpu_gfx_reset_mes_compute(struct amdgpu_device *adev,
 			goto out;
 	}
 
+after_reset:
 	/* MES doesn't detect any hung queue but we have a known bad queue
 	 * and it is not KCQ
 	 */
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index 86b70748617b..b77cfd8ba907 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -471,11 +471,6 @@ static int reset_queues_mes(struct device_queue_manager *dqm, struct queue *q)
 	if (WARN_ON(dqm->detect_hang_count > 0))
 		return 0;
 
-	if (!amdgpu_gpu_recovery) {
-		r = -ENOTRECOVERABLE;
-		goto fail;
-	}
-
 	memset(&queue_input, 0x0, sizeof(struct mes_remove_queue_input));
 	queue_input.doorbell_offset = q->properties.doorbell_off;
 	queue_input.gang_context_addr = q->gang_ctx_gpu_addr;
@@ -3609,7 +3604,9 @@ int kfd_dqm_suspend_bad_queue_mes(struct kfd_node *knode, u32 pasid, u32 doorbel
 
 		list_for_each_entry(q, &qpd->queues_list, list) {
 			if (q->doorbell_id == doorbell_id && q->properties.is_active) {
-				reset_queues_mes(dqm, q);
+				if (reset_queues_mes(dqm, q))
+					dev_dbg(knode->adev->dev,
+					"Fail to remove bad queue. doorbell_id 0x%x\n", q->doorbell_id);
 				q->properties.is_evicted = true;
 				q->properties.is_active = false;
 				decrement_queue_count(dqm, qpd, q);
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 2+ messages in thread

* Re: [PATCH v2] drm/amdkfd: Move gpu_recovery location
  2026-11-16  7:50 [PATCH v2] drm/amdkfd: Move gpu_recovery location Amber Lin
@ 2026-11-16  8:13 ` sashiko-bot
  0 siblings, 0 replies; 2+ messages in thread
From: sashiko-bot @ 2026-11-16  8:13 UTC (permalink / raw)
  To: Amber Lin, amd-gfx

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [High] drm/amdgpu: broken !amdgpu_gpu_recovery flow in
  amdgpu_gfx_reset_mes_compute()
--

commit 8a38bbe4db1b13fc0dffaa46e16598a2c6c11936
Author: Amber Lin <amber.lin@amd.com>

drm/amdkfd: Move gpu_recovery location

This commit moves the amdgpu_gpu_recovery check from reset_queues_mes()
into amdgpu_gfx_reset_mes_compute() so bad queue removal still runs when
GPU recovery is disabled.

> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
> index c5a2dcff5058a..c30fecc653b4b 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
> @@ -2594,14 +2594,17 @@ int amdgpu_gfx_reset_mes_compute(struct amdgpu_device *adev,
>  	 * reset detect will return the array of bad queue doorbells
>  	 */
>  	r = amdgpu_mes_suspend(adev, xcc_id);
> +	dev_dbg(adev->dev, "Suspend all returns %d. xcc_id %d\n", r, xcc_id);
> +
> +	if (!amdgpu_gpu_recovery)
> +		goto after_reset;
> +
>  	/* if suspend all success, it should no hang queue */
> -	if (!r) {
> -		dev_dbg(adev->dev, "Suspend all succeeds. xcc_id %d\n", xcc_id);
> +	if (!r)
>  		/* always reset the KCQ/userq since we need to signal the fence
>  		 * and we could be stuck in a loop which is preemptable.
>  		 */
>  		goto fence_reset;
> -	}
>  	r = amdgpu_mes_detect_and_reset_hung_queues(adev, AMDGPU_RING_TYPE_COMPUTE,
>  						    true, &num_hung, adev->gfx.mec.mes_hung_db_array, xcc_id);
>  	if (r) {
> @@ -2662,6 +2665,7 @@ int amdgpu_gfx_reset_mes_compute(struct amdgpu_device *adev,
>  			goto out;
>  	}
>  
> +after_reset:
>  	/* MES doesn't detect any hung queue but we have a known bad queue
>  	 * and it is not KCQ
>  	 */

[Severity: High]
Can jumping to after_reset when !amdgpu_gpu_recovery leave all kernel
compute ring DRM schedulers permanently stopped when a queue hangs?

Before calling amdgpu_mes_suspend(), amdgpu_gfx_reset_mes_compute() stops
the compute ring schedulers via amdgpu_gfx_reset_stop_compute_scheds().
When a queue is hung, amdgpu_mes_suspend() fails with a non-zero r, but
if !amdgpu_gpu_recovery is set, this jumps straight to after_reset with
num_hung == 0:

drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c:amdgpu_gfx_reset_mes_compute() {
	...
after_reset:
	if (!num_hung && queue_input && !ring) {
		...
		r = adev->mes.funcs->remove_hw_queue(&adev->mes, queue_input);
		...
	}

out:
	amdgpu_mes_resume(adev, xcc_id);
	ring_err = r;
	...
	if (!ring_err)
		amdgpu_gfx_reset_start_compute_scheds(adev, ring);
	...
}

Because suspend_all failed, the hung queue was not preempted and
remove_hw_queue() fails as well (or if another queue on the XCC caused
suspend_all to fail, remove_hw_queue() can succeed and overwrite the
non-zero suspend error r with 0).  When r is non-zero, ring_err is
non-zero and amdgpu_gfx_reset_start_compute_scheds() is skipped; since no
full GPU reset follows when !amdgpu_gpu_recovery (or when called from
kfd_dqm_suspend_bad_queue_mes()), the compute schedulers are never
restarted.

Also, if amdgpu_gfx_reset_mes_compute() is called for a KCQ or KGD user
queue (where queue_input is NULL) and amdgpu_mes_suspend() succeeds,
jumping to after_reset skips fence_reset and returns 0 without resetting
the ring or user queue.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261116075047.35150-1-amber.lin@amd.com?part=1

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2026-11-16  8:14 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-16  7:50 [PATCH v2] drm/amdkfd: Move gpu_recovery location Amber Lin
2026-11-16  8:13 ` sashiko-bot

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox