[PATCH 1/2] drm/amdgpu: Don't ignore LSDMA copy errors during remote interrupt send

[PATCH 1/2] drm/amdgpu: Don't ignore LSDMA copy errors during remote interrupt send

[Mukul Joshi]

On the LSDMA path amdgpu_ualink_send_command() ignored the return of
amdgpu_lsdma_copy_mem() and polled the remote completion for the full
timeout even when the copy failed. With a peer down, every NPA-REVOKE
send in amdgpu_ualink_exp_cleanup_worker() wasted ~2s, and the peer
stayed ESTABLISHED so each subsequent handle cleanup timed out again,
piling up past the 120s hung-task threshold.

Return immediately when an LSDMA copy fails, and mark the connection
NOT_READY on a failed revoke send so other cleanups skip the unreachable
peer via amdgpu_ualink_check_conn_ready().

Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c | 29 +++++++++++++++++++---
 1 file changed, 26 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c
index f5e1e5217fa5..f5865c5fe93e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c
@@ -2903,7 +2903,8 @@ static void amdgpu_ualink_exp_cleanup_worker(struct work_struct *work)
 	}
 
 	/* Send NPA-REVOKE to all importers which have imported this memory.
-	 * On send failure clear the bit (no response will arrive).
+	 * On send failure clear the bit (no response will arrive) and mark the
+	 * connection NOT_READY so other cleanups skip this unreachable peer.
 	 */
 	for_each_set_bit(remote_acc_id, exp_xa_node->importers_bitmap,
 				 AMDGPU_UALINK_ACCEL_MAX) {
@@ -2916,6 +2917,11 @@ static void amdgpu_ualink_exp_cleanup_worker(struct work_struct *work)
 				"EXP-CLEANUP: NPA-REVOKE send failed to remote:%u\n",
 				remote_acc_id);
 			clear_bit(remote_acc_id, exp_xa_node->npa_release_bitmap);
+
+			imp_entry = &exp_xa_node->importer_entries[remote_acc_id];
+			amdgpu_ualink_handle_connection_reset(adev, remote_acc_id,
+						AMDGPU_UALINK_CONN_NOT_READY,
+						imp_entry->generation_count);
 		}
 	}
 
@@ -5623,10 +5629,27 @@ static int amdgpu_ualink_send_command(struct amdgpu_device *adev,
 		r = amdgpu_lsdma_copy_mem(adev, src,
 					  ring->rb_npa_gart + (ring->wptr % ring->rb_size) * 64,
 					  64);
-		r = amdgpu_lsdma_copy_mem(adev, src + 64, ring->wptr_npa_gart, 8);
+		if (!r)
+			r = amdgpu_lsdma_copy_mem(adev, src + 64,
+						  ring->wptr_npa_gart, 8);
+		if (!r)
+			r = amdgpu_lsdma_copy_mem(adev, src + 72,
+						  doorbell_npa_gart, 4);
 
-		r = amdgpu_lsdma_copy_mem(adev, src + 72, doorbell_npa_gart, 4);
 		amdgpu_job_free(job);
+
+		/* Copy failed: command never reached the remote, so no
+		 * writeback will arrive. Return now instead of polling the
+		 * completion for the full remote timeout.
+		 */
+		if (r) {
+			dev_dbg(adev->dev,
+				"remote %u lsdma copy failed (r %d), skip completion wait\n",
+				remote_accel_id, r);
+			mutex_unlock(&peer->lock);
+			return r;
+		}
+
 		goto out_wait_complete;
         }
 
-- 
2.54.0

[PATCH 2/2] drm/amdgpu: Reset UALink connection on failed TLB-shootdown send

[Mukul Joshi]

If sending TLB shootdown to a remote GPU fails, reset the connection and
mark the connection as NOT_READY.

Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c
index f5865c5fe93e..5b3c7f62c34d 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c
@@ -2681,10 +2681,19 @@ static void amdgpu_ualink_send_tlb_shootdown(struct amdgpu_device *adev,
 		r = amdgpu_ualink_remote_shootdown(adev, remote_acc_id,
 					npa_addr, size,
 					AMDGPU_UALINK_HEAVYWEIGHT_TLB_SHOOTDOWN);
-		if (r)
+		if (r) {
 			dev_err(adev->dev,
 				"EXP-CLEANUP: TLB shootdown send failed to remote:%u\n",
 				remote_acc_id);
+
+			/* Send failed: mark the connection NOT_READY so the
+			 * rest of this cleanup (and other cleanups) skip the
+			 * unreachable peer via amdgpu_ualink_check_conn_ready().
+			 */
+			amdgpu_ualink_handle_connection_reset(adev, remote_acc_id,
+						AMDGPU_UALINK_CONN_NOT_READY,
+						imp_entry->generation_count);
+		}
 	}
 }
 
-- 
2.54.0

Re: [PATCH 2/2] drm/amdgpu: Reset UALink connection on failed TLB-shootdown send

[Kasiviswanathan, Harish]

[-- Attachment #1: Type: text/plain, Size: 2152 bytes --]

AMD General

This series Reviewed-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>

________________________________
From: Joshi, Mukul <Mukul.Joshi@amd.com>
Sent: Thursday, September 3, 2026 4:48 PM
To: amd-gfx@lists.freedesktop.org <amd-gfx@lists.freedesktop.org>
Cc: Kasiviswanathan, Harish <Harish.Kasiviswanathan@amd.com>; Kuehling, Felix <Felix.Kuehling@amd.com>; Joshi, Mukul <Mukul.Joshi@amd.com>
Subject: [PATCH 2/2] drm/amdgpu: Reset UALink connection on failed TLB-shootdown send

If sending TLB shootdown to a remote GPU fails, reset the connection and
mark the connection as NOT_READY.

Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c
index f5865c5fe93e..5b3c7f62c34d 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c
@@ -2681,10 +2681,19 @@ static void amdgpu_ualink_send_tlb_shootdown(struct amdgpu_device *adev,
                 r = amdgpu_ualink_remote_shootdown(adev, remote_acc_id,
                                         npa_addr, size,
                                         AMDGPU_UALINK_HEAVYWEIGHT_TLB_SHOOTDOWN);
-               if (r)
+               if (r) {
                         dev_err(adev->dev,
                                 "EXP-CLEANUP: TLB shootdown send failed to remote:%u\n",
                                 remote_acc_id);
+
+                       /* Send failed: mark the connection NOT_READY so the
+                        * rest of this cleanup (and other cleanups) skip the
+                        * unreachable peer via amdgpu_ualink_check_conn_ready().
+                        */
+                       amdgpu_ualink_handle_connection_reset(adev, remote_acc_id,
+                                               AMDGPU_UALINK_CONN_NOT_READY,
+                                               imp_entry->generation_count);
+               }
         }
 }

--
2.54.0


[-- Attachment #2: Type: text/html, Size: 5659 bytes --]