[PATCH v4 0/4] drm/amdgpu: Wire GPU events into EVENTFD notifications

[PATCH v4 0/4] drm/amdgpu: Wire GPU events into EVENTFD notifications

[Srinivasan Shanmugam]

This series wires USERQ_EOP, QUEUE_RESET, and GPU_RESET events into the
render-node EVENTFD notification infrastructure.

It builds on the render-node EVENTFD infrastructure introduced in:

  https://patchwork.freedesktop.org/series/164618/#rev7

That series provides the EVENTFD manager, subscription lifetime
management, event type definitions, and queue-aware subscription
support. This series adds the corresponding notification producers for
GPU events.

Base branch:
  origin/amd-unified-interface

Base commit:
  540d68973c0b
  ("drm/amdgpu/userq: fix NULL deref in amdgpu_userq_evict_all")

EVENTFD provides lightweight notification-only wakeups. Event payload
and metadata are retrieved separately through the corresponding event
interface.

The patches in this series:

  1. Signal USERQ_EOP notifications from the user queue fence interrupt
     path.

  2. Properly account for a successfully reset guilty queue by
     incrementing the reset counter, force-completing its fences, and
     reporting the DRM wedged event.

  3. Signal QUEUE_RESET notifications from the common successful queue
     reset handling path.

  4. Signal GPU_RESET notifications after successful hardware reset
     completion.

Changes since v3:

* Added Alex's patch:

    drm/amdgpu/userq: properly account for resets

  This handles reset accounting, fence completion, and the DRM wedged
  event for the original guilty queue after a successful queue reset.

* Reworked the QUEUE_RESET producer to introduce
  amdgpu_userq_handle_hung_queue() as the common successful queue reset
  handling helper.

* Use amdgpu_userq_handle_hung_queue() for both the original guilty
  queue and collateral queues reported by MES, ensuring both paths
  update queue state, signal QUEUE_RESET EVENTFD notifications,
  force-complete fences, and report the DRM wedged event consistently.

* Continue to resolve collateral queues directly through the doorbell
  xarray rather than scanning all user queues.

* Preserve separate reset-counter accounting in the guilty and
  collateral queue paths.

* Rebased onto the latest EVENTFD infrastructure.

The USERQ_EOP and GPU_RESET notification patches are unchanged from v3
apart from rebasing and retain Alex's Reviewed-by tags.

Only compilation tested.

Alex Deucher (1):
  drm/amdgpu/userq: properly account for resets

Srinivasan Shanmugam (3):
  drm/amdgpu: Signal USERQ_EOP EVENTFD notifications
  drm/amdgpu: Signal QUEUE_RESET EVENTFD notifications
  drm/amdgpu: Signal GPU_RESET EVENTFD notifications

 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 51 ++++++++++++++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c  | 38 +++++++++++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_userq.h  |  2 +
 drivers/gpu/drm/amd/amdgpu/mes_userqueue.c | 72 ++++++++++++++--------
 4 files changed, 134 insertions(+), 29 deletions(-)

-- 
2.34.1

[PATCH v4 1/4] drm/amdgpu: Signal USERQ_EOP EVENTFD notifications

[Srinivasan Shanmugam]

Signal USERQ_EOP EVENTFD subscriptions from the USERQ interrupt path.

The EOP interrupt already identifies the queue that completed. Use the
queue object directly to notify matching EVENTFD subscribers.

Routing notifications through the queue object keeps delivery tied to
the queue instance that generated the completion event, avoiding
ambiguities from reused queue identifiers or doorbell indices.

EVENTFD remains notification-only and carries no event payload.

v2: (per Christian)
- Move USERQ_EOP EVENTFD signaling into amdgpu_userq_process_fence_irq().
- Reuse the existing doorbell-to-queue lookup instead of duplicating it
  in the interrupt handler.
- Keep fence processing and EVENTFD notification handling together in a
  single helper.

Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Signed-off-by: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
index fbf783946f6d..ab3ef3a9f655 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
@@ -215,6 +215,7 @@ void amdgpu_userq_process_fence_irq(struct amdgpu_device *adev, u32 doorbell)
 {
 	struct xarray *xa = &adev->userq_doorbell_xa;
 	struct amdgpu_usermode_queue *queue;
+	struct amdgpu_eventfd_mgr *eventfd_mgr;
 	unsigned long flags;
 	int r;
 
@@ -232,6 +233,11 @@ void amdgpu_userq_process_fence_irq(struct amdgpu_device *adev, u32 doorbell)
 		/* Restart the timer when there are still fences pending */
 		if (r == 1)
 			amdgpu_userq_start_hang_detect_work(queue);
+
+		eventfd_mgr = amdgpu_userq_eventfd_mgr(queue->userq_mgr);
+		amdgpu_eventfd_signal(eventfd_mgr,
+				      DRM_AMDGPU_EVENT_TYPE_USERQ_EOP,
+				      queue);
 	}
 	xa_unlock_irqrestore(xa, flags);
 }
-- 
2.34.1

[PATCH v4 2/4] drm/amdgpu/userq: properly account for resets

[Srinivasan Shanmugam]

From: Alex Deucher <alexander.deucher@amd.com>

We need to increment the reset counter, force fence completion,
and set the wedged event when a user queue is reset.

mes_userq_reset_queue() handles this for collateral damage, but the
caller needs to handle this directly for the original guilty queue.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
Cc: Prike Liang <Prike.Liang@amd.com>
Cc: Sunil Khatri <sunil.khatri@amd.com>
Reviewed-by: Jesse Zhang <Jesse.Zhang@amd.com>
Reviewed-by: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
index ab3ef3a9f655..fb0d335875af 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
@@ -167,8 +167,13 @@ static void amdgpu_userq_hang_detect_work(struct work_struct *work)
 							 queue, NULL, NULL);
 		else
 			r = userq_funcs->reset(queue);
-		if (r)
+		if (r) {
 			gpu_reset = true;
+		} else {
+			atomic_inc(&adev->gpu_reset_counter);
+			amdgpu_userq_fence_driver_force_completion(queue);
+			drm_dev_wedged_event(adev_to_drm(adev), DRM_WEDGE_RECOVERY_NONE, NULL);
+		}
 	} else {
 		gpu_reset = true;
 	}
-- 
2.34.1

[PATCH v4 3/4] drm/amdgpu: Signal QUEUE_RESET EVENTFD notifications

[Srinivasan Shanmugam]

Signal QUEUE_RESET EVENTFD subscriptions when hung user queues are
detected.

MES reports the doorbell indices of hung queues. Use the existing
doorbell-to-queue mapping to resolve the affected queue without scanning
all user queues.

Consolidate the queue state update, reset accounting, EVENTFD signaling,
fence completion, and wedged event notification in a common helper. Use
the helper for both the original guilty queue and additional affected
queues.

EVENTFD remains notification-only.

v2: (per Christian)
 - Use the doorbell xarray to look up affected queues instead of scanning
   all queues.
 - Move queue reset handling into amdgpu_userq.c.
 - Consolidate queue state updates, EVENTFD signaling, and fence
   completion in a single helper.

v3: Rebase only.

v4: (per Alex)
 - Rebase on Alex's "drm/amdgpu/userq: properly account for resets".
 - Preserve the reset counter update in the common hung queue helper.
 - Use amdgpu_userq_handle_hung_queue() for both successful queue reset
   paths.

Suggested-by: Christian König <christian.koenig@amd.com>
Suggested-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c  | 39 +++++++++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_userq.h  |  2 +
 drivers/gpu/drm/amd/amdgpu/mes_userqueue.c | 72 ++++++++++++++--------
 3 files changed, 80 insertions(+), 33 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
index fb0d335875af..65b20d7a4b18 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
@@ -135,6 +135,34 @@ static void amdgpu_userq_mgr_reset_work(struct work_struct *work)
 	amdgpu_device_gpu_recover(adev, NULL, &reset_context);
 }
 
+/**
+ * amdgpu_userq_handle_hung_queue - handle a successfully reset hung queue
+ * @adev: AMDGPU device
+ * @queue: affected user queue
+ *
+ * Mark the queue as hung, account for the reset, force-complete its
+ * fences, notify matching QUEUE_RESET EVENTFD subscribers, and send
+ * the DRM wedged event.
+ */
+void amdgpu_userq_handle_hung_queue(struct amdgpu_device *adev,
+				    struct amdgpu_usermode_queue *queue)
+{
+	struct amdgpu_eventfd_mgr *eventfd_mgr;
+
+	queue->state = AMDGPU_USERQ_STATE_HUNG;
+	atomic_inc(&adev->gpu_reset_counter);
+
+	amdgpu_userq_fence_driver_force_completion(queue);
+
+	eventfd_mgr = amdgpu_userq_eventfd_mgr(queue->userq_mgr);
+	amdgpu_eventfd_signal(eventfd_mgr,
+			      DRM_AMDGPU_EVENT_TYPE_QUEUE_RESET,
+			      queue);
+
+	drm_dev_wedged_event(adev_to_drm(adev),
+			     DRM_WEDGE_RECOVERY_NONE, NULL);
+}
+
 static void amdgpu_userq_hang_detect_work(struct work_struct *work)
 {
 	struct amdgpu_usermode_queue *queue =
@@ -167,13 +195,10 @@ static void amdgpu_userq_hang_detect_work(struct work_struct *work)
 							 queue, NULL, NULL);
 		else
 			r = userq_funcs->reset(queue);
-		if (r) {
+		if (r)
 			gpu_reset = true;
-		} else {
-			atomic_inc(&adev->gpu_reset_counter);
-			amdgpu_userq_fence_driver_force_completion(queue);
-			drm_dev_wedged_event(adev_to_drm(adev), DRM_WEDGE_RECOVERY_NONE, NULL);
-		}
+		else
+			amdgpu_userq_handle_hung_queue(adev, queue);
 	} else {
 		gpu_reset = true;
 	}
@@ -247,8 +272,6 @@ void amdgpu_userq_process_fence_irq(struct amdgpu_device *adev, u32 doorbell)
 	xa_unlock_irqrestore(xa, flags);
 }
 
-
-
 int amdgpu_userq_input_va_validate(struct amdgpu_device *adev,
 				   struct amdgpu_usermode_queue *queue,
 				   u64 addr, u64 expected_size,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.h
index b69621311b80..526e007e300e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.h
@@ -180,6 +180,8 @@ void amdgpu_userq_pre_reset(struct amdgpu_device *adev);
 int amdgpu_userq_post_reset(struct amdgpu_device *adev, bool vram_lost);
 void amdgpu_userq_start_hang_detect_work(struct amdgpu_usermode_queue *queue);
 void amdgpu_userq_process_fence_irq(struct amdgpu_device *adev, u32 doorbell);
+void amdgpu_userq_handle_hung_queue(struct amdgpu_device *adev,
+				    struct amdgpu_usermode_queue *queue);
 
 /*
  * CP packs the per-process doorbell_id of the queue in
diff --git a/drivers/gpu/drm/amd/amdgpu/mes_userqueue.c b/drivers/gpu/drm/amd/amdgpu/mes_userqueue.c
index 3fad95199e0c..27adcea7b9a0 100644
--- a/drivers/gpu/drm/amd/amdgpu/mes_userqueue.c
+++ b/drivers/gpu/drm/amd/amdgpu/mes_userqueue.c
@@ -26,6 +26,7 @@
 #include "amdgpu_gfx.h"
 #include "mes_userqueue.h"
 #include "amdgpu_userq_fence.h"
+#include "amdgpu_userq_internal.h"
 #include "amdgpu_cwsr.h"
 
 #define AMDGPU_USERQ_PROC_CTX_SZ PAGE_SIZE
@@ -238,34 +239,55 @@ int mes_userq_reset_queue(struct amdgpu_device *adev,
 			  unsigned int queue,
 			  unsigned int db)
 {
+	struct xarray *xa = &adev->userq_doorbell_xa;
 	struct amdgpu_usermode_queue *uq;
 	bool use_mmio = adev->gfx.mec.use_mmio_for_reset;
-	unsigned long uq_id;
-	int r;
+	unsigned long flags;
+	int r = 0;
 
-	xa_for_each(&adev->userq_doorbell_xa, uq_id, uq) {
-		if (uq->queue_type == queue_type) {
-			if (uq == guilty_uq)
-				continue;
-			if (uq->doorbell_index == db) {
-				uq->state = AMDGPU_USERQ_STATE_HUNG;
-				if (use_mmio)
-					r = amdgpu_mes_reset_queue_mmio(adev, queue_type, 0, 1, pipe, queue, 0);
-				else
-					r = amdgpu_mes_reset_user_queue(adev, queue_type, db, 0);
-				if (r)
-					return r;
-				r = mes_userq_unmap(uq);
-				if (r)
-					return r;
-				atomic_inc(&adev->gpu_reset_counter);
-				amdgpu_userq_fence_driver_force_completion(uq);
-				drm_dev_wedged_event(adev_to_drm(adev), DRM_WEDGE_RECOVERY_NONE, NULL);
-				break;
-			}
-		}
-	}
-	return 0;
+	/*
+	 * Resolve the doorbell directly to the affected queue instead of
+	 * scanning all user queues.
+	 */
+	xa_lock_irqsave(xa, flags);
+
+	uq = xa_load(xa, db);
+	if (uq)
+		kref_get(&uq->refcount);
+
+	xa_unlock_irqrestore(xa, flags);
+
+	if (!uq)
+		return 0;
+
+	/*
+	 * The guilty queue is handled separately by the caller.
+	 */
+	if (uq == guilty_uq)
+		goto out_put_queue;
+
+	if (uq->queue_type != queue_type)
+		goto out_put_queue;
+
+	if (use_mmio)
+		r = amdgpu_mes_reset_queue_mmio(adev, queue_type,
+						0, 1, pipe, queue, 0);
+	else
+		r = amdgpu_mes_reset_user_queue(adev, queue_type, db, 0);
+
+	if (r)
+		goto out_put_queue;
+
+	r = mes_userq_unmap(uq);
+	if (r)
+		goto out_put_queue;
+
+	amdgpu_userq_handle_hung_queue(adev, uq);
+
+out_put_queue:
+	amdgpu_userq_put(uq);
+
+	return r;
 }
 
 static int mes_userq_create_ctx_space(struct amdgpu_userq_mgr *uq_mgr,
-- 
2.34.1

[PATCH v4 4/4] drm/amdgpu: Signal GPU_RESET EVENTFD notifications

[Srinivasan Shanmugam]

Signal GPU_RESET EVENTFD subscriptions after successful GPU reset
completion.

GPU reset notifications allow userspace to observe that GPU recovery has
completed and the device is operational again.

GPU_RESET is a device-scoped event, so no queue object is used. All
processes that subscribed to GPU_RESET on the device are notified.

EVENTFD remains notification-only.

v2:
- Move GPU_RESET EVENTFD signaling to the path that confirms a real
  GPU hardware reset occurred.
- Notify only after successful hardware reset completion.
- Do not signal reset-skip or recovery-only paths.

Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Signed-off-by: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 51 +++++++++++++++++++++-
 1 file changed, 49 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index c924a55fd16f..15aaceae13d1 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -5390,15 +5390,43 @@ static void amdgpu_device_halt_activities(struct amdgpu_device *adev,
 	}
 }
 
+/*
+ * Signal GPU_RESET EVENTFD subscriptions for all open DRM files.
+ *
+ * GPU reset is a device-wide event rather than being associated with a
+ * specific user queue. Notify every process that subscribed to the
+ * GPU_RESET event on this device.
+ */
+static void amdgpu_device_eventfd_signal_gpu_reset(struct amdgpu_device *adev)
+{
+	struct drm_device *ddev = adev_to_drm(adev);
+	struct drm_file *file;
+
+	mutex_lock(&ddev->filelist_mutex);
+
+	list_for_each_entry(file, &ddev->filelist, lhead) {
+		struct amdgpu_fpriv *fpriv = file->driver_priv;
+
+		if (fpriv)
+			amdgpu_eventfd_signal(&fpriv->eventfd_mgr,
+					      DRM_AMDGPU_EVENT_TYPE_GPU_RESET,
+					      NULL);
+	}
+
+	mutex_unlock(&ddev->filelist_mutex);
+}
+
 static int amdgpu_device_asic_reset(struct amdgpu_device *adev,
 			      struct list_head *device_list,
 			      struct amdgpu_reset_context *reset_context)
 {
 	struct amdgpu_device *tmp_adev = NULL;
+	bool need_full_reset, skip_hw_reset;
 	int retry_limit = AMDGPU_MAX_RETRY_LIMIT;
 	int r = 0;
 
-retry:	/* Rest of adevs pre asic reset from XGMI hive. */
+retry:
+	/* Rest of adevs pre asic reset from XGMI hive. */
 	list_for_each_entry(tmp_adev, device_list, reset_list) {
 		r = amdgpu_device_pre_asic_reset(tmp_adev, reset_context);
 		/*TODO Should we stop ?*/
@@ -5430,10 +5458,30 @@ static int amdgpu_device_asic_reset(struct amdgpu_device *adev,
 		}
 		if (r)
 			adev->asic_reset_res = r;
+		else
+			amdgpu_device_eventfd_signal_gpu_reset(adev);
 	} else {
+		/*
+		 * Save the reset decision before amdgpu_do_asic_reset(),
+		 * which may clear AMDGPU_NEED_FULL_RESET before returning.
+		 */
+		need_full_reset =
+			test_bit(AMDGPU_NEED_FULL_RESET, &reset_context->flags);
+		skip_hw_reset =
+			test_bit(AMDGPU_SKIP_HW_RESET, &reset_context->flags);
+
 		r = amdgpu_do_asic_reset(device_list, reset_context);
 		if (r && r == -EAGAIN)
 			goto retry;
+
+		/*
+		 * Notify only after a real hardware reset completed.
+		 * Do not notify recovery-only or reset-skip paths.
+		 */
+		if (!r && need_full_reset && !skip_hw_reset) {
+			list_for_each_entry(tmp_adev, device_list, reset_list)
+				amdgpu_device_eventfd_signal_gpu_reset(tmp_adev);
+		}
 	}
 
 	list_for_each_entry(tmp_adev, device_list, reset_list) {
@@ -5530,7 +5578,6 @@ static void amdgpu_device_gpu_resume(struct amdgpu_device *adev,
 	}
 }
 
-
 /**
  * amdgpu_device_gpu_recover - reset the asic and recover scheduler
  *
-- 
2.34.1

Re: [PATCH v4 1/4] drm/amdgpu: Signal USERQ_EOP EVENTFD notifications

[Christian König]

On 7/14/26 10:02, Srinivasan Shanmugam wrote:
> Signal USERQ_EOP EVENTFD subscriptions from the USERQ interrupt path.
> 
> The EOP interrupt already identifies the queue that completed. Use the
> queue object directly to notify matching EVENTFD subscribers.
> 
> Routing notifications through the queue object keeps delivery tied to
> the queue instance that generated the completion event, avoiding
> ambiguities from reused queue identifiers or doorbell indices.
> 
> EVENTFD remains notification-only and carries no event payload.
> 
> v2: (per Christian)
> - Move USERQ_EOP EVENTFD signaling into amdgpu_userq_process_fence_irq().
> - Reuse the existing doorbell-to-queue lookup instead of duplicating it
>   in the interrupt handler.
> - Keep fence processing and EVENTFD notification handling together in a
>   single helper.
> 
> Cc: Alex Deucher <alexander.deucher@amd.com>
> Cc: Christian König <christian.koenig@amd.com>
> Signed-off-by: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
> Reviewed-by: Alex Deucher <alexander.deucher@amd.com>

Reviewed-by: Christian König <christian.koenig@amd.com>

> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
> index fbf783946f6d..ab3ef3a9f655 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
> @@ -215,6 +215,7 @@ void amdgpu_userq_process_fence_irq(struct amdgpu_device *adev, u32 doorbell)
>  {
>  	struct xarray *xa = &adev->userq_doorbell_xa;
>  	struct amdgpu_usermode_queue *queue;
> +	struct amdgpu_eventfd_mgr *eventfd_mgr;
>  	unsigned long flags;
>  	int r;
>  
> @@ -232,6 +233,11 @@ void amdgpu_userq_process_fence_irq(struct amdgpu_device *adev, u32 doorbell)
>  		/* Restart the timer when there are still fences pending */
>  		if (r == 1)
>  			amdgpu_userq_start_hang_detect_work(queue);
> +
> +		eventfd_mgr = amdgpu_userq_eventfd_mgr(queue->userq_mgr);
> +		amdgpu_eventfd_signal(eventfd_mgr,
> +				      DRM_AMDGPU_EVENT_TYPE_USERQ_EOP,
> +				      queue);
>  	}
>  	xa_unlock_irqrestore(xa, flags);
>  }

Re: [PATCH v4 2/4] drm/amdgpu/userq: properly account for resets

[Christian König]

On 7/14/26 10:02, Srinivasan Shanmugam wrote:
> From: Alex Deucher <alexander.deucher@amd.com>
> 
> We need to increment the reset counter, force fence completion,
> and set the wedged event when a user queue is reset.
> 
> mes_userq_reset_queue() handles this for collateral damage, but the
> caller needs to handle this directly for the original guilty queue.
> 
> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
> Cc: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
> Cc: Prike Liang <Prike.Liang@amd.com>
> Cc: Sunil Khatri <sunil.khatri@amd.com>
> Reviewed-by: Jesse Zhang <Jesse.Zhang@amd.com>
> Reviewed-by: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>

Acked-by: Christian König <christian.koenig@amd.com>

> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
> index ab3ef3a9f655..fb0d335875af 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
> @@ -167,8 +167,13 @@ static void amdgpu_userq_hang_detect_work(struct work_struct *work)
>  							 queue, NULL, NULL);
>  		else
>  			r = userq_funcs->reset(queue);
> -		if (r)
> +		if (r) {
>  			gpu_reset = true;
> +		} else {
> +			atomic_inc(&adev->gpu_reset_counter);
> +			amdgpu_userq_fence_driver_force_completion(queue);
> +			drm_dev_wedged_event(adev_to_drm(adev), DRM_WEDGE_RECOVERY_NONE, NULL);
> +		}
>  	} else {
>  		gpu_reset = true;
>  	}

Re: [PATCH v4 3/4] drm/amdgpu: Signal QUEUE_RESET EVENTFD notifications

[Christian König]

On 7/14/26 10:02, Srinivasan Shanmugam wrote:
> Signal QUEUE_RESET EVENTFD subscriptions when hung user queues are
> detected.
> 
> MES reports the doorbell indices of hung queues. Use the existing
> doorbell-to-queue mapping to resolve the affected queue without scanning
> all user queues.
> 
> Consolidate the queue state update, reset accounting, EVENTFD signaling,
> fence completion, and wedged event notification in a common helper. Use
> the helper for both the original guilty queue and additional affected
> queues.
> 
> EVENTFD remains notification-only.
> 
> v2: (per Christian)
>  - Use the doorbell xarray to look up affected queues instead of scanning
>    all queues.
>  - Move queue reset handling into amdgpu_userq.c.
>  - Consolidate queue state updates, EVENTFD signaling, and fence
>    completion in a single helper.
> 
> v3: Rebase only.
> 
> v4: (per Alex)
>  - Rebase on Alex's "drm/amdgpu/userq: properly account for resets".
>  - Preserve the reset counter update in the common hung queue helper.
>  - Use amdgpu_userq_handle_hung_queue() for both successful queue reset
>    paths.
> 
> Suggested-by: Christian König <christian.koenig@amd.com>
> Suggested-by: Alex Deucher <alexander.deucher@amd.com>
> Signed-off-by: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>

Acked-by: Christian König <christian.koenig@amd.com>

> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c  | 39 +++++++++---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_userq.h  |  2 +
>  drivers/gpu/drm/amd/amdgpu/mes_userqueue.c | 72 ++++++++++++++--------
>  3 files changed, 80 insertions(+), 33 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
> index fb0d335875af..65b20d7a4b18 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
> @@ -135,6 +135,34 @@ static void amdgpu_userq_mgr_reset_work(struct work_struct *work)
>  	amdgpu_device_gpu_recover(adev, NULL, &reset_context);
>  }
>  
> +/**
> + * amdgpu_userq_handle_hung_queue - handle a successfully reset hung queue
> + * @adev: AMDGPU device
> + * @queue: affected user queue
> + *
> + * Mark the queue as hung, account for the reset, force-complete its
> + * fences, notify matching QUEUE_RESET EVENTFD subscribers, and send
> + * the DRM wedged event.
> + */
> +void amdgpu_userq_handle_hung_queue(struct amdgpu_device *adev,
> +				    struct amdgpu_usermode_queue *queue)
> +{
> +	struct amdgpu_eventfd_mgr *eventfd_mgr;
> +
> +	queue->state = AMDGPU_USERQ_STATE_HUNG;
> +	atomic_inc(&adev->gpu_reset_counter);
> +
> +	amdgpu_userq_fence_driver_force_completion(queue);
> +
> +	eventfd_mgr = amdgpu_userq_eventfd_mgr(queue->userq_mgr);
> +	amdgpu_eventfd_signal(eventfd_mgr,
> +			      DRM_AMDGPU_EVENT_TYPE_QUEUE_RESET,
> +			      queue);
> +
> +	drm_dev_wedged_event(adev_to_drm(adev),
> +			     DRM_WEDGE_RECOVERY_NONE, NULL);
> +}
> +
>  static void amdgpu_userq_hang_detect_work(struct work_struct *work)
>  {
>  	struct amdgpu_usermode_queue *queue =
> @@ -167,13 +195,10 @@ static void amdgpu_userq_hang_detect_work(struct work_struct *work)
>  							 queue, NULL, NULL);
>  		else
>  			r = userq_funcs->reset(queue);
> -		if (r) {
> +		if (r)
>  			gpu_reset = true;
> -		} else {
> -			atomic_inc(&adev->gpu_reset_counter);
> -			amdgpu_userq_fence_driver_force_completion(queue);
> -			drm_dev_wedged_event(adev_to_drm(adev), DRM_WEDGE_RECOVERY_NONE, NULL);
> -		}
> +		else
> +			amdgpu_userq_handle_hung_queue(adev, queue);
>  	} else {
>  		gpu_reset = true;
>  	}
> @@ -247,8 +272,6 @@ void amdgpu_userq_process_fence_irq(struct amdgpu_device *adev, u32 doorbell)
>  	xa_unlock_irqrestore(xa, flags);
>  }
>  
> -
> -
>  int amdgpu_userq_input_va_validate(struct amdgpu_device *adev,
>  				   struct amdgpu_usermode_queue *queue,
>  				   u64 addr, u64 expected_size,
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.h
> index b69621311b80..526e007e300e 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.h
> @@ -180,6 +180,8 @@ void amdgpu_userq_pre_reset(struct amdgpu_device *adev);
>  int amdgpu_userq_post_reset(struct amdgpu_device *adev, bool vram_lost);
>  void amdgpu_userq_start_hang_detect_work(struct amdgpu_usermode_queue *queue);
>  void amdgpu_userq_process_fence_irq(struct amdgpu_device *adev, u32 doorbell);
> +void amdgpu_userq_handle_hung_queue(struct amdgpu_device *adev,
> +				    struct amdgpu_usermode_queue *queue);
>  
>  /*
>   * CP packs the per-process doorbell_id of the queue in
> diff --git a/drivers/gpu/drm/amd/amdgpu/mes_userqueue.c b/drivers/gpu/drm/amd/amdgpu/mes_userqueue.c
> index 3fad95199e0c..27adcea7b9a0 100644
> --- a/drivers/gpu/drm/amd/amdgpu/mes_userqueue.c
> +++ b/drivers/gpu/drm/amd/amdgpu/mes_userqueue.c
> @@ -26,6 +26,7 @@
>  #include "amdgpu_gfx.h"
>  #include "mes_userqueue.h"
>  #include "amdgpu_userq_fence.h"
> +#include "amdgpu_userq_internal.h"
>  #include "amdgpu_cwsr.h"
>  
>  #define AMDGPU_USERQ_PROC_CTX_SZ PAGE_SIZE
> @@ -238,34 +239,55 @@ int mes_userq_reset_queue(struct amdgpu_device *adev,
>  			  unsigned int queue,
>  			  unsigned int db)
>  {
> +	struct xarray *xa = &adev->userq_doorbell_xa;
>  	struct amdgpu_usermode_queue *uq;
>  	bool use_mmio = adev->gfx.mec.use_mmio_for_reset;
> -	unsigned long uq_id;
> -	int r;
> +	unsigned long flags;
> +	int r = 0;
>  
> -	xa_for_each(&adev->userq_doorbell_xa, uq_id, uq) {
> -		if (uq->queue_type == queue_type) {
> -			if (uq == guilty_uq)
> -				continue;
> -			if (uq->doorbell_index == db) {
> -				uq->state = AMDGPU_USERQ_STATE_HUNG;
> -				if (use_mmio)
> -					r = amdgpu_mes_reset_queue_mmio(adev, queue_type, 0, 1, pipe, queue, 0);
> -				else
> -					r = amdgpu_mes_reset_user_queue(adev, queue_type, db, 0);
> -				if (r)
> -					return r;
> -				r = mes_userq_unmap(uq);
> -				if (r)
> -					return r;
> -				atomic_inc(&adev->gpu_reset_counter);
> -				amdgpu_userq_fence_driver_force_completion(uq);
> -				drm_dev_wedged_event(adev_to_drm(adev), DRM_WEDGE_RECOVERY_NONE, NULL);
> -				break;
> -			}
> -		}
> -	}
> -	return 0;
> +	/*
> +	 * Resolve the doorbell directly to the affected queue instead of
> +	 * scanning all user queues.
> +	 */
> +	xa_lock_irqsave(xa, flags);
> +
> +	uq = xa_load(xa, db);
> +	if (uq)
> +		kref_get(&uq->refcount);
> +
> +	xa_unlock_irqrestore(xa, flags);
> +
> +	if (!uq)
> +		return 0;
> +
> +	/*
> +	 * The guilty queue is handled separately by the caller.
> +	 */
> +	if (uq == guilty_uq)
> +		goto out_put_queue;
> +
> +	if (uq->queue_type != queue_type)
> +		goto out_put_queue;
> +
> +	if (use_mmio)
> +		r = amdgpu_mes_reset_queue_mmio(adev, queue_type,
> +						0, 1, pipe, queue, 0);
> +	else
> +		r = amdgpu_mes_reset_user_queue(adev, queue_type, db, 0);
> +
> +	if (r)
> +		goto out_put_queue;
> +
> +	r = mes_userq_unmap(uq);
> +	if (r)
> +		goto out_put_queue;
> +
> +	amdgpu_userq_handle_hung_queue(adev, uq);
> +
> +out_put_queue:
> +	amdgpu_userq_put(uq);
> +
> +	return r;
>  }
>  
>  static int mes_userq_create_ctx_space(struct amdgpu_userq_mgr *uq_mgr,

Re: [PATCH v4 4/4] drm/amdgpu: Signal GPU_RESET EVENTFD notifications

[Christian König]

On 7/14/26 10:02, Srinivasan Shanmugam wrote:
> Signal GPU_RESET EVENTFD subscriptions after successful GPU reset
> completion.
> 
> GPU reset notifications allow userspace to observe that GPU recovery has
> completed and the device is operational again.
> 
> GPU_RESET is a device-scoped event, so no queue object is used. All
> processes that subscribed to GPU_RESET on the device are notified.
> 
> EVENTFD remains notification-only.
> 
> v2:
> - Move GPU_RESET EVENTFD signaling to the path that confirms a real
>   GPU hardware reset occurred.
> - Notify only after successful hardware reset completion.
> - Do not signal reset-skip or recovery-only paths.
> 
> Cc: Alex Deucher <alexander.deucher@amd.com>
> Cc: Christian König <christian.koenig@amd.com>
> Signed-off-by: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
> Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 51 +++++++++++++++++++++-
>  1 file changed, 49 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> index c924a55fd16f..15aaceae13d1 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -5390,15 +5390,43 @@ static void amdgpu_device_halt_activities(struct amdgpu_device *adev,
>  	}
>  }
>  
> +/*
> + * Signal GPU_RESET EVENTFD subscriptions for all open DRM files.
> + *
> + * GPU reset is a device-wide event rather than being associated with a
> + * specific user queue. Notify every process that subscribed to the
> + * GPU_RESET event on this device.
> + */
> +static void amdgpu_device_eventfd_signal_gpu_reset(struct amdgpu_device *adev)
> +{
> +	struct drm_device *ddev = adev_to_drm(adev);
> +	struct drm_file *file;
> +
> +	mutex_lock(&ddev->filelist_mutex);
> +
> +	list_for_each_entry(file, &ddev->filelist, lhead) {

Is it allowed to grab the filelist_mutex in a device reset?

@Vitaly another one we need to handle in our lockdep checks. Lowest priority, but please put that on your TODO list.

Thanks,
Christian.

> +		struct amdgpu_fpriv *fpriv = file->driver_priv;
> +
> +		if (fpriv)
> +			amdgpu_eventfd_signal(&fpriv->eventfd_mgr,
> +					      DRM_AMDGPU_EVENT_TYPE_GPU_RESET,
> +					      NULL);
> +	}
> +
> +	mutex_unlock(&ddev->filelist_mutex);
> +}
> +
>  static int amdgpu_device_asic_reset(struct amdgpu_device *adev,
>  			      struct list_head *device_list,
>  			      struct amdgpu_reset_context *reset_context)
>  {
>  	struct amdgpu_device *tmp_adev = NULL;
> +	bool need_full_reset, skip_hw_reset;
>  	int retry_limit = AMDGPU_MAX_RETRY_LIMIT;
>  	int r = 0;
>  
> -retry:	/* Rest of adevs pre asic reset from XGMI hive. */
> +retry:
> +	/* Rest of adevs pre asic reset from XGMI hive. */
>  	list_for_each_entry(tmp_adev, device_list, reset_list) {
>  		r = amdgpu_device_pre_asic_reset(tmp_adev, reset_context);
>  		/*TODO Should we stop ?*/
> @@ -5430,10 +5458,30 @@ static int amdgpu_device_asic_reset(struct amdgpu_device *adev,
>  		}
>  		if (r)
>  			adev->asic_reset_res = r;
> +		else
> +			amdgpu_device_eventfd_signal_gpu_reset(adev);
>  	} else {
> +		/*
> +		 * Save the reset decision before amdgpu_do_asic_reset(),
> +		 * which may clear AMDGPU_NEED_FULL_RESET before returning.
> +		 */
> +		need_full_reset =
> +			test_bit(AMDGPU_NEED_FULL_RESET, &reset_context->flags);
> +		skip_hw_reset =
> +			test_bit(AMDGPU_SKIP_HW_RESET, &reset_context->flags);
> +
>  		r = amdgpu_do_asic_reset(device_list, reset_context);
>  		if (r && r == -EAGAIN)
>  			goto retry;
> +
> +		/*
> +		 * Notify only after a real hardware reset completed.
> +		 * Do not notify recovery-only or reset-skip paths.
> +		 */
> +		if (!r && need_full_reset && !skip_hw_reset) {
> +			list_for_each_entry(tmp_adev, device_list, reset_list)
> +				amdgpu_device_eventfd_signal_gpu_reset(tmp_adev);
> +		}
>  	}
>  
>  	list_for_each_entry(tmp_adev, device_list, reset_list) {
> @@ -5530,7 +5578,6 @@ static void amdgpu_device_gpu_resume(struct amdgpu_device *adev,
>  	}
>  }
>  
> -
>  /**
>   * amdgpu_device_gpu_recover - reset the asic and recover scheduler
>   *