From: "Timur Kristóf" <timur.kristof@gmail.com>
To: Jesse Zhang <Jesse.Zhang@amd.com>, amd-gfx@lists.freedesktop.org
Cc: Alexander.Deucher@amd.com,
Christian Koenig <christian.koenig@amd.com>,
Lijo.Lazar@amd.com, Mario.Novales@amd.com, Manu.Rastogi@amd.com,
Jiongyu.Zhu@amd.com
Subject: Re: [PATCH v4 03/13] drm/amdgpu: add an engine scope to the IP block soft reset
Date: Thu, 19 Nov 2026 12:35:00 +0100 [thread overview]
Message-ID: <ce4d20f811d0f9484bc2b1a281e05ec970547791.camel@gmail.com> (raw)
In-Reply-To: <20261105030612.726542-3-Jesse.Zhang@amd.com>
On Thu, 2026-11-05 at 11:05 +0800, Jesse Zhang wrote:
> Some IP blocks have engines the hardware can reset on their own, eg.
> the graphics and compute engines of the GFX IP. Add an optional
> soft_reset_engine() callback, which resets the engine of a ring type
> and
> recovers its user queues. amdgpu_device_ip_soft_reset() uses it
> instead
> of soft_reset(), and then only stops and replays the rings of that
> type,
> holding the reset domain for read: KFD can still add and remove
> queues
> on the other engines.
>
> A hung user queue has no kernel ring, so
> amdgpu_device_ip_soft_reset()
> now takes the device and the ring type, and an optional guilty ring
> and
> user queue.
>
> v3: New patch, replaces the separate engine reset type and helper
> (Timur).
> v4: Fold the engine reset into amdgpu_device_ip_soft_reset(), and
> take
> the ring type instead of a new engine enum (Timur).
>
> Suggested-by: Timur Kristóf <timur.kristof@gmail.com>
> Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_ip.c | 59 ++++++++++++++++------
> --
> drivers/gpu/drm/amd/amdgpu/amdgpu_ip.h | 7 ++-
> drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 3 +-
> drivers/gpu/drm/amd/include/amd_shared.h | 6 +++
> 4 files changed, 52 insertions(+), 23 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ip.c
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_ip.c
> index d0b0d3eaffc5..9d75bdb2c14b 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ip.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ip.c
> @@ -421,8 +421,11 @@ static u32 amdgpu_ring_mask_from_ip(const enum
> amd_ip_block_type ip_type)
> /**
> * amdgpu_device_ip_soft_reset() - Perform a graceful soft reset on
> an IP block.
> *
> - * @guilty_ring: The ring which is guilty of causing a reset.
> - * @guilty_fence: The fence which didn't signal.
> + * @adev: amdgpu device
> + * @ring_type: The type of the ring or user queue that hung.
> + * @guilty_ring: The ring which is guilty of causing a reset, or
> NULL.
> + * @guilty_fence: The fence which didn't signal on @guilty_ring.
> + * @guilty_queue: The user queue which is guilty of causing a reset,
> or NULL.
> *
> * IP block soft reset is used when attempting to recover
> * from a GPU hang in a situation where a more fine grained
> @@ -430,55 +433,71 @@ static u32 amdgpu_ring_mask_from_ip(const enum
> amd_ip_block_type ip_type)
> * resets all rings that belong to the same device IP block
> * and re-initializes the device IP block.
> *
> + * IP blocks implementing soft_reset_engine() only reset the engine
> of
> + * @ring_type and its rings, and recover its user queues.
> + *
> * The reset is handled gracefully, meaning that we try to
> * minimize collateral damage (ie. avoid rejecting non-guilty jobs)
> * as well as back up and restore the contents of all rings
> * so that the system can move on from the hang.
> */
> -int amdgpu_device_ip_soft_reset(struct amdgpu_ring *guilty_ring,
> - struct amdgpu_fence *guilty_fence)
> +int amdgpu_device_ip_soft_reset(struct amdgpu_device *adev,
> + enum amdgpu_ring_type ring_type,
> + struct amdgpu_ring *guilty_ring,
> + struct amdgpu_fence *guilty_fence,
> + struct amdgpu_usermode_queue
> *guilty_queue)
> {
> - struct amdgpu_device *adev = guilty_ring->adev;
> + const struct amd_ip_funcs *funcs;
> struct amdgpu_ip_block *ip_block;
> enum amd_ip_block_type ip_type;
> u32 ring_type_mask;
> + bool engine;
> int r;
>
> - ip_type = amdgpu_ip_from_ring(guilty_ring->funcs->type);
> + ip_type = amdgpu_ip_from_ring(ring_type);
> ip_block = amdgpu_device_ip_get_ip_block(adev, ip_type);
>
> if (unlikely(!ip_block)) {
> - dev_warn(adev->dev, "IP block not found for ring
> %s\n",
> - guilty_ring->name);
> + dev_warn(adev->dev, "IP block not found for ring
> type %d\n",
> + ring_type);
> return -EOPNOTSUPP;
> }
>
> - if (!ip_block->version->funcs->soft_reset) {
> + funcs = ip_block->version->funcs;
> + engine = funcs->soft_reset_engine;
> + if (!engine && (!funcs->soft_reset || !guilty_ring)) {
> dev_warn(adev->dev, "IP block soft reset not
> supported on %s\n",
> - ip_block->version->funcs->name);
> + funcs->name);
> return -EOPNOTSUPP;
> }
>
> - dev_err(adev->dev, "Starting %s IP block soft reset\n",
> - ip_block->version->funcs->name);
> -
> - ring_type_mask = amdgpu_ring_mask_from_ip(ip_type);
> + dev_err(adev->dev, "Starting %s IP block soft reset\n",
> funcs->name);
>
> - amdgpu_device_lock_reset_domain(adev->reset_domain);
> + /* An engine reset leaves the others running: let KFD
> add/remove queues. */
> + if (engine) {
> + ring_type_mask = BIT(ring_type);
> + down_read(&adev->reset_domain->sem);
> + } else {
> + ring_type_mask = amdgpu_ring_mask_from_ip(ip_type);
> + amdgpu_device_lock_reset_domain(adev->reset_domain);
> + }
> amdgpu_multi_ring_reset_helper_begin(adev, ring_type_mask,
> guilty_ring, guilty_fence);
>
> - r = ip_block->version->funcs->soft_reset(ip_block);
> + r = engine ? funcs->soft_reset_engine(ip_block, ring_type,
> guilty_queue) :
> + funcs->soft_reset(ip_block);
>
> r = amdgpu_multi_ring_reset_helper_end(adev, ring_type_mask,
> guilty_ring, r);
> - amdgpu_device_unlock_reset_domain(adev->reset_domain);
> + if (engine)
> + up_read(&adev->reset_domain->sem);
> + else
> + amdgpu_device_unlock_reset_domain(adev-
> >reset_domain);
>
> if (r) {
> dev_err(adev->dev, "Failed %s IP block soft reset:
> %d\n",
> - ip_block->version->funcs->name, r);
> + funcs->name, r);
> return r;
> }
>
> - dev_err(adev->dev, "Successful %s IP block soft reset\n",
> - ip_block->version->funcs->name);
> + dev_err(adev->dev, "Successful %s IP block soft reset\n",
> funcs->name);
> return 0;
> }
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ip.h
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_ip.h
> index 5dd6273982c4..6532c16b1a15 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ip.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ip.h
> @@ -158,7 +158,10 @@ int amdgpu_device_ip_wait_for_idle(struct
> amdgpu_device *adev,
> enum amd_ip_block_type
> block_type);
> bool amdgpu_device_ip_is_valid(struct amdgpu_device *adev,
> enum amd_ip_block_type block_type);
> -int amdgpu_device_ip_soft_reset(struct amdgpu_ring *guilty_ring,
> - struct amdgpu_fence *guilty_fence);
> +int amdgpu_device_ip_soft_reset(struct amdgpu_device *adev,
> + enum amdgpu_ring_type ring_type,
> + struct amdgpu_ring *guilty_ring,
> + struct amdgpu_fence *guilty_fence,
> + struct amdgpu_usermode_queue
> *guilty_queue);
>
> #endif /* __AMDGPU_IP_H__ */
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> index 8689a3c933b2..cb72d273bd52 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> @@ -154,7 +154,8 @@ static enum drm_gpu_sched_stat
> amdgpu_job_timedout(struct drm_sched_job *s_job)
> /* Attempt an IP block soft reset, if supported. */
> if (amdgpu_gpu_recovery &&
> amdgpu_ring_is_reset_type_supported(ring,
> AMDGPU_RESET_TYPE_IP_BLOCK_SOFT_RESET)) {
> - r = amdgpu_device_ip_soft_reset(ring, job-
> >hw_fence);
> + r = amdgpu_device_ip_soft_reset(adev, ring->funcs-
> >type, ring,
> + job->hw_fence,
> NULL);
> if (!r) {
> atomic_inc(&ring->adev->gpu_reset_counter);
> drm_dev_wedged_event(adev_to_drm(adev),
> DRM_WEDGE_RECOVERY_NONE, info);
> diff --git a/drivers/gpu/drm/amd/include/amd_shared.h
> b/drivers/gpu/drm/amd/include/amd_shared.h
> index e81ce9d79009..06874a338b27 100644
> --- a/drivers/gpu/drm/amd/include/amd_shared.h
> +++ b/drivers/gpu/drm/amd/include/amd_shared.h
> @@ -29,6 +29,7 @@
>
> #define AMD_MAX_USEC_TIMEOUT 1000000 /* 1000 ms */
> struct amdgpu_ip_block;
> +struct amdgpu_usermode_queue;
>
>
> /*
> @@ -421,6 +422,7 @@ enum DC_DEBUG_MASK {
> };
>
> enum amd_dpm_forced_level;
> +enum amdgpu_ring_type;
>
> /**
> * struct amd_ip_funcs - general hooks for managing amdgpu IP Blocks
> @@ -441,6 +443,7 @@ enum amd_dpm_forced_level;
> * @complete: handles IP specific changes after resume
> * @wait_for_idle: poll for idle
> * @soft_reset: soft reset the IP block
> + * @soft_reset_engine: soft reset the engine of a ring type and its
> user queues
> * @set_clockgating_state: enable/disable cg for the IP block
> * @set_powergating_state: enable/disable pg for the IP block
> * @get_clockgating_state: get current clockgating status
> @@ -469,6 +472,9 @@ struct amd_ip_funcs {
> void (*complete)(struct amdgpu_ip_block *ip_block);
> int (*wait_for_idle)(struct amdgpu_ip_block *ip_block);
> int (*soft_reset)(struct amdgpu_ip_block *ip_block);
> + int (*soft_reset_engine)(struct amdgpu_ip_block *ip_block,
> + enum amdgpu_ring_type ring_type,
> + struct amdgpu_usermode_queue
> *guilty_queue);
> int (*set_clockgating_state)(struct amdgpu_ip_block
> *ip_block,
> enum amd_clockgating_state
> state);
> int (*set_powergating_state)(struct amdgpu_ip_block
> *ip_block,
next prev parent reply other threads:[~2026-11-19 11:35 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-05 3:05 [PATCH v4 01/13] drm/amdgpu: pass the device to the multi ring reset helpers Jesse Zhang
2026-11-05 3:05 ` [PATCH v4 02/13] drm/amdgpu/ring: During multi-queue reset, only call begin helper if needed Jesse Zhang
2026-11-11 23:52 ` Daniel Campos Ramos
2026-11-19 11:34 ` Timur Kristóf
2026-11-05 3:05 ` [PATCH v4 03/13] drm/amdgpu: add an engine scope to the IP block soft reset Jesse Zhang
2026-11-19 11:35 ` Timur Kristóf [this message]
2026-11-05 3:05 ` [PATCH v4 04/13] drm/amdgpu/gfx: add callbacks to save and restore a gfx HQD Jesse Zhang
2026-11-19 11:35 ` Timur Kristóf
2026-11-05 3:05 ` [PATCH v4 05/13] drm/amdgpu/gfx12: " Jesse Zhang
2026-11-19 11:35 ` Timur Kristóf
2026-11-05 3:05 ` [PATCH v4 06/13] drm/amdgpu/gfx11: " Jesse Zhang
2026-11-19 11:36 ` Timur Kristóf
2026-11-05 3:05 ` [PATCH v4 07/13] drm/amdgpu/userq: add a helper to signal the fences of a queue up to a point Jesse Zhang
2026-11-19 11:36 ` Timur Kristóf
2026-11-05 3:05 ` [PATCH v4 08/13] drm/amdgpu/userq: add helpers to keep the user queues of one engine across a reset Jesse Zhang
2026-11-19 11:36 ` Timur Kristóf
2026-11-05 3:05 ` [PATCH v4 09/13] drm/amdgpu/gfx12: add a gfx engine soft reset Jesse Zhang
2026-11-19 11:42 ` Timur Kristóf
2026-11-05 3:05 ` [PATCH v4 10/13] drm/amdgpu/gfx11: " Jesse Zhang
2026-11-19 11:47 ` Timur Kristóf
2026-11-05 3:05 ` [PATCH v4 11/13] drm/amdgpu/userq: reset the gfx engine when a gfx user queue reset fails Jesse Zhang
2026-11-19 11:47 ` Timur Kristóf
2026-11-05 3:05 ` [PATCH v4 12/13] drm/amdgpu/gfx12: enable the gfx engine soft reset Jesse Zhang
2026-11-19 11:47 ` Timur Kristóf
2026-11-05 3:05 ` [PATCH v4 13/13] drm/amdgpu/gfx11: " Jesse Zhang
2026-11-19 11:47 ` Timur Kristóf
2026-11-10 2:06 ` [PATCH v4 01/13] drm/amdgpu: pass the device to the multi ring reset helpers Zhang, Jesse(Jie)
2026-11-11 15:28 ` Timur Kristóf
2026-11-19 11:33 ` Timur Kristóf
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ce4d20f811d0f9484bc2b1a281e05ec970547791.camel@gmail.com \
--to=timur.kristof@gmail.com \
--cc=Alexander.Deucher@amd.com \
--cc=Jesse.Zhang@amd.com \
--cc=Jiongyu.Zhu@amd.com \
--cc=Lijo.Lazar@amd.com \
--cc=Manu.Rastogi@amd.com \
--cc=Mario.Novales@amd.com \
--cc=amd-gfx@lists.freedesktop.org \
--cc=christian.koenig@amd.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.