[PATCH v2 01/15] drm/amdgpu/gfx: add a reset_engine callback

[PATCH v2 01/15] drm/amdgpu/gfx: add a reset_engine callback

[Jesse Zhang]

Some gfx hangs can't be recovered by a queue or a pipe reset, eg. a CP
front-end stuck in the middle of a packet. They currently end in a full
GPU reset, and VRAM is lost. Resetting only the graphics engine is
enough for them, and leaves the compute engine running.

Add an optional reset_engine callback to amdgpu_gfx_funcs that resets
only the graphics or only the compute engine. The split is specific to
the GFX IP, eg. CP_SOFT_RESET_CNTL has GFX_ONLY_SOFT_RESET and
CMP_ONLY_SOFT_RESET, so it doesn't belong in the amd_ip_funcs that all
IP blocks share.

Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h
index 5939e1fb3b24..8dc669e89db5 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h
@@ -345,6 +345,13 @@ struct amdgpu_gfx_shadow_info {
 	u32 eop_alignment;
 };
 
+/* The engines of the GFX IP that can be reset on their own. */
+enum amdgpu_gfx_engine {
+	AMDGPU_GFX_ENGINE_GFX,
+	AMDGPU_GFX_ENGINE_COMPUTE,
+	AMDGPU_GFX_ENGINE_NUM,
+};
+
 struct amdgpu_gfx_funcs {
 	/* get the gpu clock counter */
 	uint64_t (*get_gpu_clock_counter)(struct amdgpu_device *adev);
@@ -376,6 +383,9 @@ struct amdgpu_gfx_funcs {
 	/* find the gfx HQD slot (me/pipe/queue) for a doorbell */
 	bool (*detect_hung_queue)(struct amdgpu_device *adev, u32 doorbell_index,
 				  u32 *me, u32 *pipe, u32 *queue);
+	/* reset one engine, leaving the other one running */
+	int (*reset_engine)(struct amdgpu_device *adev,
+			    enum amdgpu_gfx_engine engine);
 };
 
 struct sq_work {
-- 
2.50.1

[PATCH v2 02/15] drm/amdgpu/gfx12: add a gfx engine reset

[Jesse Zhang]

Some hangs leave the gfx streaming engine of the CP front-end stuck,
eg. waiting for packet data that never arrives. Neither a queue reset
nor a pipe reset can stop it, so the recovery currently ends in a full
GPU reset, and VRAM is lost.

Implement the reset_engine callback for the graphics engine:
- halt PFP/ME;
- wait for the memory requests of the CP DMA to complete;
- with CP_SOFT_RESET_CNTL.GFX_ONLY_SOFT_RESET and GFX_HQD_REG_RESET set,
  pulse GRBM_SOFT_RESET.SOFT_RESET_CPF/CPG. A CPF reset alone does not
  clear the stuck stream;
- restart PFP/ME and restore CP_PFP_IB_CONTROL;
- remap the kernel gfx queue through MES from its MQD backup.

MEC and the compute part of CPF/CPG aren't reset, so the compute queues
keep running.

MES fetches its own queues through CPF, and a MES command in flight
during the reset doesn't complete: hold the MES lock across the reset.

A CP DMA request in flight during the reset leaves the ME busy after it,
and MES then fails its next commands. Wait for those requests before
the reset, and give up after the usual timeout: the DMA of a hung stream
may never complete. Wait for them after the reset too, but not for the
whole CP, as the halted PFP stays busy until it is restarted. If the CP
doesn't go idle once PFP/ME run again, fail the reset so that a full
reset follows.

The reset also clears the HQDs of the gfx user queues, the caller has to
handle them.

The callback relies on the caller to only use it when the IP reports
AMDGPU_RESET_TYPE_PER_ENGINE, which a later patch enables with MES and
async gfx rings.

Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c | 129 +++++++++++++++++++++++++
 1 file changed, 129 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
index d334fb805020..dfc409e28914 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
@@ -274,6 +274,8 @@ static void gfx_v12_0_set_imu_funcs(struct amdgpu_device *adev);
 static int gfx_v12_0_get_cu_info(struct amdgpu_device *adev,
 				 struct amdgpu_cu_info *cu_info);
 static uint64_t gfx_v12_0_get_gpu_clock_counter(struct amdgpu_device *adev);
+static int gfx_v12_0_reset_engine(struct amdgpu_device *adev,
+				  enum amdgpu_gfx_engine engine);
 static void gfx_v12_0_select_se_sh(struct amdgpu_device *adev, u32 se_num,
 				   u32 sh_num, u32 instance, int xcc_id);
 static u32 gfx_v12_0_get_wgp_active_bitmap_per_sh(struct amdgpu_device *adev);
@@ -959,6 +961,7 @@ static const struct amdgpu_gfx_funcs gfx_v12_0_gfx_funcs = {
 	.update_perfmon_mgcg = &gfx_v12_0_update_perf_clk,
 	.get_gfx_shadow_info = &gfx_v12_0_get_gfx_shadow_info,
 	.get_hdp_flush_mask = &amdgpu_gfx_get_hdp_flush_mask,
+	.reset_engine = &gfx_v12_0_reset_engine,
 };
 
 static int gfx_v12_0_gpu_early_init(struct amdgpu_device *adev)
@@ -3844,6 +3847,132 @@ static int gfx_v12_0_wait_for_idle(struct amdgpu_ip_block *ip_block)
 	return -ETIMEDOUT;
 }
 
+/* The CP DMA and the interfaces its memory requests go through */
+#define GFX12_CP_STAT_MEM_BUSY	(CP_STAT__DMA_BUSY_MASK | \
+				 CP_STAT__TCIU_BUSY_MASK | \
+				 CP_STAT__RCIU_BUSY_MASK | \
+				 CP_STAT__UTCL2IU_BUSY_MASK)
+
+/*
+ * Soft reset the gfx CP and restart PFP and ME. Returns 0, or -ETIMEDOUT if the
+ * CP doesn't go idle after PFP and ME are restarted.
+ */
+static int gfx_v12_0_soft_reset_gfx_cp(struct amdgpu_device *adev)
+{
+	u32 tmp, cntl, me_cntl, ib_control;
+	int i;
+
+	amdgpu_gfx_rlc_enter_safe_mode(adev, 0);
+
+	/* MES fetches its queues through CPF: keep it out of the reset. */
+	amdgpu_mes_lock(&adev->mes);
+
+	me_cntl = RREG32_SOC15(GC, 0, regCP_ME_CNTL);
+	/* Cleared by the reset. */
+	ib_control = RREG32_SOC15(GC, 0, regCP_PFP_IB_CONTROL);
+
+	tmp = REG_SET_FIELD(me_cntl, CP_ME_CNTL, ME_HALT, 1);
+	tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, PFP_HALT, 1);
+	WREG32_SOC15(GC, 0, regCP_ME_CNTL, tmp);
+	udelay(50);
+
+	/* A CP DMA in flight during the reset leaves the ME busy after it. */
+	for (i = 0; i < adev->usec_timeout; i++) {
+		if (!(RREG32_SOC15(GC, 0, regCP_STAT) & GFX12_CP_STAT_MEM_BUSY))
+			break;
+		udelay(1);
+	}
+
+	/* Without GFX_ONLY_SOFT_RESET, a CPF reset would also hit compute and MES. */
+	cntl = RREG32_SOC15(GC, 0, regCP_SOFT_RESET_CNTL);
+	tmp = REG_SET_FIELD(cntl, CP_SOFT_RESET_CNTL, GFX_ONLY_SOFT_RESET, 1);
+	tmp = REG_SET_FIELD(tmp, CP_SOFT_RESET_CNTL, GFX_HQD_REG_RESET, 1);
+	WREG32_SOC15(GC, 0, regCP_SOFT_RESET_CNTL, tmp);
+
+	/* A CPF reset alone leaves the streaming engine stuck. */
+	tmp = RREG32_SOC15(GC, 0, regGRBM_SOFT_RESET);
+	tmp = REG_SET_FIELD(tmp, GRBM_SOFT_RESET, SOFT_RESET_CPF, 1);
+	tmp = REG_SET_FIELD(tmp, GRBM_SOFT_RESET, SOFT_RESET_CPG, 1);
+	WREG32_SOC15(GC, 0, regGRBM_SOFT_RESET, tmp);
+	tmp = RREG32_SOC15(GC, 0, regGRBM_SOFT_RESET);
+	udelay(50);
+	tmp = REG_SET_FIELD(tmp, GRBM_SOFT_RESET, SOFT_RESET_CPF, 0);
+	tmp = REG_SET_FIELD(tmp, GRBM_SOFT_RESET, SOFT_RESET_CPG, 0);
+	WREG32_SOC15(GC, 0, regGRBM_SOFT_RESET, tmp);
+	tmp = RREG32_SOC15(GC, 0, regGRBM_SOFT_RESET);
+
+	WREG32_SOC15(GC, 0, regCP_SOFT_RESET_CNTL, cntl);
+
+	/* Not the whole CP: the halted PFP stays busy until restarted. */
+	for (i = 0; i < adev->usec_timeout; i++) {
+		if (!(RREG32_SOC15(GC, 0, regCP_STAT) & GFX12_CP_STAT_MEM_BUSY))
+			break;
+		udelay(1);
+	}
+	amdgpu_mes_unlock(&adev->mes);
+
+	gfx_v12_0_set_pfp_ucode_start_addr(adev);
+	gfx_v12_0_set_me_ucode_start_addr(adev);
+	tmp = RREG32_SOC15(GC, 0, regCP_ME_CNTL);
+	tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, CE_HALT,
+			    REG_GET_FIELD(me_cntl, CP_ME_CNTL, CE_HALT));
+	WREG32_SOC15(GC, 0, regCP_ME_CNTL, tmp);
+	WREG32_SOC15(GC, 0, regCP_PFP_IB_CONTROL, ib_control);
+	amdgpu_gfx_rlc_exit_safe_mode(adev, 0);
+
+	tmp = RREG32_SOC15(GC, 0, regCP_ME_CNTL);
+	tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, ME_HALT, 0);
+	tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, PFP_HALT, 0);
+	WREG32_SOC15(GC, 0, regCP_ME_CNTL, tmp);
+	for (i = 0; i < adev->usec_timeout; i++) {
+		if (!RREG32_SOC15(GC, 0, regCP_STAT))
+			return 0;
+		udelay(1);
+	}
+	return -ETIMEDOUT;
+}
+
+/*
+ * Reset the graphics engine: the CP front-end (CPF/CPG), PFP/ME and the gfx
+ * HQDs. MEC, MES and the compute queues keep running.
+ */
+static int gfx_v12_0_reset_engine(struct amdgpu_device *adev,
+				  enum amdgpu_gfx_engine engine)
+{
+	int i, r;
+
+	lockdep_assert_held(&adev->gfx.mec.reset_mutex);
+
+	if (engine != AMDGPU_GFX_ENGINE_GFX)
+		return -EOPNOTSUPP;
+
+	amdgpu_gfx_off_ctrl(adev, false);
+
+	r = gfx_v12_0_soft_reset_gfx_cp(adev);
+	if (r) {
+		dev_err(adev->dev, "gfx CP still busy after reset, CP_STAT 0x%08x\n",
+			RREG32_SOC15(GC, 0, regCP_STAT));
+		goto exit;
+	}
+
+	/* Map the kernel gfx queue again: the reset cleared its HQD. */
+	r = amdgpu_gfx_disable_kgq(adev, 0);
+	if (r)
+		goto exit;
+
+	for (i = 0; i < adev->gfx.num_gfx_rings; i++)
+		amdgpu_gfx_mqd_reset_restore(&adev->gfx.gfx_ring[i]);
+	r = amdgpu_gfx_enable_kgq(adev, 0);
+	if (r)
+		goto exit;
+
+	r = gfx_v12_0_cp_gfx_start(adev);
+
+exit:
+	amdgpu_gfx_off_ctrl(adev, true);
+	return r;
+}
+
 static uint64_t gfx_v12_0_get_gpu_clock_counter(struct amdgpu_device *adev)
 {
 	uint64_t clock = 0;
-- 
2.50.1

[PATCH v2 03/15] drm/amdgpu/gfx11: add a gfx engine reset

[Jesse Zhang]

Implement the reset_engine callback for the graphics engine of gfx11, as
for gfx12:
- halt PFP/ME and wait for the memory requests of the CP DMA to complete;
- with CP_SOFT_RESET_CNTL.GFX_ONLY_SOFT_RESET and GFX_HQD_REG_RESET set,
  pulse GRBM_SOFT_RESET.SOFT_RESET_CPF/CPG, holding the MES lock, and
  wait for the memory requests again;
- clear CP_VMID_RESET;
- restart PFP/ME from the start of their RS64 firmware and restore
  CP_PFP_IB_CONTROL; if the CP doesn't go idle then, fail the reset so
  that a full reset follows;
- remap the kernel gfx queue through MES from its MQD backup and restart
  it with gfx_v11_0_cp_gfx_start().

MES resets a gfx queue by writing CP_VMID_RESET, which the PFP handles.
When the PFP hung, eg. on a bad opcode, the request stays pending across
the CPF/CPG reset, and the restarted PFP then waits on it forever for
queues the reset already cleared.

gfx11 has no helper to restart PFP/ME from their start addresses outside
of the firmware load: add one. A later patch enables the reset with
MES, async gfx rings and the RS64 firmware.

Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c | 176 +++++++++++++++++++++++++
 1 file changed, 176 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
index 315dfd806d56..12b307e4106b 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
@@ -363,6 +363,8 @@ static void gfx_v11_0_ring_invalidate_tlbs(struct amdgpu_ring *ring,
 static void gfx_v11_0_set_safe_mode(struct amdgpu_device *adev, int xcc_id);
 static void gfx_v11_0_unset_safe_mode(struct amdgpu_device *adev, int xcc_id);
 static void gfx_v11_0_userq_priv_fault_work(struct work_struct *work);
+static int gfx_v11_0_reset_engine(struct amdgpu_device *adev,
+				  enum amdgpu_gfx_engine engine);
 static void gfx_v11_0_update_perf_clk(struct amdgpu_device *adev,
 				      bool enable);
 
@@ -1150,6 +1152,7 @@ static const struct amdgpu_gfx_funcs gfx_v11_0_gfx_funcs = {
 	.get_gfx_shadow_info = &gfx_v11_0_get_gfx_shadow_info,
 	.get_hdp_flush_mask = &amdgpu_gfx_get_hdp_flush_mask,
 	.detect_hung_queue = &gfx_v11_0_detect_hung_queue,
+	.reset_engine = &gfx_v11_0_reset_engine,
 };
 
 static int gfx_v11_0_gpu_early_init(struct amdgpu_device *adev)
@@ -6916,6 +6919,179 @@ static void gfx_v11_0_emit_mem_sync(struct amdgpu_ring *ring)
 	amdgpu_ring_write(ring, gcr_cntl); /* GCR_CNTL */
 }
 
+/* Restart the PFP and ME pipes from the start of their RS64 firmware. */
+static void gfx_v11_0_restart_gfx_rs64(struct amdgpu_device *adev)
+{
+	const struct gfx_firmware_header_v2_0 *pfp_hdr, *me_hdr;
+	u32 pipe_id, tmp;
+
+	pfp_hdr = (const struct gfx_firmware_header_v2_0 *)adev->gfx.pfp_fw->data;
+	me_hdr = (const struct gfx_firmware_header_v2_0 *)adev->gfx.me_fw->data;
+
+	mutex_lock(&adev->srbm_mutex);
+	for (pipe_id = 0; pipe_id < adev->gfx.me.num_pipe_per_me; pipe_id++) {
+		soc21_grbm_select(adev, 0, pipe_id, 0, 0);
+		WREG32_SOC15(GC, 0, regCP_PFP_PRGRM_CNTR_START,
+			     (pfp_hdr->ucode_start_addr_hi << 30) |
+			     (pfp_hdr->ucode_start_addr_lo >> 2));
+		WREG32_SOC15(GC, 0, regCP_PFP_PRGRM_CNTR_START_HI,
+			     pfp_hdr->ucode_start_addr_hi >> 2);
+		WREG32_SOC15(GC, 0, regCP_ME_PRGRM_CNTR_START,
+			     (me_hdr->ucode_start_addr_hi << 30) |
+			     (me_hdr->ucode_start_addr_lo >> 2));
+		WREG32_SOC15(GC, 0, regCP_ME_PRGRM_CNTR_START_HI,
+			     me_hdr->ucode_start_addr_hi >> 2);
+
+		tmp = RREG32_SOC15(GC, 0, regCP_ME_CNTL);
+		if (pipe_id == 0) {
+			tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, PFP_PIPE0_RESET, 1);
+			tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, ME_PIPE0_RESET, 1);
+		} else {
+			tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, PFP_PIPE1_RESET, 1);
+			tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, ME_PIPE1_RESET, 1);
+		}
+		WREG32_SOC15(GC, 0, regCP_ME_CNTL, tmp);
+		if (pipe_id == 0) {
+			tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, PFP_PIPE0_RESET, 0);
+			tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, ME_PIPE0_RESET, 0);
+		} else {
+			tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, PFP_PIPE1_RESET, 0);
+			tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, ME_PIPE1_RESET, 0);
+		}
+		WREG32_SOC15(GC, 0, regCP_ME_CNTL, tmp);
+	}
+	soc21_grbm_select(adev, 0, 0, 0, 0);
+	mutex_unlock(&adev->srbm_mutex);
+}
+
+/* The CP DMA and the interfaces its memory requests go through */
+#define GFX11_CP_STAT_MEM_BUSY	(CP_STAT__DMA_BUSY_MASK | \
+				 CP_STAT__TCIU_BUSY_MASK | \
+				 CP_STAT__RCIU_BUSY_MASK | \
+				 CP_STAT__UTCL2IU_BUSY_MASK)
+
+/*
+ * Soft reset the gfx CP and restart PFP and ME. Returns 0, or -ETIMEDOUT if the
+ * CP doesn't go idle after PFP and ME are restarted.
+ */
+static int gfx_v11_0_soft_reset_gfx_cp(struct amdgpu_device *adev)
+{
+	u32 tmp, cntl, me_cntl, ib_control;
+	int i;
+
+	amdgpu_gfx_rlc_enter_safe_mode(adev, 0);
+
+	/* MES fetches its queues through CPF: keep it out of the reset. */
+	amdgpu_mes_lock(&adev->mes);
+
+	me_cntl = RREG32_SOC15(GC, 0, regCP_ME_CNTL);
+	/* Cleared by the reset. */
+	ib_control = RREG32_SOC15(GC, 0, regCP_PFP_IB_CONTROL);
+
+	tmp = REG_SET_FIELD(me_cntl, CP_ME_CNTL, ME_HALT, 1);
+	tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, PFP_HALT, 1);
+	WREG32_SOC15(GC, 0, regCP_ME_CNTL, tmp);
+	udelay(50);
+
+	/* A CP DMA in flight during the reset leaves the ME busy after it. */
+	for (i = 0; i < adev->usec_timeout; i++) {
+		if (!(RREG32_SOC15(GC, 0, regCP_STAT) & GFX11_CP_STAT_MEM_BUSY))
+			break;
+		udelay(1);
+	}
+
+	/* Without GFX_ONLY_SOFT_RESET, a CPF reset would also hit compute and MES. */
+	cntl = RREG32_SOC15(GC, 0, regCP_SOFT_RESET_CNTL);
+	tmp = REG_SET_FIELD(cntl, CP_SOFT_RESET_CNTL, GFX_ONLY_SOFT_RESET, 1);
+	tmp = REG_SET_FIELD(tmp, CP_SOFT_RESET_CNTL, GFX_HQD_REG_RESET, 1);
+	WREG32_SOC15(GC, 0, regCP_SOFT_RESET_CNTL, tmp);
+
+	/* A CPF reset alone leaves the streaming engine stuck. */
+	tmp = RREG32_SOC15(GC, 0, regGRBM_SOFT_RESET);
+	tmp = REG_SET_FIELD(tmp, GRBM_SOFT_RESET, SOFT_RESET_CPF, 1);
+	tmp = REG_SET_FIELD(tmp, GRBM_SOFT_RESET, SOFT_RESET_CPG, 1);
+	WREG32_SOC15(GC, 0, regGRBM_SOFT_RESET, tmp);
+	tmp = RREG32_SOC15(GC, 0, regGRBM_SOFT_RESET);
+	udelay(50);
+	tmp = REG_SET_FIELD(tmp, GRBM_SOFT_RESET, SOFT_RESET_CPF, 0);
+	tmp = REG_SET_FIELD(tmp, GRBM_SOFT_RESET, SOFT_RESET_CPG, 0);
+	WREG32_SOC15(GC, 0, regGRBM_SOFT_RESET, tmp);
+	tmp = RREG32_SOC15(GC, 0, regGRBM_SOFT_RESET);
+
+	WREG32_SOC15(GC, 0, regCP_SOFT_RESET_CNTL, cntl);
+
+	/* Not the whole CP: the halted PFP stays busy until restarted. */
+	for (i = 0; i < adev->usec_timeout; i++) {
+		if (!(RREG32_SOC15(GC, 0, regCP_STAT) & GFX11_CP_STAT_MEM_BUSY))
+			break;
+		udelay(1);
+	}
+
+	/* A CP_VMID_RESET the hung PFP never handled survives the reset. */
+	WREG32_SOC15(GC, 0, regCP_VMID_RESET, 0);
+	amdgpu_mes_unlock(&adev->mes);
+
+	gfx_v11_0_restart_gfx_rs64(adev);
+	tmp = RREG32_SOC15(GC, 0, regCP_ME_CNTL);
+	tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, CE_HALT,
+			    REG_GET_FIELD(me_cntl, CP_ME_CNTL, CE_HALT));
+	WREG32_SOC15(GC, 0, regCP_ME_CNTL, tmp);
+	WREG32_SOC15(GC, 0, regCP_PFP_IB_CONTROL, ib_control);
+	amdgpu_gfx_rlc_exit_safe_mode(adev, 0);
+
+	tmp = RREG32_SOC15(GC, 0, regCP_ME_CNTL);
+	tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, ME_HALT, 0);
+	tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, PFP_HALT, 0);
+	WREG32_SOC15(GC, 0, regCP_ME_CNTL, tmp);
+	for (i = 0; i < adev->usec_timeout; i++) {
+		if (!RREG32_SOC15(GC, 0, regCP_STAT))
+			return 0;
+		udelay(1);
+	}
+	return -ETIMEDOUT;
+}
+
+/*
+ * Reset the graphics engine: the CP front-end (CPF/CPG), PFP/ME and the gfx
+ * HQDs. MEC, MES and the compute queues keep running.
+ */
+static int gfx_v11_0_reset_engine(struct amdgpu_device *adev,
+				  enum amdgpu_gfx_engine engine)
+{
+	int i, r;
+
+	lockdep_assert_held(&adev->gfx.mec.reset_mutex);
+
+	if (engine != AMDGPU_GFX_ENGINE_GFX)
+		return -EOPNOTSUPP;
+
+	amdgpu_gfx_off_ctrl(adev, false);
+
+	r = gfx_v11_0_soft_reset_gfx_cp(adev);
+	if (r) {
+		dev_err(adev->dev, "gfx CP still busy after reset, CP_STAT 0x%08x\n",
+			RREG32_SOC15(GC, 0, regCP_STAT));
+		goto exit;
+	}
+
+	/* Map the kernel gfx queue again: the reset cleared its HQD. */
+	r = amdgpu_gfx_disable_kgq(adev, 0);
+	if (r)
+		goto exit;
+
+	for (i = 0; i < adev->gfx.num_gfx_rings; i++)
+		amdgpu_gfx_mqd_reset_restore(&adev->gfx.gfx_ring[i]);
+	r = amdgpu_gfx_enable_kgq(adev, 0);
+	if (r)
+		goto exit;
+
+	r = gfx_v11_0_cp_gfx_start(adev);
+
+exit:
+	amdgpu_gfx_off_ctrl(adev, true);
+	return r;
+}
+
 static int gfx_v11_0_reset_kgq(struct amdgpu_ring *ring,
 			       unsigned int vmid,
 			       struct amdgpu_fence *timedout_fence)
-- 
2.50.1

[PATCH v2 04/15] drm/amdgpu/gfx: add callbacks to read and restore the ring state of a gfx queue

[Jesse Zhang]

A gfx engine reset clears the HQD registers of the gfx queues, including
the only copy of the ring position of the queues MES didn't preempt.
MES doesn't save them in the MQD either when the engine hangs.

Add struct amdgpu_gfx_hqd_ptrs for that ring state, and gfx callbacks
for the gfx IP, which knows its registers and the layout of its MQD:
- read_gfx_hqd_ptrs() reads the ring state of the gfx HQD of a doorbell;
- restore_gfx_mqd_ring() puts it in the MQD after the reset, for MES to
  map the queue again from it.

Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h
index 8dc669e89db5..ba6dcea3f7db 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h
@@ -345,6 +345,17 @@ struct amdgpu_gfx_shadow_info {
 	u32 eop_alignment;
 };
 
+/* Ring state of a gfx HQD that a gfx engine reset clears. */
+struct amdgpu_gfx_hqd_ptrs {
+	u32	doorbell_index;
+	u64	mqd_addr;	/* the MQD loaded in the HQD */
+	bool	on_hqd;
+	u64	wptr;
+	u32	offset;
+	u32	csmd_rptr;
+	bool	connected;	/* the queue the pipe executes */
+};
+
 /* The engines of the GFX IP that can be reset on their own. */
 enum amdgpu_gfx_engine {
 	AMDGPU_GFX_ENGINE_GFX,
@@ -386,6 +397,16 @@ struct amdgpu_gfx_funcs {
 	/* reset one engine, leaving the other one running */
 	int (*reset_engine)(struct amdgpu_device *adev,
 			    enum amdgpu_gfx_engine engine);
+	/* read the ring state of the gfx HQD of a doorbell, if it has one */
+	bool (*read_gfx_hqd_ptrs)(struct amdgpu_device *adev, u32 doorbell_index,
+				  struct amdgpu_gfx_hqd_ptrs *ptrs);
+	/*
+	 * write the ring state read by read_gfx_hqd_ptrs() into a gfx MQD;
+	 * true if the work up to ptrs->wptr was dropped
+	 */
+	bool (*restore_gfx_mqd_ring)(struct amdgpu_device *adev, void *mqd,
+				     u32 ring_size,
+				     struct amdgpu_gfx_hqd_ptrs *ptrs);
 };
 
 struct sq_work {
-- 
2.50.1

[PATCH v2 05/15] drm/amdgpu/gfx12: read and restore the ring state of a gfx queue

[Jesse Zhang]

Implement the gfx callbacks to read and restore the ring state of a gfx
queue for gfx12:
- read_gfx_hqd_ptrs() reads the HQD offset, CSMD read pointer and write
  pointer of the gfx HQD of a doorbell, and whether the queue is the one
  connected to its pipe;
- restore_gfx_mqd_ring() puts them in the MQD after a reset, for MES to
  map the queue again where it was. A queue switched out of the pipe
  resumes from the preemption status the PFP saved in its MQD, also in
  the middle of an IB. The queue the pipe was executing has its state in
  the PFP, which the reset cleared: it restarts after the work submitted
  so far instead, and that work is dropped.

Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c | 85 ++++++++++++++++++++++++++
 1 file changed, 85 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
index dfc409e28914..b5f03f9a4ac3 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
@@ -276,6 +276,12 @@ static int gfx_v12_0_get_cu_info(struct amdgpu_device *adev,
 static uint64_t gfx_v12_0_get_gpu_clock_counter(struct amdgpu_device *adev);
 static int gfx_v12_0_reset_engine(struct amdgpu_device *adev,
 				  enum amdgpu_gfx_engine engine);
+static bool gfx_v12_0_read_gfx_hqd_ptrs(struct amdgpu_device *adev,
+					u32 doorbell_index,
+					struct amdgpu_gfx_hqd_ptrs *ptrs);
+static bool gfx_v12_0_restore_gfx_mqd_ring(struct amdgpu_device *adev, void *m,
+					   u32 ring_size,
+					   struct amdgpu_gfx_hqd_ptrs *ptrs);
 static void gfx_v12_0_select_se_sh(struct amdgpu_device *adev, u32 se_num,
 				   u32 sh_num, u32 instance, int xcc_id);
 static u32 gfx_v12_0_get_wgp_active_bitmap_per_sh(struct amdgpu_device *adev);
@@ -962,6 +968,8 @@ static const struct amdgpu_gfx_funcs gfx_v12_0_gfx_funcs = {
 	.get_gfx_shadow_info = &gfx_v12_0_get_gfx_shadow_info,
 	.get_hdp_flush_mask = &amdgpu_gfx_get_hdp_flush_mask,
 	.reset_engine = &gfx_v12_0_reset_engine,
+	.read_gfx_hqd_ptrs = &gfx_v12_0_read_gfx_hqd_ptrs,
+	.restore_gfx_mqd_ring = &gfx_v12_0_restore_gfx_mqd_ring,
 };
 
 static int gfx_v12_0_gpu_early_init(struct amdgpu_device *adev)
@@ -3973,6 +3981,83 @@ static int gfx_v12_0_reset_engine(struct amdgpu_device *adev,
 	return r;
 }
 
+#define GFX12_HPD_QUEUE_STATE_CONNECTED_MIN	3
+#define GFX12_HPD_QUEUE_STATE_CONNECTED_MAX	15
+
+static bool gfx_v12_0_read_gfx_hqd_ptrs(struct amdgpu_device *adev,
+					u32 doorbell_index,
+					struct amdgpu_gfx_hqd_ptrs *ptrs)
+{
+	u32 me, pipe, queue, status, state;
+
+	memset(ptrs, 0, sizeof(*ptrs));
+	ptrs->doorbell_index = doorbell_index;
+	if (!gfx_v12_0_detect_hung_queue(adev, doorbell_index, &me, &pipe, &queue))
+		return true;
+
+	amdgpu_gfx_off_ctrl(adev, false);
+	mutex_lock(&adev->srbm_mutex);
+	soc24_grbm_select(adev, me, pipe, queue, 0);
+	ptrs->on_hqd = RREG32_SOC15(GC, 0, regCP_GFX_HQD_ACTIVE) &
+		       CP_GFX_HQD_ACTIVE__ACTIVE_MASK;
+	if (!ptrs->on_hqd)
+		goto out;
+	ptrs->mqd_addr = RREG32_SOC15(GC, 0, regCP_GFX_MQD_BASE_ADDR) |
+			 (u64)RREG32_SOC15(GC, 0, regCP_GFX_MQD_BASE_ADDR_HI) << 32;
+	ptrs->offset = RREG32_SOC15(GC, 0, regCP_GFX_HQD_OFFSET);
+	ptrs->csmd_rptr = RREG32_SOC15(GC, 0, regCP_GFX_HQD_CSMD_RPTR);
+	ptrs->wptr = RREG32_SOC15(GC, 0, regCP_GFX_HQD_WPTR) |
+		     (u64)RREG32_SOC15(GC, 0, regCP_GFX_HQD_WPTR_HI) << 32;
+	status = RREG32_SOC15(GC, 0, regCP_GFX_HPD_STATUS0);
+	state = REG_GET_FIELD(status, CP_GFX_HPD_STATUS0, QUEUE_STATE);
+	ptrs->connected =
+		state >= GFX12_HPD_QUEUE_STATE_CONNECTED_MIN &&
+		state <= GFX12_HPD_QUEUE_STATE_CONNECTED_MAX &&
+		REG_GET_FIELD(status, CP_GFX_HPD_STATUS0, MAPPED_QUEUE) == queue;
+out:
+	soc24_grbm_select(adev, 0, 0, 0, 0);
+	mutex_unlock(&adev->srbm_mutex);
+	amdgpu_gfx_off_ctrl(adev, true);
+
+	return true;
+}
+
+static bool gfx_v12_0_restore_gfx_mqd_ring(struct amdgpu_device *adev, void *m,
+					   u32 ring_size,
+					   struct amdgpu_gfx_hqd_ptrs *ptrs)
+{
+	struct v12_gfx_mqd *mqd = m;
+	u32 wptr = lower_32_bits(ptrs->wptr) & ((ring_size >> 2) - 1);
+	u32 *status;
+
+	/*
+	 * MQD dwords 32-41 (reserved_32..41) are the PFP preemption status,
+	 * mcbp_status_pfp: the offset and chained IB size and address of IB1
+	 * (32-35) and IB2 (36-39), rb_preempt (40: preempted in the ring) and
+	 * completion_status (41: 0 IB not started, 1 preempted inside an IB,
+	 * 2 IB completed, 3 IB resumed). The PFP saves it when the queue is
+	 * switched out of the pipe and resumes from it when the queue runs
+	 * again, at the HQD offset. That survives the reset for a queue that
+	 * was switched out. The queue the pipe executed has its state in the
+	 * PFP, which the reset cleared: restart it at the write pointer with
+	 * no IB to resume, and drop its work in flight.
+	 */
+	if (ptrs->connected) {
+		mqd->cp_gfx_hqd_offset = wptr;
+		mqd->cp_gfx_hqd_csmd_rptr = wptr;
+		for (status = &mqd->reserved_32; status <= &mqd->reserved_41; status++)
+			*status = 0;
+	} else {
+		mqd->cp_gfx_hqd_offset = ptrs->offset;
+		mqd->cp_gfx_hqd_csmd_rptr = ptrs->csmd_rptr;
+	}
+	mqd->cp_gfx_hqd_wptr = lower_32_bits(ptrs->wptr);
+	mqd->cp_gfx_hqd_wptr_hi = upper_32_bits(ptrs->wptr);
+	mqd->cp_gfx_hqd_active = 0;
+
+	return ptrs->connected;
+}
+
 static uint64_t gfx_v12_0_get_gpu_clock_counter(struct amdgpu_device *adev)
 {
 	uint64_t clock = 0;
-- 
2.50.1

[PATCH v2 06/15] drm/amdgpu/gfx11: read and restore the ring state of a gfx queue

[Jesse Zhang]

Implement the gfx callbacks to read and restore the ring state of a gfx
queue for gfx11:
- read_gfx_hqd_ptrs() reads the HQD offset, CSMD read pointer and write
  pointer of the gfx HQD of a doorbell, and whether the queue is the one
  connected to its pipe;
- restore_gfx_mqd_ring() puts them in the MQD after a reset, for MES to
  map the queue again where it was. A queue switched out of the pipe
  resumes from the preemption status the PFP saved in its MQD, also in
  the middle of an IB. The queue the pipe was executing has its state in
  the PFP, which the reset cleared: it restarts after the work submitted
  so far instead, and that work is dropped.

Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c | 85 ++++++++++++++++++++++++++
 1 file changed, 85 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
index 12b307e4106b..a2eaf416b12a 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
@@ -365,6 +365,12 @@ static void gfx_v11_0_unset_safe_mode(struct amdgpu_device *adev, int xcc_id);
 static void gfx_v11_0_userq_priv_fault_work(struct work_struct *work);
 static int gfx_v11_0_reset_engine(struct amdgpu_device *adev,
 				  enum amdgpu_gfx_engine engine);
+static bool gfx_v11_0_read_gfx_hqd_ptrs(struct amdgpu_device *adev,
+					u32 doorbell_index,
+					struct amdgpu_gfx_hqd_ptrs *ptrs);
+static bool gfx_v11_0_restore_gfx_mqd_ring(struct amdgpu_device *adev, void *m,
+					   u32 ring_size,
+					   struct amdgpu_gfx_hqd_ptrs *ptrs);
 static void gfx_v11_0_update_perf_clk(struct amdgpu_device *adev,
 				      bool enable);
 
@@ -1153,6 +1159,8 @@ static const struct amdgpu_gfx_funcs gfx_v11_0_gfx_funcs = {
 	.get_hdp_flush_mask = &amdgpu_gfx_get_hdp_flush_mask,
 	.detect_hung_queue = &gfx_v11_0_detect_hung_queue,
 	.reset_engine = &gfx_v11_0_reset_engine,
+	.read_gfx_hqd_ptrs = &gfx_v11_0_read_gfx_hqd_ptrs,
+	.restore_gfx_mqd_ring = &gfx_v11_0_restore_gfx_mqd_ring,
 };
 
 static int gfx_v11_0_gpu_early_init(struct amdgpu_device *adev)
@@ -7092,6 +7100,83 @@ static int gfx_v11_0_reset_engine(struct amdgpu_device *adev,
 	return r;
 }
 
+#define GFX11_HPD_QUEUE_STATE_CONNECTED_MIN	3
+#define GFX11_HPD_QUEUE_STATE_CONNECTED_MAX	15
+
+static bool gfx_v11_0_read_gfx_hqd_ptrs(struct amdgpu_device *adev,
+					u32 doorbell_index,
+					struct amdgpu_gfx_hqd_ptrs *ptrs)
+{
+	u32 me, pipe, queue, status, state;
+
+	memset(ptrs, 0, sizeof(*ptrs));
+	ptrs->doorbell_index = doorbell_index;
+	if (!gfx_v11_0_detect_hung_queue(adev, doorbell_index, &me, &pipe, &queue))
+		return true;
+
+	amdgpu_gfx_off_ctrl(adev, false);
+	mutex_lock(&adev->srbm_mutex);
+	soc21_grbm_select(adev, me, pipe, queue, 0);
+	ptrs->on_hqd = RREG32_SOC15(GC, 0, regCP_GFX_HQD_ACTIVE) &
+		       CP_GFX_HQD_ACTIVE__ACTIVE_MASK;
+	if (!ptrs->on_hqd)
+		goto out;
+	ptrs->mqd_addr = RREG32_SOC15(GC, 0, regCP_GFX_MQD_BASE_ADDR) |
+			 (u64)RREG32_SOC15(GC, 0, regCP_GFX_MQD_BASE_ADDR_HI) << 32;
+	ptrs->offset = RREG32_SOC15(GC, 0, regCP_GFX_HQD_OFFSET);
+	ptrs->csmd_rptr = RREG32_SOC15(GC, 0, regCP_GFX_HQD_CSMD_RPTR);
+	ptrs->wptr = RREG32_SOC15(GC, 0, regCP_GFX_HQD_WPTR) |
+		     (u64)RREG32_SOC15(GC, 0, regCP_GFX_HQD_WPTR_HI) << 32;
+	status = RREG32_SOC15(GC, 0, regCP_GFX_HPD_STATUS0);
+	state = REG_GET_FIELD(status, CP_GFX_HPD_STATUS0, QUEUE_STATE);
+	ptrs->connected =
+		state >= GFX11_HPD_QUEUE_STATE_CONNECTED_MIN &&
+		state <= GFX11_HPD_QUEUE_STATE_CONNECTED_MAX &&
+		REG_GET_FIELD(status, CP_GFX_HPD_STATUS0, MAPPED_QUEUE) == queue;
+out:
+	soc21_grbm_select(adev, 0, 0, 0, 0);
+	mutex_unlock(&adev->srbm_mutex);
+	amdgpu_gfx_off_ctrl(adev, true);
+
+	return true;
+}
+
+static bool gfx_v11_0_restore_gfx_mqd_ring(struct amdgpu_device *adev, void *m,
+					   u32 ring_size,
+					   struct amdgpu_gfx_hqd_ptrs *ptrs)
+{
+	struct v11_gfx_mqd *mqd = m;
+	u32 wptr = lower_32_bits(ptrs->wptr) & ((ring_size >> 2) - 1);
+	u32 *status;
+
+	/*
+	 * MQD dwords 32-41 (reserved_32..41) are the PFP preemption status,
+	 * mcbp_status_pfp: the offset and chained IB size and address of IB1
+	 * (32-35) and IB2 (36-39), rb_preempt (40: preempted in the ring) and
+	 * completion_status (41: 0 IB not started, 1 preempted inside an IB,
+	 * 2 IB completed, 3 IB resumed). The PFP saves it when the queue is
+	 * switched out of the pipe and resumes from it when the queue runs
+	 * again, at the HQD offset. That survives the reset for a queue that
+	 * was switched out. The queue the pipe executed has its state in the
+	 * PFP, which the reset cleared: restart it at the write pointer with
+	 * no IB to resume, and drop its work in flight.
+	 */
+	if (ptrs->connected) {
+		mqd->cp_gfx_hqd_offset = wptr;
+		mqd->cp_gfx_hqd_csmd_rptr = wptr;
+		for (status = &mqd->reserved_32; status <= &mqd->reserved_41; status++)
+			*status = 0;
+	} else {
+		mqd->cp_gfx_hqd_offset = ptrs->offset;
+		mqd->cp_gfx_hqd_csmd_rptr = ptrs->csmd_rptr;
+	}
+	mqd->cp_gfx_hqd_wptr = lower_32_bits(ptrs->wptr);
+	mqd->cp_gfx_hqd_wptr_hi = upper_32_bits(ptrs->wptr);
+	mqd->cp_gfx_hqd_active = 0;
+
+	return ptrs->connected;
+}
+
 static int gfx_v11_0_reset_kgq(struct amdgpu_ring *ring,
 			       unsigned int vmid,
 			       struct amdgpu_fence *timedout_fence)
-- 
2.50.1

[PATCH v2 07/15] drm/amdgpu/userq: add a helper to signal the fences of a queue up to a point

[Jesse Zhang]

The fence value of a user queue is the write pointer of the queue when
the work was submitted. amdgpu_userq_fence_driver_force_completion()
signals all the fences of a queue, as after a reset that dropped all its
work.

Add amdgpu_userq_fence_driver_force_completion_with_seqno(), which only
signals the fences of the work up to a write pointer, with an error, for
a reset that drops only part of the work of a queue: the work submitted
after that write pointer still runs.

Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
---
 .../gpu/drm/amd/amdgpu/amdgpu_userq_fence.c   | 31 +++++++++++++++++++
 .../gpu/drm/amd/amdgpu/amdgpu_userq_fence.h   |  2 ++
 2 files changed, 33 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c
index a2fa59a9609e..e76249c43b7a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c
@@ -459,6 +459,37 @@ amdgpu_userq_fence_driver_force_completion(struct amdgpu_usermode_queue *userq)
 	}
 }
 
+/*
+ * Signal the fences of @userq with a seqno up to @seqno with -ECANCELED, eg.
+ * when a reset dropped the work that would have signaled them.
+ */
+void
+amdgpu_userq_fence_driver_force_completion_with_seqno(struct amdgpu_usermode_queue *userq,
+						      u64 seqno)
+{
+	struct amdgpu_userq_fence_driver *fence_drv = userq->fence_drv;
+	struct amdgpu_userq_fence *fence;
+	unsigned long flags;
+
+	if (!fence_drv)
+		return;
+
+	spin_lock_irqsave(&fence_drv->fence_list_lock, flags);
+	list_for_each_entry(fence, &fence_drv->fences, link) {
+		if (fence->base.seqno > seqno)
+			break;
+		spin_lock(dma_fence_spinlock(&fence->base));
+		if (!dma_fence_is_signaled_locked(&fence->base))
+			dma_fence_set_error(&fence->base, -ECANCELED);
+		spin_unlock(dma_fence_spinlock(&fence->base));
+	}
+	if (amdgpu_userq_fence_read(fence_drv) < seqno)
+		amdgpu_userq_fence_write(fence_drv, seqno);
+	spin_unlock_irqrestore(&fence_drv->fence_list_lock, flags);
+
+	amdgpu_userq_fence_driver_process(fence_drv);
+}
+
 int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data,
 			      struct drm_file *filp)
 {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.h
index 0bd51616cef1..0407c8c2575f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.h
@@ -65,6 +65,8 @@ int amdgpu_userq_fence_driver_alloc(struct amdgpu_device *adev,
 void amdgpu_userq_fence_driver_free(struct amdgpu_usermode_queue *userq);
 int amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_drv);
 void amdgpu_userq_fence_driver_force_completion(struct amdgpu_usermode_queue *userq);
+void amdgpu_userq_fence_driver_force_completion_with_seqno(struct amdgpu_usermode_queue *userq,
+							   u64 seqno);
 void amdgpu_userq_fence_driver_destroy(struct kref *ref);
 int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data,
 			      struct drm_file *filp);
-- 
2.50.1

[PATCH v2 08/15] drm/amdgpu/userq: add helpers to keep the user queues of one engine across a reset

[Jesse Zhang]

An engine reset, eg. of the gfx engine, only resets the queues of one
engine of the GFX IP, while the queues of the other engines keep
running. amdgpu_userq_pre_reset() and amdgpu_userq_post_reset() handle
all the user queues for a full reset. Add separate helpers for the
queues of one engine, which keep them in the state they had:

- amdgpu_userq_pre_engine_reset(), called after the MES suspend, saves
  the ring state of the mapped queues still on a HQD: the reset clears
  their HQD, and MES couldn't preempt them into their MQD. The queues
  MES preempted have their state in their MQD already.
- amdgpu_userq_post_engine_reset(), called after the reset but before
  MES resumes, unmaps those queues, puts the saved ring state in their
  MQD and maps them again: MES doesn't load a queue whose HQD the reset
  cleared on its own, and the unmap may save the cleared HQD into the
  MQD. The queues keep their state and their work, except the one the
  pipe executed: its work in flight depends on pipe state the reset
  cleared, it is dropped and its fences signaled. The queue that hung,
  if any, is unmapped and hung.
- amdgpu_userq_cancel_engine_reset() drops the saved ring state for the
  full reset that follows a failed engine reset.

The helpers walk the queues and unmap and map them as
amdgpu_userq_pre_reset() and amdgpu_userq_post_reset() do. The saved
ring state is kept in the gfx IP, per HQD, and matched to a queue by its
doorbell and MQD address.

Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h   |   3 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c | 109 ++++++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_userq.h |   5 +
 3 files changed, 117 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h
index ba6dcea3f7db..5e9338563de3 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h
@@ -524,6 +524,9 @@ struct amdgpu_gfx {
 
 	/* reset mask */
 	uint32_t 			gfx_supported_reset;
+	/* ring state of the gfx user queues on a HQD, saved by an engine reset */
+	struct amdgpu_gfx_hqd_ptrs	engine_reset_hqds[AMDGPU_MAX_GFX_QUEUES];
+	u32				num_engine_reset_hqds;
 	uint32_t 			compute_supported_reset;
 
 	/* gfx off */
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
index 1a2aa85c504e..2c18175a6041 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
@@ -1597,3 +1597,112 @@ int amdgpu_userq_post_reset(struct amdgpu_device *adev, bool vram_lost)
 
 	return ret;
 }
+
+/**
+ * amdgpu_userq_pre_engine_reset - save the ring state of the user queues
+ * @adev: amdgpu device
+ * @queue_type: AMDGPU_HW_IP_* type of the queues on the reset engine
+ */
+void amdgpu_userq_pre_engine_reset(struct amdgpu_device *adev, u32 queue_type)
+{
+	struct amdgpu_gfx_hqd_ptrs ptrs;
+	struct amdgpu_usermode_queue *queue;
+	unsigned long queue_id;
+
+	adev->gfx.num_engine_reset_hqds = 0;
+	if (!adev->gfx.funcs->read_gfx_hqd_ptrs)
+		return;
+
+	xa_for_each(&adev->userq_doorbell_xa, queue_id, queue) {
+		if (adev->gfx.num_engine_reset_hqds >= AMDGPU_MAX_GFX_QUEUES)
+			break;
+		if (queue->queue_type != queue_type ||
+		    queue->state != AMDGPU_USERQ_STATE_MAPPED)
+			continue;
+		if (adev->gfx.funcs->read_gfx_hqd_ptrs(adev, queue->doorbell_index,
+						       &ptrs) && ptrs.on_hqd)
+			adev->gfx.engine_reset_hqds[adev->gfx.num_engine_reset_hqds++] = ptrs;
+	}
+}
+
+static struct amdgpu_gfx_hqd_ptrs *
+amdgpu_userq_engine_reset_hqd(struct amdgpu_device *adev,
+			      struct amdgpu_usermode_queue *queue)
+{
+	struct amdgpu_gfx_hqd_ptrs *ptrs;
+	u32 i;
+
+	for (i = 0; i < adev->gfx.num_engine_reset_hqds; i++) {
+		ptrs = &adev->gfx.engine_reset_hqds[i];
+		/* Check the MQD too: a new queue may reuse the doorbell. */
+		if (ptrs->doorbell_index == queue->doorbell_index &&
+		    ptrs->mqd_addr == queue->mqd.gpu_addr)
+			return ptrs;
+	}
+	return NULL;
+}
+
+/**
+ * amdgpu_userq_post_engine_reset - restore the user queues after an engine reset
+ * @adev: amdgpu device
+ * @queue_type: AMDGPU_HW_IP_* type of the queues on the reset engine
+ * @guilty: the queue that hung, or NULL
+ *
+ * MES doesn't load a queue whose HQD the reset cleared again: unmap it, which
+ * may save the cleared HQD into its MQD, restore its MQD, and map it again.
+ */
+void amdgpu_userq_post_engine_reset(struct amdgpu_device *adev, u32 queue_type,
+				    struct amdgpu_usermode_queue *guilty)
+{
+	const struct amdgpu_userq_funcs *userq_funcs = adev->userq_funcs[queue_type];
+	struct amdgpu_usermode_queue *queue;
+	struct amdgpu_gfx_hqd_ptrs *ptrs;
+	unsigned long queue_id;
+	int r;
+
+	xa_for_each(&adev->userq_doorbell_xa, queue_id, queue) {
+		if (queue->queue_type != queue_type ||
+		    queue->state != AMDGPU_USERQ_STATE_MAPPED)
+			continue;
+
+		if (queue == guilty) {
+			userq_funcs->unmap(queue);
+			amdgpu_userq_fence_driver_force_completion(queue);
+			trace_amdgpu_userq_state_changed(queue, AMDGPU_USERQ_STATE_HUNG);
+			queue->state = AMDGPU_USERQ_STATE_HUNG;
+			continue;
+		}
+
+		ptrs = amdgpu_userq_engine_reset_hqd(adev, queue);
+		if (!ptrs)
+			continue;
+
+		/* Unmapping may write the cleared HQD into the MQD. */
+		userq_funcs->unmap(queue);
+		if (adev->gfx.funcs->restore_gfx_mqd_ring(adev, queue->mqd.cpu_ptr,
+							  queue->userq_prop->queue_size,
+							  ptrs))
+			amdgpu_userq_fence_driver_force_completion_with_seqno(queue,
+									      ptrs->wptr);
+		r = userq_funcs->map(queue);
+		if (r) {
+			dev_err(adev->dev, "Failed to remap user queue %llu: %d\n",
+				queue->doorbell_index, r);
+			amdgpu_userq_fence_driver_force_completion(queue);
+			trace_amdgpu_userq_state_changed(queue, AMDGPU_USERQ_STATE_HUNG);
+			queue->state = AMDGPU_USERQ_STATE_HUNG;
+		}
+	}
+	adev->gfx.num_engine_reset_hqds = 0;
+}
+
+/**
+ * amdgpu_userq_cancel_engine_reset - give up an engine reset
+ * @adev: amdgpu device
+ * @queue_type: AMDGPU_HW_IP_* type of the queues on the reset engine
+ */
+void amdgpu_userq_cancel_engine_reset(struct amdgpu_device *adev,
+				      u32 queue_type)
+{
+	adev->gfx.num_engine_reset_hqds = 0;
+}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.h
index 488dc21d7c81..e07c71cd5af5 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.h
@@ -181,6 +181,11 @@ int amdgpu_userq_start_sched_for_enforce_isolation(struct amdgpu_device *adev,
 void amdgpu_userq_reset_work(struct work_struct *work);
 void amdgpu_userq_pre_reset(struct amdgpu_device *adev);
 int amdgpu_userq_post_reset(struct amdgpu_device *adev, bool vram_lost);
+void amdgpu_userq_pre_engine_reset(struct amdgpu_device *adev, u32 queue_type);
+void amdgpu_userq_post_engine_reset(struct amdgpu_device *adev, u32 queue_type,
+				    struct amdgpu_usermode_queue *guilty);
+void amdgpu_userq_cancel_engine_reset(struct amdgpu_device *adev,
+				      u32 queue_type);
 void amdgpu_userq_start_hang_detect_work(struct amdgpu_usermode_queue *queue);
 void amdgpu_userq_process_fence_irq(struct amdgpu_device *adev, u32 doorbell);
 
-- 
2.50.1

[PATCH v2 09/15] drm/amdgpu/gfx: add a gfx engine reset helper

[Jesse Zhang]

Add amdgpu_gfx_reset_gfx_engine() around the reset_engine callback, and
AMDGPU_RESET_TYPE_PER_ENGINE for a reset of a single engine of an IP
block, eg. only the graphics engine of the GFX IP, that leaves its other
engines running. The helper only resets the engine when the IP reports
that reset type, which none does yet.

As for a pipe reset, the other kernel gfx rings are backed up, tested
and replayed, and MES is suspended across the reset, so that it doesn't
schedule the queues of the compute engine meanwhile. Their work is only
paused.

MES fetches its commands through the CP front-end though, and a hung gfx
stream can keep it from completing the suspend. Reset the engine anyway:
the reset frees the front-end.

It only takes the reset domain for read. That keeps full resets away
while the compute engine keeps working. KFD, for example, adds and
removes MES queues only after a read trylock of the reset domain, and
taking it for write would fail those calls with -EIO.

The reset also clears the HQDs of the gfx user queues. Call
amdgpu_userq_pre_engine_reset() after the MES suspend, to save the ring
state of the queues still on a HQD, and amdgpu_userq_post_engine_reset()
after the reset, before MES resumes, to restore them. @guilty_queue is
the gfx user queue that hung, if any: it is hung instead. If the engine
reset fails, amdgpu_userq_cancel_engine_reset() drops the saved ring
state for the full reset that follows.

Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu.h     |  1 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c | 83 +++++++++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h |  4 ++
 3 files changed, 88 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index c32d3b0b4153..da99fa2f1b87 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -309,6 +309,7 @@ extern uint amdgpu_hdmi_hpd_debounce_delay_ms;
 #define AMDGPU_RESET_TYPE_PER_QUEUE (1 << 2) /* per queue */
 #define AMDGPU_RESET_TYPE_PER_PIPE (1 << 3) /* per pipe */
 #define AMDGPU_RESET_TYPE_IP_BLOCK_SOFT_RESET (1 << 4) /* soft-resets an IP block */
+#define AMDGPU_RESET_TYPE_PER_ENGINE (1 << 5) /* resets the engine of one ring type */
 
 /* max cursor sizes (in pixels) */
 #define CIK_CURSOR_WIDTH 128
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
index 0ae6e990c310..fecf2c02973a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
@@ -2181,6 +2181,89 @@ int amdgpu_gfx_reset_mes_gfx(struct amdgpu_device *adev,
 	return r;
 }
 
+/**
+ * amdgpu_gfx_reset_gfx_engine() - Reset the graphics engine.
+ *
+ * @adev: amdgpu device
+ * @guilty_ring: The kernel gfx ring that hung, or NULL for a user queue.
+ * @guilty_fence: The fence which didn't signal on @guilty_ring.
+ * @guilty_queue: The gfx user queue that hung, or NULL for a kernel ring.
+ *
+ * Reset only the graphics engine, when a gfx queue or pipe reset failed.
+ */
+int amdgpu_gfx_reset_gfx_engine(struct amdgpu_device *adev,
+				struct amdgpu_ring *guilty_ring,
+				struct amdgpu_fence *guilty_fence,
+				struct amdgpu_usermode_queue *guilty_queue)
+{
+	struct amdgpu_gfx_pipe_reset_ctx ctx;
+	struct amdgpu_ring *ring;
+	int i, r;
+
+	if (!(adev->gfx.gfx_supported_reset & AMDGPU_RESET_TYPE_PER_ENGINE) ||
+	    !adev->gfx.funcs->reset_engine)
+		return -EOPNOTSUPP;
+
+	dev_err(adev->dev, "Starting gfx engine reset\n");
+
+	/*
+	 * Keep a full GPU reset out. Only for read: KFD can still add and
+	 * remove its queues meanwhile.
+	 */
+	down_read(&adev->reset_domain->sem);
+	mutex_lock(&adev->gfx.mec.reset_mutex);
+
+	amdgpu_gfx_pipe_reset_prepare(adev, guilty_ring, U32_MAX, U32_MAX, &ctx);
+
+	/* The suspend may time out while the gfx engine hangs: reset anyway. */
+	amdgpu_mes_suspend(adev, 0);
+	amdgpu_userq_pre_engine_reset(adev, AMDGPU_HW_IP_GFX);
+	r = adev->gfx.funcs->reset_engine(adev, AMDGPU_GFX_ENGINE_GFX);
+	if (r) {
+		amdgpu_userq_cancel_engine_reset(adev, AMDGPU_HW_IP_GFX);
+		amdgpu_mes_resume(adev, 0);
+		goto exit;
+	}
+
+	amdgpu_userq_post_engine_reset(adev, AMDGPU_HW_IP_GFX, guilty_queue);
+	r = amdgpu_mes_resume(adev, 0);
+	if (r)
+		goto exit;
+
+	if (guilty_ring) {
+		r = amdgpu_ring_reset_helper_end(guilty_ring, guilty_fence);
+		if (r)
+			goto exit;
+	}
+
+	for (i = 0; i < adev->gfx.num_gfx_rings; i++) {
+		ring = &adev->gfx.gfx_ring[i];
+		if (ring == guilty_ring || !amdgpu_ring_sched_ready(ring))
+			continue;
+		r = amdgpu_ring_reset_helper_end(ring, NULL);
+		if (r)
+			goto exit;
+	}
+
+	/* Restart the schedulers stopped by amdgpu_gfx_pipe_reset_prepare(). */
+	for (i = 0; i < adev->gfx.num_gfx_rings; i++) {
+		ring = &adev->gfx.gfx_ring[i];
+		if (ring != guilty_ring && amdgpu_ring_sched_ready(ring) &&
+		    drm_sched_is_stopped(&ring->sched))
+			drm_sched_wqueue_start(&ring->sched);
+	}
+
+exit:
+	mutex_unlock(&adev->gfx.mec.reset_mutex);
+	up_read(&adev->reset_domain->sem);
+
+	if (r)
+		dev_err(adev->dev, "Failed gfx engine reset: %d\n", r);
+	else
+		dev_err(adev->dev, "Successful gfx engine reset\n");
+	return r;
+}
+
 int amdgpu_gfx_mes_reset_queue(struct amdgpu_ring *ring,
 			       unsigned int vmid,
 			       struct amdgpu_fence *timedout_fence,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h
index 5e9338563de3..007da939d135 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h
@@ -751,6 +751,10 @@ int amdgpu_gfx_reset_mes_gfx(struct amdgpu_device *adev,
 			     struct amdgpu_fence *guilty_fence,
 			     struct amdgpu_usermode_queue *queue,
 			     unsigned int vmid, bool use_mmio);
+int amdgpu_gfx_reset_gfx_engine(struct amdgpu_device *adev,
+				struct amdgpu_ring *guilty_ring,
+				struct amdgpu_fence *guilty_fence,
+				struct amdgpu_usermode_queue *guilty_queue);
 
 static inline const char *amdgpu_gfx_compute_mode_desc(int mode)
 {
-- 
2.50.1

[PATCH v2 10/15] drm/amdgpu/gfx: reset the gfx engine when a kernel gfx queue reset fails

[Jesse Zhang]

When neither the queue nor the pipe reset recovered a kernel gfx queue,
amdgpu_gfx_mes_reset_queue() now falls back to a gfx engine reset, if
the IP supports it. The pipe reset already escalates there, so the job
timeout handler doesn't change.

Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
index fecf2c02973a..e18a9ba6d147 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
@@ -2275,13 +2275,17 @@ int amdgpu_gfx_mes_reset_queue(struct amdgpu_ring *ring,
 	if (!amdgpu_gfx_me_pipe_reset_supported(adev)) {
 		r = amdgpu_gfx_mes_reset_queue_start(ring, vmid, timedout_fence,
 						     use_mmio);
-		if (r)
-			return r;
-		return amdgpu_ring_reset_helper_end(ring, timedout_fence);
+		if (!r)
+			r = amdgpu_ring_reset_helper_end(ring, timedout_fence);
+	} else {
+		r = amdgpu_gfx_reset_mes_gfx(adev, ring, timedout_fence, NULL,
+					     vmid, use_mmio);
 	}
 
-	return amdgpu_gfx_reset_mes_gfx(adev, ring, timedout_fence, NULL, vmid,
-				      use_mmio);
+	if (r && ring->funcs->type == AMDGPU_RING_TYPE_GFX &&
+	    adev->gfx.funcs->reset_engine)
+		r = amdgpu_gfx_reset_gfx_engine(adev, ring, timedout_fence, NULL);
+	return r;
 }
 
 static DEVICE_ATTR(run_cleaner_shader, 0200,
-- 
2.50.1

[PATCH v2 11/15] drm/amdgpu/userq: reset the gfx engine when a gfx user queue reset fails

[Jesse Zhang]

When neither the queue nor the pipe reset recovered a gfx user queue,
fall back to a gfx engine reset before a full reset, as for the kernel
gfx queues.

The hung queue can't be unmapped before the reset: pass it to the engine
reset, which unmaps it once the reset cleared its HQD, and hangs it,
while the other gfx user queues keep their state.

Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/mes_userqueue.c | 22 +++++++++++++++++++---
 1 file changed, 19 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/mes_userqueue.c b/drivers/gpu/drm/amd/amdgpu/mes_userqueue.c
index 7abb9d95ef59..d9be93677ff3 100644
--- a/drivers/gpu/drm/amd/amdgpu/mes_userqueue.c
+++ b/drivers/gpu/drm/amd/amdgpu/mes_userqueue.c
@@ -235,11 +235,19 @@ int mes_userq_reset_hw(struct amdgpu_usermode_queue *queue,
 	return 0;
 }
 
+/* The hung queue can't be unmapped now: the engine reset unmaps it after. */
+static int mes_userq_reset_gfx_engine(struct amdgpu_usermode_queue *queue)
+{
+	return amdgpu_gfx_reset_gfx_engine(queue->userq_mgr->adev, NULL, NULL,
+					   queue);
+}
+
 int mes_userq_reset(struct amdgpu_usermode_queue *queue)
 {
 	struct amdgpu_userq_mgr *uq_mgr = queue->userq_mgr;
 	struct amdgpu_device *adev = uq_mgr->adev;
 	struct mes_reset_queue_input queue_input;
+	int r;
 
 	/* already reset by an earlier job's hang-detect; just signal and bail */
 	if (queue->state == AMDGPU_USERQ_STATE_HUNG)
@@ -248,8 +256,12 @@ int mes_userq_reset(struct amdgpu_usermode_queue *queue)
 	/* A GFX pipe reset also affects the kernel rings on that pipe. */
 	if (queue->queue_type == AMDGPU_HW_IP_GFX &&
 	    adev->gfx.funcs->detect_hung_queue &&
-	    amdgpu_gfx_me_pipe_reset_supported(adev))
-		return amdgpu_gfx_reset_mes_gfx(adev, NULL, NULL, queue, 0, true);
+	    amdgpu_gfx_me_pipe_reset_supported(adev)) {
+		r = amdgpu_gfx_reset_mes_gfx(adev, NULL, NULL, queue, 0, true);
+		if (r && adev->gfx.funcs->reset_engine)
+			r = mes_userq_reset_gfx_engine(queue);
+		return r;
+	}
 
 	memset(&queue_input, 0x0, sizeof(struct mes_reset_queue_input));
 	queue_input.doorbell_offset = queue->doorbell_index;
@@ -277,7 +289,11 @@ int mes_userq_reset(struct amdgpu_usermode_queue *queue)
 		}
 	}
 
-	return mes_userq_reset_hw(queue, &queue_input);
+	r = mes_userq_reset_hw(queue, &queue_input);
+	if (r && queue->queue_type == AMDGPU_HW_IP_GFX &&
+	    adev->gfx.funcs->reset_engine)
+		r = mes_userq_reset_gfx_engine(queue);
+	return r;
 }
 
 int mes_userq_reset_queue(struct amdgpu_device *adev,
-- 
2.50.1

[PATCH v2 12/15] drm/amdgpu: show the engine reset in the reset masks

[Jesse Zhang]

Show AMDGPU_RESET_TYPE_PER_ENGINE as "engine" in the reset mask sysfs
files, eg. gfx_reset_mask.

Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index b74f9136aa2e..a09254a47373 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -6831,6 +6831,9 @@ ssize_t amdgpu_show_reset_mask(char *buf, uint32_t supported_reset)
 	if (supported_reset & AMDGPU_RESET_TYPE_PER_PIPE)
 		size += sysfs_emit_at(buf, size, "pipe ");
 
+	if (supported_reset & AMDGPU_RESET_TYPE_PER_ENGINE)
+		size += sysfs_emit_at(buf, size, "engine ");
+
 	if (supported_reset & AMDGPU_RESET_TYPE_FULL)
 		size += sysfs_emit_at(buf, size, "full ");
 
-- 
2.50.1

[PATCH v2 13/15] drm/amdgpu: add a debug_mask bit to disable the engine reset

[Jesse Zhang]

Add amdgpu.debug_mask=0x4000 to keep the IPs from reporting
AMDGPU_RESET_TYPE_PER_ENGINE, eg. to test the recovery without the
engine reset.

Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu.h     | 1 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 6 ++++++
 2 files changed, 7 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index da99fa2f1b87..69ad6be01ef3 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -951,6 +951,7 @@ struct amdgpu_device {
 	bool                            debug_enable_ce_cs;
 	bool                            debug_hibernation_thaw_resume_gpu;
 	bool                            debug_disable_ip_block_soft_reset;
+	bool                            debug_disable_engine_reset;
 
 	/* Protection for the following isolation structure */
 	struct mutex                    enforce_isolation_mutex;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index 3e5634de077f..43d3e7b7ef03 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -150,6 +150,7 @@ enum AMDGPU_DEBUG_MASK {
 	AMDGPU_DEBUG_HIBERNATION_THAW_RESUME_GPU = BIT(11),
 	AMDGPU_DEBUG_DISABLE_IP_BLOCK_SOFT_RESET = BIT(12),
 	AMDGPU_DEBUG_SDMA_RB_CMD = BIT(13),
+	AMDGPU_DEBUG_DISABLE_ENGINE_RESET = BIT(14),
 };
 
 unsigned int amdgpu_vram_limit = UINT_MAX;
@@ -2341,6 +2342,11 @@ static void amdgpu_init_debug_options(struct amdgpu_device *adev)
 		adev->debug_disable_ip_block_soft_reset = true;
 	}
 
+	if (amdgpu_debug_mask & AMDGPU_DEBUG_DISABLE_ENGINE_RESET) {
+		pr_info("debug: engine reset disabled\n");
+		adev->debug_disable_engine_reset = true;
+	}
+
 	if (amdgpu_debug_mask & AMDGPU_DEBUG_SDMA_RB_CMD) {
 		pr_info("debug: enable SDMA RB command switch\n");
 		adev->sdma.sdma_debug = true;
-- 
2.50.1

[PATCH v2 14/15] drm/amdgpu/gfx12: enable the gfx engine reset

[Jesse Zhang]

Report AMDGPU_RESET_TYPE_PER_ENGINE for the graphics engine when the
reset_engine callback can be used: with MES and async gfx rings. The
reset remaps the kernel gfx queue through MES from its MQD, and without
async gfx rings the kernel gfx queue has no MQD. It is programmed
through the CP_RB0 registers, which the reset clears. Not on SR-IOV VFs.

The kernel gfx queue and gfx user queue resets then fall back to a gfx
engine reset before a full reset.

Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
index b5f03f9a4ac3..a3dc05532316 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
@@ -274,6 +274,7 @@ static void gfx_v12_0_set_imu_funcs(struct amdgpu_device *adev);
 static int gfx_v12_0_get_cu_info(struct amdgpu_device *adev,
 				 struct amdgpu_cu_info *cu_info);
 static uint64_t gfx_v12_0_get_gpu_clock_counter(struct amdgpu_device *adev);
+static bool gfx_v12_0_engine_reset_supported(struct amdgpu_device *adev);
 static int gfx_v12_0_reset_engine(struct amdgpu_device *adev,
 				  enum amdgpu_gfx_engine engine);
 static bool gfx_v12_0_read_gfx_hqd_ptrs(struct amdgpu_device *adev,
@@ -1594,6 +1595,10 @@ static int gfx_v12_0_sw_init(struct amdgpu_ip_block *ip_block)
 		break;
 	}
 
+	if (gfx_v12_0_engine_reset_supported(adev) &&
+	    !adev->debug_disable_engine_reset)
+		adev->gfx.gfx_supported_reset |= AMDGPU_RESET_TYPE_PER_ENGINE;
+
 	if (!adev->enable_mes_kiq) {
 		r = amdgpu_gfx_kiq_init(adev, GFX12_MEC_HPD_SIZE, 0);
 		if (r) {
@@ -3855,6 +3860,12 @@ static int gfx_v12_0_wait_for_idle(struct amdgpu_ip_block *ip_block)
 	return -ETIMEDOUT;
 }
 
+/* The reset remaps the kernel gfx queue from its MQD: needs async rings. */
+static bool gfx_v12_0_engine_reset_supported(struct amdgpu_device *adev)
+{
+	return !amdgpu_sriov_vf(adev) && adev->enable_mes && amdgpu_async_gfx_ring;
+}
+
 /* The CP DMA and the interfaces its memory requests go through */
 #define GFX12_CP_STAT_MEM_BUSY	(CP_STAT__DMA_BUSY_MASK | \
 				 CP_STAT__TCIU_BUSY_MASK | \
-- 
2.50.1

[PATCH v2 15/15] drm/amdgpu/gfx11: enable the gfx engine reset

[Jesse Zhang]

Report AMDGPU_RESET_TYPE_PER_ENGINE for the graphics engine when the
reset_engine callback can be used: with MES, async gfx rings and the
RS64 firmware. The reset restarts PFP and ME from the start of their
RS64 firmware, and remaps the kernel gfx queue through MES from its
MQD, which the kernel gfx queue only has with async gfx rings. Not on
SR-IOV VFs.

The kernel gfx queue and gfx user queue resets then fall back to a gfx
engine reset before a full reset.

Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
index a2eaf416b12a..3b52eb58b124 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
@@ -363,6 +363,7 @@ static void gfx_v11_0_ring_invalidate_tlbs(struct amdgpu_ring *ring,
 static void gfx_v11_0_set_safe_mode(struct amdgpu_device *adev, int xcc_id);
 static void gfx_v11_0_unset_safe_mode(struct amdgpu_device *adev, int xcc_id);
 static void gfx_v11_0_userq_priv_fault_work(struct work_struct *work);
+static bool gfx_v11_0_engine_reset_supported(struct amdgpu_device *adev);
 static int gfx_v11_0_reset_engine(struct amdgpu_device *adev,
 				  enum amdgpu_gfx_engine engine);
 static bool gfx_v11_0_read_gfx_hqd_ptrs(struct amdgpu_device *adev,
@@ -1945,6 +1946,10 @@ static int gfx_v11_0_sw_init(struct amdgpu_ip_block *ip_block)
 		break;
 	}
 
+	if (gfx_v11_0_engine_reset_supported(adev) &&
+	    !adev->debug_disable_engine_reset)
+		adev->gfx.gfx_supported_reset |= AMDGPU_RESET_TYPE_PER_ENGINE;
+
 	if (!adev->enable_mes_kiq) {
 		r = amdgpu_gfx_kiq_init(adev, GFX11_MEC_HPD_SIZE, 0);
 		if (r) {
@@ -6927,6 +6932,16 @@ static void gfx_v11_0_emit_mem_sync(struct amdgpu_ring *ring)
 	amdgpu_ring_write(ring, gcr_cntl); /* GCR_CNTL */
 }
 
+/*
+ * The reset restarts PFP/ME through their RS64 start addresses, and remaps the
+ * kernel gfx queue from its MQD, which needs async rings.
+ */
+static bool gfx_v11_0_engine_reset_supported(struct amdgpu_device *adev)
+{
+	return !amdgpu_sriov_vf(adev) && adev->enable_mes &&
+	       amdgpu_async_gfx_ring && adev->gfx.rs64_enable;
+}
+
 /* Restart the PFP and ME pipes from the start of their RS64 firmware. */
 static void gfx_v11_0_restart_gfx_rs64(struct amdgpu_device *adev)
 {
-- 
2.50.1

Re: [PATCH v2 01/15] drm/amdgpu/gfx: add a reset_engine callback

[Timur Kristóf]

On 2026. október 22., csütörtök 2:19:41 középső államokbeli zónaidő Jesse 
Zhang wrote:
> Some gfx hangs can't be recovered by a queue or a pipe reset, eg. a CP
> front-end stuck in the middle of a packet. They currently end in a full
> GPU reset, and VRAM is lost. Resetting only the graphics engine is
> enough for them, and leaves the compute engine running.
> 
> Add an optional reset_engine callback to amdgpu_gfx_funcs that resets
> only the graphics or only the compute engine. The split is specific to
> the GFX IP, eg. CP_SOFT_RESET_CNTL has GFX_ONLY_SOFT_RESET and
> CMP_ONLY_SOFT_RESET, so it doesn't belong in the amd_ip_funcs that all
> IP blocks share.
> 

Hi Jesse,

I think this is a very good idea, in fact I wanted to implement something like 
this myself.

I notice that there is a lot of overlap between your engine reset and the pre-
existing IP block soft reset. I would prefer that the engine reset be 
integrated into the IP block soft reset instead of adding yet another reset 
type to the driver. I believe this would make the code overall better.

My suggestion is the following:

- Instead of introducing amdgpu_gfx_reset_gfx_engine(), use 
amdgpu_device_ip_soft_reset() and add the missing user queue handling there. 
Note that amdgpu_device_ip_soft_reset() already has a more sophisticated way 
to minimize collateral damage to kernel queues, so the new GPU generations 
will benefit from that as well.

- Instead of introducing amdgpu_gfx_funcs::reset_engine(), use 
amd_ip_funcs::soft_reset() and fix that up for each affected GPU generation as 
necessary.

- I think the cleanest solution is to reset the entire GFX IP block (including 
both graphics and compute) for the sake of consistency. There are some hang 
cases such as gang submissions which cannot be resolved by just resetting 
graphics or compute.

Please let me know what you think. I'd be happy to contribute to this effort.

Thanks & best regards,
Timur

> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h index 5939e1fb3b24..8dc669e89db5
> 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h
> @@ -345,6 +345,13 @@ struct amdgpu_gfx_shadow_info {
>  	u32 eop_alignment;
>  };
> 
> +/* The engines of the GFX IP that can be reset on their own. */
> +enum amdgpu_gfx_engine {
> +	AMDGPU_GFX_ENGINE_GFX,
> +	AMDGPU_GFX_ENGINE_COMPUTE,
> +	AMDGPU_GFX_ENGINE_NUM,
> +};
> +
>  struct amdgpu_gfx_funcs {
>  	/* get the gpu clock counter */
>  	uint64_t (*get_gpu_clock_counter)(struct amdgpu_device *adev);
> @@ -376,6 +383,9 @@ struct amdgpu_gfx_funcs {
>  	/* find the gfx HQD slot (me/pipe/queue) for a doorbell */
>  	bool (*detect_hung_queue)(struct amdgpu_device *adev, u32 
doorbell_index,
>  				  u32 *me, u32 *pipe, u32 
*queue);
> +	/* reset one engine, leaving the other one running */
> +	int (*reset_engine)(struct amdgpu_device *adev,
> +			    enum amdgpu_gfx_engine engine);
>  };
> 
>  struct sq_work {

Re: [PATCH v2 03/15] drm/amdgpu/gfx11: add a gfx engine reset

[Timur Kristóf]

On 2026. október 22., csütörtök 10:19:43 közép-európai téli idő Jesse Zhang 
wrote:
> Implement the reset_engine callback for the graphics engine of gfx11, as
> for gfx12:
> - halt PFP/ME and wait for the memory requests of the CP DMA to complete;
> - with CP_SOFT_RESET_CNTL.GFX_ONLY_SOFT_RESET and GFX_HQD_REG_RESET set,
>   pulse GRBM_SOFT_RESET.SOFT_RESET_CPF/CPG, holding the MES lock, and
>   wait for the memory requests again;
> - clear CP_VMID_RESET;
> - restart PFP/ME from the start of their RS64 firmware and restore
>   CP_PFP_IB_CONTROL; if the CP doesn't go idle then, fail the reset so
>   that a full reset follows;
> - remap the kernel gfx queue through MES from its MQD backup and restart
>   it with gfx_v11_0_cp_gfx_start().
> 
> MES resets a gfx queue by writing CP_VMID_RESET, which the PFP handles.
> When the PFP hung, eg. on a bad opcode, the request stays pending across
> the CPF/CPG reset, and the restarted PFP then waits on it forever for
> queues the reset already cleared.
> 
> gfx11 has no helper to restart PFP/ME from their start addresses outside
> of the firmware load: add one. A later patch enables the reset with
> MES, async gfx rings and the RS64 firmware.

I don't think it's enough to soft-reset just the CP and expect the GFX queue 
to continue. There are other parts of the GFX block which can hang, for 
example a badly written NGG shader can easily hang the PA. I think we need to 
use the SOFT_RESET_GFX bit as well.

I noticed that the soft reset implementation needs to also re-initialize the 
clock and power gating for the GFX block, otherwise the GPU can consume more 
power after the reset than before on older GPU generations (and some GPUs just 
fall off the PCIe bus). We need to check how GFX11-12 behaves. I think it would 
be best to re-initialize the clock and power gating here too.

Additionally, as I said on the other patch, I think the cleanest would be to 
reset the entire GFX block instead of just parts of it, for two reasons:
(1) gang submit can hang both gfx and compute at the same time, and those 
hangs can only be resolved by resetting both gfx and compute.
(2) if we choose to reset the whole gfx block, that allows us to reuse the 
code from the suspend() and resume() functions, which means better code reuse 
and more consistency.

What do you think?

(Same comments go for the GFX12 patch as well.)

Thanks & best regards,
Timur




> 
> Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c | 176 +++++++++++++++++++++++++
>  1 file changed, 176 insertions(+)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
> b/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c index 315dfd806d56..12b307e4106b
> 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
> @@ -363,6 +363,8 @@ static void gfx_v11_0_ring_invalidate_tlbs(struct
> amdgpu_ring *ring, static void gfx_v11_0_set_safe_mode(struct amdgpu_device
> *adev, int xcc_id); static void gfx_v11_0_unset_safe_mode(struct
> amdgpu_device *adev, int xcc_id); static void
> gfx_v11_0_userq_priv_fault_work(struct work_struct *work); +static int
> gfx_v11_0_reset_engine(struct amdgpu_device *adev,
> +				  enum amdgpu_gfx_engine engine);
>  static void gfx_v11_0_update_perf_clk(struct amdgpu_device *adev,
>  				      bool enable);
> 
> @@ -1150,6 +1152,7 @@ static const struct amdgpu_gfx_funcs
> gfx_v11_0_gfx_funcs = { .get_gfx_shadow_info =
> &gfx_v11_0_get_gfx_shadow_info,
>  	.get_hdp_flush_mask = &amdgpu_gfx_get_hdp_flush_mask,
>  	.detect_hung_queue = &gfx_v11_0_detect_hung_queue,
> +	.reset_engine = &gfx_v11_0_reset_engine,
>  };
> 
>  static int gfx_v11_0_gpu_early_init(struct amdgpu_device *adev)
> @@ -6916,6 +6919,179 @@ static void gfx_v11_0_emit_mem_sync(struct
> amdgpu_ring *ring) amdgpu_ring_write(ring, gcr_cntl); /* GCR_CNTL */
>  }
> 
> +/* Restart the PFP and ME pipes from the start of their RS64 firmware. */
> +static void gfx_v11_0_restart_gfx_rs64(struct amdgpu_device *adev)
> +{
> +	const struct gfx_firmware_header_v2_0 *pfp_hdr, *me_hdr;
> +	u32 pipe_id, tmp;
> +
> +	pfp_hdr = (const struct gfx_firmware_header_v2_0 *)adev->gfx.pfp_fw-
>data;
> +	me_hdr = (const struct gfx_firmware_header_v2_0 *)adev->gfx.me_fw-
>data;
> +
> +	mutex_lock(&adev->srbm_mutex);
> +	for (pipe_id = 0; pipe_id < adev->gfx.me.num_pipe_per_me; pipe_id+
+) {
> +		soc21_grbm_select(adev, 0, pipe_id, 0, 0);
> +		WREG32_SOC15(GC, 0, regCP_PFP_PRGRM_CNTR_START,
> +			     (pfp_hdr->ucode_start_addr_hi << 30) |
> +			     (pfp_hdr->ucode_start_addr_lo >> 2));
> +		WREG32_SOC15(GC, 0, regCP_PFP_PRGRM_CNTR_START_HI,
> +			     pfp_hdr->ucode_start_addr_hi >> 2);
> +		WREG32_SOC15(GC, 0, regCP_ME_PRGRM_CNTR_START,
> +			     (me_hdr->ucode_start_addr_hi << 30) |
> +			     (me_hdr->ucode_start_addr_lo >> 2));
> +		WREG32_SOC15(GC, 0, regCP_ME_PRGRM_CNTR_START_HI,
> +			     me_hdr->ucode_start_addr_hi >> 2);
> +
> +		tmp = RREG32_SOC15(GC, 0, regCP_ME_CNTL);
> +		if (pipe_id == 0) {
> +			tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, 
PFP_PIPE0_RESET, 1);
> +			tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, 
ME_PIPE0_RESET, 1);
> +		} else {
> +			tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, 
PFP_PIPE1_RESET, 1);
> +			tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, 
ME_PIPE1_RESET, 1);
> +		}
> +		WREG32_SOC15(GC, 0, regCP_ME_CNTL, tmp);
> +		if (pipe_id == 0) {
> +			tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, 
PFP_PIPE0_RESET, 0);
> +			tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, 
ME_PIPE0_RESET, 0);
> +		} else {
> +			tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, 
PFP_PIPE1_RESET, 0);
> +			tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, 
ME_PIPE1_RESET, 0);
> +		}
> +		WREG32_SOC15(GC, 0, regCP_ME_CNTL, tmp);
> +	}
> +	soc21_grbm_select(adev, 0, 0, 0, 0);
> +	mutex_unlock(&adev->srbm_mutex);
> +}
> +
> +/* The CP DMA and the interfaces its memory requests go through */
> +#define GFX11_CP_STAT_MEM_BUSY	(CP_STAT__DMA_BUSY_MASK | \
> +				 CP_STAT__TCIU_BUSY_MASK | \
> +				 CP_STAT__RCIU_BUSY_MASK | \
> +				 CP_STAT__UTCL2IU_BUSY_MASK)
> +
> +/*
> + * Soft reset the gfx CP and restart PFP and ME. Returns 0, or -ETIMEDOUT
> if the + * CP doesn't go idle after PFP and ME are restarted.
> + */
> +static int gfx_v11_0_soft_reset_gfx_cp(struct amdgpu_device *adev)
> +{
> +	u32 tmp, cntl, me_cntl, ib_control;
> +	int i;
> +
> +	amdgpu_gfx_rlc_enter_safe_mode(adev, 0);
> +
> +	/* MES fetches its queues through CPF: keep it out of the reset. */
> +	amdgpu_mes_lock(&adev->mes);
> +
> +	me_cntl = RREG32_SOC15(GC, 0, regCP_ME_CNTL);
> +	/* Cleared by the reset. */
> +	ib_control = RREG32_SOC15(GC, 0, regCP_PFP_IB_CONTROL);
> +
> +	tmp = REG_SET_FIELD(me_cntl, CP_ME_CNTL, ME_HALT, 1);
> +	tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, PFP_HALT, 1);
> +	WREG32_SOC15(GC, 0, regCP_ME_CNTL, tmp);
> +	udelay(50);
> +
> +	/* A CP DMA in flight during the reset leaves the ME busy after it. 
*/
> +	for (i = 0; i < adev->usec_timeout; i++) {
> +		if (!(RREG32_SOC15(GC, 0, regCP_STAT) & 
GFX11_CP_STAT_MEM_BUSY))
> +			break;
> +		udelay(1);
> +	}
> +
> +	/* Without GFX_ONLY_SOFT_RESET, a CPF reset would also hit compute 
and
> MES. */ +	cntl = RREG32_SOC15(GC, 0, regCP_SOFT_RESET_CNTL);
> +	tmp = REG_SET_FIELD(cntl, CP_SOFT_RESET_CNTL, GFX_ONLY_SOFT_RESET, 
1);
> +	tmp = REG_SET_FIELD(tmp, CP_SOFT_RESET_CNTL, GFX_HQD_REG_RESET, 1);
> +	WREG32_SOC15(GC, 0, regCP_SOFT_RESET_CNTL, tmp);
> +
> +	/* A CPF reset alone leaves the streaming engine stuck. */
> +	tmp = RREG32_SOC15(GC, 0, regGRBM_SOFT_RESET);
> +	tmp = REG_SET_FIELD(tmp, GRBM_SOFT_RESET, SOFT_RESET_CPF, 1);
> +	tmp = REG_SET_FIELD(tmp, GRBM_SOFT_RESET, SOFT_RESET_CPG, 1);
> +	WREG32_SOC15(GC, 0, regGRBM_SOFT_RESET, tmp);
> +	tmp = RREG32_SOC15(GC, 0, regGRBM_SOFT_RESET);
> +	udelay(50);
> +	tmp = REG_SET_FIELD(tmp, GRBM_SOFT_RESET, SOFT_RESET_CPF, 0);
> +	tmp = REG_SET_FIELD(tmp, GRBM_SOFT_RESET, SOFT_RESET_CPG, 0);
> +	WREG32_SOC15(GC, 0, regGRBM_SOFT_RESET, tmp);
> +	tmp = RREG32_SOC15(GC, 0, regGRBM_SOFT_RESET);
> +
> +	WREG32_SOC15(GC, 0, regCP_SOFT_RESET_CNTL, cntl);
> +
> +	/* Not the whole CP: the halted PFP stays busy until restarted. */
> +	for (i = 0; i < adev->usec_timeout; i++) {
> +		if (!(RREG32_SOC15(GC, 0, regCP_STAT) & 
GFX11_CP_STAT_MEM_BUSY))
> +			break;
> +		udelay(1);
> +	}
> +
> +	/* A CP_VMID_RESET the hung PFP never handled survives the reset. 
*/
> +	WREG32_SOC15(GC, 0, regCP_VMID_RESET, 0);
> +	amdgpu_mes_unlock(&adev->mes);
> +
> +	gfx_v11_0_restart_gfx_rs64(adev);
> +	tmp = RREG32_SOC15(GC, 0, regCP_ME_CNTL);
> +	tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, CE_HALT,
> +			    REG_GET_FIELD(me_cntl, CP_ME_CNTL, 
CE_HALT));
> +	WREG32_SOC15(GC, 0, regCP_ME_CNTL, tmp);
> +	WREG32_SOC15(GC, 0, regCP_PFP_IB_CONTROL, ib_control);
> +	amdgpu_gfx_rlc_exit_safe_mode(adev, 0);
> +
> +	tmp = RREG32_SOC15(GC, 0, regCP_ME_CNTL);
> +	tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, ME_HALT, 0);
> +	tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, PFP_HALT, 0);
> +	WREG32_SOC15(GC, 0, regCP_ME_CNTL, tmp);
> +	for (i = 0; i < adev->usec_timeout; i++) {
> +		if (!RREG32_SOC15(GC, 0, regCP_STAT))
> +			return 0;
> +		udelay(1);
> +	}
> +	return -ETIMEDOUT;
> +}
> +
> +/*
> + * Reset the graphics engine: the CP front-end (CPF/CPG), PFP/ME and the
> gfx + * HQDs. MEC, MES and the compute queues keep running.
> + */
> +static int gfx_v11_0_reset_engine(struct amdgpu_device *adev,
> +				  enum amdgpu_gfx_engine engine)
> +{
> +	int i, r;
> +
> +	lockdep_assert_held(&adev->gfx.mec.reset_mutex);
> +
> +	if (engine != AMDGPU_GFX_ENGINE_GFX)
> +		return -EOPNOTSUPP;
> +
> +	amdgpu_gfx_off_ctrl(adev, false);
> +
> +	r = gfx_v11_0_soft_reset_gfx_cp(adev);
> +	if (r) {
> +		dev_err(adev->dev, "gfx CP still busy after reset, 
CP_STAT 0x%08x\n",
> +			RREG32_SOC15(GC, 0, regCP_STAT));
> +		goto exit;
> +	}
> +
> +	/* Map the kernel gfx queue again: the reset cleared its HQD. */
> +	r = amdgpu_gfx_disable_kgq(adev, 0);
> +	if (r)
> +		goto exit;
> +
> +	for (i = 0; i < adev->gfx.num_gfx_rings; i++)
> +		amdgpu_gfx_mqd_reset_restore(&adev->gfx.gfx_ring[i]);
> +	r = amdgpu_gfx_enable_kgq(adev, 0);
> +	if (r)
> +		goto exit;
> +
> +	r = gfx_v11_0_cp_gfx_start(adev);
> +
> +exit:
> +	amdgpu_gfx_off_ctrl(adev, true);
> +	return r;
> +}
> +
>  static int gfx_v11_0_reset_kgq(struct amdgpu_ring *ring,
>  			       unsigned int vmid,
>  			       struct amdgpu_fence *timedout_fence)

RE: [PATCH v2 03/15] drm/amdgpu/gfx11: add a gfx engine reset

[Zhang, Jesse(Jie)]

AMD General

> -----Original Message-----
> From: Timur Kristóf <timur.kristof@gmail.com>
> Sent: Monday, October 26, 2026 1:10 AM
> To: amd-gfx@lists.freedesktop.org
> Cc: Deucher, Alexander <Alexander.Deucher@amd.com>; Koenig, Christian
> <Christian.Koenig@amd.com>; Lazar, Lijo <Lijo.Lazar@amd.com>; Mario
> Sopena-Novales <Mario.Novales@amd.com>; Rastogi, Manu
> <Manu.Rastogi@amd.com>; Zhu, Jiongyu (Joey) <Jiongyu.Zhu@amd.com>;
> Zhang, Jesse(Jie) <Jesse.Zhang@amd.com>; Zhang, Jesse(Jie)
> <Jesse.Zhang@amd.com>
> Subject: Re: [PATCH v2 03/15] drm/amdgpu/gfx11: add a gfx engine reset
>
> On 2026. október 22., csütörtök 10:19:43 közép-európai téli idő Jesse Zhang
> wrote:
> > Implement the reset_engine callback for the graphics engine of gfx11,
> > as for gfx12:
> > - halt PFP/ME and wait for the memory requests of the CP DMA to
> > complete;
> > - with CP_SOFT_RESET_CNTL.GFX_ONLY_SOFT_RESET and
> GFX_HQD_REG_RESET set,
> >   pulse GRBM_SOFT_RESET.SOFT_RESET_CPF/CPG, holding the MES lock, and
> >   wait for the memory requests again;
> > - clear CP_VMID_RESET;
> > - restart PFP/ME from the start of their RS64 firmware and restore
> >   CP_PFP_IB_CONTROL; if the CP doesn't go idle then, fail the reset so
> >   that a full reset follows;
> > - remap the kernel gfx queue through MES from its MQD backup and restart
> >   it with gfx_v11_0_cp_gfx_start().
> >
> > MES resets a gfx queue by writing CP_VMID_RESET, which the PFP handles.
> > When the PFP hung, eg. on a bad opcode, the request stays pending
> > across the CPF/CPG reset, and the restarted PFP then waits on it
> > forever for queues the reset already cleared.
> >
> > gfx11 has no helper to restart PFP/ME from their start addresses
> > outside of the firmware load: add one. A later patch enables the reset
> > with MES, async gfx rings and the RS64 firmware.
>
> I don't think it's enough to soft-reset just the CP and expect the GFX queue to
> continue. There are other parts of the GFX block which can hang, for example a
> badly written NGG shader can easily hang the PA. I think we need to use the
> SOFT_RESET_GFX bit as well.
>
> I noticed that the soft reset implementation needs to also re-initialize the clock
> and power gating for the GFX block, otherwise the GPU can consume more
> power after the reset than before on older GPU generations (and some GPUs
> just fall off the PCIe bus). We need to check how GFX11-12 behaves. I think it
> would be best to re-initialize the clock and power gating here too.
>
> Additionally, as I said on the other patch, I think the cleanest would be to reset
> the entire GFX block instead of just parts of it, for two reasons:
> (1) gang submit can hang both gfx and compute at the same time, and those
> hangs can only be resolved by resetting both gfx and compute.
> (2) if we choose to reset the whole gfx block, that allows us to reuse the code
> from the suspend() and resume() functions, which means better code reuse
> and more consistency.
>
> What do you think?
>

Hi Timur,

Thanks for the review. I'm glad you find the idea useful.

I would keep the gfx-only CP reset as a separate recovery level. Its purpose is to recover an isolated gfx CP hang without resetting independent compute workloads.
Replacing it with a whole GFX IP reset would lose that benefit.

There are several reasons for keeping this distinction:

- The hardware supports this narrower scope through
  CP_SOFT_RESET_CNTL.GFX_ONLY_SOFT_RESET and CMP_ONLY_SOFT_RESET.
  This gives us an intermediate recovery step when a queue or pipe reset
  cannot clear the hang.
- A whole GFX IP reset also affects compute resources and requires
  recovery of the affected MES queue state and all the KFD/ROCm queues.
  The gfx-only CP reset preserves that state and the pending compute work, although MES
  suspend/resume can temporarily pause scheduling. We have successful
  HIP and compute test runs across these resets, demonstrating the value
  of preserving independent compute workloads.

- The locking doesn't fit either: amdgpu_device_ip_soft_reset() takes
  the reset domain for write. KFD only try-locks it for read before it
  adds or removes MES queues, so those calls fail with -EIO meanwhile;
  we saw that turn into HIP failures and full resets. The engine reset
  only takes it for read. It is also reached from the user queue reset
  path, not only from a kernel ring timeout.

- gfx12 has no soft_reset callback, and the gfx11 one isn't enabled as
  a recovery step. A whole GFX reset for these generations needs new
  work and validation in any case.

I agree that there is overlap in the queue recovery code. We should reuse the common preparation,
backup and replay helpers while preserving the gfx-only reset scope.

About gang submissions: before the engine reset, the schedulers of the
gfx rings are stopped, so nothing new is submitted, and the hung gfx job
can't complete anyway. If the compute part of the gang then hangs
waiting on it, the compute ring times out on its own and goes through
its own recovery, starting with a compute queue reset firstly.

Thanks,
Jesse
> (Same comments go for the GFX12 patch as well.)
>
> Thanks & best regards,
> Timur
>
>
>
>
> >
> > Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
> > ---
> >  drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c | 176
> > +++++++++++++++++++++++++
> >  1 file changed, 176 insertions(+)
> >
> > diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
> > b/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c index
> > 315dfd806d56..12b307e4106b
> > 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
> > @@ -363,6 +363,8 @@ static void gfx_v11_0_ring_invalidate_tlbs(struct
> > amdgpu_ring *ring, static void gfx_v11_0_set_safe_mode(struct
> > amdgpu_device *adev, int xcc_id); static void
> > gfx_v11_0_unset_safe_mode(struct amdgpu_device *adev, int xcc_id);
> > static void gfx_v11_0_userq_priv_fault_work(struct work_struct *work);
> > +static int gfx_v11_0_reset_engine(struct amdgpu_device *adev,
> > +                             enum amdgpu_gfx_engine engine);
> >  static void gfx_v11_0_update_perf_clk(struct amdgpu_device *adev,
> >                                   bool enable);
> >
> > @@ -1150,6 +1152,7 @@ static const struct amdgpu_gfx_funcs
> > gfx_v11_0_gfx_funcs = { .get_gfx_shadow_info =
> > &gfx_v11_0_get_gfx_shadow_info,
> >     .get_hdp_flush_mask = &amdgpu_gfx_get_hdp_flush_mask,
> >     .detect_hung_queue = &gfx_v11_0_detect_hung_queue,
> > +   .reset_engine = &gfx_v11_0_reset_engine,
> >  };
> >
> >  static int gfx_v11_0_gpu_early_init(struct amdgpu_device *adev) @@
> > -6916,6 +6919,179 @@ static void gfx_v11_0_emit_mem_sync(struct
> > amdgpu_ring *ring) amdgpu_ring_write(ring, gcr_cntl); /* GCR_CNTL */
> > }
> >
> > +/* Restart the PFP and ME pipes from the start of their RS64
> > +firmware. */ static void gfx_v11_0_restart_gfx_rs64(struct
> > +amdgpu_device *adev) {
> > +   const struct gfx_firmware_header_v2_0 *pfp_hdr, *me_hdr;
> > +   u32 pipe_id, tmp;
> > +
> > +   pfp_hdr = (const struct gfx_firmware_header_v2_0 *)adev-
> >gfx.pfp_fw-
> >data;
> > +   me_hdr = (const struct gfx_firmware_header_v2_0 *)adev-
> >gfx.me_fw-
> >data;
> > +
> > +   mutex_lock(&adev->srbm_mutex);
> > +   for (pipe_id = 0; pipe_id < adev->gfx.me.num_pipe_per_me; pipe_id+
> +) {
> > +           soc21_grbm_select(adev, 0, pipe_id, 0, 0);
> > +           WREG32_SOC15(GC, 0, regCP_PFP_PRGRM_CNTR_START,
> > +                        (pfp_hdr->ucode_start_addr_hi << 30) |
> > +                        (pfp_hdr->ucode_start_addr_lo >> 2));
> > +           WREG32_SOC15(GC, 0, regCP_PFP_PRGRM_CNTR_START_HI,
> > +                        pfp_hdr->ucode_start_addr_hi >> 2);
> > +           WREG32_SOC15(GC, 0, regCP_ME_PRGRM_CNTR_START,
> > +                        (me_hdr->ucode_start_addr_hi << 30) |
> > +                        (me_hdr->ucode_start_addr_lo >> 2));
> > +           WREG32_SOC15(GC, 0, regCP_ME_PRGRM_CNTR_START_HI,
> > +                        me_hdr->ucode_start_addr_hi >> 2);
> > +
> > +           tmp = RREG32_SOC15(GC, 0, regCP_ME_CNTL);
> > +           if (pipe_id == 0) {
> > +                   tmp = REG_SET_FIELD(tmp, CP_ME_CNTL,
> PFP_PIPE0_RESET, 1);
> > +                   tmp = REG_SET_FIELD(tmp, CP_ME_CNTL,
> ME_PIPE0_RESET, 1);
> > +           } else {
> > +                   tmp = REG_SET_FIELD(tmp, CP_ME_CNTL,
> PFP_PIPE1_RESET, 1);
> > +                   tmp = REG_SET_FIELD(tmp, CP_ME_CNTL,
> ME_PIPE1_RESET, 1);
> > +           }
> > +           WREG32_SOC15(GC, 0, regCP_ME_CNTL, tmp);
> > +           if (pipe_id == 0) {
> > +                   tmp = REG_SET_FIELD(tmp, CP_ME_CNTL,
> PFP_PIPE0_RESET, 0);
> > +                   tmp = REG_SET_FIELD(tmp, CP_ME_CNTL,
> ME_PIPE0_RESET, 0);
> > +           } else {
> > +                   tmp = REG_SET_FIELD(tmp, CP_ME_CNTL,
> PFP_PIPE1_RESET, 0);
> > +                   tmp = REG_SET_FIELD(tmp, CP_ME_CNTL,
> ME_PIPE1_RESET, 0);
> > +           }
> > +           WREG32_SOC15(GC, 0, regCP_ME_CNTL, tmp);
> > +   }
> > +   soc21_grbm_select(adev, 0, 0, 0, 0);
> > +   mutex_unlock(&adev->srbm_mutex);
> > +}
> > +
> > +/* The CP DMA and the interfaces its memory requests go through */
> > +#define GFX11_CP_STAT_MEM_BUSY
>       (CP_STAT__DMA_BUSY_MASK | \
> > +                            CP_STAT__TCIU_BUSY_MASK | \
> > +                            CP_STAT__RCIU_BUSY_MASK | \
> > +                            CP_STAT__UTCL2IU_BUSY_MASK)
> > +
> > +/*
> > + * Soft reset the gfx CP and restart PFP and ME. Returns 0, or
> > +-ETIMEDOUT
> > if the + * CP doesn't go idle after PFP and ME are restarted.
> > + */
> > +static int gfx_v11_0_soft_reset_gfx_cp(struct amdgpu_device *adev) {
> > +   u32 tmp, cntl, me_cntl, ib_control;
> > +   int i;
> > +
> > +   amdgpu_gfx_rlc_enter_safe_mode(adev, 0);
> > +
> > +   /* MES fetches its queues through CPF: keep it out of the reset. */
> > +   amdgpu_mes_lock(&adev->mes);
> > +
> > +   me_cntl = RREG32_SOC15(GC, 0, regCP_ME_CNTL);
> > +   /* Cleared by the reset. */
> > +   ib_control = RREG32_SOC15(GC, 0, regCP_PFP_IB_CONTROL);
> > +
> > +   tmp = REG_SET_FIELD(me_cntl, CP_ME_CNTL, ME_HALT, 1);
> > +   tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, PFP_HALT, 1);
> > +   WREG32_SOC15(GC, 0, regCP_ME_CNTL, tmp);
> > +   udelay(50);
> > +
> > +   /* A CP DMA in flight during the reset leaves the ME busy after it.
> */
> > +   for (i = 0; i < adev->usec_timeout; i++) {
> > +           if (!(RREG32_SOC15(GC, 0, regCP_STAT) &
> GFX11_CP_STAT_MEM_BUSY))
> > +                   break;
> > +           udelay(1);
> > +   }
> > +
> > +   /* Without GFX_ONLY_SOFT_RESET, a CPF reset would also hit
> compute
> and
> > MES. */ +   cntl = RREG32_SOC15(GC, 0, regCP_SOFT_RESET_CNTL);
> > +   tmp = REG_SET_FIELD(cntl, CP_SOFT_RESET_CNTL,
> GFX_ONLY_SOFT_RESET,
> 1);
> > +   tmp = REG_SET_FIELD(tmp, CP_SOFT_RESET_CNTL,
> GFX_HQD_REG_RESET, 1);
> > +   WREG32_SOC15(GC, 0, regCP_SOFT_RESET_CNTL, tmp);
> > +
> > +   /* A CPF reset alone leaves the streaming engine stuck. */
> > +   tmp = RREG32_SOC15(GC, 0, regGRBM_SOFT_RESET);
> > +   tmp = REG_SET_FIELD(tmp, GRBM_SOFT_RESET, SOFT_RESET_CPF, 1);
> > +   tmp = REG_SET_FIELD(tmp, GRBM_SOFT_RESET, SOFT_RESET_CPG, 1);
> > +   WREG32_SOC15(GC, 0, regGRBM_SOFT_RESET, tmp);
> > +   tmp = RREG32_SOC15(GC, 0, regGRBM_SOFT_RESET);
> > +   udelay(50);
> > +   tmp = REG_SET_FIELD(tmp, GRBM_SOFT_RESET, SOFT_RESET_CPF, 0);
> > +   tmp = REG_SET_FIELD(tmp, GRBM_SOFT_RESET, SOFT_RESET_CPG, 0);
> > +   WREG32_SOC15(GC, 0, regGRBM_SOFT_RESET, tmp);
> > +   tmp = RREG32_SOC15(GC, 0, regGRBM_SOFT_RESET);
> > +
> > +   WREG32_SOC15(GC, 0, regCP_SOFT_RESET_CNTL, cntl);
> > +
> > +   /* Not the whole CP: the halted PFP stays busy until restarted. */
> > +   for (i = 0; i < adev->usec_timeout; i++) {
> > +           if (!(RREG32_SOC15(GC, 0, regCP_STAT) &
> GFX11_CP_STAT_MEM_BUSY))
> > +                   break;
> > +           udelay(1);
> > +   }
> > +
> > +   /* A CP_VMID_RESET the hung PFP never handled survives the reset.
> */
> > +   WREG32_SOC15(GC, 0, regCP_VMID_RESET, 0);
> > +   amdgpu_mes_unlock(&adev->mes);
> > +
> > +   gfx_v11_0_restart_gfx_rs64(adev);
> > +   tmp = RREG32_SOC15(GC, 0, regCP_ME_CNTL);
> > +   tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, CE_HALT,
> > +                       REG_GET_FIELD(me_cntl, CP_ME_CNTL,
> CE_HALT));
> > +   WREG32_SOC15(GC, 0, regCP_ME_CNTL, tmp);
> > +   WREG32_SOC15(GC, 0, regCP_PFP_IB_CONTROL, ib_control);
> > +   amdgpu_gfx_rlc_exit_safe_mode(adev, 0);
> > +
> > +   tmp = RREG32_SOC15(GC, 0, regCP_ME_CNTL);
> > +   tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, ME_HALT, 0);
> > +   tmp = REG_SET_FIELD(tmp, CP_ME_CNTL, PFP_HALT, 0);
> > +   WREG32_SOC15(GC, 0, regCP_ME_CNTL, tmp);
> > +   for (i = 0; i < adev->usec_timeout; i++) {
> > +           if (!RREG32_SOC15(GC, 0, regCP_STAT))
> > +                   return 0;
> > +           udelay(1);
> > +   }
> > +   return -ETIMEDOUT;
> > +}
> > +
> > +/*
> > + * Reset the graphics engine: the CP front-end (CPF/CPG), PFP/ME and
> > +the
> > gfx + * HQDs. MEC, MES and the compute queues keep running.
> > + */
> > +static int gfx_v11_0_reset_engine(struct amdgpu_device *adev,
> > +                             enum amdgpu_gfx_engine engine)
> > +{
> > +   int i, r;
> > +
> > +   lockdep_assert_held(&adev->gfx.mec.reset_mutex);
> > +
> > +   if (engine != AMDGPU_GFX_ENGINE_GFX)
> > +           return -EOPNOTSUPP;
> > +
> > +   amdgpu_gfx_off_ctrl(adev, false);
> > +
> > +   r = gfx_v11_0_soft_reset_gfx_cp(adev);
> > +   if (r) {
> > +           dev_err(adev->dev, "gfx CP still busy after reset,
> CP_STAT 0x%08x\n",
> > +                   RREG32_SOC15(GC, 0, regCP_STAT));
> > +           goto exit;
> > +   }
> > +
> > +   /* Map the kernel gfx queue again: the reset cleared its HQD. */
> > +   r = amdgpu_gfx_disable_kgq(adev, 0);
> > +   if (r)
> > +           goto exit;
> > +
> > +   for (i = 0; i < adev->gfx.num_gfx_rings; i++)
> > +           amdgpu_gfx_mqd_reset_restore(&adev->gfx.gfx_ring[i]);
> > +   r = amdgpu_gfx_enable_kgq(adev, 0);
> > +   if (r)
> > +           goto exit;
> > +
> > +   r = gfx_v11_0_cp_gfx_start(adev);
> > +
> > +exit:
> > +   amdgpu_gfx_off_ctrl(adev, true);
> > +   return r;
> > +}
> > +
> >  static int gfx_v11_0_reset_kgq(struct amdgpu_ring *ring,
> >                            unsigned int vmid,
> >                            struct amdgpu_fence *timedout_fence)
>
>
>

Re: [PATCH v2 03/15] drm/amdgpu/gfx11: add a gfx engine reset

[Timur Kristóf]

On Mon, 2026-10-26 at 08:35 +0000, Zhang, Jesse(Jie) wrote:
> > 
> > On 2026. október 22., csütörtök 10:19:43 közép-európai téli idő
> > Jesse Zhang
> > wrote:
> > > Implement the reset_engine callback for the graphics engine of
> > > gfx11,
> > > as for gfx12:
> > > - halt PFP/ME and wait for the memory requests of the CP DMA to
> > > complete;
> > > - with CP_SOFT_RESET_CNTL.GFX_ONLY_SOFT_RESET and
> > GFX_HQD_REG_RESET set,
> > >   pulse GRBM_SOFT_RESET.SOFT_RESET_CPF/CPG, holding the MES lock,
> > > and
> > >   wait for the memory requests again;
> > > - clear CP_VMID_RESET;
> > > - restart PFP/ME from the start of their RS64 firmware and
> > > restore
> > >   CP_PFP_IB_CONTROL; if the CP doesn't go idle then, fail the
> > > reset so
> > >   that a full reset follows;
> > > - remap the kernel gfx queue through MES from its MQD backup and
> > > restart
> > >   it with gfx_v11_0_cp_gfx_start().
> > > 
> > > MES resets a gfx queue by writing CP_VMID_RESET, which the PFP
> > > handles.
> > > When the PFP hung, eg. on a bad opcode, the request stays pending
> > > across the CPF/CPG reset, and the restarted PFP then waits on it
> > > forever for queues the reset already cleared.
> > > 
> > > gfx11 has no helper to restart PFP/ME from their start addresses
> > > outside of the firmware load: add one. A later patch enables the
> > > reset
> > > with MES, async gfx rings and the RS64 firmware.
> > 
> > I don't think it's enough to soft-reset just the CP and expect the
> > GFX queue to
> > continue. There are other parts of the GFX block which can hang,
> > for example a
> > badly written NGG shader can easily hang the PA. I think we need to
> > use the
> > SOFT_RESET_GFX bit as well.
> > 
> > I noticed that the soft reset implementation needs to also re-
> > initialize the clock
> > and power gating for the GFX block, otherwise the GPU can consume
> > more
> > power after the reset than before on older GPU generations (and
> > some GPUs
> > just fall off the PCIe bus). We need to check how GFX11-12 behaves.
> > I think it
> > would be best to re-initialize the clock and power gating here too.
> > 
> > Additionally, as I said on the other patch, I think the cleanest
> > would be to reset
> > the entire GFX block instead of just parts of it, for two reasons:
> > (1) gang submit can hang both gfx and compute at the same time, and
> > those
> > hangs can only be resolved by resetting both gfx and compute.
> > (2) if we choose to reset the whole gfx block, that allows us to
> > reuse the code
> > from the suspend() and resume() functions, which means better code
> > reuse
> > and more consistency.
> > 
> > What do you think?
> > 
> 
> Hi Timur,
> 
> Thanks for the review. I'm glad you find the idea useful.
> 
> I would keep the gfx-only CP reset as a separate recovery level. Its
> purpose is to recover an isolated gfx CP hang without resetting
> independent compute workloads.
> Replacing it with a whole GFX IP reset would lose that benefit.

I see your point. I'm skeptical because the GFX7-8 soft reset code was
also supposed to do that, but it never actually worked and nobody
noticed it for many years until I fixed it recently.

> 
> There are several reasons for keeping this distinction:
> 
> - The hardware supports this narrower scope through
>   CP_SOFT_RESET_CNTL.GFX_ONLY_SOFT_RESET and CMP_ONLY_SOFT_RESET.
>   This gives us an intermediate recovery step when a queue or pipe
> reset
>   cannot clear the hang.

We need to test each code path very thoroughly to make sure it can
recover from all possible hangs on both graphics and compute queues.
There should also be a code path that resets all engines at the same
time if needed.

> - A whole GFX IP reset also affects compute resources and requires
>   recovery of the affected MES queue state and all the KFD/ROCm
> queues.

The soft reset code already handles that for kernel queues.
It just needs to be extended to also consider user queues.

>   The gfx-only CP reset preserves that state and the pending compute
> work, although MES
>   suspend/resume can temporarily pause scheduling. We have successful
>   HIP and compute test runs across these resets, demonstrating the
> value
>   of preserving independent compute workloads.

Nice! That's impressive.

> 
> - The locking doesn't fit either: amdgpu_device_ip_soft_reset() takes
>   the reset domain for write. [...]

We can change that.

> It is also reached from the user queue
> reset path, not only from a kernel ring timeout.

I was planning to extend the soft reset code to be reachable when user
queues hang the GPU. Just haven't done it yet as currently user queues
don't yet work well on any supported GPU.

> - gfx12 has no soft_reset callback, and the gfx11 one isn't enabled
> as
>   a recovery step. A whole GFX reset for these generations needs new
>   work and validation in any case.

If we really want to have the capability to reset graphics and compute
separately, then I think the right way to go about it is to make soft
reset more fine-grained instead of introducing another reset type.
I think that would simplify the code and benefit both implementations.

In my opinion, we should do the following:

1. amdgpu_device_ip_soft_reset() needs to be extended to handle user
queues. It should be able to save and recover non-guilty user queues
(it already does that for kernel queues), and it should be reachable
when user queues hang.

2. We should add an IP block function called soft_reset_engine() and
move your engine reset implementation there. The argument should be a
bit mask, each bit representing an engine so that the function should
be able to reset multiple engines at once if needed (when all bits are
set, it should reset all engines at the same time, ie. equivalent to a
full IP block reset).

3. amdgpu_device_ip_soft_reset() should call soft_reset_engine() if it
exists.

I'm happy to help with the code if you want. Can you share what is your
test case and how to reproduce it?

Best regards,
Timur

RE: [PATCH v2 03/15] drm/amdgpu/gfx11: add a gfx engine reset

[Zhang, Jesse(Jie)]

AMD General

> -----Original Message-----
> From: Timur Kristóf <timur.kristof@gmail.com>
> Sent: Tuesday, October 27, 2026 7:51 AM
> To: Zhang, Jesse(Jie) <Jesse.Zhang@amd.com>; amd-
> gfx@lists.freedesktop.org
> Cc: Deucher, Alexander <Alexander.Deucher@amd.com>; Koenig, Christian
> <Christian.Koenig@amd.com>; Lazar, Lijo <Lijo.Lazar@amd.com>; Mario
> Sopena-Novales <Mario.Novales@amd.com>; Rastogi, Manu
> <Manu.Rastogi@amd.com>; Zhu, Jiongyu (Joey) <Jiongyu.Zhu@amd.com>
> Subject: Re: [PATCH v2 03/15] drm/amdgpu/gfx11: add a gfx engine reset
>
> On Mon, 2026-10-26 at 08:35 +0000, Zhang, Jesse(Jie) wrote:
> > >
> > > On 2026. október 22., csütörtök 10:19:43 közép-európai téli idő
> > > Jesse Zhang
> > > wrote:
> > > > Implement the reset_engine callback for the graphics engine of
> > > > gfx11, as for gfx12:
> > > > - halt PFP/ME and wait for the memory requests of the CP DMA to
> > > > complete;
> > > > - with CP_SOFT_RESET_CNTL.GFX_ONLY_SOFT_RESET and
> > > GFX_HQD_REG_RESET set,
> > > >   pulse GRBM_SOFT_RESET.SOFT_RESET_CPF/CPG, holding the MES lock,
> > > > and
> > > >   wait for the memory requests again;
> > > > - clear CP_VMID_RESET;
> > > > - restart PFP/ME from the start of their RS64 firmware and restore
> > > >   CP_PFP_IB_CONTROL; if the CP doesn't go idle then, fail the
> > > > reset so
> > > >   that a full reset follows;
> > > > - remap the kernel gfx queue through MES from its MQD backup and
> > > > restart
> > > >   it with gfx_v11_0_cp_gfx_start().
> > > >
> > > > MES resets a gfx queue by writing CP_VMID_RESET, which the PFP
> > > > handles.
> > > > When the PFP hung, eg. on a bad opcode, the request stays pending
> > > > across the CPF/CPG reset, and the restarted PFP then waits on it
> > > > forever for queues the reset already cleared.
> > > >
> > > > gfx11 has no helper to restart PFP/ME from their start addresses
> > > > outside of the firmware load: add one. A later patch enables the
> > > > reset with MES, async gfx rings and the RS64 firmware.
> > >
> > > I don't think it's enough to soft-reset just the CP and expect the
> > > GFX queue to continue. There are other parts of the GFX block which
> > > can hang, for example a badly written NGG shader can easily hang the
> > > PA. I think we need to use the SOFT_RESET_GFX bit as well.
> > >
> > > I noticed that the soft reset implementation needs to also re-
> > > initialize the clock and power gating for the GFX block, otherwise
> > > the GPU can consume more power after the reset than before on older
> > > GPU generations (and some GPUs just fall off the PCIe bus). We need
> > > to check how GFX11-12 behaves.
> > > I think it
> > > would be best to re-initialize the clock and power gating here too.
> > >
> > > Additionally, as I said on the other patch, I think the cleanest
> > > would be to reset the entire GFX block instead of just parts of it,
> > > for two reasons:
> > > (1) gang submit can hang both gfx and compute at the same time, and
> > > those hangs can only be resolved by resetting both gfx and compute.
> > > (2) if we choose to reset the whole gfx block, that allows us to
> > > reuse the code from the suspend() and resume() functions, which
> > > means better code reuse and more consistency.
> > >
> > > What do you think?
> > >
> >
> > Hi Timur,
> >
> > Thanks for the review. I'm glad you find the idea useful.
> >
> > I would keep the gfx-only CP reset as a separate recovery level. Its
> > purpose is to recover an isolated gfx CP hang without resetting
> > independent compute workloads.
> > Replacing it with a whole GFX IP reset would lose that benefit.
>
> I see your point. I'm skeptical because the GFX7-8 soft reset code was also
> supposed to do that, but it never actually worked and nobody noticed it for
> many years until I fixed it recently.
>
> >
> > There are several reasons for keeping this distinction:
> >
> > - The hardware supports this narrower scope through
> >   CP_SOFT_RESET_CNTL.GFX_ONLY_SOFT_RESET and
> CMP_ONLY_SOFT_RESET.
> >   This gives us an intermediate recovery step when a queue or pipe
> > reset
> >   cannot clear the hang.
>
> We need to test each code path very thoroughly to make sure it can recover
> from all possible hangs on both graphics and compute queues.
> There should also be a code path that resets all engines at the same time if
> needed.
>
> > - A whole GFX IP reset also affects compute resources and requires
> >   recovery of the affected MES queue state and all the KFD/ROCm
> > queues.
>
> The soft reset code already handles that for kernel queues.
> It just needs to be extended to also consider user queues.
>
> >   The gfx-only CP reset preserves that state and the pending compute
> > work, although MES
> >   suspend/resume can temporarily pause scheduling. We have successful
> >   HIP and compute test runs across these resets, demonstrating the
> > value
> >   of preserving independent compute workloads.
>
> Nice! That's impressive.
>
> >
> > - The locking doesn't fit either: amdgpu_device_ip_soft_reset() takes
> >   the reset domain for write. [...]
>
> We can change that.
>
> > It is also reached from the user queue reset path, not only from a
> > kernel ring timeout.
>
> I was planning to extend the soft reset code to be reachable when user queues
> hang the GPU. Just haven't done it yet as currently user queues don't yet work
> well on any supported GPU.
>
> > - gfx12 has no soft_reset callback, and the gfx11 one isn't enabled as
> >   a recovery step. A whole GFX reset for these generations needs new
> >   work and validation in any case.
>
> If we really want to have the capability to reset graphics and compute
> separately, then I think the right way to go about it is to make soft reset more
> fine-grained instead of introducing another reset type.
> I think that would simplify the code and benefit both implementations.
>
> In my opinion, we should do the following:
>
> 1. amdgpu_device_ip_soft_reset() needs to be extended to handle user
> queues. It should be able to save and recover non-guilty user queues (it
> already does that for kernel queues), and it should be reachable when user
> queues hang.

This series already does that for gfx user queues: the gfx engine reset
saves the ring state of the non-guilty gfx user queues before the reset
and maps them again after it, and it is reached from the gfx user queue
reset path when the queue and pipe resets fail.
Compute user queues would be the next step, together with a compute engine reset.

>
> 2. We should add an IP block function called soft_reset_engine() and move
> your engine reset implementation there. The argument should be a bit mask,
> each bit representing an engine so that the function should be able to reset
> multiple engines at once if needed (when all bits are set, it should reset all
> engines at the same time, ie. equivalent to a full IP block reset).

That's close to how I implemented it at first. I'm not sure we need a
bit mask though: the GFX IP is the only one where we need to choose
between engines, gfx or compute, as the hardware can reset them
separately (CP_SOFT_RESET_CNTL.GFX_ONLY_SOFT_RESET /
CMP_ONLY_SOFT_RESET). Other IPs like SDMA or VCN reset per instance and
can keep using soft_reset(). So an engine selection for the GFX IP
(gfx, compute, or all) would be enough, and the new callback would be
optional for the other IPs.

>
> 3. amdgpu_device_ip_soft_reset() should call soft_reset_engine() if it exists.
I am fine with this way .

One more requirement: for a gfx-only reset, the reset domain has to be
taken for read. KFD only try-locks it for read before it adds or removes
MES queues, so under the write lock those calls fail with -EIO; we saw
that with an earlier version of this series.

 @Deucher, Alexander,  do you have any concerns with this direction?
I'll rework the series along these lines and send a v3.

Thanks
Jesse
>
> I'm happy to help with the code if you want. Can you share what is your test
> case and how to reproduce it?
>
> Best regards,
> Timur