* [PATCH v2 1/5] drm/amdgpu: add gpu_recovery_allowed flag to amdgpu_device
@ 2026-08-13 17:06 Pierre-Eric Pelloux-Prayer
2026-08-13 17:06 ` [PATCH v2 2/5] drm/amdgpu: honor gpu_recovery_allowed in amdgpu_job_timedout Pierre-Eric Pelloux-Prayer
` (4 more replies)
0 siblings, 5 replies; 7+ messages in thread
From: Pierre-Eric Pelloux-Prayer @ 2026-08-13 17:06 UTC (permalink / raw)
To: Alex Deucher, Christian König, David Airlie, Simona Vetter
Cc: Pierre-Eric Pelloux-Prayer, amd-gfx, dri-devel, linux-kernel
Add a per-device boolean to control whether GPU recovery is attempted
on a hang, independently of the global amdgpu.gpu_recovery module
parameter. It defaults to true and is exposed as a write to the
existing amdgpu_gpu_recover debugfs file, so a single device can have
auto-recovery disabled without affecting every other GPU in the
system.
amdgpu_device_should_recover_gpu() now takes this flag into account.
Assisted-by: Claude:Sonnet 5
Signed-off-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 7 +++++++
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 5 +++++
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 21 +++++++++++++++++----
3 files changed, 29 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index 7974f9b7944f..21b33dc34edf 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -957,6 +957,13 @@ struct amdgpu_device {
struct amdgpu_uma_carveout_info uma_info;
+ /* Whether this device is allowed to attempt GPU recovery on a hang.
+ * Defaults to true; can be turned off per-device (e.g. via the
+ * amdgpu_gpu_recover debugfs file) independently of the global
+ * amdgpu.gpu_recovery module parameter.
+ */
+ bool gpu_recovery_allowed;
+
/* KFD
* Must be last --ends in a flexible-array member.
*/
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 964efec0d335..5578d5f64937 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -4034,6 +4034,8 @@ int amdgpu_device_init(struct amdgpu_device *adev,
}
}
+ adev->gpu_recovery_allowed = true;
+
fence_driver_init:
/* Fence driver */
r = amdgpu_fence_driver_sw_init(adev);
@@ -4834,6 +4836,9 @@ bool amdgpu_device_should_recover_gpu(struct amdgpu_device *adev)
if (amdgpu_gpu_recovery == 0)
goto disabled;
+ if (!adev->gpu_recovery_allowed)
+ goto disabled;
+
/* Skip soft reset check in fatal error mode */
if (!amdgpu_ras_is_poison_mode_supported(adev))
return true;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
index 3043ad041bb4..707e69d8bb11 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
@@ -975,9 +975,13 @@ static int amdgpu_debugfs_fence_info_show(struct seq_file *m, void *unused)
}
/*
- * amdgpu_debugfs_gpu_recover - manually trigger a gpu reset & recover
+ * amdgpu_debugfs_gpu_recover - manually trigger a gpu reset & recover,
+ * and control whether this device is allowed to auto-recover from a hang.
*
- * Manually trigger a gpu reset at the next fence wait.
+ * Read triggers a gpu reset at the next fence wait.
+ *
+ * Write 0/1 to disable/enable auto GPU recovery for this device
+ * (equivalent to amdgpu.gpu_recovery=0, but scoped to this device only).
*/
static int gpu_recover_get(void *data, u64 *val)
{
@@ -1001,8 +1005,17 @@ static int gpu_recover_get(void *data, u64 *val)
return 0;
}
+static int gpu_recover_set(void *data, u64 val)
+{
+ struct amdgpu_device *adev = (struct amdgpu_device *)data;
+
+ adev->gpu_recovery_allowed = !!val;
+
+ return 0;
+}
+
DEFINE_SHOW_ATTRIBUTE(amdgpu_debugfs_fence_info);
-DEFINE_DEBUGFS_ATTRIBUTE(amdgpu_debugfs_gpu_recover_fops, gpu_recover_get, NULL,
+DEFINE_DEBUGFS_ATTRIBUTE(amdgpu_debugfs_gpu_recover_fops, gpu_recover_get, gpu_recover_set,
"%lld\n");
static void amdgpu_debugfs_reset_work(struct work_struct *work)
@@ -1037,7 +1050,7 @@ void amdgpu_debugfs_fence_init(struct amdgpu_device *adev)
if (!amdgpu_sriov_vf(adev)) {
INIT_WORK(&adev->reset_work, amdgpu_debugfs_reset_work);
- debugfs_create_file("amdgpu_gpu_recover", 0444, root, adev,
+ debugfs_create_file("amdgpu_gpu_recover", 0644, root, adev,
&amdgpu_debugfs_gpu_recover_fops);
}
#endif
--
2.43.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v2 2/5] drm/amdgpu: honor gpu_recovery_allowed in amdgpu_job_timedout
2026-08-13 17:06 [PATCH v2 1/5] drm/amdgpu: add gpu_recovery_allowed flag to amdgpu_device Pierre-Eric Pelloux-Prayer
@ 2026-08-13 17:06 ` Pierre-Eric Pelloux-Prayer
2026-08-13 17:07 ` [PATCH v2 3/5] drm/amdgpu: add wedge event implementation Pierre-Eric Pelloux-Prayer
` (3 subsequent siblings)
4 siblings, 0 replies; 7+ messages in thread
From: Pierre-Eric Pelloux-Prayer @ 2026-08-13 17:06 UTC (permalink / raw)
To: Alex Deucher, Christian König, David Airlie, Simona Vetter
Cc: Pierre-Eric Pelloux-Prayer, amd-gfx, dri-devel, linux-kernel
The soft-recovery, per-ring-reset and IP-block-reset attempts in
amdgpu_job_timedout() were only gated on the global amdgpu_gpu_recovery
module parameter. Introduce a single can_reset variable that also
takes the per-device adev->gpu_recovery_allowed flag into account, and
use it for all three attempts instead of reading amdgpu_gpu_recovery
directly.
Assisted-by: Claude:Sonnet 5
Signed-off-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
index d97cf1212e0f..43511e0419a1 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
@@ -92,6 +92,7 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job)
struct drm_wedge_task_info *info = NULL;
struct amdgpu_task_info *ti = NULL;
struct amdgpu_device *adev = ring->adev;
+ bool can_reset = amdgpu_gpu_recovery && adev->gpu_recovery_allowed;
int idx, r;
if (!drm_dev_enter(adev_to_drm(adev), &idx)) {
@@ -111,7 +112,7 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job)
if (!amdgpu_sriov_vf(adev))
amdgpu_job_core_dump(adev, job);
- if (amdgpu_gpu_recovery &&
+ if (can_reset &&
amdgpu_ring_is_reset_type_supported(ring, AMDGPU_RESET_TYPE_SOFT_RECOVERY) &&
amdgpu_ring_soft_recovery(ring, job->vmid, s_job->s_fence->parent)) {
dev_err(adev->dev, "ring %s timeout, but soft recovered\n",
@@ -130,7 +131,7 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job)
}
/* attempt a per ring reset */
- if (amdgpu_gpu_recovery &&
+ if (can_reset &&
amdgpu_ring_is_reset_type_supported(ring, AMDGPU_RESET_TYPE_PER_QUEUE) &&
ring->funcs->reset) {
dev_err(adev->dev, "Starting %s ring reset\n",
@@ -152,7 +153,7 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job)
}
/* Attempt an IP block soft reset, if supported. */
- if (amdgpu_gpu_recovery &&
+ if (can_reset &&
amdgpu_ring_is_reset_type_supported(ring, AMDGPU_RESET_TYPE_IP_BLOCK_SOFT_RESET)) {
r = amdgpu_device_ip_soft_reset(ring, job->hw_fence);
if (!r) {
--
2.43.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v2 3/5] drm/amdgpu: add wedge event implementation
2026-08-13 17:06 [PATCH v2 1/5] drm/amdgpu: add gpu_recovery_allowed flag to amdgpu_device Pierre-Eric Pelloux-Prayer
2026-08-13 17:06 ` [PATCH v2 2/5] drm/amdgpu: honor gpu_recovery_allowed in amdgpu_job_timedout Pierre-Eric Pelloux-Prayer
@ 2026-08-13 17:07 ` Pierre-Eric Pelloux-Prayer
2026-08-13 17:07 ` [PATCH v2 4/5] drm/amdgpu: skip amdgpu_gart_unbind if the device is wedged Pierre-Eric Pelloux-Prayer
` (2 subsequent siblings)
4 siblings, 0 replies; 7+ messages in thread
From: Pierre-Eric Pelloux-Prayer @ 2026-08-13 17:07 UTC (permalink / raw)
To: Alex Deucher, Christian König, David Airlie, Simona Vetter,
Felix Kuehling
Cc: Pierre-Eric Pelloux-Prayer, amd-gfx, dri-devel, linux-kernel
When a job times out the kernel tries to reset the GPU to get
the system back to a normal state.
Sometimes it's useful to disable this process - for instance to
be able to inspect the hardware state at the time of the hang.
Since leaving fence unsignalled might affect the system's
stability, this commit enables the drm wedge framework for amdgpu.
When a hang condition is detected, the GPU isn't reset but all
the pending fences are signalled and the hung GPU cannot receive
new work. For KFD it's implemented by making
kfd_process_device_data_by_id return NULL if the requested GPU
is wedged.
When the user is done, it's possible to trigger a GPU
reset.
The auto-recovery is still enabled by default, and to use this
feature one has to either:
* boot with amdgpu.gpu_recovery=0: not recommended as it disables
recovery for all GPUs
* write 0 to /sys/kernel/debug/dri/X/amdgpu_gpu_recover to disable
auto-recovery for one GPU only
Runtime power management is disabled when the device is wedged
(because we can't submit any work to the GPU).
Assisted-by: Claude:Sonnet 5
Signed-off-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 13 +++++++++
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 4 +++
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 5 ++++
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 3 ++
drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 34 ++++++++++++++++++++--
drivers/gpu/drm/amd/amdkfd/kfd_process.c | 6 +++-
6 files changed, 61 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index 21b33dc34edf..039a92f2eb0f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -964,6 +964,13 @@ struct amdgpu_device {
*/
bool gpu_recovery_allowed;
+ /* Set to 1 when the device has been declared wedged following a
+ * hang: it no longer accepts new work and pending fences have been
+ * force-signalled. Reset to 0 once the device is reset back to a
+ * working state.
+ */
+ atomic_t wedge_status;
+
/* KFD
* Must be last --ends in a flexible-array member.
*/
@@ -1349,4 +1356,10 @@ void amdgpu_device_set_uid(struct amdgpu_uid *uid_info,
uint64_t uid);
uint64_t amdgpu_device_get_uid(struct amdgpu_uid *uid_info,
enum amdgpu_uid_type type, uint8_t inst);
+
+static inline bool amdgpu_device_is_wedged(struct amdgpu_device *adev)
+{
+ return atomic_read(&adev->wedge_status);
+}
+
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 5578d5f64937..949de13d7997 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -4035,6 +4035,8 @@ int amdgpu_device_init(struct amdgpu_device *adev,
}
adev->gpu_recovery_allowed = true;
+ if (atomic_xchg(&adev->wedge_status, 0))
+ pm_runtime_put_autosuspend(adev->dev);
fence_driver_init:
/* Fence driver */
@@ -5745,6 +5747,8 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,
dev_info(adev->dev, "GPU reset end with ret = %d\n", r);
atomic_set(&adev->reset_domain->reset_res, r);
+ if (atomic_xchg(&adev->wedge_status, 0))
+ pm_runtime_put_autosuspend(adev->dev);
if (!r) {
struct amdgpu_task_info *ti = NULL;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index 0ab380ca7e64..399e935df7b2 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -3006,10 +3006,15 @@ long amdgpu_drm_ioctl(struct file *filp,
unsigned int cmd, unsigned long arg)
{
struct drm_file *file_priv = filp->private_data;
+ struct amdgpu_device *adev;
struct drm_device *dev;
long ret;
dev = file_priv->minor->dev;
+ adev = drm_to_adev(dev);
+ if (amdgpu_device_is_wedged(adev))
+ return -ENODEV;
+
ret = pm_runtime_get_sync(dev->dev);
if (ret < 0)
goto out;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
index 3dc8faa091d7..f43d85ba4b78 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
@@ -716,6 +716,9 @@ void amdgpu_gmc_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid,
struct amdgpu_job *job;
int r;
+ if (amdgpu_device_is_wedged(adev))
+ return;
+
ring = to_amdgpu_ring(adev->mman.buffer_funcs_scheds[0]);
if (!hub->sdma_invalidation_workaround || vmid ||
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
index 43511e0419a1..41d083646da5 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
@@ -24,6 +24,7 @@
#include <linux/kthread.h>
#include <linux/wait.h>
#include <linux/sched.h>
+#include <linux/pm_runtime.h>
#include <drm/drm_drv.h>
@@ -185,9 +186,35 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job)
if (r)
dev_err(adev->dev, "GPU Recovery Failed: %d\n", r);
} else {
- drm_sched_suspend_timeout(&ring->sched);
- if (amdgpu_sriov_vf(adev))
+ if (amdgpu_sriov_vf(adev)) {
+ drm_sched_suspend_timeout(&ring->sched);
adev->virt.tdr_debug = true;
+ } else {
+ struct drm_gpu_scheduler *sched;
+ struct amdgpu_fence *guilty_fence;
+
+ /* Declare the device as wedged if it's not already. */
+ if (!atomic_xchg(&adev->wedge_status, 1)) {
+ pm_runtime_get_sync(adev->dev);
+
+ pci_clear_master(adev->pdev);
+
+ drm_dev_wedged_event(&adev->ddev, DRM_WEDGE_RECOVERY_REBIND |
+ DRM_WEDGE_RECOVERY_BUS_RESET, NULL);
+ }
+
+ guilty_fence = to_amdgpu_job(s_job)->hw_fence;
+
+ sched = &ring->sched;
+
+ /* Prevent anybody else from touching the ring buffer. */
+ drm_sched_wqueue_stop(sched);
+
+ amdgpu_fence_driver_force_completion(ring, &guilty_fence->base);
+
+ /* Start the scheduler again */
+ drm_sched_wqueue_start(sched);
+ }
}
exit:
@@ -453,7 +480,8 @@ static struct dma_fence *amdgpu_job_run(struct drm_sched_job *sched_job)
/* Skip job if VRAM is lost and never resubmit gangs */
if (job->generation != amdgpu_vm_generation(adev, job->vm) ||
- (job->job_run_counter && job->gang_submit))
+ (job->job_run_counter && job->gang_submit) ||
+ amdgpu_device_is_wedged(adev))
dma_fence_set_error(finished, -ECANCELED);
if (finished->error < 0) {
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index 0a7c1900da95..544dc960833b 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -2433,8 +2433,12 @@ struct kfd_process_device *kfd_process_device_data_by_id(struct kfd_process *p,
for (i = 0; i < p->n_pdds; i++) {
struct kfd_process_device *pdd = p->pdds[i];
- if (pdd->user_gpu_id == gpu_id)
+ if (pdd->user_gpu_id == gpu_id) {
+ if (amdgpu_device_is_wedged(pdd->dev->adev))
+ return NULL;
+
return pdd;
+ }
}
}
return NULL;
--
2.43.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v2 4/5] drm/amdgpu: skip amdgpu_gart_unbind if the device is wedged
2026-08-13 17:06 [PATCH v2 1/5] drm/amdgpu: add gpu_recovery_allowed flag to amdgpu_device Pierre-Eric Pelloux-Prayer
2026-08-13 17:06 ` [PATCH v2 2/5] drm/amdgpu: honor gpu_recovery_allowed in amdgpu_job_timedout Pierre-Eric Pelloux-Prayer
2026-08-13 17:07 ` [PATCH v2 3/5] drm/amdgpu: add wedge event implementation Pierre-Eric Pelloux-Prayer
@ 2026-08-13 17:07 ` Pierre-Eric Pelloux-Prayer
2026-08-14 7:49 ` Christian König
2026-08-13 17:07 ` [PATCH v2 5/5] drm/amdgpu: skip amdgpu_gmc_flush_gpu_tlb_pasid if " Pierre-Eric Pelloux-Prayer
2026-08-14 7:47 ` [PATCH v2 1/5] drm/amdgpu: add gpu_recovery_allowed flag to amdgpu_device Christian König
4 siblings, 1 reply; 7+ messages in thread
From: Pierre-Eric Pelloux-Prayer @ 2026-08-13 17:07 UTC (permalink / raw)
To: Alex Deucher, Christian König, David Airlie, Simona Vetter
Cc: Pierre-Eric Pelloux-Prayer, amd-gfx, dri-devel, linux-kernel
This operation requires access to the GPU.
Signed-off-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
index c4c21dbbbdbf..09ddfccb8174 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
@@ -315,7 +315,7 @@ void amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset,
uint64_t flags = 0;
int idx;
- if (!adev->gart.ptr)
+ if (!adev->gart.ptr || amdgpu_device_is_wedged(adev))
return;
if (!drm_dev_enter(adev_to_drm(adev), &idx))
--
2.43.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v2 5/5] drm/amdgpu: skip amdgpu_gmc_flush_gpu_tlb_pasid if device is wedged
2026-08-13 17:06 [PATCH v2 1/5] drm/amdgpu: add gpu_recovery_allowed flag to amdgpu_device Pierre-Eric Pelloux-Prayer
` (2 preceding siblings ...)
2026-08-13 17:07 ` [PATCH v2 4/5] drm/amdgpu: skip amdgpu_gart_unbind if the device is wedged Pierre-Eric Pelloux-Prayer
@ 2026-08-13 17:07 ` Pierre-Eric Pelloux-Prayer
2026-08-14 7:47 ` [PATCH v2 1/5] drm/amdgpu: add gpu_recovery_allowed flag to amdgpu_device Christian König
4 siblings, 0 replies; 7+ messages in thread
From: Pierre-Eric Pelloux-Prayer @ 2026-08-13 17:07 UTC (permalink / raw)
To: Alex Deucher, Christian König, David Airlie, Simona Vetter
Cc: Pierre-Eric Pelloux-Prayer, amd-gfx, dri-devel, linux-kernel
Otherwise it'll fail with: "failed to reg_write_reg_wait" and we're
going to fill the MES buffer which will prevent the reset to succeed.
Signed-off-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
index f43d85ba4b78..c33320a6413e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
@@ -786,6 +786,9 @@ int amdgpu_gmc_flush_gpu_tlb_pasid(struct amdgpu_device *adev, uint16_t pasid,
int r, cnt = 0;
uint32_t seq;
+ if (amdgpu_device_is_wedged(adev))
+ return -ENODEV;
+
/*
* A GPU reset should flush all TLBs anyway, so no need to do
* this while one is ongoing.
--
2.43.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH v2 1/5] drm/amdgpu: add gpu_recovery_allowed flag to amdgpu_device
2026-08-13 17:06 [PATCH v2 1/5] drm/amdgpu: add gpu_recovery_allowed flag to amdgpu_device Pierre-Eric Pelloux-Prayer
` (3 preceding siblings ...)
2026-08-13 17:07 ` [PATCH v2 5/5] drm/amdgpu: skip amdgpu_gmc_flush_gpu_tlb_pasid if " Pierre-Eric Pelloux-Prayer
@ 2026-08-14 7:47 ` Christian König
4 siblings, 0 replies; 7+ messages in thread
From: Christian König @ 2026-08-14 7:47 UTC (permalink / raw)
To: Pierre-Eric Pelloux-Prayer, Alex Deucher, David Airlie,
Simona Vetter
Cc: amd-gfx, dri-devel, linux-kernel
On 8/13/26 19:06, Pierre-Eric Pelloux-Prayer wrote:
> Add a per-device boolean to control whether GPU recovery is attempted
> on a hang, independently of the global amdgpu.gpu_recovery module
> parameter. It defaults to true and is exposed as a write to the
> existing amdgpu_gpu_recover debugfs file, so a single device can have
> auto-recovery disabled without affecting every other GPU in the
> system.
>
> amdgpu_device_should_recover_gpu() now takes this flag into account.
>
> Assisted-by: Claude:Sonnet 5
> Signed-off-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 7 +++++++
> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 5 +++++
> drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 21 +++++++++++++++++----
> 3 files changed, 29 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> index 7974f9b7944f..21b33dc34edf 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> @@ -957,6 +957,13 @@ struct amdgpu_device {
>
> struct amdgpu_uma_carveout_info uma_info;
>
> + /* Whether this device is allowed to attempt GPU recovery on a hang.
> + * Defaults to true; can be turned off per-device (e.g. via the
> + * amdgpu_gpu_recover debugfs file) independently of the global
> + * amdgpu.gpu_recovery module parameter.
> + */
> + bool gpu_recovery_allowed;
The funcationality sounds sane to me, but the naming is just horrible.
We should probably use something like gpu_recovery_enabled instead.
Regards,
Christian.
> +
> /* KFD
> * Must be last --ends in a flexible-array member.
> */
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> index 964efec0d335..5578d5f64937 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -4034,6 +4034,8 @@ int amdgpu_device_init(struct amdgpu_device *adev,
> }
> }
>
> + adev->gpu_recovery_allowed = true;
> +
> fence_driver_init:
> /* Fence driver */
> r = amdgpu_fence_driver_sw_init(adev);
> @@ -4834,6 +4836,9 @@ bool amdgpu_device_should_recover_gpu(struct amdgpu_device *adev)
> if (amdgpu_gpu_recovery == 0)
> goto disabled;
>
> + if (!adev->gpu_recovery_allowed)
> + goto disabled;
> +
> /* Skip soft reset check in fatal error mode */
> if (!amdgpu_ras_is_poison_mode_supported(adev))
> return true;
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
> index 3043ad041bb4..707e69d8bb11 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
> @@ -975,9 +975,13 @@ static int amdgpu_debugfs_fence_info_show(struct seq_file *m, void *unused)
> }
>
> /*
> - * amdgpu_debugfs_gpu_recover - manually trigger a gpu reset & recover
> + * amdgpu_debugfs_gpu_recover - manually trigger a gpu reset & recover,
> + * and control whether this device is allowed to auto-recover from a hang.
> *
> - * Manually trigger a gpu reset at the next fence wait.
> + * Read triggers a gpu reset at the next fence wait.
> + *
> + * Write 0/1 to disable/enable auto GPU recovery for this device
> + * (equivalent to amdgpu.gpu_recovery=0, but scoped to this device only).
> */
> static int gpu_recover_get(void *data, u64 *val)
> {
> @@ -1001,8 +1005,17 @@ static int gpu_recover_get(void *data, u64 *val)
> return 0;
> }
>
> +static int gpu_recover_set(void *data, u64 val)
> +{
> + struct amdgpu_device *adev = (struct amdgpu_device *)data;
> +
> + adev->gpu_recovery_allowed = !!val;
> +
> + return 0;
> +}
> +
> DEFINE_SHOW_ATTRIBUTE(amdgpu_debugfs_fence_info);
> -DEFINE_DEBUGFS_ATTRIBUTE(amdgpu_debugfs_gpu_recover_fops, gpu_recover_get, NULL,
> +DEFINE_DEBUGFS_ATTRIBUTE(amdgpu_debugfs_gpu_recover_fops, gpu_recover_get, gpu_recover_set,
> "%lld\n");
>
> static void amdgpu_debugfs_reset_work(struct work_struct *work)
> @@ -1037,7 +1050,7 @@ void amdgpu_debugfs_fence_init(struct amdgpu_device *adev)
> if (!amdgpu_sriov_vf(adev)) {
>
> INIT_WORK(&adev->reset_work, amdgpu_debugfs_reset_work);
> - debugfs_create_file("amdgpu_gpu_recover", 0444, root, adev,
> + debugfs_create_file("amdgpu_gpu_recover", 0644, root, adev,
> &amdgpu_debugfs_gpu_recover_fops);
> }
> #endif
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v2 4/5] drm/amdgpu: skip amdgpu_gart_unbind if the device is wedged
2026-08-13 17:07 ` [PATCH v2 4/5] drm/amdgpu: skip amdgpu_gart_unbind if the device is wedged Pierre-Eric Pelloux-Prayer
@ 2026-08-14 7:49 ` Christian König
0 siblings, 0 replies; 7+ messages in thread
From: Christian König @ 2026-08-14 7:49 UTC (permalink / raw)
To: Pierre-Eric Pelloux-Prayer, Alex Deucher, David Airlie,
Simona Vetter
Cc: amd-gfx, dri-devel, linux-kernel
On 8/13/26 19:07, Pierre-Eric Pelloux-Prayer wrote:
> This operation requires access to the GPU.
>
> Signed-off-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
> index c4c21dbbbdbf..09ddfccb8174 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
> @@ -315,7 +315,7 @@ void amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset,
> uint64_t flags = 0;
> int idx;
>
> - if (!adev->gart.ptr)
> + if (!adev->gart.ptr || amdgpu_device_is_wedged(adev))
That is a good start but certainly not sufficient.
We need that on every occasion drm_dev_enter() is called in the driver.
Regards,
Christian.
> return;
>
> if (!drm_dev_enter(adev_to_drm(adev), &idx))
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2026-08-14 7:49 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-08-13 17:06 [PATCH v2 1/5] drm/amdgpu: add gpu_recovery_allowed flag to amdgpu_device Pierre-Eric Pelloux-Prayer
2026-08-13 17:06 ` [PATCH v2 2/5] drm/amdgpu: honor gpu_recovery_allowed in amdgpu_job_timedout Pierre-Eric Pelloux-Prayer
2026-08-13 17:07 ` [PATCH v2 3/5] drm/amdgpu: add wedge event implementation Pierre-Eric Pelloux-Prayer
2026-08-13 17:07 ` [PATCH v2 4/5] drm/amdgpu: skip amdgpu_gart_unbind if the device is wedged Pierre-Eric Pelloux-Prayer
2026-08-14 7:49 ` Christian König
2026-08-13 17:07 ` [PATCH v2 5/5] drm/amdgpu: skip amdgpu_gmc_flush_gpu_tlb_pasid if " Pierre-Eric Pelloux-Prayer
2026-08-14 7:47 ` [PATCH v2 1/5] drm/amdgpu: add gpu_recovery_allowed flag to amdgpu_device Christian König
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox