* [PATCH 1/3] drm/amdkfd: only flush the TLB of devices whose queues got suspended
@ 2026-11-18 5:49 Jesse Zhang
2026-11-18 5:49 ` [PATCH 2/3] drm/amdkfd: return the memory violation data to the debugger Jesse Zhang
2026-11-18 5:50 ` [PATCH 3/3] drm/amdkfd: don't keep the debugger pointer when DBG_TRAP_ENABLE fails Jesse Zhang
0 siblings, 2 replies; 5+ messages in thread
From: Jesse Zhang @ 2026-11-18 5:49 UTC (permalink / raw)
To: amd-gfx; +Cc: Alexander.Deucher, Felix Kuehling, Jonathan Kim, Jesse Zhang
suspend_queues() goes through all devices of the process, and on MES does
the debug memory fence and a heavy-weight TLB flush when total_suspended,
the number of queues suspended so far on any device, is non-zero. On a
later device without suspended queues that flushes the TLB of a VM the
process may not even have: kfd_flush_tlb() dereferences pdd->drm_priv,
which is NULL when the process never acquired a VM on that GPU.
That is a process using only one GPU of a system with more, e.g. a dGPU
plus the APU's iGPU, stopped by a debugger. On navi31 with the iGPU,
a debugger suspending a compute queue of a ptraced child that only
acquired the navi31 VM:
Oops: general protection fault, probably for non-canonical address 0xdffffc000000002f: 0000 [#1] SMP KASAN NOPTI
KASAN: null-ptr-deref in range [0x0000000000000178-0x000000000000017f]
RIP: 0010:suspend_queues+0x104b/0x1a80 [amdgpu]
Call Trace:
kfd_ioctl_set_debug_trap+0x1234/0x1720 [amdgpu]
kfd_ioctl+0x46a/0xb70 [amdgpu]
and the debugger ends up in D state.
Count the queues MES suspended per device and only fence and flush for
those devices. With that the queue is suspended and resumed (status 0) and
the kfdtest KFDDBGTest tests pass.
Fixes: ff114e705d6a ("drm/amdkfd: Add TLB flush after MES queue eviction/suspension")
Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index b5d7d13f31a5..a50a49d6ed01 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -3952,7 +3952,7 @@ int suspend_queues(struct kfd_process *p,
struct device *dev = dqm->dev->adev->dev;
struct qcm_process_device *qpd = &pdd->qpd;
struct queue *q;
- int r, per_device_suspended = 0;
+ int r, per_device_suspended = 0, mes_suspended = 0;
mutex_lock(&p->event_mutex);
dqm_lock(dqm);
@@ -3974,7 +3974,7 @@ int suspend_queues(struct kfd_process *p,
~exception_clear_mask;
if (is_mes)
- total_suspended++;
+ mes_suspended++;
else
per_device_suspended++;
} else if (err != -EBUSY) {
@@ -3987,7 +3987,12 @@ int suspend_queues(struct kfd_process *p,
if (!per_device_suspended) {
dqm_unlock(dqm);
mutex_unlock(&p->event_mutex);
- if (total_suspended) {
+ /*
+ * Only for the queues of this device, the process may
+ * not even have a VM on the other devices.
+ */
+ if (mes_suspended) {
+ total_suspended += mes_suspended;
amdgpu_amdkfd_debug_mem_fence(dqm->dev->adev);
/* Heavy-weight TLB flush after MES suspends queues */
kfd_flush_tlb(pdd);
--
2.50.1
^ permalink raw reply related [flat|nested] 5+ messages in thread* [PATCH 2/3] drm/amdkfd: return the memory violation data to the debugger
2026-11-18 5:49 [PATCH 1/3] drm/amdkfd: only flush the TLB of devices whose queues got suspended Jesse Zhang
@ 2026-11-18 5:49 ` Jesse Zhang
2026-11-18 6:06 ` sashiko-bot
2026-11-18 5:50 ` [PATCH 3/3] drm/amdkfd: don't keep the debugger pointer when DBG_TRAP_ENABLE fails Jesse Zhang
1 sibling, 1 reply; 5+ messages in thread
From: Jesse Zhang @ 2026-11-18 5:49 UTC (permalink / raw)
To: amd-gfx; +Cc: Alexander.Deucher, Felix Kuehling, Jonathan Kim, Jesse Zhang
kfd_dbg_ev_raise() saves the memory exception data of a GPU VM fault for a
debugger subscribed to EC_DEVICE_MEMORY_VIOLATION, but never sets
vm_fault_exc_data_size. QUERY_EXCEPTION_INFO copies that many bytes, so the
debugger always gets 0 bytes and no fault address.
On navi48, with a process debugging itself and a KFD SDMA queue writing to
an unmapped VA:
amdgpu 0000:03:00.0: [gfxhub] page fault (src_id:0 ring:40 vmid:8 pasid:63)
amdgpu 0000:03:00.0: in page starting at address 0x0000000060200000 from client 10
amdgpu 0000:03:00.0: Faulty UTCL2 client ID: SDMA0 (0xd)
memory violation info: 0 bytes, va 0x0 gpu_id 0, fault at 0x60200000
With the size set:
memory violation info: 32 bytes, va 0x60200000 gpu_id 51573, fault at 0x60200000
The data is also only freed by a query clearing the exception, free it with
the process device as well.
Fixes: 2b36de971d25 ("drm/amdkfd: add debug query exception info operation")
Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_debug.c | 4 +++-
drivers/gpu/drm/amd/amdkfd/kfd_process.c | 1 +
2 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_debug.c b/drivers/gpu/drm/amd/amdkfd/kfd_debug.c
index d5c87de519ac..14e7ee30cbdf 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_debug.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_debug.c
@@ -143,7 +143,9 @@ bool kfd_dbg_ev_raise(uint64_t event_mask,
exception_data,
exception_data_size,
GFP_KERNEL);
- if (!pdd->vm_fault_exc_data)
+ if (pdd->vm_fault_exc_data)
+ pdd->vm_fault_exc_data_size = exception_data_size;
+ else
pr_debug("Failed to allocate exception data memory");
} else {
pr_debug("Debugger exception data not saved\n");
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index f959f19d92d2..d57eebec063c 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -1288,6 +1288,7 @@ static void kfd_process_destroy_pdds(struct kfd_process *p)
atomic_dec(&pdd->dev->kfd->kfd_processes_count);
mutex_destroy(&pdd->doorbell_mutex);
+ kfree(pdd->vm_fault_exc_data);
kfree(pdd);
p->pdds[i] = NULL;
}
--
2.50.1
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [PATCH 2/3] drm/amdkfd: return the memory violation data to the debugger
2026-11-18 5:49 ` [PATCH 2/3] drm/amdkfd: return the memory violation data to the debugger Jesse Zhang
@ 2026-11-18 6:06 ` sashiko-bot
0 siblings, 0 replies; 5+ messages in thread
From: sashiko-bot @ 2026-11-18 6:06 UTC (permalink / raw)
To: Jesse Zhang, amd-gfx
Thank you for your contribution! Sashiko AI review found no new issues
in this patch, but noticed 1 related pre-existing issue it does not
introduce:
- [Critical] drm/amdkfd: use-after-free in kfd_process_destroy_pdds()
[1]
--
commit 27e966c319a74439c4a48d13a51742cea89473d5
Author: Jesse Zhang <Jesse.Zhang@amd.com>
drm/amdkfd: return the memory violation data to the debugger
This commit records the size of GPU VM fault exception data in
kfd_dbg_ev_raise() and frees vm_fault_exc_data when destroying the process
device.
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> index 546721b4d59ac..16131fdc55468 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> @@ -1253,6 +1253,7 @@ static void kfd_process_destroy_pdds(struct kfd_process *p)
> atomic_dec(&pdd->dev->kfd->kfd_processes_count);
> mutex_destroy(&pdd->doorbell_mutex);
>
> + kfree(pdd->vm_fault_exc_data);
> kfree(pdd);
> p->pdds[i] = NULL;
> }
[Severity: Critical]
This problem wasn't introduced by this patch, but can
kfd_process_destroy_pdds() dereference freed pdd->dev and pdd->dev->kfd
memory [1] during device teardown?
When a KFD process exits, it is removed from kfd_processes_table before its
release work runs on kfd_process_wq. If kgd2kfd_teardown_processes() runs
after the last process is removed from kfd_processes_table, it returns early
and skips waiting for kfd_processes_count to reach zero:
drivers/gpu/drm/amd/amdkfd/kfd_device.c:kgd2kfd_teardown_processes() {
...
mutex_lock(&kfd_processes_mutex);
if (hash_empty(kfd_processes_table)) {
mutex_unlock(&kfd_processes_mutex);
return;
}
...
mutex_unlock(&kfd_processes_mutex);
/* wait all kfd processes use adev terminate */
while (!!atomic_read(&adev->kfd.dev->kfd_processes_count))
cond_resched();
}
CPU0 CPU1 (kworker)
kgd2kfd_teardown_processes()
if (hash_empty(kfd_processes_table))
return;
<device teardown frees kfd_node/kfd_dev>
kfd_process_wq_release()
kfd_process_destroy_pdds()
atomic_dec(&pdd->dev->kfd->...);
<-- use-after-free
[1] https://sashiko.dev/#/bug/linux-89a2e0c3-8eea-4d17-956c-8aac02272705
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261118055009.3005630-1-Jesse.Zhang@amd.com?part=2
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 3/3] drm/amdkfd: don't keep the debugger pointer when DBG_TRAP_ENABLE fails
2026-11-18 5:49 [PATCH 1/3] drm/amdkfd: only flush the TLB of devices whose queues got suspended Jesse Zhang
2026-11-18 5:49 ` [PATCH 2/3] drm/amdkfd: return the memory violation data to the debugger Jesse Zhang
@ 2026-11-18 5:50 ` Jesse Zhang
2026-11-18 6:07 ` sashiko-bot
1 sibling, 1 reply; 5+ messages in thread
From: Jesse Zhang @ 2026-11-18 5:50 UTC (permalink / raw)
To: amd-gfx; +Cc: Alexander.Deucher, Felix Kuehling, Jonathan Kim, Jesse Zhang
KFD_IOC_DBG_TRAP_ENABLE stores the caller in target->debugger_process
before calling kfd_dbg_trap_enable(), and nothing resets it when that
fails (-EALREADY, -ENODEV, -EBUSY, -EBADF for a bad dbg_fd, -EFAULT).
The permission check at the top of kfd_ioctl_set_debug_trap() accepts
any caller equal to target->debugger_process, so the caller keeps
debug access after it stopped ptracing the target. The pointer is also
not reference counted, so it dangles once the caller exits, and a later
successful ENABLE by the target itself increments
debugged_process_count of the freed process.
On navi31, a parent that ptraces its child, tries to enable debugging
with dbg_fd = -1 (-EBADF) and then detaches can still enable debugging
on the child with a valid fd:
1. ptraced, ENABLE with dbg_fd=-1 -> -9 (Bad file descriptor)
2. ptrace detached
3. not ptraced, ENABLE with valid fd -> 0 (success)
Restore the previous debugger when enabling fails. Step 3 now fails with
-EPERM.
Return -EALREADY for an already enabled session before touching the
pointer at all: the exit cleanup of the current debugger looks for its
targets through target->debugger_process without holding
target->mutex, so even a temporary change of the pointer could make it
skip the target and leave a dangling pointer behind.
Fixes: 0ab2d7532b05 ("drm/amdkfd: prepare per-process debug enable and disable")
Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 18 +++++++++++++++++-
1 file changed, 17 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index 3d2136adb62b..c42c30b459a3 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -3226,7 +3226,19 @@ static int kfd_ioctl_set_debug_trap(struct file *filep, struct kfd_process *p, v
}
switch (args->op) {
- case KFD_IOC_DBG_TRAP_ENABLE:
+ case KFD_IOC_DBG_TRAP_ENABLE: {
+ struct kfd_process *old_debugger = target->debugger_process;
+
+ /*
+ * Don't touch the debugger of a session that is already enabled,
+ * not even temporarily: its exit cleanup looks the pointer up
+ * without target->mutex.
+ */
+ if (target->debug_trap_enabled) {
+ r = -EALREADY;
+ break;
+ }
+
if (target != p)
target->debugger_process = p;
@@ -3236,8 +3248,12 @@ static int kfd_ioctl_set_debug_trap(struct file *filep, struct kfd_process *p, v
&args->enable.rinfo_size);
if (!r)
target->exception_enable_mask = args->enable.exception_mask;
+ else
+ /* the debugger check above trusts this pointer */
+ target->debugger_process = old_debugger;
break;
+ }
case KFD_IOC_DBG_TRAP_DISABLE:
r = kfd_dbg_trap_disable(target, true);
break;
--
2.50.1
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [PATCH 3/3] drm/amdkfd: don't keep the debugger pointer when DBG_TRAP_ENABLE fails
2026-11-18 5:50 ` [PATCH 3/3] drm/amdkfd: don't keep the debugger pointer when DBG_TRAP_ENABLE fails Jesse Zhang
@ 2026-11-18 6:07 ` sashiko-bot
0 siblings, 0 replies; 5+ messages in thread
From: sashiko-bot @ 2026-11-18 6:07 UTC (permalink / raw)
To: Jesse Zhang, amd-gfx
Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] drm/amdkfd: TOCTOU race on debugger_process in
kfd_ioctl_set_debug_trap()
--
commit 7ecd0279fcac9bb56764baeea1894a732f3b3ff3
Author: Jesse Zhang <Jesse.Zhang@amd.com>
drm/amdkfd: don't keep the debugger pointer when DBG_TRAP_ENABLE fails
This commit restores target->debugger_process when kfd_dbg_trap_enable()
fails and returns -EALREADY before modifying target->debugger_process if
debugging is already enabled.
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
> index 5a7f4f0cd0ce7..9969d48ca6c7f 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
> @@ -3226,7 +3226,19 @@ static int kfd_ioctl_set_debug_trap(struct file *filep, struct kfd_process *p, v
> }
>
> switch (args->op) {
> - case KFD_IOC_DBG_TRAP_ENABLE:
> + case KFD_IOC_DBG_TRAP_ENABLE: {
> + struct kfd_process *old_debugger = target->debugger_process;
> +
> + /*
> + * Don't touch the debugger of a session that is already enabled,
> + * not even temporarily: its exit cleanup looks the pointer up
> + * without target->mutex.
> + */
> + if (target->debug_trap_enabled) {
> + r = -EALREADY;
> + break;
> + }
> +
> if (target != p)
> target->debugger_process = p;
>
> @@ -3236,8 +3248,12 @@ static int kfd_ioctl_set_debug_trap(struct file *filep, struct kfd_process *p, v
> &args->enable.rinfo_size);
> if (!r)
> target->exception_enable_mask = args->enable.exception_mask;
> + else
> + /* the debugger check above trusts this pointer */
> + target->debugger_process = old_debugger;
>
> break;
> + }
> case KFD_IOC_DBG_TRAP_DISABLE:
> r = kfd_dbg_trap_disable(target, true);
> break;
[Severity: Medium]
Can a second thread in the debugger process still bypass the ptrace check
by reading target->debugger_process before kfd_dbg_trap_enable() rolls it
back?
kfd_ioctl_set_debug_trap() checks target->debugger_process before
acquiring target->mutex and does not re-check it after locking:
drivers/gpu/drm/amd/amdkfd/kfd_chardev.c:kfd_ioctl_set_debug_trap() {
...
rcu_read_lock();
if (target != p && ptrace_parent(target->lead_thread) != current
&& target->debugger_process != p) {
pr_err("PID %i is not PTRACED and cannot be debugged\n", args->pid);
r = -EPERM;
}
rcu_read_unlock();
if (r)
goto out;
mutex_lock(&target->mutex);
...
}
Because target->debugger_process is set to p while kfd_dbg_trap_enable()
sleeps in copy_to_user(), another thread in p can pass the unlocked
permission check and wait on target->mutex:
CPU0 (ptrace thread in p) CPU1 (other thread in p)
kfd_ioctl_set_debug_trap(ENABLE)
mutex_lock(&target->mutex)
target->debugger_process = p
kfd_dbg_trap_enable()
copy_to_user(...)
kfd_ioctl_set_debug_trap(ENABLE)
/* sees target->debugger_process == p */
mutex_lock(&target->mutex)
/* fails with -EFAULT */
target->debugger_process = NULL
mutex_unlock(&target->mutex)
/* detaches ptrace */
target->debugger_process = p
kfd_dbg_trap_enable() <-- succeeds without ptrace
The same window exists during KFD_IOC_DBG_TRAP_DISABLE while
kfd_dbg_trap_disable() sleeps in cancel_work_sync() under target->mutex
before clearing target->debugger_process. Performing the permission check
inside target->mutex and only setting target->debugger_process after
kfd_dbg_trap_enable() succeeds (passing p into kfd_dbg_trap_enable() or
updating debugged_process_count afterwards) would close this race.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261118055009.3005630-1-Jesse.Zhang@amd.com?part=3
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2026-11-18 6:07 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-18 5:49 [PATCH 1/3] drm/amdkfd: only flush the TLB of devices whose queues got suspended Jesse Zhang
2026-11-18 5:49 ` [PATCH 2/3] drm/amdkfd: return the memory violation data to the debugger Jesse Zhang
2026-11-18 6:06 ` sashiko-bot
2026-11-18 5:50 ` [PATCH 3/3] drm/amdkfd: don't keep the debugger pointer when DBG_TRAP_ENABLE fails Jesse Zhang
2026-11-18 6:07 ` sashiko-bot
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.