* [PATCH V4 00/18] amdkfd: Implement kfd multiple contexts
@ 2025-09-23 7:25 Zhu Lingshan
2025-09-23 7:25 ` [PATCH V4 01/18] amdkfd: enlarge the hashtable of kfd_process Zhu Lingshan
` (17 more replies)
0 siblings, 18 replies; 28+ messages in thread
From: Zhu Lingshan @ 2025-09-23 7:25 UTC (permalink / raw)
To: felix.kuehling, alexander.deucher; +Cc: ray.huang, amd-gfx, Zhu Lingshan
Currently kfd manages kfd_process in a one context (kfd_process)
per program manner, thus each user space program
only onws one kfd context (kfd_process).
This model works fine for most of the programs, but imperfect
for a hypervisor like QEMU. Because all programs in the guest
user space share the same only one kfd context, which is
problematic, including but not limited to:
As illustrated in Figure 1, all guest user space programs share the same fd of /dev/kfd
and the same kfd_process, and the same PASID leading to the same
GPU_VM address space. Therefore the IOVA range of each
guest user space programs are not isolated,
they can attack each other through GPU DMA.
+----------------------------------------------------------------------------------+
| |
| +-----------+ +-----------+ +------------+ +------------+ |
| | | | | | | | | |
| | Program 1 | | Program 2 | | Program 3 | | Program N | |
| | | | | | | | | |
| +----+------+ +--------+--+ +--+---------+ +-----+------+ |
| | | | | |
| | | | | Guest |
| | | | | |
+-------+----------------------+------------+----------------------+---------------+
| | | |
| | | |
| | | |
| | | |
| +--+------------+---+ |
| | file descriptor | |
+-------------------+ of /dev/kfd +------------------+
| opened by QEMU |
| |
+---------+---------+ User Space
| QEMU
|
---------------------------------------+-----------------------------------------------------
| Kernel Space
| KFD Module
|
+--------+--------+
| |
| kfd_process |<------------------The only one KFD context
| |
+--------+--------+
|
+--------+--------+
| PASID |
+--------+--------+
|
+--------+--------+
| GPU_VM |
+-----------------+
Fiture 1
This series implements a multiple contexts solution:
- Allow each program to create and hold multiple contexts (kfd processes)
- Each context has its own fd of /dev/kfd and an exclusive kfd_process,
which is a secondary kfd context. So that PASID/GPU VM isolates their IOVA address spaces.
Therefore, they can not attack each other through GPU DMA.
The design is illustrated in Figure 2 below:
+---------------------------------------------------------------------------------------------------------+
| |
| |
| |
| +----------------------------------------------------------------------------------+ |
| | | |
| | +-----------+ +-----------+ +-----------+ +-----------+ | |
| | | | | | | | | | | |
| | | Program 1 | | Program 2 | | Program 3 | | Program N | | |
| | | | | | | | | | | |
| | +-----+-----+ +-----+-----+ +-----+-----+ +-----+-----+ | |
| | | | | | | |
| | | | | | Guest | |
| | | | | | | |
| +-------+------------------+-----------------+----------------+--------------------+ |
| | | | | QEMU |
| | | | | |
+---------------+------------------+-----------------+----------------+--------------------------+--------+
| | | | |
| | | | |
| | | | |
+---+----+ +---+----+ +---+----+ +---+----+ +---+-----+
| | | | | | | | | Primary |
| FD 1 | | FD 2 | | FD 3 | | FD 4 | | FD |
| | | | | | | | | |
+---+----+ +---+----+ +---+----+ +----+---+ +----+----+
| | | | | User Space
| | | | |
-------------------+------------------+-----------------+-----------------+--------------------------+----------------------------
| | | | | Kernel SPace
| | | | |
| | | | |
+--------------------------------------------------------------------------------------------------------------------------+
| +------+------+ +------+------+ +------+------+ +------+------+ +------+------+ |
| | Secondary | | Secondary | | Secondary | | Secondary | | Primary | KFD Module |
| |kfd_process 1| |kfd_process 2| |kfd_process 3| |kfd_process 4| | kfd_process | |
| | | | | | | | | | | |
| +------+------+ +------+------+ +------+------+ +------+------+ +------+------+ |
| | | | | | |
| +------+------+ +------+------+ +------+------+ +------+------+ +------+------+ |
| | PASID | | PASID | | PASID | | PASID | | PASID | |
| +------+------+ +------+------+ +------+------+ +------+------+ +------+------+ |
| | | | | | |
| | | | | | |
| +------+------+ +------+------+ +------+------+ +------+------+ +------+------+ |
| | GPU_VM | | GPU_VM | | GPU_VM | | GPU_VM | | GPU_VM | |
| +-------------+ +-------------+ +-------------+ +-------------+ +-------------+ |
| |
+--------------------------------------------------------------------------------------------------------------------------+
Figure 2
The relevant reference user space rocm changes could be found at:
https://github.com/AMD-ROCm-Internal/rocm-systems/pull/78
https://github.com/AMD-ROCm-Internal/rocm-systems/pull/110
Thanks!
Zhu Lingshan (18):
amdkfd: enlarge the hashtable of kfd_process
amdkfd: mark the first kfd_process as the primary one
amdkfd: find_process_by_mm always return the primary context
amdkfd: Introduce kfd_create_process_sysfs as a separate function
amdkfd: destroy kfd secondary contexts through fd close
amdkfd: process svm ioctl only on the primary kfd process
amdkfd: process USERPTR allocation only on the primary kfd process
amdkfd: identify a secondary kfd process by its id
amdkfd: find kfd_process by filep->private_data in kfd_mmap
amdkfd: remove DIQ support
amdkfd: process pointer of a HIQ should be NULL
amdkfd: remove test_kq
amdkfd: introduce new helper kfd_lookup_process_by_id
amdkfd: record kfd process id into kfd process_info
amdkfd: record kfd process id in amdkfd_fence
amdkfd: fence handler evict and restore a kfd process by its id
amdkfd: set_debug_trap ioctl only works on a primary kfd_process
target
amdkfd: introduce new ioctl AMDKFD_IOC_CREATE_PROCESS
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 8 +-
.../gpu/drm/amd/amdgpu/amdgpu_amdkfd_fence.c | 10 +-
.../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 8 +-
drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 102 +++++++-
drivers/gpu/drm/amd/amdkfd/kfd_device.c | 7 +-
.../drm/amd/amdkfd/kfd_device_queue_manager.c | 6 +-
drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c | 61 +----
.../drm/amd/amdkfd/kfd_packet_manager_v9.c | 4 -
.../drm/amd/amdkfd/kfd_packet_manager_vi.c | 4 -
drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 17 +-
drivers/gpu/drm/amd/amdkfd/kfd_process.c | 244 +++++++++++++-----
.../amd/amdkfd/kfd_process_queue_manager.c | 39 +--
drivers/gpu/drm/amd/amdkfd/kfd_svm.c | 2 +-
include/uapi/linux/kfd_ioctl.h | 8 +-
14 files changed, 321 insertions(+), 199 deletions(-)
--
2.51.0
^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH V4 01/18] amdkfd: enlarge the hashtable of kfd_process
2025-09-23 7:25 [PATCH V4 00/18] amdkfd: Implement kfd multiple contexts Zhu Lingshan
@ 2025-09-23 7:25 ` Zhu Lingshan
2025-09-23 7:25 ` [PATCH V4 02/18] amdkfd: mark the first kfd_process as the primary one Zhu Lingshan
` (16 subsequent siblings)
17 siblings, 0 replies; 28+ messages in thread
From: Zhu Lingshan @ 2025-09-23 7:25 UTC (permalink / raw)
To: felix.kuehling, alexander.deucher; +Cc: ray.huang, amd-gfx, Zhu Lingshan
This commit enlarges the hashtable size of
kfd_process to 256, because of the multiple
contexts feature allowing each application
create multiple kfd_processes
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
index 67694bcd9464..8a33e6ee6369 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
@@ -1011,7 +1011,7 @@ struct kfd_process {
bool gpu_page_fault;
};
-#define KFD_PROCESS_TABLE_SIZE 5 /* bits: 32 entries */
+#define KFD_PROCESS_TABLE_SIZE 8 /* bits: 256 entries */
extern DECLARE_HASHTABLE(kfd_processes_table, KFD_PROCESS_TABLE_SIZE);
extern struct srcu_struct kfd_processes_srcu;
--
2.51.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH V4 02/18] amdkfd: mark the first kfd_process as the primary one
2025-09-23 7:25 [PATCH V4 00/18] amdkfd: Implement kfd multiple contexts Zhu Lingshan
2025-09-23 7:25 ` [PATCH V4 01/18] amdkfd: enlarge the hashtable of kfd_process Zhu Lingshan
@ 2025-09-23 7:25 ` Zhu Lingshan
2025-09-23 7:25 ` [PATCH V4 03/18] amdkfd: find_process_by_mm always return the primary context Zhu Lingshan
` (15 subsequent siblings)
17 siblings, 0 replies; 28+ messages in thread
From: Zhu Lingshan @ 2025-09-23 7:25 UTC (permalink / raw)
To: felix.kuehling, alexander.deucher; +Cc: ray.huang, amd-gfx, Zhu Lingshan
The first kfd_process is created through open(),
this commit marks it as the primary kfd_process.
Only the primary process should register the mmu_notifier.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 3 +++
drivers/gpu/drm/amd/amdkfd/kfd_process.c | 20 ++++++++++++--------
2 files changed, 15 insertions(+), 8 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
index 8a33e6ee6369..2b95f37c1af8 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
@@ -1009,6 +1009,9 @@ struct kfd_process {
/* if gpu page fault sent to KFD */
bool gpu_page_fault;
+
+ /* indicating whether this is a primary kfd_process */
+ bool primary;
};
#define KFD_PROCESS_TABLE_SIZE 8 /* bits: 256 entries */
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index 5be28c6c4f6a..762e306d85db 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -68,7 +68,7 @@ static struct workqueue_struct *kfd_restore_wq;
static struct kfd_process *find_process(const struct task_struct *thread,
bool ref);
static void kfd_process_ref_release(struct kref *ref);
-static struct kfd_process *create_process(const struct task_struct *thread);
+static struct kfd_process *create_process(const struct task_struct *thread, bool primary);
static void evict_process_worker(struct work_struct *work);
static void restore_process_worker(struct work_struct *work);
@@ -867,7 +867,7 @@ struct kfd_process *kfd_create_process(struct task_struct *thread)
if (process) {
pr_debug("Process already found\n");
} else {
- process = create_process(thread);
+ process = create_process(thread, true);
if (IS_ERR(process))
goto out;
@@ -1510,7 +1510,7 @@ void kfd_process_set_trap_debug_flag(struct qcm_process_device *qpd,
* On return the kfd_process is fully operational and will be freed when the
* mm is released
*/
-static struct kfd_process *create_process(const struct task_struct *thread)
+static struct kfd_process *create_process(const struct task_struct *thread, bool primary)
{
struct kfd_process *process;
struct mmu_notifier *mn;
@@ -1526,6 +1526,8 @@ static struct kfd_process *create_process(const struct task_struct *thread)
process->lead_thread = thread->group_leader;
process->n_pdds = 0;
process->queues_paused = false;
+ process->primary = primary;
+
INIT_DELAYED_WORK(&process->eviction_work, evict_process_worker);
INIT_DELAYED_WORK(&process->restore_work, restore_process_worker);
process->last_restore_timestamp = get_jiffies_64();
@@ -1569,12 +1571,14 @@ static struct kfd_process *create_process(const struct task_struct *thread)
* After this point, mmu_notifier_put will trigger the cleanup by
* dropping the last process reference in the free_notifier.
*/
- mn = mmu_notifier_get(&kfd_process_mmu_notifier_ops, process->mm);
- if (IS_ERR(mn)) {
- err = PTR_ERR(mn);
- goto err_register_notifier;
+ if (primary) {
+ mn = mmu_notifier_get(&kfd_process_mmu_notifier_ops, process->mm);
+ if (IS_ERR(mn)) {
+ err = PTR_ERR(mn);
+ goto err_register_notifier;
+ }
+ BUG_ON(mn != &process->mmu_notifier);
}
- BUG_ON(mn != &process->mmu_notifier);
kfd_unref_process(process);
get_task_struct(process->lead_thread);
--
2.51.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH V4 03/18] amdkfd: find_process_by_mm always return the primary context
2025-09-23 7:25 [PATCH V4 00/18] amdkfd: Implement kfd multiple contexts Zhu Lingshan
2025-09-23 7:25 ` [PATCH V4 01/18] amdkfd: enlarge the hashtable of kfd_process Zhu Lingshan
2025-09-23 7:25 ` [PATCH V4 02/18] amdkfd: mark the first kfd_process as the primary one Zhu Lingshan
@ 2025-09-23 7:25 ` Zhu Lingshan
2025-09-23 7:25 ` [PATCH V4 04/18] amdkfd: Introduce kfd_create_process_sysfs as a separate function Zhu Lingshan
` (14 subsequent siblings)
17 siblings, 0 replies; 28+ messages in thread
From: Zhu Lingshan @ 2025-09-23 7:25 UTC (permalink / raw)
To: felix.kuehling, alexander.deucher; +Cc: ray.huang, amd-gfx, Zhu Lingshan
Up until this commit, the kfd multiple contexts feature has
not been fully implemented in mainline kernel yet.
For backawrd compatibility, not break existing use cases,
this commit changes function find_process_by_mm, let it
always return the primary kfd_process.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_process.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index 762e306d85db..88421e57a072 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -935,7 +935,7 @@ static struct kfd_process *find_process_by_mm(const struct mm_struct *mm)
hash_for_each_possible_rcu(kfd_processes_table, process,
kfd_processes, (uintptr_t)mm)
- if (process->mm == mm)
+ if (process->mm == mm && process->primary)
return process;
return NULL;
--
2.51.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH V4 04/18] amdkfd: Introduce kfd_create_process_sysfs as a separate function
2025-09-23 7:25 [PATCH V4 00/18] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (2 preceding siblings ...)
2025-09-23 7:25 ` [PATCH V4 03/18] amdkfd: find_process_by_mm always return the primary context Zhu Lingshan
@ 2025-09-23 7:25 ` Zhu Lingshan
2025-09-23 7:25 ` [PATCH V4 05/18] amdkfd: destroy kfd secondary contexts through fd close Zhu Lingshan
` (13 subsequent siblings)
17 siblings, 0 replies; 28+ messages in thread
From: Zhu Lingshan @ 2025-09-23 7:25 UTC (permalink / raw)
To: felix.kuehling, alexander.deucher; +Cc: ray.huang, amd-gfx, Zhu Lingshan
KFD creates sysfs entries for a kfd_process in
function kfd_create_process when creating it.
This commit extracts the code creating sysfs
entries to a separate function because it
would be invoked in other code path like
creating secondary kfd contexts (kfd_process).
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 1 +
drivers/gpu/drm/amd/amdkfd/kfd_process.c | 66 +++++++++++++++---------
2 files changed, 42 insertions(+), 25 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
index 2b95f37c1af8..0818705820c6 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
@@ -1043,6 +1043,7 @@ int kfd_process_create_wq(void);
void kfd_process_destroy_wq(void);
void kfd_cleanup_processes(void);
struct kfd_process *kfd_create_process(struct task_struct *thread);
+int kfd_create_process_sysfs(struct kfd_process *process);
struct kfd_process *kfd_get_process(const struct task_struct *task);
struct kfd_process *kfd_lookup_process_by_pasid(u32 pasid,
struct kfd_process_device **pdd);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index 88421e57a072..bce7e35a15c9 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -825,6 +825,44 @@ static void kfd_process_device_destroy_ib_mem(struct kfd_process_device *pdd)
kfd_process_free_gpuvm(qpd->ib_mem, pdd, &qpd->ib_kaddr);
}
+int kfd_create_process_sysfs(struct kfd_process *process)
+{
+ int ret;
+
+ if (process->kobj) {
+ pr_warn("kobject already exsists for the kfd_process\n");
+ return -EINVAL;
+ }
+
+ process->kobj = kfd_alloc_struct(process->kobj);
+ if (!process->kobj) {
+ pr_warn("Creating procfs kobject failed");
+ return -ENOMEM;
+ }
+ ret = kobject_init_and_add(process->kobj, &procfs_type,
+ procfs.kobj, "%d",
+ (int)process->lead_thread->pid);
+ if (ret) {
+ pr_warn("Creating procfs pid directory failed");
+ kobject_put(process->kobj);
+ return ret;
+ }
+
+ kfd_sysfs_create_file(process->kobj, &process->attr_pasid,
+ "pasid");
+
+ process->kobj_queues = kobject_create_and_add("queues",
+ process->kobj);
+ if (!process->kobj_queues)
+ pr_warn("Creating KFD proc/queues folder failed");
+
+ kfd_procfs_add_sysfs_stats(process);
+ kfd_procfs_add_sysfs_files(process);
+ kfd_procfs_add_sysfs_counters(process);
+
+ return 0;
+}
+
struct kfd_process *kfd_create_process(struct task_struct *thread)
{
struct kfd_process *process;
@@ -874,31 +912,9 @@ struct kfd_process *kfd_create_process(struct task_struct *thread)
if (!procfs.kobj)
goto out;
- process->kobj = kfd_alloc_struct(process->kobj);
- if (!process->kobj) {
- pr_warn("Creating procfs kobject failed");
- goto out;
- }
- ret = kobject_init_and_add(process->kobj, &procfs_type,
- procfs.kobj, "%d",
- (int)process->lead_thread->pid);
- if (ret) {
- pr_warn("Creating procfs pid directory failed");
- kobject_put(process->kobj);
- goto out;
- }
-
- kfd_sysfs_create_file(process->kobj, &process->attr_pasid,
- "pasid");
-
- process->kobj_queues = kobject_create_and_add("queues",
- process->kobj);
- if (!process->kobj_queues)
- pr_warn("Creating KFD proc/queues folder failed");
-
- kfd_procfs_add_sysfs_stats(process);
- kfd_procfs_add_sysfs_files(process);
- kfd_procfs_add_sysfs_counters(process);
+ ret = kfd_create_process_sysfs(process);
+ if (ret)
+ pr_warn("Failed to create sysfs entry for the kfd_process");
kfd_debugfs_add_process(process);
--
2.51.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH V4 05/18] amdkfd: destroy kfd secondary contexts through fd close
2025-09-23 7:25 [PATCH V4 00/18] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (3 preceding siblings ...)
2025-09-23 7:25 ` [PATCH V4 04/18] amdkfd: Introduce kfd_create_process_sysfs as a separate function Zhu Lingshan
@ 2025-09-23 7:25 ` Zhu Lingshan
2025-09-23 7:25 ` [PATCH V4 06/18] amdkfd: process svm ioctl only on the primary kfd process Zhu Lingshan
` (12 subsequent siblings)
17 siblings, 0 replies; 28+ messages in thread
From: Zhu Lingshan @ 2025-09-23 7:25 UTC (permalink / raw)
To: felix.kuehling, alexander.deucher; +Cc: ray.huang, amd-gfx, Zhu Lingshan
Life cycle of a KFD secondary context(kfd_process) is tied
to the opened file. Therefore this commit destroy a kfd
secondary context when close the fd it belonging to.
This commit extracts the code removing the kfd_process
from the kfd_process_table to a separate function and
call it in kfd_process_notifier_release_internal unconditionally.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 9 ++++--
drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 1 +
drivers/gpu/drm/amd/amdkfd/kfd_process.c | 41 +++++++++++++-----------
3 files changed, 31 insertions(+), 20 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index 828a9ceef1e7..e8c6273de99b 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -164,8 +164,13 @@ static int kfd_release(struct inode *inode, struct file *filep)
{
struct kfd_process *process = filep->private_data;
- if (process)
- kfd_unref_process(process);
+ if (!process)
+ return 0;
+
+ if (!process->primary)
+ kfd_process_notifier_release_internal(process);
+
+ kfd_unref_process(process);
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
index 0818705820c6..d1436f1f527d 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
@@ -1085,6 +1085,7 @@ bool kfd_process_xnack_mode(struct kfd_process *p, bool supported);
int kfd_reserved_mem_mmap(struct kfd_node *dev, struct kfd_process *process,
struct vm_area_struct *vma);
+void kfd_process_notifier_release_internal(struct kfd_process *p);
/* KFD process API for creating and translating handles */
int kfd_process_device_create_obj_handle(struct kfd_process_device *pdd,
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index bce7e35a15c9..5d59a4d994d5 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -1233,10 +1233,30 @@ static void kfd_process_free_notifier(struct mmu_notifier *mn)
kfd_unref_process(container_of(mn, struct kfd_process, mmu_notifier));
}
-static void kfd_process_notifier_release_internal(struct kfd_process *p)
+static void kfd_process_table_remove(struct kfd_process *p)
+{
+ mutex_lock(&kfd_processes_mutex);
+ /*
+ * Do early return if table is empty.
+ *
+ * This could potentially happen if this function is called concurrently
+ * by mmu_notifier and by kfd_cleanup_pocesses.
+ *
+ */
+ if (hash_empty(kfd_processes_table)) {
+ mutex_unlock(&kfd_processes_mutex);
+ return;
+ }
+ hash_del_rcu(&p->kfd_processes);
+ mutex_unlock(&kfd_processes_mutex);
+ synchronize_srcu(&kfd_processes_srcu);
+}
+
+void kfd_process_notifier_release_internal(struct kfd_process *p)
{
int i;
+ kfd_process_table_remove(p);
cancel_delayed_work_sync(&p->eviction_work);
cancel_delayed_work_sync(&p->restore_work);
@@ -1270,7 +1290,8 @@ static void kfd_process_notifier_release_internal(struct kfd_process *p)
srcu_read_unlock(&kfd_processes_srcu, idx);
}
- mmu_notifier_put(&p->mmu_notifier);
+ if (p->primary)
+ mmu_notifier_put(&p->mmu_notifier);
}
static void kfd_process_notifier_release(struct mmu_notifier *mn,
@@ -1286,22 +1307,6 @@ static void kfd_process_notifier_release(struct mmu_notifier *mn,
if (WARN_ON(p->mm != mm))
return;
- mutex_lock(&kfd_processes_mutex);
- /*
- * Do early return if table is empty.
- *
- * This could potentially happen if this function is called concurrently
- * by mmu_notifier and by kfd_cleanup_pocesses.
- *
- */
- if (hash_empty(kfd_processes_table)) {
- mutex_unlock(&kfd_processes_mutex);
- return;
- }
- hash_del_rcu(&p->kfd_processes);
- mutex_unlock(&kfd_processes_mutex);
- synchronize_srcu(&kfd_processes_srcu);
-
kfd_process_notifier_release_internal(p);
}
--
2.51.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH V4 06/18] amdkfd: process svm ioctl only on the primary kfd process
2025-09-23 7:25 [PATCH V4 00/18] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (4 preceding siblings ...)
2025-09-23 7:25 ` [PATCH V4 05/18] amdkfd: destroy kfd secondary contexts through fd close Zhu Lingshan
@ 2025-09-23 7:25 ` Zhu Lingshan
2025-09-23 7:25 ` [PATCH V4 07/18] amdkfd: process USERPTR allocation " Zhu Lingshan
` (11 subsequent siblings)
17 siblings, 0 replies; 28+ messages in thread
From: Zhu Lingshan @ 2025-09-23 7:25 UTC (permalink / raw)
To: felix.kuehling, alexander.deucher; +Cc: ray.huang, amd-gfx, Zhu Lingshan
svm ioctl should only be processed on the primary
kfd process because only the lifecycle of the
primary kfd process is tied to the user space
applicaiton.
Another reason is in virtualization the hypervisor owns
the primary kfd process as a privileged one.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index e8c6273de99b..f56faf11ee43 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -1719,6 +1719,12 @@ static int kfd_ioctl_svm(struct file *filep, struct kfd_process *p, void *data)
struct kfd_ioctl_svm_args *args = data;
int r = 0;
+ if (!p->primary) {
+ pr_debug("SVM ioctl not supported on non-primary kfd process\n");
+
+ return -EOPNOTSUPP;
+ }
+
pr_debug("start 0x%llx size 0x%llx op 0x%x nattr 0x%x\n",
args->start_addr, args->size, args->op, args->nattr);
--
2.51.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH V4 07/18] amdkfd: process USERPTR allocation only on the primary kfd process
2025-09-23 7:25 [PATCH V4 00/18] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (5 preceding siblings ...)
2025-09-23 7:25 ` [PATCH V4 06/18] amdkfd: process svm ioctl only on the primary kfd process Zhu Lingshan
@ 2025-09-23 7:25 ` Zhu Lingshan
2025-09-23 7:25 ` [PATCH V4 08/18] amdkfd: identify a secondary kfd process by its id Zhu Lingshan
` (10 subsequent siblings)
17 siblings, 0 replies; 28+ messages in thread
From: Zhu Lingshan @ 2025-09-23 7:25 UTC (permalink / raw)
To: felix.kuehling, alexander.deucher; +Cc: ray.huang, amd-gfx, Zhu Lingshan
The lifecycle of the primary kfd process is tied to
the user space program, all secondary kfd process
would be destroyed when fd close. Thus only the primary
kfd process should process USERPTR memory allocation.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index f56faf11ee43..28df35a63c29 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -1068,6 +1068,12 @@ static int kfd_ioctl_alloc_memory_of_gpu(struct file *filep,
if (args->size == 0)
return -EINVAL;
+ if (!p->primary && (flags & KFD_IOC_ALLOC_MEM_FLAGS_USERPTR)) {
+ pr_debug("USERPTR is not supported on non-primary kfd_process\n");
+
+ return -EOPNOTSUPP;
+ }
+
#if IS_ENABLED(CONFIG_HSA_AMD_SVM)
/* Flush pending deferred work to avoid racing with deferred actions
* from previous memory map changes (e.g. munmap).
--
2.51.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH V4 08/18] amdkfd: identify a secondary kfd process by its id
2025-09-23 7:25 [PATCH V4 00/18] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (6 preceding siblings ...)
2025-09-23 7:25 ` [PATCH V4 07/18] amdkfd: process USERPTR allocation " Zhu Lingshan
@ 2025-09-23 7:25 ` Zhu Lingshan
2025-09-24 21:41 ` Kuehling, Felix
2025-09-23 7:25 ` [PATCH V4 09/18] amdkfd: find kfd_process by filep->private_data in kfd_mmap Zhu Lingshan
` (9 subsequent siblings)
17 siblings, 1 reply; 28+ messages in thread
From: Zhu Lingshan @ 2025-09-23 7:25 UTC (permalink / raw)
To: felix.kuehling, alexander.deucher; +Cc: ray.huang, amd-gfx, Zhu Lingshan
This commit introduces a new id field for
struct kfd process, which helps identify
a kfd process among multiple contexts that
all belong to a single user space program.
The sysfs entry of a secondary kfd process
is placed under the sysfs entry folder of
its primary kfd process.
The naming format of the sysfs entry of a secondary
kfd process is "context_%u" where %u is the process id.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 6 ++
drivers/gpu/drm/amd/amdkfd/kfd_process.c | 83 +++++++++++++++++++++++-
2 files changed, 86 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
index d1436f1f527d..d140463e221b 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
@@ -998,6 +998,9 @@ struct kfd_process {
/* Tracks debug per-vmid request for debug flags */
u32 dbg_flags;
+ /* kfd process id */
+ u16 id;
+
atomic_t poison;
/* Queues are in paused stated because we are in the process of doing a CRIU checkpoint */
bool queues_paused;
@@ -1012,6 +1015,9 @@ struct kfd_process {
/* indicating whether this is a primary kfd_process */
bool primary;
+
+ /* The primary kfd_process allocating IDs for its secondary kfd_process, 0 for primary kfd_process */
+ struct ida id_table;
};
#define KFD_PROCESS_TABLE_SIZE 8 /* bits: 256 entries */
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index 5d59a4d994d5..8e498fd35b8c 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -54,6 +54,9 @@ DEFINE_MUTEX(kfd_processes_mutex);
DEFINE_SRCU(kfd_processes_srcu);
+#define KFD_PROCESS_ID_MIN 1
+#define KFD_PROCESS_ID_WIDTH 16
+
/* For process termination handling */
static struct workqueue_struct *kfd_process_wq;
@@ -827,6 +830,7 @@ static void kfd_process_device_destroy_ib_mem(struct kfd_process_device *pdd)
int kfd_create_process_sysfs(struct kfd_process *process)
{
+ struct kfd_process *primary_process;
int ret;
if (process->kobj) {
@@ -839,9 +843,22 @@ int kfd_create_process_sysfs(struct kfd_process *process)
pr_warn("Creating procfs kobject failed");
return -ENOMEM;
}
- ret = kobject_init_and_add(process->kobj, &procfs_type,
- procfs.kobj, "%d",
- (int)process->lead_thread->pid);
+
+ if (process->primary)
+ ret = kobject_init_and_add(process->kobj, &procfs_type,
+ procfs.kobj, "%d",
+ (int)process->lead_thread->pid);
+ else {
+ primary_process = kfd_lookup_process_by_mm(process->lead_thread->mm);
+ if (!primary_process)
+ return -ESRCH;
+
+ ret = kobject_init_and_add(process->kobj, &procfs_type,
+ primary_process->kobj, "context_%u",
+ process->id);
+ kfd_unref_process(primary_process);
+ }
+
if (ret) {
pr_warn("Creating procfs pid directory failed");
kobject_put(process->kobj);
@@ -863,6 +880,51 @@ int kfd_create_process_sysfs(struct kfd_process *process)
return 0;
}
+static int kfd_process_alloc_id(struct kfd_process *process)
+{
+ int ret;
+ struct kfd_process *primary_process;
+
+ if (process->primary) {
+ process->id = 0;
+
+ return 0;
+ }
+
+ primary_process = kfd_lookup_process_by_mm(process->lead_thread->mm);
+ if (!primary_process)
+ return -ESRCH;
+
+ ret = ida_alloc_range(&primary_process->id_table, KFD_PROCESS_ID_MIN,
+ (1 << KFD_PROCESS_ID_WIDTH) - 1, GFP_KERNEL);
+ if (ret < 0)
+ goto out;
+
+ process->id = ret;
+ ret = 0;
+
+out:
+ kfd_unref_process(primary_process);
+
+ return ret;
+}
+
+static void kfd_process_free_id(struct kfd_process *process)
+{
+ struct kfd_process *primary_process;
+
+ if (process->primary)
+ return;
+
+ primary_process = kfd_lookup_process_by_mm(process->lead_thread->mm);
+ if (!primary_process)
+ return;
+
+ ida_free(&primary_process->id_table, process->id);
+
+ kfd_unref_process(primary_process);
+}
+
struct kfd_process *kfd_create_process(struct task_struct *thread)
{
struct kfd_process *process;
@@ -1193,6 +1255,11 @@ static void kfd_process_wq_release(struct work_struct *work)
if (ef)
dma_fence_signal(ef);
+ if (!p->primary)
+ kfd_process_free_id(p);
+ else
+ ida_destroy(&p->id_table);
+
kfd_process_remove_sysfs(p);
kfd_debugfs_remove_process(p);
@@ -1549,6 +1616,12 @@ static struct kfd_process *create_process(const struct task_struct *thread, bool
process->queues_paused = false;
process->primary = primary;
+ err = kfd_process_alloc_id(process);
+ if (err) {
+ pr_err("Creating kfd process: failed to alloc an id\n");
+ goto err_alloc_id;
+ }
+
INIT_DELAYED_WORK(&process->eviction_work, evict_process_worker);
INIT_DELAYED_WORK(&process->restore_work, restore_process_worker);
process->last_restore_timestamp = get_jiffies_64();
@@ -1599,6 +1672,8 @@ static struct kfd_process *create_process(const struct task_struct *thread, bool
goto err_register_notifier;
}
BUG_ON(mn != &process->mmu_notifier);
+
+ ida_init(&process->id_table);
}
kfd_unref_process(process);
@@ -1619,6 +1694,8 @@ static struct kfd_process *create_process(const struct task_struct *thread, bool
err_process_pqm_init:
kfd_event_free_process(process);
err_event_init:
+ kfd_process_free_id(process);
+err_alloc_id:
mutex_destroy(&process->mutex);
kfree(process);
err_alloc_process:
--
2.51.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH V4 09/18] amdkfd: find kfd_process by filep->private_data in kfd_mmap
2025-09-23 7:25 [PATCH V4 00/18] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (7 preceding siblings ...)
2025-09-23 7:25 ` [PATCH V4 08/18] amdkfd: identify a secondary kfd process by its id Zhu Lingshan
@ 2025-09-23 7:25 ` Zhu Lingshan
2025-09-23 7:26 ` [PATCH V4 10/18] amdkfd: remove DIQ support Zhu Lingshan
` (8 subsequent siblings)
17 siblings, 0 replies; 28+ messages in thread
From: Zhu Lingshan @ 2025-09-23 7:25 UTC (permalink / raw)
To: felix.kuehling, alexander.deucher; +Cc: ray.huang, amd-gfx, Zhu Lingshan
This commit finds the proper kfd_process by
filep->private_data in kfd_mmap,
because the function kfd_get_process()
can not locate a specific kfd process among
multiple contexts.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 11 +++++++----
1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index 28df35a63c29..662b181f1fd2 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -3407,16 +3407,19 @@ static int kfd_mmio_mmap(struct kfd_node *dev, struct kfd_process *process,
}
-static int kfd_mmap(struct file *filp, struct vm_area_struct *vma)
+static int kfd_mmap(struct file *filep, struct vm_area_struct *vma)
{
struct kfd_process *process;
struct kfd_node *dev = NULL;
unsigned long mmap_offset;
unsigned int gpu_id;
- process = kfd_get_process(current);
- if (IS_ERR(process))
- return PTR_ERR(process);
+ process = filep->private_data;
+ if (!process)
+ return -ESRCH;
+
+ if (process->lead_thread != current->group_leader)
+ return -EBADF;
mmap_offset = vma->vm_pgoff << PAGE_SHIFT;
gpu_id = KFD_MMAP_GET_GPU_ID(mmap_offset);
--
2.51.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH V4 10/18] amdkfd: remove DIQ support
2025-09-23 7:25 [PATCH V4 00/18] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (8 preceding siblings ...)
2025-09-23 7:25 ` [PATCH V4 09/18] amdkfd: find kfd_process by filep->private_data in kfd_mmap Zhu Lingshan
@ 2025-09-23 7:26 ` Zhu Lingshan
2025-09-24 21:41 ` Kuehling, Felix
2025-09-23 7:26 ` [PATCH V4 11/18] amdkfd: process pointer of a HIQ should be NULL Zhu Lingshan
` (7 subsequent siblings)
17 siblings, 1 reply; 28+ messages in thread
From: Zhu Lingshan @ 2025-09-23 7:26 UTC (permalink / raw)
To: felix.kuehling, alexander.deucher; +Cc: ray.huang, amd-gfx, Zhu Lingshan
This commit remove DIQ support because it has been
marked as DEPRECATED since 2022 in commit 5bdd3eb2
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
---
.../drm/amd/amdkfd/kfd_device_queue_manager.c | 6 +--
drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c | 29 +-------------
.../drm/amd/amdkfd/kfd_packet_manager_v9.c | 4 --
.../drm/amd/amdkfd/kfd_packet_manager_vi.c | 4 --
drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 1 -
.../amd/amdkfd/kfd_process_queue_manager.c | 39 +------------------
6 files changed, 6 insertions(+), 77 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index 6c5c7c1bf5ed..ac6052d7a4cb 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -399,8 +399,7 @@ static void increment_queue_count(struct device_queue_manager *dqm,
struct queue *q)
{
dqm->active_queue_count++;
- if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE ||
- q->properties.type == KFD_QUEUE_TYPE_DIQ)
+ if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE)
dqm->active_cp_queue_count++;
if (q->properties.is_gws) {
@@ -414,8 +413,7 @@ static void decrement_queue_count(struct device_queue_manager *dqm,
struct queue *q)
{
dqm->active_queue_count--;
- if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE ||
- q->properties.type == KFD_QUEUE_TYPE_DIQ)
+ if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE)
dqm->active_cp_queue_count--;
if (q->properties.is_gws) {
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
index 2b0a830f5b29..4c7ab7a711be 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
@@ -46,7 +46,7 @@ static bool kq_initialize(struct kernel_queue *kq, struct kfd_node *dev,
int retval;
union PM4_MES_TYPE_3_HEADER nop;
- if (WARN_ON(type != KFD_QUEUE_TYPE_DIQ && type != KFD_QUEUE_TYPE_HIQ))
+ if (WARN_ON(type != KFD_QUEUE_TYPE_HIQ))
return false;
pr_debug("Initializing queue type %d size %d\n", KFD_QUEUE_TYPE_HIQ,
@@ -61,17 +61,7 @@ static bool kq_initialize(struct kernel_queue *kq, struct kfd_node *dev,
kq->dev = dev;
kq->nop_packet = nop.u32all;
- switch (type) {
- case KFD_QUEUE_TYPE_DIQ:
- kq->mqd_mgr = dev->dqm->mqd_mgrs[KFD_MQD_TYPE_DIQ];
- break;
- case KFD_QUEUE_TYPE_HIQ:
- kq->mqd_mgr = dev->dqm->mqd_mgrs[KFD_MQD_TYPE_HIQ];
- break;
- default:
- dev_err(dev->adev->dev, "Invalid queue type %d\n", type);
- return false;
- }
+ kq->mqd_mgr = dev->dqm->mqd_mgrs[KFD_MQD_TYPE_HIQ];
if (!kq->mqd_mgr)
return false;
@@ -162,24 +152,11 @@ static bool kq_initialize(struct kernel_queue *kq, struct kfd_node *dev,
kq->mqd_mgr->load_mqd(kq->mqd_mgr, kq->queue->mqd,
kq->queue->pipe, kq->queue->queue,
&kq->queue->properties, NULL);
- } else {
- /* allocate fence for DIQ */
-
- retval = kfd_gtt_sa_allocate(dev, sizeof(uint32_t),
- &kq->fence_mem_obj);
-
- if (retval != 0)
- goto err_alloc_fence;
-
- kq->fence_kernel_address = kq->fence_mem_obj->cpu_ptr;
- kq->fence_gpu_addr = kq->fence_mem_obj->gpu_addr;
}
print_queue(kq->queue);
return true;
-err_alloc_fence:
- kq->mqd_mgr->free_mqd(kq->mqd_mgr, kq->queue->mqd, kq->queue->mqd_mem_obj);
err_allocate_mqd:
uninit_queue(kq->queue);
err_init_queue:
@@ -209,8 +186,6 @@ static void kq_uninitialize(struct kernel_queue *kq)
kq->queue->queue);
up_read(&kq->dev->adev->reset_domain->sem);
}
- else if (kq->queue->properties.type == KFD_QUEUE_TYPE_DIQ)
- kfd_gtt_sa_free(kq->dev, kq->fence_mem_obj);
kq->mqd_mgr->free_mqd(kq->mqd_mgr, kq->queue->mqd,
kq->queue->mqd_mem_obj);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c
index 505036968a77..3d2375817c3e 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c
@@ -252,10 +252,6 @@ static int pm_map_queues_v9(struct packet_manager *pm, uint32_t *buffer,
packet->bitfields2.queue_type =
queue_type__mes_map_queues__normal_latency_static_queue_vi;
break;
- case KFD_QUEUE_TYPE_DIQ:
- packet->bitfields2.queue_type =
- queue_type__mes_map_queues__debug_interface_queue_vi;
- break;
case KFD_QUEUE_TYPE_SDMA:
case KFD_QUEUE_TYPE_SDMA_XGMI:
if (q->properties.sdma_engine_id < 2 &&
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c
index a1de5d7e173a..60086e7cc258 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c
@@ -166,10 +166,6 @@ static int pm_map_queues_vi(struct packet_manager *pm, uint32_t *buffer,
packet->bitfields2.queue_type =
queue_type__mes_map_queues__normal_latency_static_queue_vi;
break;
- case KFD_QUEUE_TYPE_DIQ:
- packet->bitfields2.queue_type =
- queue_type__mes_map_queues__debug_interface_queue_vi;
- break;
case KFD_QUEUE_TYPE_SDMA:
case KFD_QUEUE_TYPE_SDMA_XGMI:
packet->bitfields2.engine_sel = q->properties.sdma_engine_id +
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
index d140463e221b..a2f8b1c24fc3 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
@@ -425,7 +425,6 @@ enum kfd_queue_type {
KFD_QUEUE_TYPE_COMPUTE,
KFD_QUEUE_TYPE_SDMA,
KFD_QUEUE_TYPE_HIQ,
- KFD_QUEUE_TYPE_DIQ,
KFD_QUEUE_TYPE_SDMA_XGMI,
KFD_QUEUE_TYPE_SDMA_BY_ENG_ID
};
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
index 7fbb5c274ccc..8ddc33abf230 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
@@ -345,7 +345,7 @@ int pqm_create_queue(struct process_queue_manager *pqm,
* If we are just about to create DIQ, the is_debug flag is not set yet
* Hence we also check the type as well
*/
- if ((pdd->qpd.is_debug) || (type == KFD_QUEUE_TYPE_DIQ))
+ if ((pdd->qpd.is_debug))
max_queues = dev->kfd->device_info.max_no_of_hqd/2;
if (pdd->qpd.queue_count >= max_queues)
@@ -426,22 +426,6 @@ int pqm_create_queue(struct process_queue_manager *pqm,
restore_mqd, restore_ctl_stack);
print_queue(q);
break;
- case KFD_QUEUE_TYPE_DIQ:
- kq = kernel_queue_init(dev, KFD_QUEUE_TYPE_DIQ);
- if (!kq) {
- retval = -ENOMEM;
- goto err_create_queue;
- }
- kq->queue->properties.queue_id = *qid;
- pqn->kq = kq;
- pqn->q = NULL;
- retval = kfd_process_drain_interrupts(pdd);
- if (retval)
- break;
-
- retval = dev->dqm->ops.create_kernel_queue(dev->dqm,
- kq, &pdd->qpd);
- break;
default:
WARN(1, "Invalid queue type %d", type);
retval = -EINVAL;
@@ -1131,32 +1115,13 @@ int pqm_debugfs_mqds(struct seq_file *m, void *data)
break;
default:
seq_printf(m,
- " Bad user queue type %d on device %x\n",
+ " Qeueu node with bad user queue type %d on device %x\n",
q->properties.type, q->device->id);
continue;
}
mqd_mgr = q->device->dqm->mqd_mgrs[mqd_type];
size = mqd_mgr->mqd_stride(mqd_mgr,
&q->properties);
- } else if (pqn->kq) {
- q = pqn->kq->queue;
- mqd_mgr = pqn->kq->mqd_mgr;
- switch (q->properties.type) {
- case KFD_QUEUE_TYPE_DIQ:
- seq_printf(m, " DIQ on device %x\n",
- pqn->kq->dev->id);
- break;
- default:
- seq_printf(m,
- " Bad kernel queue type %d on device %x\n",
- q->properties.type,
- pqn->kq->dev->id);
- continue;
- }
- } else {
- seq_printf(m,
- " Weird: Queue node with neither kernel nor user queue\n");
- continue;
}
for (xcc = 0; xcc < num_xccs; xcc++) {
--
2.51.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH V4 11/18] amdkfd: process pointer of a HIQ should be NULL
2025-09-23 7:25 [PATCH V4 00/18] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (9 preceding siblings ...)
2025-09-23 7:26 ` [PATCH V4 10/18] amdkfd: remove DIQ support Zhu Lingshan
@ 2025-09-23 7:26 ` Zhu Lingshan
2025-09-24 21:43 ` Kuehling, Felix
2025-09-23 7:26 ` [PATCH V4 12/18] amdkfd: remove test_kq Zhu Lingshan
` (6 subsequent siblings)
17 siblings, 1 reply; 28+ messages in thread
From: Zhu Lingshan @ 2025-09-23 7:26 UTC (permalink / raw)
To: felix.kuehling, alexander.deucher; +Cc: ray.huang, amd-gfx, Zhu Lingshan
In kq_initialize, queue->process of a HIQ should
be NULL as initialized, because it does not belong
to any kfd_process.
This commit decommisions the function kfd_get_process() because
it can not locate a specific kfd_process among multiple
contexts and not any code path calls it after this commit.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c | 1 -
drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 1 -
drivers/gpu/drm/amd/amdkfd/kfd_process.c | 18 ------------------
3 files changed, 20 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
index 4c7ab7a711be..0a4f8e8dd77e 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
@@ -134,7 +134,6 @@ static bool kq_initialize(struct kernel_queue *kq, struct kfd_node *dev,
goto err_init_queue;
kq->queue->device = dev;
- kq->queue->process = kfd_get_process(current);
kq->queue->mqd_mem_obj = kq->mqd_mgr->allocate_mqd(kq->mqd_mgr->dev,
&kq->queue->properties);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
index a2f8b1c24fc3..e09c8bd3e138 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
@@ -1049,7 +1049,6 @@ void kfd_process_destroy_wq(void);
void kfd_cleanup_processes(void);
struct kfd_process *kfd_create_process(struct task_struct *thread);
int kfd_create_process_sysfs(struct kfd_process *process);
-struct kfd_process *kfd_get_process(const struct task_struct *task);
struct kfd_process *kfd_lookup_process_by_pasid(u32 pasid,
struct kfd_process_device **pdd);
struct kfd_process *kfd_lookup_process_by_mm(const struct mm_struct *mm);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index 8e498fd35b8c..0c3f0cc16bf4 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -989,24 +989,6 @@ struct kfd_process *kfd_create_process(struct task_struct *thread)
return process;
}
-struct kfd_process *kfd_get_process(const struct task_struct *thread)
-{
- struct kfd_process *process;
-
- if (!thread->mm)
- return ERR_PTR(-EINVAL);
-
- /* Only the pthreads threading model is supported. */
- if (thread->group_leader->mm != thread->mm)
- return ERR_PTR(-EINVAL);
-
- process = find_process(thread, false);
- if (!process)
- return ERR_PTR(-EINVAL);
-
- return process;
-}
-
static struct kfd_process *find_process_by_mm(const struct mm_struct *mm)
{
struct kfd_process *process;
--
2.51.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH V4 12/18] amdkfd: remove test_kq
2025-09-23 7:25 [PATCH V4 00/18] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (10 preceding siblings ...)
2025-09-23 7:26 ` [PATCH V4 11/18] amdkfd: process pointer of a HIQ should be NULL Zhu Lingshan
@ 2025-09-23 7:26 ` Zhu Lingshan
2025-09-23 7:26 ` [PATCH V4 13/18] amdkfd: introduce new helper kfd_lookup_process_by_id Zhu Lingshan
` (5 subsequent siblings)
17 siblings, 0 replies; 28+ messages in thread
From: Zhu Lingshan @ 2025-09-23 7:26 UTC (permalink / raw)
To: felix.kuehling, alexander.deucher; +Cc: ray.huang, amd-gfx, Zhu Lingshan
This commit removes test_kq() function becuse it has been
marked as unused since 2014 and no other functions calls it.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c | 31 -------------------
1 file changed, 31 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
index 0a4f8e8dd77e..887cd7f2cd65 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
@@ -332,34 +332,3 @@ void kernel_queue_uninit(struct kernel_queue *kq)
kq_uninitialize(kq);
kfree(kq);
}
-
-/* FIXME: Can this test be removed? */
-static __attribute__((unused)) void test_kq(struct kfd_node *dev)
-{
- struct kernel_queue *kq;
- uint32_t *buffer, i;
- int retval;
-
- dev_err(dev->adev->dev, "Starting kernel queue test\n");
-
- kq = kernel_queue_init(dev, KFD_QUEUE_TYPE_HIQ);
- if (unlikely(!kq)) {
- dev_err(dev->adev->dev, " Failed to initialize HIQ\n");
- dev_err(dev->adev->dev, "Kernel queue test failed\n");
- return;
- }
-
- retval = kq_acquire_packet_buffer(kq, 5, &buffer);
- if (unlikely(retval != 0)) {
- dev_err(dev->adev->dev, " Failed to acquire packet buffer\n");
- dev_err(dev->adev->dev, "Kernel queue test failed\n");
- return;
- }
- for (i = 0; i < 5; i++)
- buffer[i] = kq->nop_packet;
- kq_submit_packet(kq);
-
- dev_err(dev->adev->dev, "Ending kernel queue test\n");
-}
-
-
--
2.51.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH V4 13/18] amdkfd: introduce new helper kfd_lookup_process_by_id
2025-09-23 7:25 [PATCH V4 00/18] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (11 preceding siblings ...)
2025-09-23 7:26 ` [PATCH V4 12/18] amdkfd: remove test_kq Zhu Lingshan
@ 2025-09-23 7:26 ` Zhu Lingshan
2025-09-23 7:26 ` [PATCH V4 14/18] amdkfd: record kfd process id into kfd process_info Zhu Lingshan
` (4 subsequent siblings)
17 siblings, 0 replies; 28+ messages in thread
From: Zhu Lingshan @ 2025-09-23 7:26 UTC (permalink / raw)
To: felix.kuehling, alexander.deucher; +Cc: ray.huang, amd-gfx, Zhu Lingshan
This commit introduces a new helper function
kfd_lookup_process_by_id which can find a
kfd process that identified by its id from
the kfd process table
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 1 +
drivers/gpu/drm/amd/amdkfd/kfd_process.c | 21 +++++++++++++++++++++
2 files changed, 22 insertions(+)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
index e09c8bd3e138..65cf4c366124 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
@@ -1052,6 +1052,7 @@ int kfd_create_process_sysfs(struct kfd_process *process);
struct kfd_process *kfd_lookup_process_by_pasid(u32 pasid,
struct kfd_process_device **pdd);
struct kfd_process *kfd_lookup_process_by_mm(const struct mm_struct *mm);
+struct kfd_process *kfd_lookup_process_by_id(const struct mm_struct *mm, u16 id);
int kfd_process_gpuidx_from_gpuid(struct kfd_process *p, uint32_t gpu_id);
int kfd_process_gpuid_from_node(struct kfd_process *p, struct kfd_node *node,
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index 0c3f0cc16bf4..e469ed951579 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -1956,6 +1956,27 @@ struct kfd_process *kfd_lookup_process_by_mm(const struct mm_struct *mm)
return p;
}
+/* This increments the process->ref counter. */
+struct kfd_process *kfd_lookup_process_by_id(const struct mm_struct *mm, u16 id)
+{
+ struct kfd_process *p, *ret_p = NULL;
+ unsigned int temp;
+
+ int idx = srcu_read_lock(&kfd_processes_srcu);
+
+ hash_for_each_rcu(kfd_processes_table, temp, p, kfd_processes) {
+ if (p->mm == mm && p->id == id) {
+ kref_get(&p->ref);
+ ret_p = p;
+ break;
+ }
+ }
+
+ srcu_read_unlock(&kfd_processes_srcu, idx);
+
+ return ret_p;
+}
+
/* kfd_process_evict_queues - Evict all user queues of a process
*
* Eviction is reference-counted per process-device. This means multiple
--
2.51.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH V4 14/18] amdkfd: record kfd process id into kfd process_info
2025-09-23 7:25 [PATCH V4 00/18] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (12 preceding siblings ...)
2025-09-23 7:26 ` [PATCH V4 13/18] amdkfd: introduce new helper kfd_lookup_process_by_id Zhu Lingshan
@ 2025-09-23 7:26 ` Zhu Lingshan
2025-09-24 21:45 ` Kuehling, Felix
2025-09-23 7:26 ` [PATCH V4 15/18] amdkfd: record kfd process id in amdkfd_fence Zhu Lingshan
` (3 subsequent siblings)
17 siblings, 1 reply; 28+ messages in thread
From: Zhu Lingshan @ 2025-09-23 7:26 UTC (permalink / raw)
To: felix.kuehling, alexander.deucher; +Cc: ray.huang, amd-gfx, Zhu Lingshan
This commit records the id of the owner
kfd_process into a kfd process_info when
create it.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 2 ++
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 4 ++++
2 files changed, 6 insertions(+)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
index aa88bad7416b..d867984a68da 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
@@ -146,6 +146,8 @@ struct amdkfd_process_info {
/* MMU-notifier related fields */
struct mutex notifier_lock;
uint32_t evicted_bos;
+ /* kfd process id */
+ u16 process_id;
struct delayed_work restore_userptr_work;
struct pid *pid;
bool block_mmu_notifications;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index b16cce7c22c3..723d34921c12 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -1382,8 +1382,10 @@ static int init_kfd_vm(struct amdgpu_vm *vm, void **process_info,
struct dma_fence **ef)
{
struct amdkfd_process_info *info = NULL;
+ struct kfd_process *process = NULL;
int ret;
+ process = container_of(process_info, struct kfd_process, kgd_process_info);
if (!*process_info) {
info = kzalloc(sizeof(*info), GFP_KERNEL);
if (!info)
@@ -1410,6 +1412,8 @@ static int init_kfd_vm(struct amdgpu_vm *vm, void **process_info,
INIT_DELAYED_WORK(&info->restore_userptr_work,
amdgpu_amdkfd_restore_userptr_worker);
+ info->process_id = process->id;
+
*process_info = info;
}
--
2.51.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH V4 15/18] amdkfd: record kfd process id in amdkfd_fence
2025-09-23 7:25 [PATCH V4 00/18] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (13 preceding siblings ...)
2025-09-23 7:26 ` [PATCH V4 14/18] amdkfd: record kfd process id into kfd process_info Zhu Lingshan
@ 2025-09-23 7:26 ` Zhu Lingshan
2025-09-23 7:26 ` [PATCH V4 16/18] amdkfd: fence handler evict and restore a kfd process by its id Zhu Lingshan
` (2 subsequent siblings)
17 siblings, 0 replies; 28+ messages in thread
From: Zhu Lingshan @ 2025-09-23 7:26 UTC (permalink / raw)
To: felix.kuehling, alexander.deucher; +Cc: ray.huang, amd-gfx, Zhu Lingshan
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 4 +++-
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_fence.c | 4 +++-
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 4 ++--
drivers/gpu/drm/amd/amdkfd/kfd_svm.c | 2 +-
4 files changed, 9 insertions(+), 5 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
index d867984a68da..f3e497da106c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
@@ -98,6 +98,7 @@ struct amdgpu_amdkfd_fence {
spinlock_t lock;
char timeline_name[TASK_COMM_LEN];
struct svm_range_bo *svm_bo;
+ uint16_t process_id;
};
struct amdgpu_kfd_dev {
@@ -188,7 +189,8 @@ int amdgpu_queue_mask_bit_to_set_resource_bit(struct amdgpu_device *adev,
struct amdgpu_amdkfd_fence *amdgpu_amdkfd_fence_create(u64 context,
struct mm_struct *mm,
- struct svm_range_bo *svm_bo);
+ struct svm_range_bo *svm_bo,
+ u16 kfd_process_id);
int amdgpu_amdkfd_drm_client_create(struct amdgpu_device *adev);
#if defined(CONFIG_DEBUG_FS)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_fence.c
index 1ef758ac5076..a6ff4d82b029 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_fence.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_fence.c
@@ -62,7 +62,8 @@ static atomic_t fence_seq = ATOMIC_INIT(0);
struct amdgpu_amdkfd_fence *amdgpu_amdkfd_fence_create(u64 context,
struct mm_struct *mm,
- struct svm_range_bo *svm_bo)
+ struct svm_range_bo *svm_bo,
+ u16 process_id)
{
struct amdgpu_amdkfd_fence *fence;
@@ -76,6 +77,7 @@ struct amdgpu_amdkfd_fence *amdgpu_amdkfd_fence_create(u64 context,
get_task_comm(fence->timeline_name, current);
spin_lock_init(&fence->lock);
fence->svm_bo = svm_bo;
+ fence->process_id = process_id;
dma_fence_init(&fence->base, &amdkfd_fence_ops, &fence->lock,
context, atomic_inc_return(&fence_seq));
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index 723d34921c12..6eb568dde5b6 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -1401,7 +1401,7 @@ static int init_kfd_vm(struct amdgpu_vm *vm, void **process_info,
info->eviction_fence =
amdgpu_amdkfd_fence_create(dma_fence_context_alloc(1),
current->mm,
- NULL);
+ NULL, process->id);
if (!info->eviction_fence) {
pr_err("Failed to create eviction fence\n");
ret = -ENOMEM;
@@ -3034,7 +3034,7 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence __rcu *
amdgpu_amdkfd_fence_create(
process_info->eviction_fence->base.context,
process_info->eviction_fence->mm,
- NULL);
+ NULL, process_info->process_id);
if (!new_fence) {
pr_err("Failed to create eviction fence\n");
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
index a0f22ea6d15a..f3e649e8d518 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
@@ -585,7 +585,7 @@ svm_range_vram_node_new(struct kfd_node *node, struct svm_range *prange,
svm_bo->eviction_fence =
amdgpu_amdkfd_fence_create(dma_fence_context_alloc(1),
mm,
- svm_bo);
+ svm_bo, p->id);
mmput(mm);
INIT_WORK(&svm_bo->eviction_work, svm_range_evict_svm_bo_worker);
svm_bo->evicting = 0;
--
2.51.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH V4 16/18] amdkfd: fence handler evict and restore a kfd process by its id
2025-09-23 7:25 [PATCH V4 00/18] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (14 preceding siblings ...)
2025-09-23 7:26 ` [PATCH V4 15/18] amdkfd: record kfd process id in amdkfd_fence Zhu Lingshan
@ 2025-09-23 7:26 ` Zhu Lingshan
2025-09-23 7:26 ` [PATCH V4 17/18] amdkfd: set_debug_trap ioctl only works on a primary kfd_process target Zhu Lingshan
2025-09-23 7:26 ` [PATCH V4 18/18] amdkfd: introduce new ioctl AMDKFD_IOC_CREATE_PROCESS Zhu Lingshan
17 siblings, 0 replies; 28+ messages in thread
From: Zhu Lingshan @ 2025-09-23 7:26 UTC (permalink / raw)
To: felix.kuehling, alexander.deucher; +Cc: ray.huang, amd-gfx, Zhu Lingshan
In fence enable signaling handler, kfd evicts
and restores the corresponding kfd_process,
this commit helps find the kfd_process by
both its mm and id.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 2 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_fence.c | 6 +++++-
drivers/gpu/drm/amd/amdkfd/kfd_device.c | 7 ++++---
3 files changed, 10 insertions(+), 5 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
index f3e497da106c..0c28acf2fd07 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
@@ -409,7 +409,7 @@ int kgd2kfd_init_zone_device(struct amdgpu_device *adev)
int kgd2kfd_quiesce_mm(struct mm_struct *mm, uint32_t trigger);
int kgd2kfd_resume_mm(struct mm_struct *mm);
int kgd2kfd_schedule_evict_and_restore_process(struct mm_struct *mm,
- struct dma_fence *fence);
+ u16 process_id, struct dma_fence *fence);
#if IS_ENABLED(CONFIG_HSA_AMD)
int kgd2kfd_init(void);
void kgd2kfd_exit(void);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_fence.c
index a6ff4d82b029..1599eebe5c3e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_fence.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_fence.c
@@ -128,8 +128,12 @@ static bool amdkfd_fence_enable_signaling(struct dma_fence *f)
if (dma_fence_is_signaled(f))
return true;
+ /* if fence->svm_bo is NULL, means this fence is created through
+ * init_kfd_vm() or amdgpu_amdkfd_gpuvm_restore_process_bos().
+ * Therefore, this fence is amdgpu_amdkfd_fence->eviction_fence.
+ */
if (!fence->svm_bo) {
- if (!kgd2kfd_schedule_evict_and_restore_process(fence->mm, f))
+ if (!kgd2kfd_schedule_evict_and_restore_process(fence->mm, fence->process_id, f))
return true;
} else {
if (!svm_range_schedule_evict_svm_bo(fence))
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device.c b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
index 349c351e242b..3377b6d53e47 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
@@ -1193,12 +1193,13 @@ int kgd2kfd_resume_mm(struct mm_struct *mm)
* prepare for safe eviction of KFD BOs that belong to the specified
* process.
*
- * @mm: mm_struct that identifies the specified KFD process
+ * @mm: mm_struct that identifies a group of KFD processes
+ * @process_id: an id that identifies a specific KFD process in the above group
* @fence: eviction fence attached to KFD process BOs
*
*/
int kgd2kfd_schedule_evict_and_restore_process(struct mm_struct *mm,
- struct dma_fence *fence)
+ u16 process_id, struct dma_fence *fence)
{
struct kfd_process *p;
unsigned long active_time;
@@ -1210,7 +1211,7 @@ int kgd2kfd_schedule_evict_and_restore_process(struct mm_struct *mm,
if (dma_fence_is_signaled(fence))
return 0;
- p = kfd_lookup_process_by_mm(mm);
+ p = kfd_lookup_process_by_id(mm, process_id);
if (!p)
return -ENODEV;
--
2.51.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH V4 17/18] amdkfd: set_debug_trap ioctl only works on a primary kfd_process target
2025-09-23 7:25 [PATCH V4 00/18] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (15 preceding siblings ...)
2025-09-23 7:26 ` [PATCH V4 16/18] amdkfd: fence handler evict and restore a kfd process by its id Zhu Lingshan
@ 2025-09-23 7:26 ` Zhu Lingshan
2025-09-24 21:50 ` Kuehling, Felix
2025-09-23 7:26 ` [PATCH V4 18/18] amdkfd: introduce new ioctl AMDKFD_IOC_CREATE_PROCESS Zhu Lingshan
17 siblings, 1 reply; 28+ messages in thread
From: Zhu Lingshan @ 2025-09-23 7:26 UTC (permalink / raw)
To: felix.kuehling, alexander.deucher; +Cc: ray.huang, amd-gfx, Zhu Lingshan
The user space program pass down a pid to kfd
through set_debug_trap ioctl, which can help
find the corresponding user space program and
its mm struct.
However, these information is insufficient to
locate a specific kfd process among the
multiple kfd_process that belong to the
same user space program.
For correctness and backward compatibilities,
this commit only allow set_debut_trap ioctl
work for a user space program which does not
own any secondary kfd_process.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 29 ++++++++++++++++++++++++
1 file changed, 29 insertions(+)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index 662b181f1fd2..2df095e25c2e 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -36,6 +36,7 @@
#include <linux/ptrace.h>
#include <linux/dma-buf.h>
#include <linux/processor.h>
+#include <linux/fdtable.h>
#include "kfd_priv.h"
#include "kfd_device_queue_manager.h"
#include "kfd_svm.h"
@@ -2918,6 +2919,25 @@ static int kfd_ioctl_runtime_enable(struct file *filep, struct kfd_process *p, v
return r;
}
+static int kfd_process_count_cb(const void *num, struct file *filep, unsigned int n)
+{
+ u16 *ret = (u16 *)num;
+
+ if (filep->f_op == &kfd_fops)
+ (*ret)++;
+
+ return 0;
+}
+
+static u16 kfd_process_count(struct task_struct *thread)
+{
+ u16 count = 0;
+
+ iterate_fd(thread->files, 0, kfd_process_count_cb, (const void *)&count);
+
+ return count;
+}
+
static int kfd_ioctl_set_debug_trap(struct file *filep, struct kfd_process *p, void *data)
{
struct kfd_ioctl_dbg_trap_args *args = data;
@@ -2927,6 +2947,7 @@ static int kfd_ioctl_set_debug_trap(struct file *filep, struct kfd_process *p, v
struct kfd_process *target = NULL;
struct kfd_process_device *pdd = NULL;
int r = 0;
+ u16 process_count = 0;
if (sched_policy == KFD_SCHED_POLICY_NO_HWS) {
pr_err("Debugging does not support sched_policy %i", sched_policy);
@@ -2984,6 +3005,14 @@ static int kfd_ioctl_set_debug_trap(struct file *filep, struct kfd_process *p, v
if (r)
goto out;
+ process_count = kfd_process_count(thread);
+ /* The user space program has secondary contexts */
+ if (process_count > 1) {
+ pr_err("Detect multiple kfd_process owned by PID %i, please consider close any secondary kfd_process and try again\n", args->pid);
+ r = -EINVAL;
+ goto out;
+ }
+
mutex_lock(&target->mutex);
if (args->op != KFD_IOC_DBG_TRAP_ENABLE && !target->debug_trap_enabled) {
--
2.51.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH V4 18/18] amdkfd: introduce new ioctl AMDKFD_IOC_CREATE_PROCESS
2025-09-23 7:25 [PATCH V4 00/18] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (16 preceding siblings ...)
2025-09-23 7:26 ` [PATCH V4 17/18] amdkfd: set_debug_trap ioctl only works on a primary kfd_process target Zhu Lingshan
@ 2025-09-23 7:26 ` Zhu Lingshan
17 siblings, 0 replies; 28+ messages in thread
From: Zhu Lingshan @ 2025-09-23 7:26 UTC (permalink / raw)
To: felix.kuehling, alexander.deucher; +Cc: ray.huang, amd-gfx, Zhu Lingshan
This commit implemetns a new ioctl AMDKFD_IOC_CREATE_PROCESS
that creates a new secondary kfd_progress on the FD.
To keep backward compatibility, userspace programs need to invoke
this ioctl explicitly on a FD to create a secondary
kfd_process which replacing its primary kfd_process.
This commit bumps ioctl minor version.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 41 ++++++++++++++++++++++++
drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 1 +
drivers/gpu/drm/amd/amdkfd/kfd_process.c | 3 +-
include/uapi/linux/kfd_ioctl.h | 8 +++--
4 files changed, 49 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index 2df095e25c2e..a228cb32591e 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -3164,6 +3164,44 @@ static int kfd_ioctl_set_debug_trap(struct file *filep, struct kfd_process *p, v
return r;
}
+/* userspace programs need to invoke this ioctl explicitly on a FD to
+ * create a secondary kfd_process which replacing its primary kfd_process
+ */
+static int kfd_ioctl_create_process(struct file *filep, struct kfd_process *p, void *data)
+{
+ struct kfd_process *process;
+ int ret;
+
+ if (!filep->private_data || !p)
+ return -EINVAL;
+
+ if (p != filep->private_data)
+ return -EINVAL;
+
+ /* Each FD owns only one kfd_process */
+ if (!p->primary)
+ return -EINVAL;
+
+ mutex_lock(&kfd_processes_mutex);
+ process = create_process(current, false);
+ mutex_unlock(&kfd_processes_mutex);
+
+ if (IS_ERR(process))
+ return PTR_ERR(process);
+
+ /* Each open() increases kref of the primary kfd_process,
+ * so we need to reduce it here before we create a new secondary process replacing it
+ */
+ kfd_unref_process(p);
+
+ filep->private_data = process;
+ ret = kfd_create_process_sysfs(process);
+ if (ret)
+ pr_warn("Failed to create sysfs entry for the kfd_process");
+
+ return 0;
+}
+
#define AMDKFD_IOCTL_DEF(ioctl, _func, _flags) \
[_IOC_NR(ioctl)] = {.cmd = ioctl, .func = _func, .flags = _flags, \
.cmd_drv = 0, .name = #ioctl}
@@ -3282,6 +3320,9 @@ static const struct amdkfd_ioctl_desc amdkfd_ioctls[] = {
AMDKFD_IOCTL_DEF(AMDKFD_IOC_DBG_TRAP,
kfd_ioctl_set_debug_trap, 0),
+
+ AMDKFD_IOCTL_DEF(AMDKFD_IOC_CREATE_PROCESS,
+ kfd_ioctl_create_process, 0),
};
#define AMDKFD_CORE_IOCTL_COUNT ARRAY_SIZE(amdkfd_ioctls)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
index 65cf4c366124..9bab74758cf9 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
@@ -1044,6 +1044,7 @@ struct amdkfd_ioctl_desc {
};
bool kfd_dev_is_large_bar(struct kfd_node *dev);
+struct kfd_process *create_process(const struct task_struct *thread, bool primary);
int kfd_process_create_wq(void);
void kfd_process_destroy_wq(void);
void kfd_cleanup_processes(void);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index e469ed951579..6ca7081cb817 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -71,7 +71,6 @@ static struct workqueue_struct *kfd_restore_wq;
static struct kfd_process *find_process(const struct task_struct *thread,
bool ref);
static void kfd_process_ref_release(struct kref *ref);
-static struct kfd_process *create_process(const struct task_struct *thread, bool primary);
static void evict_process_worker(struct work_struct *work);
static void restore_process_worker(struct work_struct *work);
@@ -1580,7 +1579,7 @@ void kfd_process_set_trap_debug_flag(struct qcm_process_device *qpd,
* On return the kfd_process is fully operational and will be freed when the
* mm is released
*/
-static struct kfd_process *create_process(const struct task_struct *thread, bool primary)
+struct kfd_process *create_process(const struct task_struct *thread, bool primary)
{
struct kfd_process *process;
struct mmu_notifier *mn;
diff --git a/include/uapi/linux/kfd_ioctl.h b/include/uapi/linux/kfd_ioctl.h
index 04c7d283dc7d..1d206ecc831e 100644
--- a/include/uapi/linux/kfd_ioctl.h
+++ b/include/uapi/linux/kfd_ioctl.h
@@ -44,9 +44,10 @@
* - 1.16 - Add contiguous VRAM allocation flag
* - 1.17 - Add SDMA queue creation with target SDMA engine ID
* - 1.18 - Rename pad in set_memory_policy_args to misc_process_flag
+ * - 1.19 - Add a new ioctl to craete secondary kfd processes
*/
#define KFD_IOCTL_MAJOR_VERSION 1
-#define KFD_IOCTL_MINOR_VERSION 18
+#define KFD_IOCTL_MINOR_VERSION 19
struct kfd_ioctl_get_version_args {
__u32 major_version; /* from KFD */
@@ -1671,7 +1672,10 @@ struct kfd_ioctl_dbg_trap_args {
#define AMDKFD_IOC_DBG_TRAP \
AMDKFD_IOWR(0x26, struct kfd_ioctl_dbg_trap_args)
+#define AMDKFD_IOC_CREATE_PROCESS \
+ AMDKFD_IO(0x27)
+
#define AMDKFD_COMMAND_START 0x01
-#define AMDKFD_COMMAND_END 0x27
+#define AMDKFD_COMMAND_END 0x28
#endif
--
2.51.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* Re: [PATCH V4 08/18] amdkfd: identify a secondary kfd process by its id
2025-09-23 7:25 ` [PATCH V4 08/18] amdkfd: identify a secondary kfd process by its id Zhu Lingshan
@ 2025-09-24 21:41 ` Kuehling, Felix
2025-09-26 2:58 ` Zhu, Lingshan
0 siblings, 1 reply; 28+ messages in thread
From: Kuehling, Felix @ 2025-09-24 21:41 UTC (permalink / raw)
To: Zhu Lingshan, alexander.deucher; +Cc: ray.huang, amd-gfx
On 2025-09-23 03:25, Zhu Lingshan wrote:
> This commit introduces a new id field for
> struct kfd process, which helps identify
> a kfd process among multiple contexts that
> all belong to a single user space program.
>
> The sysfs entry of a secondary kfd process
> is placed under the sysfs entry folder of
> its primary kfd process.
>
> The naming format of the sysfs entry of a secondary
> kfd process is "context_%u" where %u is the process id.
>
> Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
> Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
> ---
> drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 6 ++
> drivers/gpu/drm/amd/amdkfd/kfd_process.c | 83 +++++++++++++++++++++++-
> 2 files changed, 86 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> index d1436f1f527d..d140463e221b 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> @@ -998,6 +998,9 @@ struct kfd_process {
> /* Tracks debug per-vmid request for debug flags */
> u32 dbg_flags;
>
> + /* kfd process id */
> + u16 id;
Can this subsume the "primary" flag? E.g. process->id == 0 could mean
"primary context", and all the secondary contexts could have non-0 IDs.
Regards,
Felix
> +
> atomic_t poison;
> /* Queues are in paused stated because we are in the process of doing a CRIU checkpoint */
> bool queues_paused;
> @@ -1012,6 +1015,9 @@ struct kfd_process {
>
> /* indicating whether this is a primary kfd_process */
> bool primary;
> +
> + /* The primary kfd_process allocating IDs for its secondary kfd_process, 0 for primary kfd_process */
> + struct ida id_table;
> };
>
> #define KFD_PROCESS_TABLE_SIZE 8 /* bits: 256 entries */
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> index 5d59a4d994d5..8e498fd35b8c 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> @@ -54,6 +54,9 @@ DEFINE_MUTEX(kfd_processes_mutex);
>
> DEFINE_SRCU(kfd_processes_srcu);
>
> +#define KFD_PROCESS_ID_MIN 1
> +#define KFD_PROCESS_ID_WIDTH 16
> +
> /* For process termination handling */
> static struct workqueue_struct *kfd_process_wq;
>
> @@ -827,6 +830,7 @@ static void kfd_process_device_destroy_ib_mem(struct kfd_process_device *pdd)
>
> int kfd_create_process_sysfs(struct kfd_process *process)
> {
> + struct kfd_process *primary_process;
> int ret;
>
> if (process->kobj) {
> @@ -839,9 +843,22 @@ int kfd_create_process_sysfs(struct kfd_process *process)
> pr_warn("Creating procfs kobject failed");
> return -ENOMEM;
> }
> - ret = kobject_init_and_add(process->kobj, &procfs_type,
> - procfs.kobj, "%d",
> - (int)process->lead_thread->pid);
> +
> + if (process->primary)
> + ret = kobject_init_and_add(process->kobj, &procfs_type,
> + procfs.kobj, "%d",
> + (int)process->lead_thread->pid);
> + else {
> + primary_process = kfd_lookup_process_by_mm(process->lead_thread->mm);
> + if (!primary_process)
> + return -ESRCH;
> +
> + ret = kobject_init_and_add(process->kobj, &procfs_type,
> + primary_process->kobj, "context_%u",
> + process->id);
> + kfd_unref_process(primary_process);
> + }
> +
> if (ret) {
> pr_warn("Creating procfs pid directory failed");
> kobject_put(process->kobj);
> @@ -863,6 +880,51 @@ int kfd_create_process_sysfs(struct kfd_process *process)
> return 0;
> }
>
> +static int kfd_process_alloc_id(struct kfd_process *process)
> +{
> + int ret;
> + struct kfd_process *primary_process;
> +
> + if (process->primary) {
> + process->id = 0;
> +
> + return 0;
> + }
> +
> + primary_process = kfd_lookup_process_by_mm(process->lead_thread->mm);
> + if (!primary_process)
> + return -ESRCH;
> +
> + ret = ida_alloc_range(&primary_process->id_table, KFD_PROCESS_ID_MIN,
> + (1 << KFD_PROCESS_ID_WIDTH) - 1, GFP_KERNEL);
> + if (ret < 0)
> + goto out;
> +
> + process->id = ret;
> + ret = 0;
> +
> +out:
> + kfd_unref_process(primary_process);
> +
> + return ret;
> +}
> +
> +static void kfd_process_free_id(struct kfd_process *process)
> +{
> + struct kfd_process *primary_process;
> +
> + if (process->primary)
> + return;
> +
> + primary_process = kfd_lookup_process_by_mm(process->lead_thread->mm);
> + if (!primary_process)
> + return;
> +
> + ida_free(&primary_process->id_table, process->id);
> +
> + kfd_unref_process(primary_process);
> +}
> +
> struct kfd_process *kfd_create_process(struct task_struct *thread)
> {
> struct kfd_process *process;
> @@ -1193,6 +1255,11 @@ static void kfd_process_wq_release(struct work_struct *work)
> if (ef)
> dma_fence_signal(ef);
>
> + if (!p->primary)
> + kfd_process_free_id(p);
> + else
> + ida_destroy(&p->id_table);
> +
> kfd_process_remove_sysfs(p);
> kfd_debugfs_remove_process(p);
>
> @@ -1549,6 +1616,12 @@ static struct kfd_process *create_process(const struct task_struct *thread, bool
> process->queues_paused = false;
> process->primary = primary;
>
> + err = kfd_process_alloc_id(process);
> + if (err) {
> + pr_err("Creating kfd process: failed to alloc an id\n");
> + goto err_alloc_id;
> + }
> +
> INIT_DELAYED_WORK(&process->eviction_work, evict_process_worker);
> INIT_DELAYED_WORK(&process->restore_work, restore_process_worker);
> process->last_restore_timestamp = get_jiffies_64();
> @@ -1599,6 +1672,8 @@ static struct kfd_process *create_process(const struct task_struct *thread, bool
> goto err_register_notifier;
> }
> BUG_ON(mn != &process->mmu_notifier);
> +
> + ida_init(&process->id_table);
> }
>
> kfd_unref_process(process);
> @@ -1619,6 +1694,8 @@ static struct kfd_process *create_process(const struct task_struct *thread, bool
> err_process_pqm_init:
> kfd_event_free_process(process);
> err_event_init:
> + kfd_process_free_id(process);
> +err_alloc_id:
> mutex_destroy(&process->mutex);
> kfree(process);
> err_alloc_process:
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH V4 10/18] amdkfd: remove DIQ support
2025-09-23 7:26 ` [PATCH V4 10/18] amdkfd: remove DIQ support Zhu Lingshan
@ 2025-09-24 21:41 ` Kuehling, Felix
0 siblings, 0 replies; 28+ messages in thread
From: Kuehling, Felix @ 2025-09-24 21:41 UTC (permalink / raw)
To: Zhu Lingshan, alexander.deucher; +Cc: ray.huang, amd-gfx
On 2025-09-23 03:26, Zhu Lingshan wrote:
> This commit remove DIQ support because it has been
> marked as DEPRECATED since 2022 in commit 5bdd3eb2
>
> Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
> ---
> .../drm/amd/amdkfd/kfd_device_queue_manager.c | 6 +--
> drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c | 29 +-------------
> .../drm/amd/amdkfd/kfd_packet_manager_v9.c | 4 --
> .../drm/amd/amdkfd/kfd_packet_manager_vi.c | 4 --
> drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 1 -
> .../amd/amdkfd/kfd_process_queue_manager.c | 39 +------------------
> 6 files changed, 6 insertions(+), 77 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> index 6c5c7c1bf5ed..ac6052d7a4cb 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> @@ -399,8 +399,7 @@ static void increment_queue_count(struct device_queue_manager *dqm,
> struct queue *q)
> {
> dqm->active_queue_count++;
> - if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE ||
> - q->properties.type == KFD_QUEUE_TYPE_DIQ)
> + if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE)
> dqm->active_cp_queue_count++;
>
> if (q->properties.is_gws) {
> @@ -414,8 +413,7 @@ static void decrement_queue_count(struct device_queue_manager *dqm,
> struct queue *q)
> {
> dqm->active_queue_count--;
> - if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE ||
> - q->properties.type == KFD_QUEUE_TYPE_DIQ)
> + if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE)
> dqm->active_cp_queue_count--;
>
> if (q->properties.is_gws) {
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
> index 2b0a830f5b29..4c7ab7a711be 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
> @@ -46,7 +46,7 @@ static bool kq_initialize(struct kernel_queue *kq, struct kfd_node *dev,
> int retval;
> union PM4_MES_TYPE_3_HEADER nop;
>
> - if (WARN_ON(type != KFD_QUEUE_TYPE_DIQ && type != KFD_QUEUE_TYPE_HIQ))
> + if (WARN_ON(type != KFD_QUEUE_TYPE_HIQ))
> return false;
>
> pr_debug("Initializing queue type %d size %d\n", KFD_QUEUE_TYPE_HIQ,
> @@ -61,17 +61,7 @@ static bool kq_initialize(struct kernel_queue *kq, struct kfd_node *dev,
>
> kq->dev = dev;
> kq->nop_packet = nop.u32all;
> - switch (type) {
> - case KFD_QUEUE_TYPE_DIQ:
> - kq->mqd_mgr = dev->dqm->mqd_mgrs[KFD_MQD_TYPE_DIQ];
> - break;
> - case KFD_QUEUE_TYPE_HIQ:
> - kq->mqd_mgr = dev->dqm->mqd_mgrs[KFD_MQD_TYPE_HIQ];
> - break;
> - default:
> - dev_err(dev->adev->dev, "Invalid queue type %d\n", type);
> - return false;
> - }
> + kq->mqd_mgr = dev->dqm->mqd_mgrs[KFD_MQD_TYPE_HIQ];
>
> if (!kq->mqd_mgr)
> return false;
> @@ -162,24 +152,11 @@ static bool kq_initialize(struct kernel_queue *kq, struct kfd_node *dev,
> kq->mqd_mgr->load_mqd(kq->mqd_mgr, kq->queue->mqd,
> kq->queue->pipe, kq->queue->queue,
> &kq->queue->properties, NULL);
> - } else {
> - /* allocate fence for DIQ */
> -
> - retval = kfd_gtt_sa_allocate(dev, sizeof(uint32_t),
> - &kq->fence_mem_obj);
> -
> - if (retval != 0)
> - goto err_alloc_fence;
> -
> - kq->fence_kernel_address = kq->fence_mem_obj->cpu_ptr;
> - kq->fence_gpu_addr = kq->fence_mem_obj->gpu_addr;
> }
>
> print_queue(kq->queue);
>
> return true;
> -err_alloc_fence:
> - kq->mqd_mgr->free_mqd(kq->mqd_mgr, kq->queue->mqd, kq->queue->mqd_mem_obj);
> err_allocate_mqd:
> uninit_queue(kq->queue);
> err_init_queue:
> @@ -209,8 +186,6 @@ static void kq_uninitialize(struct kernel_queue *kq)
> kq->queue->queue);
> up_read(&kq->dev->adev->reset_domain->sem);
> }
> - else if (kq->queue->properties.type == KFD_QUEUE_TYPE_DIQ)
> - kfd_gtt_sa_free(kq->dev, kq->fence_mem_obj);
>
> kq->mqd_mgr->free_mqd(kq->mqd_mgr, kq->queue->mqd,
> kq->queue->mqd_mem_obj);
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c
> index 505036968a77..3d2375817c3e 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c
> @@ -252,10 +252,6 @@ static int pm_map_queues_v9(struct packet_manager *pm, uint32_t *buffer,
> packet->bitfields2.queue_type =
> queue_type__mes_map_queues__normal_latency_static_queue_vi;
> break;
> - case KFD_QUEUE_TYPE_DIQ:
> - packet->bitfields2.queue_type =
> - queue_type__mes_map_queues__debug_interface_queue_vi;
> - break;
> case KFD_QUEUE_TYPE_SDMA:
> case KFD_QUEUE_TYPE_SDMA_XGMI:
> if (q->properties.sdma_engine_id < 2 &&
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c
> index a1de5d7e173a..60086e7cc258 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c
> @@ -166,10 +166,6 @@ static int pm_map_queues_vi(struct packet_manager *pm, uint32_t *buffer,
> packet->bitfields2.queue_type =
> queue_type__mes_map_queues__normal_latency_static_queue_vi;
> break;
> - case KFD_QUEUE_TYPE_DIQ:
> - packet->bitfields2.queue_type =
> - queue_type__mes_map_queues__debug_interface_queue_vi;
> - break;
> case KFD_QUEUE_TYPE_SDMA:
> case KFD_QUEUE_TYPE_SDMA_XGMI:
> packet->bitfields2.engine_sel = q->properties.sdma_engine_id +
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> index d140463e221b..a2f8b1c24fc3 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> @@ -425,7 +425,6 @@ enum kfd_queue_type {
> KFD_QUEUE_TYPE_COMPUTE,
> KFD_QUEUE_TYPE_SDMA,
> KFD_QUEUE_TYPE_HIQ,
> - KFD_QUEUE_TYPE_DIQ,
> KFD_QUEUE_TYPE_SDMA_XGMI,
> KFD_QUEUE_TYPE_SDMA_BY_ENG_ID
> };
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
> index 7fbb5c274ccc..8ddc33abf230 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
> @@ -345,7 +345,7 @@ int pqm_create_queue(struct process_queue_manager *pqm,
> * If we are just about to create DIQ, the is_debug flag is not set yet
> * Hence we also check the type as well
> */
> - if ((pdd->qpd.is_debug) || (type == KFD_QUEUE_TYPE_DIQ))
> + if ((pdd->qpd.is_debug))
> max_queues = dev->kfd->device_info.max_no_of_hqd/2;
>
> if (pdd->qpd.queue_count >= max_queues)
> @@ -426,22 +426,6 @@ int pqm_create_queue(struct process_queue_manager *pqm,
> restore_mqd, restore_ctl_stack);
> print_queue(q);
> break;
> - case KFD_QUEUE_TYPE_DIQ:
> - kq = kernel_queue_init(dev, KFD_QUEUE_TYPE_DIQ);
> - if (!kq) {
> - retval = -ENOMEM;
> - goto err_create_queue;
> - }
> - kq->queue->properties.queue_id = *qid;
> - pqn->kq = kq;
> - pqn->q = NULL;
> - retval = kfd_process_drain_interrupts(pdd);
> - if (retval)
> - break;
> -
> - retval = dev->dqm->ops.create_kernel_queue(dev->dqm,
> - kq, &pdd->qpd);
> - break;
> default:
> WARN(1, "Invalid queue type %d", type);
> retval = -EINVAL;
> @@ -1131,32 +1115,13 @@ int pqm_debugfs_mqds(struct seq_file *m, void *data)
> break;
> default:
> seq_printf(m,
> - " Bad user queue type %d on device %x\n",
> + " Qeueu node with bad user queue type %d on device %x\n",
> q->properties.type, q->device->id);
> continue;
> }
> mqd_mgr = q->device->dqm->mqd_mgrs[mqd_type];
> size = mqd_mgr->mqd_stride(mqd_mgr,
> &q->properties);
> - } else if (pqn->kq) {
> - q = pqn->kq->queue;
> - mqd_mgr = pqn->kq->mqd_mgr;
> - switch (q->properties.type) {
> - case KFD_QUEUE_TYPE_DIQ:
> - seq_printf(m, " DIQ on device %x\n",
> - pqn->kq->dev->id);
> - break;
> - default:
> - seq_printf(m,
> - " Bad kernel queue type %d on device %x\n",
> - q->properties.type,
> - pqn->kq->dev->id);
> - continue;
> - }
> - } else {
> - seq_printf(m,
> - " Weird: Queue node with neither kernel nor user queue\n");
> - continue;
> }
>
> for (xcc = 0; xcc < num_xccs; xcc++) {
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH V4 11/18] amdkfd: process pointer of a HIQ should be NULL
2025-09-23 7:26 ` [PATCH V4 11/18] amdkfd: process pointer of a HIQ should be NULL Zhu Lingshan
@ 2025-09-24 21:43 ` Kuehling, Felix
0 siblings, 0 replies; 28+ messages in thread
From: Kuehling, Felix @ 2025-09-24 21:43 UTC (permalink / raw)
To: Zhu Lingshan, alexander.deucher; +Cc: ray.huang, amd-gfx
On 2025-09-23 03:26, Zhu Lingshan wrote:
> In kq_initialize, queue->process of a HIQ should
> be NULL as initialized, because it does not belong
> to any kfd_process.
>
> This commit decommisions the function kfd_get_process() because
> it can not locate a specific kfd_process among multiple
> contexts and not any code path calls it after this commit.
>
> Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
> ---
> drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c | 1 -
> drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 1 -
> drivers/gpu/drm/amd/amdkfd/kfd_process.c | 18 ------------------
> 3 files changed, 20 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
> index 4c7ab7a711be..0a4f8e8dd77e 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
> @@ -134,7 +134,6 @@ static bool kq_initialize(struct kernel_queue *kq, struct kfd_node *dev,
> goto err_init_queue;
>
> kq->queue->device = dev;
> - kq->queue->process = kfd_get_process(current);
>
> kq->queue->mqd_mem_obj = kq->mqd_mgr->allocate_mqd(kq->mqd_mgr->dev,
> &kq->queue->properties);
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> index a2f8b1c24fc3..e09c8bd3e138 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> @@ -1049,7 +1049,6 @@ void kfd_process_destroy_wq(void);
> void kfd_cleanup_processes(void);
> struct kfd_process *kfd_create_process(struct task_struct *thread);
> int kfd_create_process_sysfs(struct kfd_process *process);
> -struct kfd_process *kfd_get_process(const struct task_struct *task);
> struct kfd_process *kfd_lookup_process_by_pasid(u32 pasid,
> struct kfd_process_device **pdd);
> struct kfd_process *kfd_lookup_process_by_mm(const struct mm_struct *mm);
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> index 8e498fd35b8c..0c3f0cc16bf4 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> @@ -989,24 +989,6 @@ struct kfd_process *kfd_create_process(struct task_struct *thread)
> return process;
> }
>
> -struct kfd_process *kfd_get_process(const struct task_struct *thread)
> -{
> - struct kfd_process *process;
> -
> - if (!thread->mm)
> - return ERR_PTR(-EINVAL);
> -
> - /* Only the pthreads threading model is supported. */
> - if (thread->group_leader->mm != thread->mm)
> - return ERR_PTR(-EINVAL);
> -
> - process = find_process(thread, false);
> - if (!process)
> - return ERR_PTR(-EINVAL);
> -
> - return process;
> -}
> -
> static struct kfd_process *find_process_by_mm(const struct mm_struct *mm)
> {
> struct kfd_process *process;
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH V4 14/18] amdkfd: record kfd process id into kfd process_info
2025-09-23 7:26 ` [PATCH V4 14/18] amdkfd: record kfd process id into kfd process_info Zhu Lingshan
@ 2025-09-24 21:45 ` Kuehling, Felix
2025-09-26 2:51 ` Zhu, Lingshan
0 siblings, 1 reply; 28+ messages in thread
From: Kuehling, Felix @ 2025-09-24 21:45 UTC (permalink / raw)
To: Zhu Lingshan, alexander.deucher; +Cc: ray.huang, amd-gfx
On 2025-09-23 03:26, Zhu Lingshan wrote:
> This commit records the id of the owner
> kfd_process into a kfd process_info when
> create it.
>
> Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 2 ++
> drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 4 ++++
> 2 files changed, 6 insertions(+)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
> index aa88bad7416b..d867984a68da 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
> @@ -146,6 +146,8 @@ struct amdkfd_process_info {
> /* MMU-notifier related fields */
> struct mutex notifier_lock;
> uint32_t evicted_bos;
> + /* kfd process id */
> + u16 process_id;
The name "process_id" is a bit misleading. I would prefer something like
"context_id" or "secondary_id" to make it cleare that this identifies
secondary contexts and has nothing to do with the PID. Maybe use the
same name in struct kfd_process as well for clarity.
Regards,
Felix
> struct delayed_work restore_userptr_work;
> struct pid *pid;
> bool block_mmu_notifications;
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
> index b16cce7c22c3..723d34921c12 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
> @@ -1382,8 +1382,10 @@ static int init_kfd_vm(struct amdgpu_vm *vm, void **process_info,
> struct dma_fence **ef)
> {
> struct amdkfd_process_info *info = NULL;
> + struct kfd_process *process = NULL;
> int ret;
>
> + process = container_of(process_info, struct kfd_process, kgd_process_info);
> if (!*process_info) {
> info = kzalloc(sizeof(*info), GFP_KERNEL);
> if (!info)
> @@ -1410,6 +1412,8 @@ static int init_kfd_vm(struct amdgpu_vm *vm, void **process_info,
> INIT_DELAYED_WORK(&info->restore_userptr_work,
> amdgpu_amdkfd_restore_userptr_worker);
>
> + info->process_id = process->id;
> +
> *process_info = info;
> }
>
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH V4 17/18] amdkfd: set_debug_trap ioctl only works on a primary kfd_process target
2025-09-23 7:26 ` [PATCH V4 17/18] amdkfd: set_debug_trap ioctl only works on a primary kfd_process target Zhu Lingshan
@ 2025-09-24 21:50 ` Kuehling, Felix
2025-09-26 2:49 ` Zhu, Lingshan
0 siblings, 1 reply; 28+ messages in thread
From: Kuehling, Felix @ 2025-09-24 21:50 UTC (permalink / raw)
To: Zhu Lingshan, alexander.deucher; +Cc: ray.huang, amd-gfx
On 2025-09-23 03:26, Zhu Lingshan wrote:
> The user space program pass down a pid to kfd
> through set_debug_trap ioctl, which can help
> find the corresponding user space program and
> its mm struct.
>
> However, these information is insufficient to
> locate a specific kfd process among the
> multiple kfd_process that belong to the
> same user space program.
>
> For correctness and backward compatibilities,
> this commit only allow set_debut_trap ioctl
> work for a user space program which does not
> own any secondary kfd_process.
What happens if a secondary context is created after the debugger
attaches to a process?
It may be simpler and more consistent to allow debugging of the primary
context, even if secondary contexts exist. Just ignore any secondary
contexts for the purpose of the debug API.
Regards,
Felix
>
> Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
> ---
> drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 29 ++++++++++++++++++++++++
> 1 file changed, 29 insertions(+)
>
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
> index 662b181f1fd2..2df095e25c2e 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
> @@ -36,6 +36,7 @@
> #include <linux/ptrace.h>
> #include <linux/dma-buf.h>
> #include <linux/processor.h>
> +#include <linux/fdtable.h>
> #include "kfd_priv.h"
> #include "kfd_device_queue_manager.h"
> #include "kfd_svm.h"
> @@ -2918,6 +2919,25 @@ static int kfd_ioctl_runtime_enable(struct file *filep, struct kfd_process *p, v
> return r;
> }
>
> +static int kfd_process_count_cb(const void *num, struct file *filep, unsigned int n)
> +{
> + u16 *ret = (u16 *)num;
> +
> + if (filep->f_op == &kfd_fops)
> + (*ret)++;
> +
> + return 0;
> +}
> +
> +static u16 kfd_process_count(struct task_struct *thread)
> +{
> + u16 count = 0;
> +
> + iterate_fd(thread->files, 0, kfd_process_count_cb, (const void *)&count);
> +
> + return count;
> +}
> +
> static int kfd_ioctl_set_debug_trap(struct file *filep, struct kfd_process *p, void *data)
> {
> struct kfd_ioctl_dbg_trap_args *args = data;
> @@ -2927,6 +2947,7 @@ static int kfd_ioctl_set_debug_trap(struct file *filep, struct kfd_process *p, v
> struct kfd_process *target = NULL;
> struct kfd_process_device *pdd = NULL;
> int r = 0;
> + u16 process_count = 0;
>
> if (sched_policy == KFD_SCHED_POLICY_NO_HWS) {
> pr_err("Debugging does not support sched_policy %i", sched_policy);
> @@ -2984,6 +3005,14 @@ static int kfd_ioctl_set_debug_trap(struct file *filep, struct kfd_process *p, v
> if (r)
> goto out;
>
> + process_count = kfd_process_count(thread);
> + /* The user space program has secondary contexts */
> + if (process_count > 1) {
> + pr_err("Detect multiple kfd_process owned by PID %i, please consider close any secondary kfd_process and try again\n", args->pid);
> + r = -EINVAL;
> + goto out;
> + }
> +
> mutex_lock(&target->mutex);
>
> if (args->op != KFD_IOC_DBG_TRAP_ENABLE && !target->debug_trap_enabled) {
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH V4 17/18] amdkfd: set_debug_trap ioctl only works on a primary kfd_process target
2025-09-24 21:50 ` Kuehling, Felix
@ 2025-09-26 2:49 ` Zhu, Lingshan
2025-10-01 5:42 ` Kuehling, Felix
0 siblings, 1 reply; 28+ messages in thread
From: Zhu, Lingshan @ 2025-09-26 2:49 UTC (permalink / raw)
To: Kuehling, Felix, alexander.deucher; +Cc: ray.huang, amd-gfx
[-- Attachment #1: Type: text/plain, Size: 3671 bytes --]
On 9/25/2025 5:50 AM, Kuehling, Felix wrote:
> On 2025-09-23 03:26, Zhu Lingshan wrote:
>> The user space program pass down a pid to kfd
>> through set_debug_trap ioctl, which can help
>> find the corresponding user space program and
>> its mm struct.
>>
>> However, these information is insufficient to
>> locate a specific kfd process among the
>> multiple kfd_process that belong to the
>> same user space program.
>>
>> For correctness and backward compatibilities,
>> this commit only allow set_debut_trap ioctl
>> work for a user space program which does not
>> own any secondary kfd_process.
>
> What happens if a secondary context is created after the debugger
> attaches to a process?
>
> It may be simpler and more consistent to allow debugging of the
> primary context, even if secondary contexts exist. Just ignore any
> secondary contexts for the purpose of the debug API.
Do you mean reject any set_debug_trap ioctls on secondary contexts, and only allow debugging on the primary context, just like how we handle SVM and user ptr?
Thanks
Lingshan
>
> Regards,
> Felix
>
>
>>
>> Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
>> ---
>> drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 29 ++++++++++++++++++++++++
>> 1 file changed, 29 insertions(+)
>>
>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
>> b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
>> index 662b181f1fd2..2df095e25c2e 100644
>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
>> @@ -36,6 +36,7 @@
>> #include <linux/ptrace.h>
>> #include <linux/dma-buf.h>
>> #include <linux/processor.h>
>> +#include <linux/fdtable.h>
>> #include "kfd_priv.h"
>> #include "kfd_device_queue_manager.h"
>> #include "kfd_svm.h"
>> @@ -2918,6 +2919,25 @@ static int kfd_ioctl_runtime_enable(struct
>> file *filep, struct kfd_process *p, v
>> return r;
>> }
>> +static int kfd_process_count_cb(const void *num, struct file
>> *filep, unsigned int n)
>> +{
>> + u16 *ret = (u16 *)num;
>> +
>> + if (filep->f_op == &kfd_fops)
>> + (*ret)++;
>> +
>> + return 0;
>> +}
>> +
>> +static u16 kfd_process_count(struct task_struct *thread)
>> +{
>> + u16 count = 0;
>> +
>> + iterate_fd(thread->files, 0, kfd_process_count_cb, (const void
>> *)&count);
>> +
>> + return count;
>> +}
>> +
>> static int kfd_ioctl_set_debug_trap(struct file *filep, struct
>> kfd_process *p, void *data)
>> {
>> struct kfd_ioctl_dbg_trap_args *args = data;
>> @@ -2927,6 +2947,7 @@ static int kfd_ioctl_set_debug_trap(struct file
>> *filep, struct kfd_process *p, v
>> struct kfd_process *target = NULL;
>> struct kfd_process_device *pdd = NULL;
>> int r = 0;
>> + u16 process_count = 0;
>> if (sched_policy == KFD_SCHED_POLICY_NO_HWS) {
>> pr_err("Debugging does not support sched_policy %i",
>> sched_policy);
>> @@ -2984,6 +3005,14 @@ static int kfd_ioctl_set_debug_trap(struct
>> file *filep, struct kfd_process *p, v
>> if (r)
>> goto out;
>> + process_count = kfd_process_count(thread);
>> + /* The user space program has secondary contexts */
>> + if (process_count > 1) {
>> + pr_err("Detect multiple kfd_process owned by PID %i, please
>> consider close any secondary kfd_process and try again\n", args->pid);
>> + r = -EINVAL;
>> + goto out;
>> + }
>> +
>> mutex_lock(&target->mutex);
>> if (args->op != KFD_IOC_DBG_TRAP_ENABLE &&
>> !target->debug_trap_enabled) {
[-- Attachment #2: Type: text/html, Size: 6449 bytes --]
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH V4 14/18] amdkfd: record kfd process id into kfd process_info
2025-09-24 21:45 ` Kuehling, Felix
@ 2025-09-26 2:51 ` Zhu, Lingshan
0 siblings, 0 replies; 28+ messages in thread
From: Zhu, Lingshan @ 2025-09-26 2:51 UTC (permalink / raw)
To: Kuehling, Felix, alexander.deucher; +Cc: ray.huang, amd-gfx
[-- Attachment #1: Type: text/plain, Size: 2558 bytes --]
On 9/25/2025 5:45 AM, Kuehling, Felix wrote:
> On 2025-09-23 03:26, Zhu Lingshan wrote:
>> This commit records the id of the owner
>> kfd_process into a kfd process_info when
>> create it.
>>
>> Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
>> ---
>> drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 2 ++
>> drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 4 ++++
>> 2 files changed, 6 insertions(+)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
>> index aa88bad7416b..d867984a68da 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
>> @@ -146,6 +146,8 @@ struct amdkfd_process_info {
>> /* MMU-notifier related fields */
>> struct mutex notifier_lock;
>> uint32_t evicted_bos;
>> + /* kfd process id */
>> + u16 process_id;
>
> The name "process_id" is a bit misleading. I would prefer something
> like "context_id" or "secondary_id" to make it cleare that this
> identifies secondary contexts and has nothing to do with the PID.
> Maybe use the same name in struct kfd_process as well for clarity.
I will change it to context_id, Thanks!
>
> Regards,
> Felix
>
>
>> struct delayed_work restore_userptr_work;
>> struct pid *pid;
>> bool block_mmu_notifications;
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> index b16cce7c22c3..723d34921c12 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> @@ -1382,8 +1382,10 @@ static int init_kfd_vm(struct amdgpu_vm *vm,
>> void **process_info,
>> struct dma_fence **ef)
>> {
>> struct amdkfd_process_info *info = NULL;
>> + struct kfd_process *process = NULL;
>> int ret;
>> + process = container_of(process_info, struct kfd_process,
>> kgd_process_info);
>> if (!*process_info) {
>> info = kzalloc(sizeof(*info), GFP_KERNEL);
>> if (!info)
>> @@ -1410,6 +1412,8 @@ static int init_kfd_vm(struct amdgpu_vm *vm,
>> void **process_info,
>> INIT_DELAYED_WORK(&info->restore_userptr_work,
>> amdgpu_amdkfd_restore_userptr_worker);
>> + info->process_id = process->id;
>> +
>> *process_info = info;
>> }
>>
[-- Attachment #2: Type: text/html, Size: 4657 bytes --]
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH V4 08/18] amdkfd: identify a secondary kfd process by its id
2025-09-24 21:41 ` Kuehling, Felix
@ 2025-09-26 2:58 ` Zhu, Lingshan
0 siblings, 0 replies; 28+ messages in thread
From: Zhu, Lingshan @ 2025-09-26 2:58 UTC (permalink / raw)
To: Kuehling, Felix, alexander.deucher; +Cc: ray.huang, amd-gfx
[-- Attachment #1: Type: text/plain, Size: 7772 bytes --]
On 9/25/2025 5:41 AM, Kuehling, Felix wrote:
> On 2025-09-23 03:25, Zhu Lingshan wrote:
>> This commit introduces a new id field for
>> struct kfd process, which helps identify
>> a kfd process among multiple contexts that
>> all belong to a single user space program.
>>
>> The sysfs entry of a secondary kfd process
>> is placed under the sysfs entry folder of
>> its primary kfd process.
>>
>> The naming format of the sysfs entry of a secondary
>> kfd process is "context_%u" where %u is the process id.
>>
>> Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
>> Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
>> ---
>> drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 6 ++
>> drivers/gpu/drm/amd/amdkfd/kfd_process.c | 83 +++++++++++++++++++++++-
>> 2 files changed, 86 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
>> b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
>> index d1436f1f527d..d140463e221b 100644
>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
>> @@ -998,6 +998,9 @@ struct kfd_process {
>> /* Tracks debug per-vmid request for debug flags */
>> u32 dbg_flags;
>> + /* kfd process id */
>> + u16 id;
>
> Can this subsume the "primary" flag? E.g. process->id == 0 could mean
> "primary context", and all the secondary contexts could have non-0 IDs.
I will remove this primary flag and try using process->id to identify the contexts, so here 0 is not a good
default value for this process->id of the primary kfd context anymore, because 0 is the default initialized
value for all kfd contexts, I will assign another default value 0xFFFF to the primary kfd context.
This change will affect some other patches in this series, I will remove their "reviewed-by" tag.
Thanks
Lingshan
>
> Regards,
> Felix
>
>
>> +
>> atomic_t poison;
>> /* Queues are in paused stated because we are in the process of
>> doing a CRIU checkpoint */
>> bool queues_paused;
>> @@ -1012,6 +1015,9 @@ struct kfd_process {
>> /* indicating whether this is a primary kfd_process */
>> bool primary;
>> +
>> + /* The primary kfd_process allocating IDs for its secondary
>> kfd_process, 0 for primary kfd_process */
>> + struct ida id_table;
>> };
>> #define KFD_PROCESS_TABLE_SIZE 8 /* bits: 256 entries */
>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
>> b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
>> index 5d59a4d994d5..8e498fd35b8c 100644
>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
>> @@ -54,6 +54,9 @@ DEFINE_MUTEX(kfd_processes_mutex);
>> DEFINE_SRCU(kfd_processes_srcu);
>> +#define KFD_PROCESS_ID_MIN 1
>> +#define KFD_PROCESS_ID_WIDTH 16
>> +
>> /* For process termination handling */
>> static struct workqueue_struct *kfd_process_wq;
>> @@ -827,6 +830,7 @@ static void
>> kfd_process_device_destroy_ib_mem(struct kfd_process_device *pdd)
>> int kfd_create_process_sysfs(struct kfd_process *process)
>> {
>> + struct kfd_process *primary_process;
>> int ret;
>> if (process->kobj) {
>> @@ -839,9 +843,22 @@ int kfd_create_process_sysfs(struct kfd_process
>> *process)
>> pr_warn("Creating procfs kobject failed");
>> return -ENOMEM;
>> }
>> - ret = kobject_init_and_add(process->kobj, &procfs_type,
>> - procfs.kobj, "%d",
>> - (int)process->lead_thread->pid);
>> +
>> + if (process->primary)
>> + ret = kobject_init_and_add(process->kobj, &procfs_type,
>> + procfs.kobj, "%d",
>> + (int)process->lead_thread->pid);
>> + else {
>> + primary_process =
>> kfd_lookup_process_by_mm(process->lead_thread->mm);
>> + if (!primary_process)
>> + return -ESRCH;
>> +
>> + ret = kobject_init_and_add(process->kobj, &procfs_type,
>> + primary_process->kobj, "context_%u",
>> + process->id);
>> + kfd_unref_process(primary_process);
>> + }
>> +
>> if (ret) {
>> pr_warn("Creating procfs pid directory failed");
>> kobject_put(process->kobj);
>> @@ -863,6 +880,51 @@ int kfd_create_process_sysfs(struct kfd_process
>> *process)
>> return 0;
>> }
>> +static int kfd_process_alloc_id(struct kfd_process *process)
>> +{
>> + int ret;
>> + struct kfd_process *primary_process;
>> +
>> + if (process->primary) {
>> + process->id = 0;
>> +
>> + return 0;
>> + }
>> +
>> + primary_process =
>> kfd_lookup_process_by_mm(process->lead_thread->mm);
>> + if (!primary_process)
>> + return -ESRCH;
>> +
>> + ret = ida_alloc_range(&primary_process->id_table,
>> KFD_PROCESS_ID_MIN,
>> + (1 << KFD_PROCESS_ID_WIDTH) - 1, GFP_KERNEL);
>> + if (ret < 0)
>> + goto out;
>> +
>> + process->id = ret;
>> + ret = 0;
>> +
>> +out:
>> + kfd_unref_process(primary_process);
>> +
>> + return ret;
>> +}
>> +
>> +static void kfd_process_free_id(struct kfd_process *process)
>> +{
>> + struct kfd_process *primary_process;
>> +
>> + if (process->primary)
>> + return;
>> +
>> + primary_process =
>> kfd_lookup_process_by_mm(process->lead_thread->mm);
>> + if (!primary_process)
>> + return;
>> +
>> + ida_free(&primary_process->id_table, process->id);
>> +
>> + kfd_unref_process(primary_process);
>> +}
>> +
>> struct kfd_process *kfd_create_process(struct task_struct *thread)
>> {
>> struct kfd_process *process;
>> @@ -1193,6 +1255,11 @@ static void kfd_process_wq_release(struct
>> work_struct *work)
>> if (ef)
>> dma_fence_signal(ef);
>> + if (!p->primary)
>> + kfd_process_free_id(p);
>> + else
>> + ida_destroy(&p->id_table);
>> +
>> kfd_process_remove_sysfs(p);
>> kfd_debugfs_remove_process(p);
>> @@ -1549,6 +1616,12 @@ static struct kfd_process
>> *create_process(const struct task_struct *thread, bool
>> process->queues_paused = false;
>> process->primary = primary;
>> + err = kfd_process_alloc_id(process);
>> + if (err) {
>> + pr_err("Creating kfd process: failed to alloc an id\n");
>> + goto err_alloc_id;
>> + }
>> +
>> INIT_DELAYED_WORK(&process->eviction_work, evict_process_worker);
>> INIT_DELAYED_WORK(&process->restore_work, restore_process_worker);
>> process->last_restore_timestamp = get_jiffies_64();
>> @@ -1599,6 +1672,8 @@ static struct kfd_process *create_process(const
>> struct task_struct *thread, bool
>> goto err_register_notifier;
>> }
>> BUG_ON(mn != &process->mmu_notifier);
>> +
>> + ida_init(&process->id_table);
>> }
>> kfd_unref_process(process);
>> @@ -1619,6 +1694,8 @@ static struct kfd_process *create_process(const
>> struct task_struct *thread, bool
>> err_process_pqm_init:
>> kfd_event_free_process(process);
>> err_event_init:
>> + kfd_process_free_id(process);
>> +err_alloc_id:
>> mutex_destroy(&process->mutex);
>> kfree(process);
>> err_alloc_process:
[-- Attachment #2: Type: text/html, Size: 14356 bytes --]
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH V4 17/18] amdkfd: set_debug_trap ioctl only works on a primary kfd_process target
2025-09-26 2:49 ` Zhu, Lingshan
@ 2025-10-01 5:42 ` Kuehling, Felix
0 siblings, 0 replies; 28+ messages in thread
From: Kuehling, Felix @ 2025-10-01 5:42 UTC (permalink / raw)
To: Zhu, Lingshan, alexander.deucher; +Cc: ray.huang, amd-gfx
[-- Attachment #1: Type: text/plain, Size: 3887 bytes --]
On 2025-09-25 22:49, Zhu, Lingshan wrote:
> On 9/25/2025 5:50 AM, Kuehling, Felix wrote:
>> On 2025-09-23 03:26, Zhu Lingshan wrote:
>>> The user space program pass down a pid to kfd
>>> through set_debug_trap ioctl, which can help
>>> find the corresponding user space program and
>>> its mm struct.
>>>
>>> However, these information is insufficient to
>>> locate a specific kfd process among the
>>> multiple kfd_process that belong to the
>>> same user space program.
>>>
>>> For correctness and backward compatibilities,
>>> this commit only allow set_debut_trap ioctl
>>> work for a user space program which does not
>>> own any secondary kfd_process.
>>
>> What happens if a secondary context is created after the debugger
>> attaches to a process?
>>
>> It may be simpler and more consistent to allow debugging of the
>> primary context, even if secondary contexts exist. Just ignore any
>> secondary contexts for the purpose of the debug API.
> Do you mean reject any set_debug_trap ioctls on secondary contexts, and only allow
> debugging on the primary context, just like how we handle SVM and user ptr?
Yes, that is what I meant.
Regards,
Felix
>
> Thanks
> Lingshan
>>
>> Regards,
>> Felix
>>
>>
>>>
>>> Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
>>> ---
>>> drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 29
>>> ++++++++++++++++++++++++
>>> 1 file changed, 29 insertions(+)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
>>> b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
>>> index 662b181f1fd2..2df095e25c2e 100644
>>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
>>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
>>> @@ -36,6 +36,7 @@
>>> #include <linux/ptrace.h>
>>> #include <linux/dma-buf.h>
>>> #include <linux/processor.h>
>>> +#include <linux/fdtable.h>
>>> #include "kfd_priv.h"
>>> #include "kfd_device_queue_manager.h"
>>> #include "kfd_svm.h"
>>> @@ -2918,6 +2919,25 @@ static int kfd_ioctl_runtime_enable(struct
>>> file *filep, struct kfd_process *p, v
>>> return r;
>>> }
>>> +static int kfd_process_count_cb(const void *num, struct file
>>> *filep, unsigned int n)
>>> +{
>>> + u16 *ret = (u16 *)num;
>>> +
>>> + if (filep->f_op == &kfd_fops)
>>> + (*ret)++;
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +static u16 kfd_process_count(struct task_struct *thread)
>>> +{
>>> + u16 count = 0;
>>> +
>>> + iterate_fd(thread->files, 0, kfd_process_count_cb, (const void
>>> *)&count);
>>> +
>>> + return count;
>>> +}
>>> +
>>> static int kfd_ioctl_set_debug_trap(struct file *filep, struct
>>> kfd_process *p, void *data)
>>> {
>>> struct kfd_ioctl_dbg_trap_args *args = data;
>>> @@ -2927,6 +2947,7 @@ static int kfd_ioctl_set_debug_trap(struct
>>> file *filep, struct kfd_process *p, v
>>> struct kfd_process *target = NULL;
>>> struct kfd_process_device *pdd = NULL;
>>> int r = 0;
>>> + u16 process_count = 0;
>>> if (sched_policy == KFD_SCHED_POLICY_NO_HWS) {
>>> pr_err("Debugging does not support sched_policy %i",
>>> sched_policy);
>>> @@ -2984,6 +3005,14 @@ static int kfd_ioctl_set_debug_trap(struct
>>> file *filep, struct kfd_process *p, v
>>> if (r)
>>> goto out;
>>> + process_count = kfd_process_count(thread);
>>> + /* The user space program has secondary contexts */
>>> + if (process_count > 1) {
>>> + pr_err("Detect multiple kfd_process owned by PID %i, please
>>> consider close any secondary kfd_process and try again\n", args->pid);
>>> + r = -EINVAL;
>>> + goto out;
>>> + }
>>> +
>>> mutex_lock(&target->mutex);
>>> if (args->op != KFD_IOC_DBG_TRAP_ENABLE &&
>>> !target->debug_trap_enabled) {
[-- Attachment #2: Type: text/html, Size: 6529 bytes --]
^ permalink raw reply [flat|nested] 28+ messages in thread
end of thread, other threads:[~2025-10-01 5:42 UTC | newest]
Thread overview: 28+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-09-23 7:25 [PATCH V4 00/18] amdkfd: Implement kfd multiple contexts Zhu Lingshan
2025-09-23 7:25 ` [PATCH V4 01/18] amdkfd: enlarge the hashtable of kfd_process Zhu Lingshan
2025-09-23 7:25 ` [PATCH V4 02/18] amdkfd: mark the first kfd_process as the primary one Zhu Lingshan
2025-09-23 7:25 ` [PATCH V4 03/18] amdkfd: find_process_by_mm always return the primary context Zhu Lingshan
2025-09-23 7:25 ` [PATCH V4 04/18] amdkfd: Introduce kfd_create_process_sysfs as a separate function Zhu Lingshan
2025-09-23 7:25 ` [PATCH V4 05/18] amdkfd: destroy kfd secondary contexts through fd close Zhu Lingshan
2025-09-23 7:25 ` [PATCH V4 06/18] amdkfd: process svm ioctl only on the primary kfd process Zhu Lingshan
2025-09-23 7:25 ` [PATCH V4 07/18] amdkfd: process USERPTR allocation " Zhu Lingshan
2025-09-23 7:25 ` [PATCH V4 08/18] amdkfd: identify a secondary kfd process by its id Zhu Lingshan
2025-09-24 21:41 ` Kuehling, Felix
2025-09-26 2:58 ` Zhu, Lingshan
2025-09-23 7:25 ` [PATCH V4 09/18] amdkfd: find kfd_process by filep->private_data in kfd_mmap Zhu Lingshan
2025-09-23 7:26 ` [PATCH V4 10/18] amdkfd: remove DIQ support Zhu Lingshan
2025-09-24 21:41 ` Kuehling, Felix
2025-09-23 7:26 ` [PATCH V4 11/18] amdkfd: process pointer of a HIQ should be NULL Zhu Lingshan
2025-09-24 21:43 ` Kuehling, Felix
2025-09-23 7:26 ` [PATCH V4 12/18] amdkfd: remove test_kq Zhu Lingshan
2025-09-23 7:26 ` [PATCH V4 13/18] amdkfd: introduce new helper kfd_lookup_process_by_id Zhu Lingshan
2025-09-23 7:26 ` [PATCH V4 14/18] amdkfd: record kfd process id into kfd process_info Zhu Lingshan
2025-09-24 21:45 ` Kuehling, Felix
2025-09-26 2:51 ` Zhu, Lingshan
2025-09-23 7:26 ` [PATCH V4 15/18] amdkfd: record kfd process id in amdkfd_fence Zhu Lingshan
2025-09-23 7:26 ` [PATCH V4 16/18] amdkfd: fence handler evict and restore a kfd process by its id Zhu Lingshan
2025-09-23 7:26 ` [PATCH V4 17/18] amdkfd: set_debug_trap ioctl only works on a primary kfd_process target Zhu Lingshan
2025-09-24 21:50 ` Kuehling, Felix
2025-09-26 2:49 ` Zhu, Lingshan
2025-10-01 5:42 ` Kuehling, Felix
2025-09-23 7:26 ` [PATCH V4 18/18] amdkfd: introduce new ioctl AMDKFD_IOC_CREATE_PROCESS Zhu Lingshan
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.