* [PATCH V3 00/13] amdkfd: Implement kfd multiple contexts
@ 2025-08-04 11:05 Zhu Lingshan
2025-08-04 11:05 ` [PATCH V3 01/13] amdkfd: enlarge the hashtable of kfd_process Zhu Lingshan
` (12 more replies)
0 siblings, 13 replies; 21+ messages in thread
From: Zhu Lingshan @ 2025-08-04 11:05 UTC (permalink / raw)
To: alexander.deucher, felix.kuehling; +Cc: ray.huang, amd-gfx, Zhu Lingshan
Currently kfd manages kfd_process in a one context (kfd_process)
per program manner, thus each user space program
only onws one kfd context (kfd_process).
This model works fine for most of the programs, but imperfect
for a hypervisor like QEMU. Because all programs in the guest
user space share the same only one kfd context, which is
problematic, including but not limited to:
As illustrated in Figure 1, all guest user space programs share the same fd of /dev/kfd
and the same kfd_process, and the same PASID leading to the same
GPU_VM address space. Therefore the IOVA range of each
guest user space programs are not isolated,
they can attack each other through GPU DMA.
+----------------------------------------------------------------------------------+
| |
| +-----------+ +-----------+ +------------+ +------------+ |
| | | | | | | | | |
| | Program 1 | | Program 2 | | Program 3 | | Program N | |
| | | | | | | | | |
| +----+------+ +--------+--+ +--+---------+ +-----+------+ |
| | | | | |
| | | | | Guest |
| | | | | |
+-------+----------------------+------------+----------------------+---------------+
| | | |
| | | |
| | | |
| | | |
| +--+------------+---+ |
| | file descriptor | |
+-------------------+ of /dev/kfd +------------------+
| opened by QEMU |
| |
+---------+---------+ User Space
| QEMU
|
---------------------------------------+-----------------------------------------------------
| Kernel Space
| KFD Module
|
+--------+--------+
| |
| kfd_process |<------------------The only one KFD context
| |
+--------+--------+
|
+--------+--------+
| PASID |
+--------+--------+
|
+--------+--------+
| GPU_VM |
+-----------------+
Fiture 1
This series implements a multiple contexts solution:
- Allow each program to create and hold multiple contexts (kfd processes)
- Each context has its own fd of /dev/kfd and an exclusive kfd_process,
which is a secondary kfd context. So that PASID/GPU VM isolates their IOVA address spaces.
Therefore, they can not attack each other through GPU DMA.
The design is illustrated in Figure 2 below:
+---------------------------------------------------------------------------------------------------------+
| |
| |
| |
| +----------------------------------------------------------------------------------+ |
| | | |
| | +-----------+ +-----------+ +-----------+ +-----------+ | |
| | | | | | | | | | | |
| | | Program 1 | | Program 2 | | Program 3 | | Program N | | |
| | | | | | | | | | | |
| | +-----+-----+ +-----+-----+ +-----+-----+ +-----+-----+ | |
| | | | | | | |
| | | | | | Guest | |
| | | | | | | |
| +-------+------------------+-----------------+----------------+--------------------+ |
| | | | | QEMU |
| | | | | |
+---------------+------------------+-----------------+----------------+--------------------------+--------+
| | | | |
| | | | |
| | | | |
+---+----+ +---+----+ +---+----+ +---+----+ +---+-----+
| | | | | | | | | Primary |
| FD 1 | | FD 2 | | FD 3 | | FD 4 | | FD |
| | | | | | | | | |
+---+----+ +---+----+ +---+----+ +----+---+ +----+----+
| | | | | User Space
| | | | |
-------------------+------------------+-----------------+-----------------+--------------------------+----------------------------
| | | | | Kernel SPace
| | | | |
| | | | |
+--------------------------------------------------------------------------------------------------------------------------+
| +------+------+ +------+------+ +------+------+ +------+------+ +------+------+ |
| | Secondary | | Secondary | | Secondary | | Secondary | | Primary | KFD Module |
| |kfd_process 1| |kfd_process 2| |kfd_process 3| |kfd_process 4| | kfd_process | |
| | | | | | | | | | | |
| +------+------+ +------+------+ +------+------+ +------+------+ +------+------+ |
| | | | | | |
| +------+------+ +------+------+ +------+------+ +------+------+ +------+------+ |
| | PASID | | PASID | | PASID | | PASID | | PASID | |
| +------+------+ +------+------+ +------+------+ +------+------+ +------+------+ |
| | | | | | |
| | | | | | |
| +------+------+ +------+------+ +------+------+ +------+------+ +------+------+ |
| | GPU_VM | | GPU_VM | | GPU_VM | | GPU_VM | | GPU_VM | |
| +-------------+ +-------------+ +-------------+ +-------------+ +-------------+ |
| |
+--------------------------------------------------------------------------------------------------------------------------+
Figure 2
Appendix, a minimal test program:
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
#include <errno.h>
#include <sys/ioctl.h>
#include <linux/kfd_ioctl.h>
int main() {
int fd1, fd2, ret;
// open FDs
fd1 = open("/dev/kfd", O_RDWR);
if (fd1 < 0) {
perror("Failed to open FD1 /dev/kfd");
return EXIT_FAILURE;
}
printf("FD1 == %d /dev/kfd opened successfully\n", fd1);
getchar();
fd2 = open("/dev/kfd", O_RDWR);
if (fd2 < 0) {
perror("Failed to open FD2 /dev/kfd");
return EXIT_FAILURE;
}
printf("FD2 == %d /dev/kfd opened successfully\n", fd2);
getchar();
// create a new secondary context
ret = ioctl(fd2, AMDKFD_IOC_CREATE_PROCESS);
printf("AMDKFD_IOC_CREATE_PROCESS ioctl ret = %d\n", ret);
getchar();
// close FDs
close(fd2);
getchar();
close(fd1);
getchar();
return EXIT_SUCCESS;
}
Zhu Lingshan (13):
amdkfd: enlarge the hashtable of kfd_process
amdkfd: mark the first kfd_process as the primary one
amdkfd: find_process_by_mm always return the primary context
amdkfd: Introduce kfd_create_process_sysfs as a separate function
amdkfd: destroy kfd secondary contexts through fd close
amdkfd: process svm ioctl only on the primary kfd process
amdkfd: process USERPTR allocation only on the primary kfd process
amdkfd: identify a secondary kfd process by its id
amdkfd: find kfd_process by filep->private_data in kfd_mmap
amdkfd: process pointer of a HIQ should be set to NULL
amdkfd: remove DIQ support
amdkfd: remove test_kq
amdkfd: introduce new ioctl AMDKFD_IOC_CREATE_PROCESS
drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 73 +++++-
.../drm/amd/amdkfd/kfd_device_queue_manager.c | 6 +-
drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c | 60 +----
.../drm/amd/amdkfd/kfd_packet_manager_v9.c | 4 -
.../drm/amd/amdkfd/kfd_packet_manager_vi.c | 4 -
drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 15 +-
drivers/gpu/drm/amd/amdkfd/kfd_process.c | 223 ++++++++++++------
.../amd/amdkfd/kfd_process_queue_manager.c | 39 +--
include/uapi/linux/kfd_ioctl.h | 8 +-
9 files changed, 248 insertions(+), 184 deletions(-)
--
2.47.1
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH V3 01/13] amdkfd: enlarge the hashtable of kfd_process
2025-08-04 11:05 [PATCH V3 00/13] amdkfd: Implement kfd multiple contexts Zhu Lingshan
@ 2025-08-04 11:05 ` Zhu Lingshan
2025-08-04 11:05 ` [PATCH V3 02/13] amdkfd: mark the first kfd_process as the primary one Zhu Lingshan
` (11 subsequent siblings)
12 siblings, 0 replies; 21+ messages in thread
From: Zhu Lingshan @ 2025-08-04 11:05 UTC (permalink / raw)
To: alexander.deucher, felix.kuehling; +Cc: ray.huang, amd-gfx, Zhu Lingshan
This commit enlarges the hashtable size of
kfd_process to 256, because of the multiple
contexts feature allowing each application
create multiple kfd_processes
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
index 67694bcd9464..8a33e6ee6369 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
@@ -1011,7 +1011,7 @@ struct kfd_process {
bool gpu_page_fault;
};
-#define KFD_PROCESS_TABLE_SIZE 5 /* bits: 32 entries */
+#define KFD_PROCESS_TABLE_SIZE 8 /* bits: 256 entries */
extern DECLARE_HASHTABLE(kfd_processes_table, KFD_PROCESS_TABLE_SIZE);
extern struct srcu_struct kfd_processes_srcu;
--
2.47.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH V3 02/13] amdkfd: mark the first kfd_process as the primary one
2025-08-04 11:05 [PATCH V3 00/13] amdkfd: Implement kfd multiple contexts Zhu Lingshan
2025-08-04 11:05 ` [PATCH V3 01/13] amdkfd: enlarge the hashtable of kfd_process Zhu Lingshan
@ 2025-08-04 11:05 ` Zhu Lingshan
2025-08-04 11:05 ` [PATCH V3 03/13] amdkfd: find_process_by_mm always return the primary context Zhu Lingshan
` (10 subsequent siblings)
12 siblings, 0 replies; 21+ messages in thread
From: Zhu Lingshan @ 2025-08-04 11:05 UTC (permalink / raw)
To: alexander.deucher, felix.kuehling; +Cc: ray.huang, amd-gfx, Zhu Lingshan
The first kfd_process is created through open(),
this commit marks it as the primary kfd_process.
Only the primary process should register the mmu_notifier.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 3 +++
drivers/gpu/drm/amd/amdkfd/kfd_process.c | 20 ++++++++++++--------
2 files changed, 15 insertions(+), 8 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
index 8a33e6ee6369..2b95f37c1af8 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
@@ -1009,6 +1009,9 @@ struct kfd_process {
/* if gpu page fault sent to KFD */
bool gpu_page_fault;
+
+ /* indicating whether this is a primary kfd_process */
+ bool primary;
};
#define KFD_PROCESS_TABLE_SIZE 8 /* bits: 256 entries */
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index 5be28c6c4f6a..762e306d85db 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -68,7 +68,7 @@ static struct workqueue_struct *kfd_restore_wq;
static struct kfd_process *find_process(const struct task_struct *thread,
bool ref);
static void kfd_process_ref_release(struct kref *ref);
-static struct kfd_process *create_process(const struct task_struct *thread);
+static struct kfd_process *create_process(const struct task_struct *thread, bool primary);
static void evict_process_worker(struct work_struct *work);
static void restore_process_worker(struct work_struct *work);
@@ -867,7 +867,7 @@ struct kfd_process *kfd_create_process(struct task_struct *thread)
if (process) {
pr_debug("Process already found\n");
} else {
- process = create_process(thread);
+ process = create_process(thread, true);
if (IS_ERR(process))
goto out;
@@ -1510,7 +1510,7 @@ void kfd_process_set_trap_debug_flag(struct qcm_process_device *qpd,
* On return the kfd_process is fully operational and will be freed when the
* mm is released
*/
-static struct kfd_process *create_process(const struct task_struct *thread)
+static struct kfd_process *create_process(const struct task_struct *thread, bool primary)
{
struct kfd_process *process;
struct mmu_notifier *mn;
@@ -1526,6 +1526,8 @@ static struct kfd_process *create_process(const struct task_struct *thread)
process->lead_thread = thread->group_leader;
process->n_pdds = 0;
process->queues_paused = false;
+ process->primary = primary;
+
INIT_DELAYED_WORK(&process->eviction_work, evict_process_worker);
INIT_DELAYED_WORK(&process->restore_work, restore_process_worker);
process->last_restore_timestamp = get_jiffies_64();
@@ -1569,12 +1571,14 @@ static struct kfd_process *create_process(const struct task_struct *thread)
* After this point, mmu_notifier_put will trigger the cleanup by
* dropping the last process reference in the free_notifier.
*/
- mn = mmu_notifier_get(&kfd_process_mmu_notifier_ops, process->mm);
- if (IS_ERR(mn)) {
- err = PTR_ERR(mn);
- goto err_register_notifier;
+ if (primary) {
+ mn = mmu_notifier_get(&kfd_process_mmu_notifier_ops, process->mm);
+ if (IS_ERR(mn)) {
+ err = PTR_ERR(mn);
+ goto err_register_notifier;
+ }
+ BUG_ON(mn != &process->mmu_notifier);
}
- BUG_ON(mn != &process->mmu_notifier);
kfd_unref_process(process);
get_task_struct(process->lead_thread);
--
2.47.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH V3 03/13] amdkfd: find_process_by_mm always return the primary context
2025-08-04 11:05 [PATCH V3 00/13] amdkfd: Implement kfd multiple contexts Zhu Lingshan
2025-08-04 11:05 ` [PATCH V3 01/13] amdkfd: enlarge the hashtable of kfd_process Zhu Lingshan
2025-08-04 11:05 ` [PATCH V3 02/13] amdkfd: mark the first kfd_process as the primary one Zhu Lingshan
@ 2025-08-04 11:05 ` Zhu Lingshan
2025-08-04 11:05 ` [PATCH V3 04/13] amdkfd: Introduce kfd_create_process_sysfs as a separate function Zhu Lingshan
` (9 subsequent siblings)
12 siblings, 0 replies; 21+ messages in thread
From: Zhu Lingshan @ 2025-08-04 11:05 UTC (permalink / raw)
To: alexander.deucher, felix.kuehling; +Cc: ray.huang, amd-gfx, Zhu Lingshan
Up until this commit, the kfd multiple contexts feature has
not been fully implemented in mainline kernel yet.
For backawrd compatibility, not break existing use cases,
this commit changes function find_process_by_mm, let it
always return the primary kfd_process.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_process.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index 762e306d85db..88421e57a072 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -935,7 +935,7 @@ static struct kfd_process *find_process_by_mm(const struct mm_struct *mm)
hash_for_each_possible_rcu(kfd_processes_table, process,
kfd_processes, (uintptr_t)mm)
- if (process->mm == mm)
+ if (process->mm == mm && process->primary)
return process;
return NULL;
--
2.47.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH V3 04/13] amdkfd: Introduce kfd_create_process_sysfs as a separate function
2025-08-04 11:05 [PATCH V3 00/13] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (2 preceding siblings ...)
2025-08-04 11:05 ` [PATCH V3 03/13] amdkfd: find_process_by_mm always return the primary context Zhu Lingshan
@ 2025-08-04 11:05 ` Zhu Lingshan
2025-08-04 11:05 ` [PATCH V3 05/13] amdkfd: destroy kfd secondary contexts through fd close Zhu Lingshan
` (8 subsequent siblings)
12 siblings, 0 replies; 21+ messages in thread
From: Zhu Lingshan @ 2025-08-04 11:05 UTC (permalink / raw)
To: alexander.deucher, felix.kuehling; +Cc: ray.huang, amd-gfx, Zhu Lingshan
KFD creates sysfs entries for a kfd_process in
function kfd_create_process when creating it.
This commit extracts the code creating sysfs
entries to a separate function because it
would be invoked in other code path like
creating secondary kfd contexts (kfd_process).
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 1 +
drivers/gpu/drm/amd/amdkfd/kfd_process.c | 66 +++++++++++++++---------
2 files changed, 42 insertions(+), 25 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
index 2b95f37c1af8..0818705820c6 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
@@ -1043,6 +1043,7 @@ int kfd_process_create_wq(void);
void kfd_process_destroy_wq(void);
void kfd_cleanup_processes(void);
struct kfd_process *kfd_create_process(struct task_struct *thread);
+int kfd_create_process_sysfs(struct kfd_process *process);
struct kfd_process *kfd_get_process(const struct task_struct *task);
struct kfd_process *kfd_lookup_process_by_pasid(u32 pasid,
struct kfd_process_device **pdd);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index 88421e57a072..bce7e35a15c9 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -825,6 +825,44 @@ static void kfd_process_device_destroy_ib_mem(struct kfd_process_device *pdd)
kfd_process_free_gpuvm(qpd->ib_mem, pdd, &qpd->ib_kaddr);
}
+int kfd_create_process_sysfs(struct kfd_process *process)
+{
+ int ret;
+
+ if (process->kobj) {
+ pr_warn("kobject already exsists for the kfd_process\n");
+ return -EINVAL;
+ }
+
+ process->kobj = kfd_alloc_struct(process->kobj);
+ if (!process->kobj) {
+ pr_warn("Creating procfs kobject failed");
+ return -ENOMEM;
+ }
+ ret = kobject_init_and_add(process->kobj, &procfs_type,
+ procfs.kobj, "%d",
+ (int)process->lead_thread->pid);
+ if (ret) {
+ pr_warn("Creating procfs pid directory failed");
+ kobject_put(process->kobj);
+ return ret;
+ }
+
+ kfd_sysfs_create_file(process->kobj, &process->attr_pasid,
+ "pasid");
+
+ process->kobj_queues = kobject_create_and_add("queues",
+ process->kobj);
+ if (!process->kobj_queues)
+ pr_warn("Creating KFD proc/queues folder failed");
+
+ kfd_procfs_add_sysfs_stats(process);
+ kfd_procfs_add_sysfs_files(process);
+ kfd_procfs_add_sysfs_counters(process);
+
+ return 0;
+}
+
struct kfd_process *kfd_create_process(struct task_struct *thread)
{
struct kfd_process *process;
@@ -874,31 +912,9 @@ struct kfd_process *kfd_create_process(struct task_struct *thread)
if (!procfs.kobj)
goto out;
- process->kobj = kfd_alloc_struct(process->kobj);
- if (!process->kobj) {
- pr_warn("Creating procfs kobject failed");
- goto out;
- }
- ret = kobject_init_and_add(process->kobj, &procfs_type,
- procfs.kobj, "%d",
- (int)process->lead_thread->pid);
- if (ret) {
- pr_warn("Creating procfs pid directory failed");
- kobject_put(process->kobj);
- goto out;
- }
-
- kfd_sysfs_create_file(process->kobj, &process->attr_pasid,
- "pasid");
-
- process->kobj_queues = kobject_create_and_add("queues",
- process->kobj);
- if (!process->kobj_queues)
- pr_warn("Creating KFD proc/queues folder failed");
-
- kfd_procfs_add_sysfs_stats(process);
- kfd_procfs_add_sysfs_files(process);
- kfd_procfs_add_sysfs_counters(process);
+ ret = kfd_create_process_sysfs(process);
+ if (ret)
+ pr_warn("Failed to create sysfs entry for the kfd_process");
kfd_debugfs_add_process(process);
--
2.47.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH V3 05/13] amdkfd: destroy kfd secondary contexts through fd close
2025-08-04 11:05 [PATCH V3 00/13] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (3 preceding siblings ...)
2025-08-04 11:05 ` [PATCH V3 04/13] amdkfd: Introduce kfd_create_process_sysfs as a separate function Zhu Lingshan
@ 2025-08-04 11:05 ` Zhu Lingshan
2025-08-04 11:05 ` [PATCH V3 06/13] amdkfd: process svm ioctl only on the primary kfd process Zhu Lingshan
` (7 subsequent siblings)
12 siblings, 0 replies; 21+ messages in thread
From: Zhu Lingshan @ 2025-08-04 11:05 UTC (permalink / raw)
To: alexander.deucher, felix.kuehling; +Cc: ray.huang, amd-gfx, Zhu Lingshan
Life cycle of a KFD secondary context(kfd_process) is tied
to the opened file. Therefore this commit destroy a kfd
secondary context when close the fd it belonging to.
This commit extracts the code removing the kfd_process
from the kfd_process_table to a separate function and
call it in kfd_process_notifier_release_internal unconditionally.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 9 ++++--
drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 1 +
drivers/gpu/drm/amd/amdkfd/kfd_process.c | 41 +++++++++++++-----------
3 files changed, 31 insertions(+), 20 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index 828a9ceef1e7..e8c6273de99b 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -164,8 +164,13 @@ static int kfd_release(struct inode *inode, struct file *filep)
{
struct kfd_process *process = filep->private_data;
- if (process)
- kfd_unref_process(process);
+ if (!process)
+ return 0;
+
+ if (!process->primary)
+ kfd_process_notifier_release_internal(process);
+
+ kfd_unref_process(process);
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
index 0818705820c6..d1436f1f527d 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
@@ -1085,6 +1085,7 @@ bool kfd_process_xnack_mode(struct kfd_process *p, bool supported);
int kfd_reserved_mem_mmap(struct kfd_node *dev, struct kfd_process *process,
struct vm_area_struct *vma);
+void kfd_process_notifier_release_internal(struct kfd_process *p);
/* KFD process API for creating and translating handles */
int kfd_process_device_create_obj_handle(struct kfd_process_device *pdd,
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index bce7e35a15c9..5d59a4d994d5 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -1233,10 +1233,30 @@ static void kfd_process_free_notifier(struct mmu_notifier *mn)
kfd_unref_process(container_of(mn, struct kfd_process, mmu_notifier));
}
-static void kfd_process_notifier_release_internal(struct kfd_process *p)
+static void kfd_process_table_remove(struct kfd_process *p)
+{
+ mutex_lock(&kfd_processes_mutex);
+ /*
+ * Do early return if table is empty.
+ *
+ * This could potentially happen if this function is called concurrently
+ * by mmu_notifier and by kfd_cleanup_pocesses.
+ *
+ */
+ if (hash_empty(kfd_processes_table)) {
+ mutex_unlock(&kfd_processes_mutex);
+ return;
+ }
+ hash_del_rcu(&p->kfd_processes);
+ mutex_unlock(&kfd_processes_mutex);
+ synchronize_srcu(&kfd_processes_srcu);
+}
+
+void kfd_process_notifier_release_internal(struct kfd_process *p)
{
int i;
+ kfd_process_table_remove(p);
cancel_delayed_work_sync(&p->eviction_work);
cancel_delayed_work_sync(&p->restore_work);
@@ -1270,7 +1290,8 @@ static void kfd_process_notifier_release_internal(struct kfd_process *p)
srcu_read_unlock(&kfd_processes_srcu, idx);
}
- mmu_notifier_put(&p->mmu_notifier);
+ if (p->primary)
+ mmu_notifier_put(&p->mmu_notifier);
}
static void kfd_process_notifier_release(struct mmu_notifier *mn,
@@ -1286,22 +1307,6 @@ static void kfd_process_notifier_release(struct mmu_notifier *mn,
if (WARN_ON(p->mm != mm))
return;
- mutex_lock(&kfd_processes_mutex);
- /*
- * Do early return if table is empty.
- *
- * This could potentially happen if this function is called concurrently
- * by mmu_notifier and by kfd_cleanup_pocesses.
- *
- */
- if (hash_empty(kfd_processes_table)) {
- mutex_unlock(&kfd_processes_mutex);
- return;
- }
- hash_del_rcu(&p->kfd_processes);
- mutex_unlock(&kfd_processes_mutex);
- synchronize_srcu(&kfd_processes_srcu);
-
kfd_process_notifier_release_internal(p);
}
--
2.47.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH V3 06/13] amdkfd: process svm ioctl only on the primary kfd process
2025-08-04 11:05 [PATCH V3 00/13] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (4 preceding siblings ...)
2025-08-04 11:05 ` [PATCH V3 05/13] amdkfd: destroy kfd secondary contexts through fd close Zhu Lingshan
@ 2025-08-04 11:05 ` Zhu Lingshan
2025-08-04 11:05 ` [PATCH V3 07/13] amdkfd: process USERPTR allocation " Zhu Lingshan
` (6 subsequent siblings)
12 siblings, 0 replies; 21+ messages in thread
From: Zhu Lingshan @ 2025-08-04 11:05 UTC (permalink / raw)
To: alexander.deucher, felix.kuehling; +Cc: ray.huang, amd-gfx, Zhu Lingshan
svm ioctl should only be processed on the primary
kfd process because only the lifecycle of the
primary kfd process is tied to the user space
applicaiton.
Another reason is in virtualization the hypervisor owns
the primary kfd process as a privileged one.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index e8c6273de99b..f56faf11ee43 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -1719,6 +1719,12 @@ static int kfd_ioctl_svm(struct file *filep, struct kfd_process *p, void *data)
struct kfd_ioctl_svm_args *args = data;
int r = 0;
+ if (!p->primary) {
+ pr_debug("SVM ioctl not supported on non-primary kfd process\n");
+
+ return -EOPNOTSUPP;
+ }
+
pr_debug("start 0x%llx size 0x%llx op 0x%x nattr 0x%x\n",
args->start_addr, args->size, args->op, args->nattr);
--
2.47.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH V3 07/13] amdkfd: process USERPTR allocation only on the primary kfd process
2025-08-04 11:05 [PATCH V3 00/13] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (5 preceding siblings ...)
2025-08-04 11:05 ` [PATCH V3 06/13] amdkfd: process svm ioctl only on the primary kfd process Zhu Lingshan
@ 2025-08-04 11:05 ` Zhu Lingshan
2025-08-04 11:05 ` [PATCH V3 08/13] amdkfd: identify a secondary kfd process by its id Zhu Lingshan
` (5 subsequent siblings)
12 siblings, 0 replies; 21+ messages in thread
From: Zhu Lingshan @ 2025-08-04 11:05 UTC (permalink / raw)
To: alexander.deucher, felix.kuehling; +Cc: ray.huang, amd-gfx, Zhu Lingshan
The lifecycle of the primary kfd process is tied to
the user space program, all secondary kfd process
would be destroyed when fd close. Thus only the primary
kfd process should process USERPTR memory allocation.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index f56faf11ee43..28df35a63c29 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -1068,6 +1068,12 @@ static int kfd_ioctl_alloc_memory_of_gpu(struct file *filep,
if (args->size == 0)
return -EINVAL;
+ if (!p->primary && (flags & KFD_IOC_ALLOC_MEM_FLAGS_USERPTR)) {
+ pr_debug("USERPTR is not supported on non-primary kfd_process\n");
+
+ return -EOPNOTSUPP;
+ }
+
#if IS_ENABLED(CONFIG_HSA_AMD_SVM)
/* Flush pending deferred work to avoid racing with deferred actions
* from previous memory map changes (e.g. munmap).
--
2.47.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH V3 08/13] amdkfd: identify a secondary kfd process by its id
2025-08-04 11:05 [PATCH V3 00/13] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (6 preceding siblings ...)
2025-08-04 11:05 ` [PATCH V3 07/13] amdkfd: process USERPTR allocation " Zhu Lingshan
@ 2025-08-04 11:05 ` Zhu Lingshan
2025-08-07 20:49 ` Felix Kuehling
2025-08-04 11:05 ` [PATCH V3 09/13] amdkfd: find kfd_process by filep->private_data in kfd_mmap Zhu Lingshan
` (4 subsequent siblings)
12 siblings, 1 reply; 21+ messages in thread
From: Zhu Lingshan @ 2025-08-04 11:05 UTC (permalink / raw)
To: alexander.deucher, felix.kuehling; +Cc: ray.huang, amd-gfx, Zhu Lingshan
This commit introduces a new id field for
struct kfd process, which helps identify
a kfd process among multiple contexts that
all belong to a single user space program.
The sysfs entry of a secondary kfd process
is placed under the sysfs entry folder of
its primary kfd process.
The naming format of the sysfs entry of a secondary
kfd process is "context_%u" where %u is the process id.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 6 ++
drivers/gpu/drm/amd/amdkfd/kfd_process.c | 83 +++++++++++++++++++++++-
2 files changed, 86 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
index d1436f1f527d..d140463e221b 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
@@ -998,6 +998,9 @@ struct kfd_process {
/* Tracks debug per-vmid request for debug flags */
u32 dbg_flags;
+ /* kfd process id */
+ u16 id;
+
atomic_t poison;
/* Queues are in paused stated because we are in the process of doing a CRIU checkpoint */
bool queues_paused;
@@ -1012,6 +1015,9 @@ struct kfd_process {
/* indicating whether this is a primary kfd_process */
bool primary;
+
+ /* The primary kfd_process allocating IDs for its secondary kfd_process, 0 for primary kfd_process */
+ struct ida id_table;
};
#define KFD_PROCESS_TABLE_SIZE 8 /* bits: 256 entries */
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index 5d59a4d994d5..8e498fd35b8c 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -54,6 +54,9 @@ DEFINE_MUTEX(kfd_processes_mutex);
DEFINE_SRCU(kfd_processes_srcu);
+#define KFD_PROCESS_ID_MIN 1
+#define KFD_PROCESS_ID_WIDTH 16
+
/* For process termination handling */
static struct workqueue_struct *kfd_process_wq;
@@ -827,6 +830,7 @@ static void kfd_process_device_destroy_ib_mem(struct kfd_process_device *pdd)
int kfd_create_process_sysfs(struct kfd_process *process)
{
+ struct kfd_process *primary_process;
int ret;
if (process->kobj) {
@@ -839,9 +843,22 @@ int kfd_create_process_sysfs(struct kfd_process *process)
pr_warn("Creating procfs kobject failed");
return -ENOMEM;
}
- ret = kobject_init_and_add(process->kobj, &procfs_type,
- procfs.kobj, "%d",
- (int)process->lead_thread->pid);
+
+ if (process->primary)
+ ret = kobject_init_and_add(process->kobj, &procfs_type,
+ procfs.kobj, "%d",
+ (int)process->lead_thread->pid);
+ else {
+ primary_process = kfd_lookup_process_by_mm(process->lead_thread->mm);
+ if (!primary_process)
+ return -ESRCH;
+
+ ret = kobject_init_and_add(process->kobj, &procfs_type,
+ primary_process->kobj, "context_%u",
+ process->id);
+ kfd_unref_process(primary_process);
+ }
+
if (ret) {
pr_warn("Creating procfs pid directory failed");
kobject_put(process->kobj);
@@ -863,6 +880,51 @@ int kfd_create_process_sysfs(struct kfd_process *process)
return 0;
}
+static int kfd_process_alloc_id(struct kfd_process *process)
+{
+ int ret;
+ struct kfd_process *primary_process;
+
+ if (process->primary) {
+ process->id = 0;
+
+ return 0;
+ }
+
+ primary_process = kfd_lookup_process_by_mm(process->lead_thread->mm);
+ if (!primary_process)
+ return -ESRCH;
+
+ ret = ida_alloc_range(&primary_process->id_table, KFD_PROCESS_ID_MIN,
+ (1 << KFD_PROCESS_ID_WIDTH) - 1, GFP_KERNEL);
+ if (ret < 0)
+ goto out;
+
+ process->id = ret;
+ ret = 0;
+
+out:
+ kfd_unref_process(primary_process);
+
+ return ret;
+}
+
+static void kfd_process_free_id(struct kfd_process *process)
+{
+ struct kfd_process *primary_process;
+
+ if (process->primary)
+ return;
+
+ primary_process = kfd_lookup_process_by_mm(process->lead_thread->mm);
+ if (!primary_process)
+ return;
+
+ ida_free(&primary_process->id_table, process->id);
+
+ kfd_unref_process(primary_process);
+}
+
struct kfd_process *kfd_create_process(struct task_struct *thread)
{
struct kfd_process *process;
@@ -1193,6 +1255,11 @@ static void kfd_process_wq_release(struct work_struct *work)
if (ef)
dma_fence_signal(ef);
+ if (!p->primary)
+ kfd_process_free_id(p);
+ else
+ ida_destroy(&p->id_table);
+
kfd_process_remove_sysfs(p);
kfd_debugfs_remove_process(p);
@@ -1549,6 +1616,12 @@ static struct kfd_process *create_process(const struct task_struct *thread, bool
process->queues_paused = false;
process->primary = primary;
+ err = kfd_process_alloc_id(process);
+ if (err) {
+ pr_err("Creating kfd process: failed to alloc an id\n");
+ goto err_alloc_id;
+ }
+
INIT_DELAYED_WORK(&process->eviction_work, evict_process_worker);
INIT_DELAYED_WORK(&process->restore_work, restore_process_worker);
process->last_restore_timestamp = get_jiffies_64();
@@ -1599,6 +1672,8 @@ static struct kfd_process *create_process(const struct task_struct *thread, bool
goto err_register_notifier;
}
BUG_ON(mn != &process->mmu_notifier);
+
+ ida_init(&process->id_table);
}
kfd_unref_process(process);
@@ -1619,6 +1694,8 @@ static struct kfd_process *create_process(const struct task_struct *thread, bool
err_process_pqm_init:
kfd_event_free_process(process);
err_event_init:
+ kfd_process_free_id(process);
+err_alloc_id:
mutex_destroy(&process->mutex);
kfree(process);
err_alloc_process:
--
2.47.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH V3 09/13] amdkfd: find kfd_process by filep->private_data in kfd_mmap
2025-08-04 11:05 [PATCH V3 00/13] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (7 preceding siblings ...)
2025-08-04 11:05 ` [PATCH V3 08/13] amdkfd: identify a secondary kfd process by its id Zhu Lingshan
@ 2025-08-04 11:05 ` Zhu Lingshan
2025-08-07 20:55 ` Felix Kuehling
2025-08-04 11:05 ` [PATCH V3 10/13] amdkfd: process pointer of a HIQ should be set to NULL Zhu Lingshan
` (3 subsequent siblings)
12 siblings, 1 reply; 21+ messages in thread
From: Zhu Lingshan @ 2025-08-04 11:05 UTC (permalink / raw)
To: alexander.deucher, felix.kuehling; +Cc: ray.huang, amd-gfx, Zhu Lingshan
This commit finds the proper kfd_process by
filep->private_data in kfd_mmap,
because the function kfd_get_process()
can not locate a specific kfd process among
multiple contexts.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 11 +++++++----
1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index 28df35a63c29..662b181f1fd2 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -3407,16 +3407,19 @@ static int kfd_mmio_mmap(struct kfd_node *dev, struct kfd_process *process,
}
-static int kfd_mmap(struct file *filp, struct vm_area_struct *vma)
+static int kfd_mmap(struct file *filep, struct vm_area_struct *vma)
{
struct kfd_process *process;
struct kfd_node *dev = NULL;
unsigned long mmap_offset;
unsigned int gpu_id;
- process = kfd_get_process(current);
- if (IS_ERR(process))
- return PTR_ERR(process);
+ process = filep->private_data;
+ if (!process)
+ return -ESRCH;
+
+ if (process->lead_thread != current->group_leader)
+ return -EBADF;
mmap_offset = vma->vm_pgoff << PAGE_SHIFT;
gpu_id = KFD_MMAP_GET_GPU_ID(mmap_offset);
--
2.47.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH V3 10/13] amdkfd: process pointer of a HIQ should be set to NULL
2025-08-04 11:05 [PATCH V3 00/13] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (8 preceding siblings ...)
2025-08-04 11:05 ` [PATCH V3 09/13] amdkfd: find kfd_process by filep->private_data in kfd_mmap Zhu Lingshan
@ 2025-08-04 11:05 ` Zhu Lingshan
2025-08-07 20:55 ` Felix Kuehling
2025-08-04 11:05 ` [PATCH V3 11/13] amdkfd: remove DIQ support Zhu Lingshan
` (2 subsequent siblings)
12 siblings, 1 reply; 21+ messages in thread
From: Zhu Lingshan @ 2025-08-04 11:05 UTC (permalink / raw)
To: alexander.deucher, felix.kuehling; +Cc: ray.huang, amd-gfx, Zhu Lingshan
In kq_initialize, queue->process of a HIQ should
be set to NULL because it does not belong to any kfd_process.
This commit decommisions the function kfd_get_process() because
it can not locate a specific kfd_process among multiple
contexts and not any code path calls it after this commit.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c | 3 ++-
drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 1 -
drivers/gpu/drm/amd/amdkfd/kfd_process.c | 18 ------------------
3 files changed, 2 insertions(+), 20 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
index 2b0a830f5b29..ebee37937783 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
@@ -144,7 +144,8 @@ static bool kq_initialize(struct kernel_queue *kq, struct kfd_node *dev,
goto err_init_queue;
kq->queue->device = dev;
- kq->queue->process = kfd_get_process(current);
+ if (type == KFD_QUEUE_TYPE_HIQ)
+ kq->queue->process = NULL;
kq->queue->mqd_mem_obj = kq->mqd_mgr->allocate_mqd(kq->mqd_mgr->dev,
&kq->queue->properties);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
index d140463e221b..25534473c28f 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
@@ -1050,7 +1050,6 @@ void kfd_process_destroy_wq(void);
void kfd_cleanup_processes(void);
struct kfd_process *kfd_create_process(struct task_struct *thread);
int kfd_create_process_sysfs(struct kfd_process *process);
-struct kfd_process *kfd_get_process(const struct task_struct *task);
struct kfd_process *kfd_lookup_process_by_pasid(u32 pasid,
struct kfd_process_device **pdd);
struct kfd_process *kfd_lookup_process_by_mm(const struct mm_struct *mm);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index 8e498fd35b8c..0c3f0cc16bf4 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -989,24 +989,6 @@ struct kfd_process *kfd_create_process(struct task_struct *thread)
return process;
}
-struct kfd_process *kfd_get_process(const struct task_struct *thread)
-{
- struct kfd_process *process;
-
- if (!thread->mm)
- return ERR_PTR(-EINVAL);
-
- /* Only the pthreads threading model is supported. */
- if (thread->group_leader->mm != thread->mm)
- return ERR_PTR(-EINVAL);
-
- process = find_process(thread, false);
- if (!process)
- return ERR_PTR(-EINVAL);
-
- return process;
-}
-
static struct kfd_process *find_process_by_mm(const struct mm_struct *mm)
{
struct kfd_process *process;
--
2.47.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH V3 11/13] amdkfd: remove DIQ support
2025-08-04 11:05 [PATCH V3 00/13] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (9 preceding siblings ...)
2025-08-04 11:05 ` [PATCH V3 10/13] amdkfd: process pointer of a HIQ should be set to NULL Zhu Lingshan
@ 2025-08-04 11:05 ` Zhu Lingshan
2025-08-07 21:03 ` Felix Kuehling
2025-08-04 11:05 ` [PATCH V3 12/13] amdkfd: remove test_kq Zhu Lingshan
2025-08-04 11:05 ` [PATCH V3 13/13] amdkfd: introduce new ioctl AMDKFD_IOC_CREATE_PROCESS Zhu Lingshan
12 siblings, 1 reply; 21+ messages in thread
From: Zhu Lingshan @ 2025-08-04 11:05 UTC (permalink / raw)
To: alexander.deucher, felix.kuehling; +Cc: ray.huang, amd-gfx, Zhu Lingshan
This commit remove DIQ support because it has been
marked as DEPRECATED since 2022 in commit 5bdd3eb2
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
---
.../drm/amd/amdkfd/kfd_device_queue_manager.c | 6 +--
drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c | 26 ++-----------
.../drm/amd/amdkfd/kfd_packet_manager_v9.c | 4 --
.../drm/amd/amdkfd/kfd_packet_manager_vi.c | 4 --
.../amd/amdkfd/kfd_process_queue_manager.c | 39 +------------------
5 files changed, 7 insertions(+), 72 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index 2d91027e2a74..58e47e14cf07 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -399,8 +399,7 @@ static void increment_queue_count(struct device_queue_manager *dqm,
struct queue *q)
{
dqm->active_queue_count++;
- if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE ||
- q->properties.type == KFD_QUEUE_TYPE_DIQ)
+ if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE)
dqm->active_cp_queue_count++;
if (q->properties.is_gws) {
@@ -414,8 +413,7 @@ static void decrement_queue_count(struct device_queue_manager *dqm,
struct queue *q)
{
dqm->active_queue_count--;
- if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE ||
- q->properties.type == KFD_QUEUE_TYPE_DIQ)
+ if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE)
dqm->active_cp_queue_count--;
if (q->properties.is_gws) {
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
index ebee37937783..f676b7419984 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
@@ -46,7 +46,7 @@ static bool kq_initialize(struct kernel_queue *kq, struct kfd_node *dev,
int retval;
union PM4_MES_TYPE_3_HEADER nop;
- if (WARN_ON(type != KFD_QUEUE_TYPE_DIQ && type != KFD_QUEUE_TYPE_HIQ))
+ if (WARN_ON(type != KFD_QUEUE_TYPE_HIQ))
return false;
pr_debug("Initializing queue type %d size %d\n", KFD_QUEUE_TYPE_HIQ,
@@ -61,14 +61,9 @@ static bool kq_initialize(struct kernel_queue *kq, struct kfd_node *dev,
kq->dev = dev;
kq->nop_packet = nop.u32all;
- switch (type) {
- case KFD_QUEUE_TYPE_DIQ:
- kq->mqd_mgr = dev->dqm->mqd_mgrs[KFD_MQD_TYPE_DIQ];
- break;
- case KFD_QUEUE_TYPE_HIQ:
+ if (type == KFD_QUEUE_TYPE_HIQ)
kq->mqd_mgr = dev->dqm->mqd_mgrs[KFD_MQD_TYPE_HIQ];
- break;
- default:
+ else {
dev_err(dev->adev->dev, "Invalid queue type %d\n", type);
return false;
}
@@ -163,24 +158,11 @@ static bool kq_initialize(struct kernel_queue *kq, struct kfd_node *dev,
kq->mqd_mgr->load_mqd(kq->mqd_mgr, kq->queue->mqd,
kq->queue->pipe, kq->queue->queue,
&kq->queue->properties, NULL);
- } else {
- /* allocate fence for DIQ */
-
- retval = kfd_gtt_sa_allocate(dev, sizeof(uint32_t),
- &kq->fence_mem_obj);
-
- if (retval != 0)
- goto err_alloc_fence;
-
- kq->fence_kernel_address = kq->fence_mem_obj->cpu_ptr;
- kq->fence_gpu_addr = kq->fence_mem_obj->gpu_addr;
}
print_queue(kq->queue);
return true;
-err_alloc_fence:
- kq->mqd_mgr->free_mqd(kq->mqd_mgr, kq->queue->mqd, kq->queue->mqd_mem_obj);
err_allocate_mqd:
uninit_queue(kq->queue);
err_init_queue:
@@ -210,8 +192,6 @@ static void kq_uninitialize(struct kernel_queue *kq)
kq->queue->queue);
up_read(&kq->dev->adev->reset_domain->sem);
}
- else if (kq->queue->properties.type == KFD_QUEUE_TYPE_DIQ)
- kfd_gtt_sa_free(kq->dev, kq->fence_mem_obj);
kq->mqd_mgr->free_mqd(kq->mqd_mgr, kq->queue->mqd,
kq->queue->mqd_mem_obj);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c
index 505036968a77..3d2375817c3e 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c
@@ -252,10 +252,6 @@ static int pm_map_queues_v9(struct packet_manager *pm, uint32_t *buffer,
packet->bitfields2.queue_type =
queue_type__mes_map_queues__normal_latency_static_queue_vi;
break;
- case KFD_QUEUE_TYPE_DIQ:
- packet->bitfields2.queue_type =
- queue_type__mes_map_queues__debug_interface_queue_vi;
- break;
case KFD_QUEUE_TYPE_SDMA:
case KFD_QUEUE_TYPE_SDMA_XGMI:
if (q->properties.sdma_engine_id < 2 &&
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c
index a1de5d7e173a..60086e7cc258 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c
@@ -166,10 +166,6 @@ static int pm_map_queues_vi(struct packet_manager *pm, uint32_t *buffer,
packet->bitfields2.queue_type =
queue_type__mes_map_queues__normal_latency_static_queue_vi;
break;
- case KFD_QUEUE_TYPE_DIQ:
- packet->bitfields2.queue_type =
- queue_type__mes_map_queues__debug_interface_queue_vi;
- break;
case KFD_QUEUE_TYPE_SDMA:
case KFD_QUEUE_TYPE_SDMA_XGMI:
packet->bitfields2.engine_sel = q->properties.sdma_engine_id +
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
index c643e0ccec52..e36950e7e14f 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
@@ -345,7 +345,7 @@ int pqm_create_queue(struct process_queue_manager *pqm,
* If we are just about to create DIQ, the is_debug flag is not set yet
* Hence we also check the type as well
*/
- if ((pdd->qpd.is_debug) || (type == KFD_QUEUE_TYPE_DIQ))
+ if ((pdd->qpd.is_debug))
max_queues = dev->kfd->device_info.max_no_of_hqd/2;
if (pdd->qpd.queue_count >= max_queues)
@@ -426,22 +426,6 @@ int pqm_create_queue(struct process_queue_manager *pqm,
restore_mqd, restore_ctl_stack);
print_queue(q);
break;
- case KFD_QUEUE_TYPE_DIQ:
- kq = kernel_queue_init(dev, KFD_QUEUE_TYPE_DIQ);
- if (!kq) {
- retval = -ENOMEM;
- goto err_create_queue;
- }
- kq->queue->properties.queue_id = *qid;
- pqn->kq = kq;
- pqn->q = NULL;
- retval = kfd_process_drain_interrupts(pdd);
- if (retval)
- break;
-
- retval = dev->dqm->ops.create_kernel_queue(dev->dqm,
- kq, &pdd->qpd);
- break;
default:
WARN(1, "Invalid queue type %d", type);
retval = -EINVAL;
@@ -1121,32 +1105,13 @@ int pqm_debugfs_mqds(struct seq_file *m, void *data)
break;
default:
seq_printf(m,
- " Bad user queue type %d on device %x\n",
+ " Qeueu node with bad user queue type %d on device %x\n",
q->properties.type, q->device->id);
continue;
}
mqd_mgr = q->device->dqm->mqd_mgrs[mqd_type];
size = mqd_mgr->mqd_stride(mqd_mgr,
&q->properties);
- } else if (pqn->kq) {
- q = pqn->kq->queue;
- mqd_mgr = pqn->kq->mqd_mgr;
- switch (q->properties.type) {
- case KFD_QUEUE_TYPE_DIQ:
- seq_printf(m, " DIQ on device %x\n",
- pqn->kq->dev->id);
- break;
- default:
- seq_printf(m,
- " Bad kernel queue type %d on device %x\n",
- q->properties.type,
- pqn->kq->dev->id);
- continue;
- }
- } else {
- seq_printf(m,
- " Weird: Queue node with neither kernel nor user queue\n");
- continue;
}
for (xcc = 0; xcc < num_xccs; xcc++) {
--
2.47.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH V3 12/13] amdkfd: remove test_kq
2025-08-04 11:05 [PATCH V3 00/13] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (10 preceding siblings ...)
2025-08-04 11:05 ` [PATCH V3 11/13] amdkfd: remove DIQ support Zhu Lingshan
@ 2025-08-04 11:05 ` Zhu Lingshan
2025-08-07 21:06 ` Felix Kuehling
2025-08-04 11:05 ` [PATCH V3 13/13] amdkfd: introduce new ioctl AMDKFD_IOC_CREATE_PROCESS Zhu Lingshan
12 siblings, 1 reply; 21+ messages in thread
From: Zhu Lingshan @ 2025-08-04 11:05 UTC (permalink / raw)
To: alexander.deucher, felix.kuehling; +Cc: ray.huang, amd-gfx, Zhu Lingshan
This commit removes test_kq() function becuse it has been
marked as unused since 2014 and no other functions calls it.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c | 31 -------------------
1 file changed, 31 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
index f676b7419984..6aa8b0348bad 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
@@ -339,34 +339,3 @@ void kernel_queue_uninit(struct kernel_queue *kq)
kq_uninitialize(kq);
kfree(kq);
}
-
-/* FIXME: Can this test be removed? */
-static __attribute__((unused)) void test_kq(struct kfd_node *dev)
-{
- struct kernel_queue *kq;
- uint32_t *buffer, i;
- int retval;
-
- dev_err(dev->adev->dev, "Starting kernel queue test\n");
-
- kq = kernel_queue_init(dev, KFD_QUEUE_TYPE_HIQ);
- if (unlikely(!kq)) {
- dev_err(dev->adev->dev, " Failed to initialize HIQ\n");
- dev_err(dev->adev->dev, "Kernel queue test failed\n");
- return;
- }
-
- retval = kq_acquire_packet_buffer(kq, 5, &buffer);
- if (unlikely(retval != 0)) {
- dev_err(dev->adev->dev, " Failed to acquire packet buffer\n");
- dev_err(dev->adev->dev, "Kernel queue test failed\n");
- return;
- }
- for (i = 0; i < 5; i++)
- buffer[i] = kq->nop_packet;
- kq_submit_packet(kq);
-
- dev_err(dev->adev->dev, "Ending kernel queue test\n");
-}
-
-
--
2.47.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH V3 13/13] amdkfd: introduce new ioctl AMDKFD_IOC_CREATE_PROCESS
2025-08-04 11:05 [PATCH V3 00/13] amdkfd: Implement kfd multiple contexts Zhu Lingshan
` (11 preceding siblings ...)
2025-08-04 11:05 ` [PATCH V3 12/13] amdkfd: remove test_kq Zhu Lingshan
@ 2025-08-04 11:05 ` Zhu Lingshan
12 siblings, 0 replies; 21+ messages in thread
From: Zhu Lingshan @ 2025-08-04 11:05 UTC (permalink / raw)
To: alexander.deucher, felix.kuehling; +Cc: ray.huang, amd-gfx, Zhu Lingshan
This commit implemetns a new ioctl AMDKFD_IOC_CREATE_PROCESS
that creates a new secondary kfd_progress on the FD.
To keep backward compatibility, userspace programs need to invoke
this ioctl explicitly on a FD to create a secondary
kfd_process which replacing its primary kfd_process.
This commit bumps ioctl minor version.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 41 ++++++++++++++++++++++++
drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 1 +
drivers/gpu/drm/amd/amdkfd/kfd_process.c | 3 +-
include/uapi/linux/kfd_ioctl.h | 8 +++--
4 files changed, 49 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index 662b181f1fd2..aeb5d4b31e9f 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -3135,6 +3135,44 @@ static int kfd_ioctl_set_debug_trap(struct file *filep, struct kfd_process *p, v
return r;
}
+/* userspace programs need to invoke this ioctl explicitly on a FD to
+ * create a secondary kfd_process which replacing its primary kfd_process
+ */
+static int kfd_ioctl_create_process(struct file *filep, struct kfd_process *p, void *data)
+{
+ struct kfd_process *process;
+ int ret;
+
+ if (!filep->private_data || !p)
+ return -EINVAL;
+
+ if (p != filep->private_data)
+ return -EINVAL;
+
+ /* Each FD owns only one kfd_process */
+ if (!p->primary)
+ return -EINVAL;
+
+ mutex_lock(&kfd_processes_mutex);
+ process = create_process(current, false);
+ mutex_unlock(&kfd_processes_mutex);
+
+ if (IS_ERR(process))
+ return PTR_ERR(process);
+
+ /* Each open() increases kref of the primary kfd_process,
+ * so we need to reduce it here before we create a new secondary process replacing it
+ */
+ kfd_unref_process(p);
+
+ filep->private_data = process;
+ ret = kfd_create_process_sysfs(process);
+ if (ret)
+ pr_warn("Failed to create sysfs entry for the kfd_process");
+
+ return 0;
+}
+
#define AMDKFD_IOCTL_DEF(ioctl, _func, _flags) \
[_IOC_NR(ioctl)] = {.cmd = ioctl, .func = _func, .flags = _flags, \
.cmd_drv = 0, .name = #ioctl}
@@ -3253,6 +3291,9 @@ static const struct amdkfd_ioctl_desc amdkfd_ioctls[] = {
AMDKFD_IOCTL_DEF(AMDKFD_IOC_DBG_TRAP,
kfd_ioctl_set_debug_trap, 0),
+
+ AMDKFD_IOCTL_DEF(AMDKFD_IOC_CREATE_PROCESS,
+ kfd_ioctl_create_process, 0),
};
#define AMDKFD_CORE_IOCTL_COUNT ARRAY_SIZE(amdkfd_ioctls)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
index 25534473c28f..95c7292bdd02 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
@@ -1053,6 +1053,7 @@ int kfd_create_process_sysfs(struct kfd_process *process);
struct kfd_process *kfd_lookup_process_by_pasid(u32 pasid,
struct kfd_process_device **pdd);
struct kfd_process *kfd_lookup_process_by_mm(const struct mm_struct *mm);
+struct kfd_process *create_process(const struct task_struct *thread, bool primary);
int kfd_process_gpuidx_from_gpuid(struct kfd_process *p, uint32_t gpu_id);
int kfd_process_gpuid_from_node(struct kfd_process *p, struct kfd_node *node,
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index 0c3f0cc16bf4..f51e3f4aaf48 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -71,7 +71,6 @@ static struct workqueue_struct *kfd_restore_wq;
static struct kfd_process *find_process(const struct task_struct *thread,
bool ref);
static void kfd_process_ref_release(struct kref *ref);
-static struct kfd_process *create_process(const struct task_struct *thread, bool primary);
static void evict_process_worker(struct work_struct *work);
static void restore_process_worker(struct work_struct *work);
@@ -1580,7 +1579,7 @@ void kfd_process_set_trap_debug_flag(struct qcm_process_device *qpd,
* On return the kfd_process is fully operational and will be freed when the
* mm is released
*/
-static struct kfd_process *create_process(const struct task_struct *thread, bool primary)
+struct kfd_process *create_process(const struct task_struct *thread, bool primary)
{
struct kfd_process *process;
struct mmu_notifier *mn;
diff --git a/include/uapi/linux/kfd_ioctl.h b/include/uapi/linux/kfd_ioctl.h
index 04c7d283dc7d..1d206ecc831e 100644
--- a/include/uapi/linux/kfd_ioctl.h
+++ b/include/uapi/linux/kfd_ioctl.h
@@ -44,9 +44,10 @@
* - 1.16 - Add contiguous VRAM allocation flag
* - 1.17 - Add SDMA queue creation with target SDMA engine ID
* - 1.18 - Rename pad in set_memory_policy_args to misc_process_flag
+ * - 1.19 - Add a new ioctl to craete secondary kfd processes
*/
#define KFD_IOCTL_MAJOR_VERSION 1
-#define KFD_IOCTL_MINOR_VERSION 18
+#define KFD_IOCTL_MINOR_VERSION 19
struct kfd_ioctl_get_version_args {
__u32 major_version; /* from KFD */
@@ -1671,7 +1672,10 @@ struct kfd_ioctl_dbg_trap_args {
#define AMDKFD_IOC_DBG_TRAP \
AMDKFD_IOWR(0x26, struct kfd_ioctl_dbg_trap_args)
+#define AMDKFD_IOC_CREATE_PROCESS \
+ AMDKFD_IO(0x27)
+
#define AMDKFD_COMMAND_START 0x01
-#define AMDKFD_COMMAND_END 0x27
+#define AMDKFD_COMMAND_END 0x28
#endif
--
2.47.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [PATCH V3 08/13] amdkfd: identify a secondary kfd process by its id
2025-08-04 11:05 ` [PATCH V3 08/13] amdkfd: identify a secondary kfd process by its id Zhu Lingshan
@ 2025-08-07 20:49 ` Felix Kuehling
0 siblings, 0 replies; 21+ messages in thread
From: Felix Kuehling @ 2025-08-07 20:49 UTC (permalink / raw)
To: Zhu Lingshan, alexander.deucher; +Cc: ray.huang, amd-gfx
On 2025-08-04 7:05, Zhu Lingshan wrote:
> This commit introduces a new id field for
> struct kfd process, which helps identify
> a kfd process among multiple contexts that
> all belong to a single user space program.
>
> The sysfs entry of a secondary kfd process
> is placed under the sysfs entry folder of
> its primary kfd process.
>
> The naming format of the sysfs entry of a secondary
> kfd process is "context_%u" where %u is the process id.
>
> Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
> ---
> drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 6 ++
> drivers/gpu/drm/amd/amdkfd/kfd_process.c | 83 +++++++++++++++++++++++-
> 2 files changed, 86 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> index d1436f1f527d..d140463e221b 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> @@ -998,6 +998,9 @@ struct kfd_process {
> /* Tracks debug per-vmid request for debug flags */
> u32 dbg_flags;
>
> + /* kfd process id */
> + u16 id;
> +
> atomic_t poison;
> /* Queues are in paused stated because we are in the process of doing a CRIU checkpoint */
> bool queues_paused;
> @@ -1012,6 +1015,9 @@ struct kfd_process {
>
> /* indicating whether this is a primary kfd_process */
> bool primary;
> +
> + /* The primary kfd_process allocating IDs for its secondary kfd_process, 0 for primary kfd_process */
> + struct ida id_table;
> };
>
> #define KFD_PROCESS_TABLE_SIZE 8 /* bits: 256 entries */
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> index 5d59a4d994d5..8e498fd35b8c 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> @@ -54,6 +54,9 @@ DEFINE_MUTEX(kfd_processes_mutex);
>
> DEFINE_SRCU(kfd_processes_srcu);
>
> +#define KFD_PROCESS_ID_MIN 1
> +#define KFD_PROCESS_ID_WIDTH 16
> +
> /* For process termination handling */
> static struct workqueue_struct *kfd_process_wq;
>
> @@ -827,6 +830,7 @@ static void kfd_process_device_destroy_ib_mem(struct kfd_process_device *pdd)
>
> int kfd_create_process_sysfs(struct kfd_process *process)
> {
> + struct kfd_process *primary_process;
> int ret;
>
> if (process->kobj) {
> @@ -839,9 +843,22 @@ int kfd_create_process_sysfs(struct kfd_process *process)
> pr_warn("Creating procfs kobject failed");
> return -ENOMEM;
> }
> - ret = kobject_init_and_add(process->kobj, &procfs_type,
> - procfs.kobj, "%d",
> - (int)process->lead_thread->pid);
> +
> + if (process->primary)
> + ret = kobject_init_and_add(process->kobj, &procfs_type,
> + procfs.kobj, "%d",
> + (int)process->lead_thread->pid);
> + else {
> + primary_process = kfd_lookup_process_by_mm(process->lead_thread->mm);
> + if (!primary_process)
> + return -ESRCH;
> +
> + ret = kobject_init_and_add(process->kobj, &procfs_type,
> + primary_process->kobj, "context_%u",
> + process->id);
> + kfd_unref_process(primary_process);
> + }
> +
> if (ret) {
> pr_warn("Creating procfs pid directory failed");
> kobject_put(process->kobj);
> @@ -863,6 +880,51 @@ int kfd_create_process_sysfs(struct kfd_process *process)
> return 0;
> }
>
> +static int kfd_process_alloc_id(struct kfd_process *process)
> +{
> + int ret;
> + struct kfd_process *primary_process;
> +
> + if (process->primary) {
> + process->id = 0;
> +
> + return 0;
> + }
> +
> + primary_process = kfd_lookup_process_by_mm(process->lead_thread->mm);
> + if (!primary_process)
> + return -ESRCH;
> +
> + ret = ida_alloc_range(&primary_process->id_table, KFD_PROCESS_ID_MIN,
> + (1 << KFD_PROCESS_ID_WIDTH) - 1, GFP_KERNEL);
> + if (ret < 0)
> + goto out;
> +
> + process->id = ret;
> + ret = 0;
> +
> +out:
> + kfd_unref_process(primary_process);
> +
> + return ret;
> +}
> +
> +static void kfd_process_free_id(struct kfd_process *process)
> +{
> + struct kfd_process *primary_process;
> +
> + if (process->primary)
> + return;
> +
> + primary_process = kfd_lookup_process_by_mm(process->lead_thread->mm);
> + if (!primary_process)
> + return;
> +
> + ida_free(&primary_process->id_table, process->id);
> +
> + kfd_unref_process(primary_process);
> +}
> +
> struct kfd_process *kfd_create_process(struct task_struct *thread)
> {
> struct kfd_process *process;
> @@ -1193,6 +1255,11 @@ static void kfd_process_wq_release(struct work_struct *work)
> if (ef)
> dma_fence_signal(ef);
>
> + if (!p->primary)
> + kfd_process_free_id(p);
> + else
> + ida_destroy(&p->id_table);
> +
> kfd_process_remove_sysfs(p);
> kfd_debugfs_remove_process(p);
>
> @@ -1549,6 +1616,12 @@ static struct kfd_process *create_process(const struct task_struct *thread, bool
> process->queues_paused = false;
> process->primary = primary;
>
> + err = kfd_process_alloc_id(process);
> + if (err) {
> + pr_err("Creating kfd process: failed to alloc an id\n");
> + goto err_alloc_id;
> + }
> +
> INIT_DELAYED_WORK(&process->eviction_work, evict_process_worker);
> INIT_DELAYED_WORK(&process->restore_work, restore_process_worker);
> process->last_restore_timestamp = get_jiffies_64();
> @@ -1599,6 +1672,8 @@ static struct kfd_process *create_process(const struct task_struct *thread, bool
> goto err_register_notifier;
> }
> BUG_ON(mn != &process->mmu_notifier);
> +
> + ida_init(&process->id_table);
> }
>
> kfd_unref_process(process);
> @@ -1619,6 +1694,8 @@ static struct kfd_process *create_process(const struct task_struct *thread, bool
> err_process_pqm_init:
> kfd_event_free_process(process);
> err_event_init:
> + kfd_process_free_id(process);
> +err_alloc_id:
> mutex_destroy(&process->mutex);
> kfree(process);
> err_alloc_process:
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH V3 10/13] amdkfd: process pointer of a HIQ should be set to NULL
2025-08-04 11:05 ` [PATCH V3 10/13] amdkfd: process pointer of a HIQ should be set to NULL Zhu Lingshan
@ 2025-08-07 20:55 ` Felix Kuehling
2025-08-08 7:48 ` Zhu, Lingshan
0 siblings, 1 reply; 21+ messages in thread
From: Felix Kuehling @ 2025-08-07 20:55 UTC (permalink / raw)
To: Zhu Lingshan, alexander.deucher; +Cc: ray.huang, amd-gfx
On 2025-08-04 7:05, Zhu Lingshan wrote:
> In kq_initialize, queue->process of a HIQ should
> be set to NULL because it does not belong to any kfd_process.
>
> This commit decommisions the function kfd_get_process() because
> it can not locate a specific kfd_process among multiple
> contexts and not any code path calls it after this commit.
>
> Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
> ---
> drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c | 3 ++-
> drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 1 -
> drivers/gpu/drm/amd/amdkfd/kfd_process.c | 18 ------------------
> 3 files changed, 2 insertions(+), 20 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
> index 2b0a830f5b29..ebee37937783 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
> @@ -144,7 +144,8 @@ static bool kq_initialize(struct kernel_queue *kq, struct kfd_node *dev,
> goto err_init_queue;
>
> kq->queue->device = dev;
> - kq->queue->process = kfd_get_process(current);
> + if (type == KFD_QUEUE_TYPE_HIQ)
> + kq->queue->process = NULL;
I think this should either unconditionally set the process to NULL for kernel queues, or not touch it at all, because it's probably 0-initialized at allocation time.
Either way, this commit should come after the one that removes the DIQ type, because DIQ did belong to a process.
Regards,
Felix
>
> kq->queue->mqd_mem_obj = kq->mqd_mgr->allocate_mqd(kq->mqd_mgr->dev,
> &kq->queue->properties);
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> index d140463e221b..25534473c28f 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> @@ -1050,7 +1050,6 @@ void kfd_process_destroy_wq(void);
> void kfd_cleanup_processes(void);
> struct kfd_process *kfd_create_process(struct task_struct *thread);
> int kfd_create_process_sysfs(struct kfd_process *process);
> -struct kfd_process *kfd_get_process(const struct task_struct *task);
> struct kfd_process *kfd_lookup_process_by_pasid(u32 pasid,
> struct kfd_process_device **pdd);
> struct kfd_process *kfd_lookup_process_by_mm(const struct mm_struct *mm);
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> index 8e498fd35b8c..0c3f0cc16bf4 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> @@ -989,24 +989,6 @@ struct kfd_process *kfd_create_process(struct task_struct *thread)
> return process;
> }
>
> -struct kfd_process *kfd_get_process(const struct task_struct *thread)
> -{
> - struct kfd_process *process;
> -
> - if (!thread->mm)
> - return ERR_PTR(-EINVAL);
> -
> - /* Only the pthreads threading model is supported. */
> - if (thread->group_leader->mm != thread->mm)
> - return ERR_PTR(-EINVAL);
> -
> - process = find_process(thread, false);
> - if (!process)
> - return ERR_PTR(-EINVAL);
> -
> - return process;
> -}
> -
> static struct kfd_process *find_process_by_mm(const struct mm_struct *mm)
> {
> struct kfd_process *process;
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH V3 09/13] amdkfd: find kfd_process by filep->private_data in kfd_mmap
2025-08-04 11:05 ` [PATCH V3 09/13] amdkfd: find kfd_process by filep->private_data in kfd_mmap Zhu Lingshan
@ 2025-08-07 20:55 ` Felix Kuehling
0 siblings, 0 replies; 21+ messages in thread
From: Felix Kuehling @ 2025-08-07 20:55 UTC (permalink / raw)
To: Zhu Lingshan, alexander.deucher; +Cc: ray.huang, amd-gfx
On 2025-08-04 7:05, Zhu Lingshan wrote:
> This commit finds the proper kfd_process by
> filep->private_data in kfd_mmap,
> because the function kfd_get_process()
> can not locate a specific kfd process among
> multiple contexts.
>
> Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
> ---
> drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 11 +++++++----
> 1 file changed, 7 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
> index 28df35a63c29..662b181f1fd2 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
> @@ -3407,16 +3407,19 @@ static int kfd_mmio_mmap(struct kfd_node *dev, struct kfd_process *process,
> }
>
>
> -static int kfd_mmap(struct file *filp, struct vm_area_struct *vma)
> +static int kfd_mmap(struct file *filep, struct vm_area_struct *vma)
> {
> struct kfd_process *process;
> struct kfd_node *dev = NULL;
> unsigned long mmap_offset;
> unsigned int gpu_id;
>
> - process = kfd_get_process(current);
> - if (IS_ERR(process))
> - return PTR_ERR(process);
> + process = filep->private_data;
> + if (!process)
> + return -ESRCH;
> +
> + if (process->lead_thread != current->group_leader)
> + return -EBADF;
>
> mmap_offset = vma->vm_pgoff << PAGE_SHIFT;
> gpu_id = KFD_MMAP_GET_GPU_ID(mmap_offset);
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH V3 11/13] amdkfd: remove DIQ support
2025-08-04 11:05 ` [PATCH V3 11/13] amdkfd: remove DIQ support Zhu Lingshan
@ 2025-08-07 21:03 ` Felix Kuehling
2025-08-08 7:45 ` Zhu, Lingshan
0 siblings, 1 reply; 21+ messages in thread
From: Felix Kuehling @ 2025-08-07 21:03 UTC (permalink / raw)
To: Zhu Lingshan, alexander.deucher; +Cc: ray.huang, amd-gfx
On 2025-08-04 7:05, Zhu Lingshan wrote:
> This commit remove DIQ support because it has been
> marked as DEPRECATED since 2022 in commit 5bdd3eb2
I think you can also remove KFD_QUEUE_TYPE_DIQ from enum kfd_queue_type in kfd_priv.h.
See two more comments inline ...
>
> Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
> ---
> .../drm/amd/amdkfd/kfd_device_queue_manager.c | 6 +--
> drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c | 26 ++-----------
> .../drm/amd/amdkfd/kfd_packet_manager_v9.c | 4 --
> .../drm/amd/amdkfd/kfd_packet_manager_vi.c | 4 --
> .../amd/amdkfd/kfd_process_queue_manager.c | 39 +------------------
> 5 files changed, 7 insertions(+), 72 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> index 2d91027e2a74..58e47e14cf07 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> @@ -399,8 +399,7 @@ static void increment_queue_count(struct device_queue_manager *dqm,
> struct queue *q)
> {
> dqm->active_queue_count++;
> - if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE ||
> - q->properties.type == KFD_QUEUE_TYPE_DIQ)
> + if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE)
> dqm->active_cp_queue_count++;
>
> if (q->properties.is_gws) {
> @@ -414,8 +413,7 @@ static void decrement_queue_count(struct device_queue_manager *dqm,
> struct queue *q)
> {
> dqm->active_queue_count--;
> - if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE ||
> - q->properties.type == KFD_QUEUE_TYPE_DIQ)
> + if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE)
> dqm->active_cp_queue_count--;
>
> if (q->properties.is_gws) {
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
> index ebee37937783..f676b7419984 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
> @@ -46,7 +46,7 @@ static bool kq_initialize(struct kernel_queue *kq, struct kfd_node *dev,
> int retval;
> union PM4_MES_TYPE_3_HEADER nop;
>
> - if (WARN_ON(type != KFD_QUEUE_TYPE_DIQ && type != KFD_QUEUE_TYPE_HIQ))
> + if (WARN_ON(type != KFD_QUEUE_TYPE_HIQ))
> return false;
You're returning early here for invalid queue types. That's fine, but see below ...
>
> pr_debug("Initializing queue type %d size %d\n", KFD_QUEUE_TYPE_HIQ,
> @@ -61,14 +61,9 @@ static bool kq_initialize(struct kernel_queue *kq, struct kfd_node *dev,
>
> kq->dev = dev;
> kq->nop_packet = nop.u32all;
> - switch (type) {
> - case KFD_QUEUE_TYPE_DIQ:
> - kq->mqd_mgr = dev->dqm->mqd_mgrs[KFD_MQD_TYPE_DIQ];
> - break;
> - case KFD_QUEUE_TYPE_HIQ:
> + if (type == KFD_QUEUE_TYPE_HIQ)
> kq->mqd_mgr = dev->dqm->mqd_mgrs[KFD_MQD_TYPE_HIQ];
> - break;
> - default:
> + else {
> dev_err(dev->adev->dev, "Invalid queue type %d\n", type);
> return false;
I think this code is unreachable because you returned above for any invalid queue type. You can just remove both the if and the else branch. Just assign the kq->mqd_mgr unconditionally.
Regards,
Felix
> }
> @@ -163,24 +158,11 @@ static bool kq_initialize(struct kernel_queue *kq, struct kfd_node *dev,
> kq->mqd_mgr->load_mqd(kq->mqd_mgr, kq->queue->mqd,
> kq->queue->pipe, kq->queue->queue,
> &kq->queue->properties, NULL);
> - } else {
> - /* allocate fence for DIQ */
> -
> - retval = kfd_gtt_sa_allocate(dev, sizeof(uint32_t),
> - &kq->fence_mem_obj);
> -
> - if (retval != 0)
> - goto err_alloc_fence;
> -
> - kq->fence_kernel_address = kq->fence_mem_obj->cpu_ptr;
> - kq->fence_gpu_addr = kq->fence_mem_obj->gpu_addr;
> }
>
> print_queue(kq->queue);
>
> return true;
> -err_alloc_fence:
> - kq->mqd_mgr->free_mqd(kq->mqd_mgr, kq->queue->mqd, kq->queue->mqd_mem_obj);
> err_allocate_mqd:
> uninit_queue(kq->queue);
> err_init_queue:
> @@ -210,8 +192,6 @@ static void kq_uninitialize(struct kernel_queue *kq)
> kq->queue->queue);
> up_read(&kq->dev->adev->reset_domain->sem);
> }
> - else if (kq->queue->properties.type == KFD_QUEUE_TYPE_DIQ)
> - kfd_gtt_sa_free(kq->dev, kq->fence_mem_obj);
>
> kq->mqd_mgr->free_mqd(kq->mqd_mgr, kq->queue->mqd,
> kq->queue->mqd_mem_obj);
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c
> index 505036968a77..3d2375817c3e 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c
> @@ -252,10 +252,6 @@ static int pm_map_queues_v9(struct packet_manager *pm, uint32_t *buffer,
> packet->bitfields2.queue_type =
> queue_type__mes_map_queues__normal_latency_static_queue_vi;
> break;
> - case KFD_QUEUE_TYPE_DIQ:
> - packet->bitfields2.queue_type =
> - queue_type__mes_map_queues__debug_interface_queue_vi;
> - break;
> case KFD_QUEUE_TYPE_SDMA:
> case KFD_QUEUE_TYPE_SDMA_XGMI:
> if (q->properties.sdma_engine_id < 2 &&
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c
> index a1de5d7e173a..60086e7cc258 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c
> @@ -166,10 +166,6 @@ static int pm_map_queues_vi(struct packet_manager *pm, uint32_t *buffer,
> packet->bitfields2.queue_type =
> queue_type__mes_map_queues__normal_latency_static_queue_vi;
> break;
> - case KFD_QUEUE_TYPE_DIQ:
> - packet->bitfields2.queue_type =
> - queue_type__mes_map_queues__debug_interface_queue_vi;
> - break;
> case KFD_QUEUE_TYPE_SDMA:
> case KFD_QUEUE_TYPE_SDMA_XGMI:
> packet->bitfields2.engine_sel = q->properties.sdma_engine_id +
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
> index c643e0ccec52..e36950e7e14f 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
> @@ -345,7 +345,7 @@ int pqm_create_queue(struct process_queue_manager *pqm,
> * If we are just about to create DIQ, the is_debug flag is not set yet
> * Hence we also check the type as well
> */
> - if ((pdd->qpd.is_debug) || (type == KFD_QUEUE_TYPE_DIQ))
> + if ((pdd->qpd.is_debug))
> max_queues = dev->kfd->device_info.max_no_of_hqd/2;
>
> if (pdd->qpd.queue_count >= max_queues)
> @@ -426,22 +426,6 @@ int pqm_create_queue(struct process_queue_manager *pqm,
> restore_mqd, restore_ctl_stack);
> print_queue(q);
> break;
> - case KFD_QUEUE_TYPE_DIQ:
> - kq = kernel_queue_init(dev, KFD_QUEUE_TYPE_DIQ);
> - if (!kq) {
> - retval = -ENOMEM;
> - goto err_create_queue;
> - }
> - kq->queue->properties.queue_id = *qid;
> - pqn->kq = kq;
> - pqn->q = NULL;
> - retval = kfd_process_drain_interrupts(pdd);
> - if (retval)
> - break;
> -
> - retval = dev->dqm->ops.create_kernel_queue(dev->dqm,
> - kq, &pdd->qpd);
> - break;
> default:
> WARN(1, "Invalid queue type %d", type);
> retval = -EINVAL;
> @@ -1121,32 +1105,13 @@ int pqm_debugfs_mqds(struct seq_file *m, void *data)
> break;
> default:
> seq_printf(m,
> - " Bad user queue type %d on device %x\n",
> + " Qeueu node with bad user queue type %d on device %x\n",
> q->properties.type, q->device->id);
> continue;
> }
> mqd_mgr = q->device->dqm->mqd_mgrs[mqd_type];
> size = mqd_mgr->mqd_stride(mqd_mgr,
> &q->properties);
> - } else if (pqn->kq) {
> - q = pqn->kq->queue;
> - mqd_mgr = pqn->kq->mqd_mgr;
> - switch (q->properties.type) {
> - case KFD_QUEUE_TYPE_DIQ:
> - seq_printf(m, " DIQ on device %x\n",
> - pqn->kq->dev->id);
> - break;
> - default:
> - seq_printf(m,
> - " Bad kernel queue type %d on device %x\n",
> - q->properties.type,
> - pqn->kq->dev->id);
> - continue;
> - }
> - } else {
> - seq_printf(m,
> - " Weird: Queue node with neither kernel nor user queue\n");
> - continue;
> }
>
> for (xcc = 0; xcc < num_xccs; xcc++) {
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH V3 12/13] amdkfd: remove test_kq
2025-08-04 11:05 ` [PATCH V3 12/13] amdkfd: remove test_kq Zhu Lingshan
@ 2025-08-07 21:06 ` Felix Kuehling
0 siblings, 0 replies; 21+ messages in thread
From: Felix Kuehling @ 2025-08-07 21:06 UTC (permalink / raw)
To: Zhu Lingshan, alexander.deucher; +Cc: ray.huang, amd-gfx
On 2025-08-04 7:05, Zhu Lingshan wrote:
> This commit removes test_kq() function becuse it has been
> marked as unused since 2014 and no other functions calls it.
>
> Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
> ---
> drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c | 31 -------------------
> 1 file changed, 31 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
> index f676b7419984..6aa8b0348bad 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
> @@ -339,34 +339,3 @@ void kernel_queue_uninit(struct kernel_queue *kq)
> kq_uninitialize(kq);
> kfree(kq);
> }
> -
> -/* FIXME: Can this test be removed? */
> -static __attribute__((unused)) void test_kq(struct kfd_node *dev)
> -{
> - struct kernel_queue *kq;
> - uint32_t *buffer, i;
> - int retval;
> -
> - dev_err(dev->adev->dev, "Starting kernel queue test\n");
> -
> - kq = kernel_queue_init(dev, KFD_QUEUE_TYPE_HIQ);
> - if (unlikely(!kq)) {
> - dev_err(dev->adev->dev, " Failed to initialize HIQ\n");
> - dev_err(dev->adev->dev, "Kernel queue test failed\n");
> - return;
> - }
> -
> - retval = kq_acquire_packet_buffer(kq, 5, &buffer);
> - if (unlikely(retval != 0)) {
> - dev_err(dev->adev->dev, " Failed to acquire packet buffer\n");
> - dev_err(dev->adev->dev, "Kernel queue test failed\n");
> - return;
> - }
> - for (i = 0; i < 5; i++)
> - buffer[i] = kq->nop_packet;
> - kq_submit_packet(kq);
> -
> - dev_err(dev->adev->dev, "Ending kernel queue test\n");
> -}
> -
> -
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH V3 11/13] amdkfd: remove DIQ support
2025-08-07 21:03 ` Felix Kuehling
@ 2025-08-08 7:45 ` Zhu, Lingshan
0 siblings, 0 replies; 21+ messages in thread
From: Zhu, Lingshan @ 2025-08-08 7:45 UTC (permalink / raw)
To: Felix Kuehling, alexander.deucher; +Cc: ray.huang, amd-gfx
[-- Attachment #1: Type: text/plain, Size: 8731 bytes --]
On 8/8/2025 5:03 AM, Felix Kuehling wrote:
> On 2025-08-04 7:05, Zhu Lingshan wrote:
>> This commit remove DIQ support because it has been
>> marked as DEPRECATED since 2022 in commit 5bdd3eb2
> I think you can also remove KFD_QUEUE_TYPE_DIQ from enum kfd_queue_type in kfd_priv.h.
I was thinking about whether there can be any customer maintained out-of-tree
modules rely on hard code offset in this enum. Maybe I am overthinking because
it is named priv.h, I will remove DIQ in the enmu in the next version.
>
> See two more comments inline ...
>
>> Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
>> ---
>> .../drm/amd/amdkfd/kfd_device_queue_manager.c | 6 +--
>> drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c | 26 ++-----------
>> .../drm/amd/amdkfd/kfd_packet_manager_v9.c | 4 --
>> .../drm/amd/amdkfd/kfd_packet_manager_vi.c | 4 --
>> .../amd/amdkfd/kfd_process_queue_manager.c | 39 +------------------
>> 5 files changed, 7 insertions(+), 72 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
>> index 2d91027e2a74..58e47e14cf07 100644
>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
>> @@ -399,8 +399,7 @@ static void increment_queue_count(struct device_queue_manager *dqm,
>> struct queue *q)
>> {
>> dqm->active_queue_count++;
>> - if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE ||
>> - q->properties.type == KFD_QUEUE_TYPE_DIQ)
>> + if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE)
>> dqm->active_cp_queue_count++;
>>
>> if (q->properties.is_gws) {
>> @@ -414,8 +413,7 @@ static void decrement_queue_count(struct device_queue_manager *dqm,
>> struct queue *q)
>> {
>> dqm->active_queue_count--;
>> - if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE ||
>> - q->properties.type == KFD_QUEUE_TYPE_DIQ)
>> + if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE)
>> dqm->active_cp_queue_count--;
>>
>> if (q->properties.is_gws) {
>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
>> index ebee37937783..f676b7419984 100644
>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
>> @@ -46,7 +46,7 @@ static bool kq_initialize(struct kernel_queue *kq, struct kfd_node *dev,
>> int retval;
>> union PM4_MES_TYPE_3_HEADER nop;
>>
>> - if (WARN_ON(type != KFD_QUEUE_TYPE_DIQ && type != KFD_QUEUE_TYPE_HIQ))
>> + if (WARN_ON(type != KFD_QUEUE_TYPE_HIQ))
>> return false;
> You're returning early here for invalid queue types. That's fine, but see below ...
>
>
>>
>> pr_debug("Initializing queue type %d size %d\n", KFD_QUEUE_TYPE_HIQ,
>> @@ -61,14 +61,9 @@ static bool kq_initialize(struct kernel_queue *kq, struct kfd_node *dev,
>>
>> kq->dev = dev;
>> kq->nop_packet = nop.u32all;
>> - switch (type) {
>> - case KFD_QUEUE_TYPE_DIQ:
>> - kq->mqd_mgr = dev->dqm->mqd_mgrs[KFD_MQD_TYPE_DIQ];
>> - break;
>> - case KFD_QUEUE_TYPE_HIQ:
>> + if (type == KFD_QUEUE_TYPE_HIQ)
>> kq->mqd_mgr = dev->dqm->mqd_mgrs[KFD_MQD_TYPE_HIQ];
>> - break;
>> - default:
>> + else {
>> dev_err(dev->adev->dev, "Invalid queue type %d\n", type);
>> return false;
> I think this code is unreachable because you returned above for any invalid queue type. You can just remove both the if and the else branch. Just assign the kq->mqd_mgr unconditionally.
Yes, will improve.
Thanks
Lingshan
>
> Regards,
> Felix
>
>
>> }
>> @@ -163,24 +158,11 @@ static bool kq_initialize(struct kernel_queue *kq, struct kfd_node *dev,
>> kq->mqd_mgr->load_mqd(kq->mqd_mgr, kq->queue->mqd,
>> kq->queue->pipe, kq->queue->queue,
>> &kq->queue->properties, NULL);
>> - } else {
>> - /* allocate fence for DIQ */
>> -
>> - retval = kfd_gtt_sa_allocate(dev, sizeof(uint32_t),
>> - &kq->fence_mem_obj);
>> -
>> - if (retval != 0)
>> - goto err_alloc_fence;
>> -
>> - kq->fence_kernel_address = kq->fence_mem_obj->cpu_ptr;
>> - kq->fence_gpu_addr = kq->fence_mem_obj->gpu_addr;
>> }
>>
>> print_queue(kq->queue);
>>
>> return true;
>> -err_alloc_fence:
>> - kq->mqd_mgr->free_mqd(kq->mqd_mgr, kq->queue->mqd, kq->queue->mqd_mem_obj);
>> err_allocate_mqd:
>> uninit_queue(kq->queue);
>> err_init_queue:
>> @@ -210,8 +192,6 @@ static void kq_uninitialize(struct kernel_queue *kq)
>> kq->queue->queue);
>> up_read(&kq->dev->adev->reset_domain->sem);
>> }
>> - else if (kq->queue->properties.type == KFD_QUEUE_TYPE_DIQ)
>> - kfd_gtt_sa_free(kq->dev, kq->fence_mem_obj);
>>
>> kq->mqd_mgr->free_mqd(kq->mqd_mgr, kq->queue->mqd,
>> kq->queue->mqd_mem_obj);
>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c
>> index 505036968a77..3d2375817c3e 100644
>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c
>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c
>> @@ -252,10 +252,6 @@ static int pm_map_queues_v9(struct packet_manager *pm, uint32_t *buffer,
>> packet->bitfields2.queue_type =
>> queue_type__mes_map_queues__normal_latency_static_queue_vi;
>> break;
>> - case KFD_QUEUE_TYPE_DIQ:
>> - packet->bitfields2.queue_type =
>> - queue_type__mes_map_queues__debug_interface_queue_vi;
>> - break;
>> case KFD_QUEUE_TYPE_SDMA:
>> case KFD_QUEUE_TYPE_SDMA_XGMI:
>> if (q->properties.sdma_engine_id < 2 &&
>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c
>> index a1de5d7e173a..60086e7cc258 100644
>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c
>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c
>> @@ -166,10 +166,6 @@ static int pm_map_queues_vi(struct packet_manager *pm, uint32_t *buffer,
>> packet->bitfields2.queue_type =
>> queue_type__mes_map_queues__normal_latency_static_queue_vi;
>> break;
>> - case KFD_QUEUE_TYPE_DIQ:
>> - packet->bitfields2.queue_type =
>> - queue_type__mes_map_queues__debug_interface_queue_vi;
>> - break;
>> case KFD_QUEUE_TYPE_SDMA:
>> case KFD_QUEUE_TYPE_SDMA_XGMI:
>> packet->bitfields2.engine_sel = q->properties.sdma_engine_id +
>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
>> index c643e0ccec52..e36950e7e14f 100644
>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
>> @@ -345,7 +345,7 @@ int pqm_create_queue(struct process_queue_manager *pqm,
>> * If we are just about to create DIQ, the is_debug flag is not set yet
>> * Hence we also check the type as well
>> */
>> - if ((pdd->qpd.is_debug) || (type == KFD_QUEUE_TYPE_DIQ))
>> + if ((pdd->qpd.is_debug))
>> max_queues = dev->kfd->device_info.max_no_of_hqd/2;
>>
>> if (pdd->qpd.queue_count >= max_queues)
>> @@ -426,22 +426,6 @@ int pqm_create_queue(struct process_queue_manager *pqm,
>> restore_mqd, restore_ctl_stack);
>> print_queue(q);
>> break;
>> - case KFD_QUEUE_TYPE_DIQ:
>> - kq = kernel_queue_init(dev, KFD_QUEUE_TYPE_DIQ);
>> - if (!kq) {
>> - retval = -ENOMEM;
>> - goto err_create_queue;
>> - }
>> - kq->queue->properties.queue_id = *qid;
>> - pqn->kq = kq;
>> - pqn->q = NULL;
>> - retval = kfd_process_drain_interrupts(pdd);
>> - if (retval)
>> - break;
>> -
>> - retval = dev->dqm->ops.create_kernel_queue(dev->dqm,
>> - kq, &pdd->qpd);
>> - break;
>> default:
>> WARN(1, "Invalid queue type %d", type);
>> retval = -EINVAL;
>> @@ -1121,32 +1105,13 @@ int pqm_debugfs_mqds(struct seq_file *m, void *data)
>> break;
>> default:
>> seq_printf(m,
>> - " Bad user queue type %d on device %x\n",
>> + " Qeueu node with bad user queue type %d on device %x\n",
>> q->properties.type, q->device->id);
>> continue;
>> }
>> mqd_mgr = q->device->dqm->mqd_mgrs[mqd_type];
>> size = mqd_mgr->mqd_stride(mqd_mgr,
>> &q->properties);
>> - } else if (pqn->kq) {
>> - q = pqn->kq->queue;
>> - mqd_mgr = pqn->kq->mqd_mgr;
>> - switch (q->properties.type) {
>> - case KFD_QUEUE_TYPE_DIQ:
>> - seq_printf(m, " DIQ on device %x\n",
>> - pqn->kq->dev->id);
>> - break;
>> - default:
>> - seq_printf(m,
>> - " Bad kernel queue type %d on device %x\n",
>> - q->properties.type,
>> - pqn->kq->dev->id);
>> - continue;
>> - }
>> - } else {
>> - seq_printf(m,
>> - " Weird: Queue node with neither kernel nor user queue\n");
>> - continue;
>> }
>>
>> for (xcc = 0; xcc < num_xccs; xcc++) {
[-- Attachment #2: Type: text/html, Size: 9943 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH V3 10/13] amdkfd: process pointer of a HIQ should be set to NULL
2025-08-07 20:55 ` Felix Kuehling
@ 2025-08-08 7:48 ` Zhu, Lingshan
0 siblings, 0 replies; 21+ messages in thread
From: Zhu, Lingshan @ 2025-08-08 7:48 UTC (permalink / raw)
To: Felix Kuehling, alexander.deucher; +Cc: ray.huang, amd-gfx
[-- Attachment #1: Type: text/plain, Size: 3385 bytes --]
On 8/8/2025 4:55 AM, Felix Kuehling wrote:
> On 2025-08-04 7:05, Zhu Lingshan wrote:
>> In kq_initialize, queue->process of a HIQ should
>> be set to NULL because it does not belong to any kfd_process.
>>
>> This commit decommisions the function kfd_get_process() because
>> it can not locate a specific kfd_process among multiple
>> contexts and not any code path calls it after this commit.
>>
>> Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
>> ---
>> drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c | 3 ++-
>> drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 1 -
>> drivers/gpu/drm/amd/amdkfd/kfd_process.c | 18 ------------------
>> 3 files changed, 2 insertions(+), 20 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
>> index 2b0a830f5b29..ebee37937783 100644
>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
>> @@ -144,7 +144,8 @@ static bool kq_initialize(struct kernel_queue *kq, struct kfd_node *dev,
>> goto err_init_queue;
>>
>> kq->queue->device = dev;
>> - kq->queue->process = kfd_get_process(current);
>> + if (type == KFD_QUEUE_TYPE_HIQ)
>> + kq->queue->process = NULL;
> I think this should either unconditionally set the process to NULL for kernel queues, or not touch it at all, because it's probably 0-initialized at allocation time.
I will leave it untouched.
>
> Either way, this commit should come after the one that removes the DIQ type, because DIQ did belong to a process.
Yes
Thanks
Lingshan
>
> Regards,
> Felix
>
>
>>
>> kq->queue->mqd_mem_obj = kq->mqd_mgr->allocate_mqd(kq->mqd_mgr->dev,
>> &kq->queue->properties);
>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
>> index d140463e221b..25534473c28f 100644
>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
>> @@ -1050,7 +1050,6 @@ void kfd_process_destroy_wq(void);
>> void kfd_cleanup_processes(void);
>> struct kfd_process *kfd_create_process(struct task_struct *thread);
>> int kfd_create_process_sysfs(struct kfd_process *process);
>> -struct kfd_process *kfd_get_process(const struct task_struct *task);
>> struct kfd_process *kfd_lookup_process_by_pasid(u32 pasid,
>> struct kfd_process_device **pdd);
>> struct kfd_process *kfd_lookup_process_by_mm(const struct mm_struct *mm);
>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
>> index 8e498fd35b8c..0c3f0cc16bf4 100644
>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
>> @@ -989,24 +989,6 @@ struct kfd_process *kfd_create_process(struct task_struct *thread)
>> return process;
>> }
>>
>> -struct kfd_process *kfd_get_process(const struct task_struct *thread)
>> -{
>> - struct kfd_process *process;
>> -
>> - if (!thread->mm)
>> - return ERR_PTR(-EINVAL);
>> -
>> - /* Only the pthreads threading model is supported. */
>> - if (thread->group_leader->mm != thread->mm)
>> - return ERR_PTR(-EINVAL);
>> -
>> - process = find_process(thread, false);
>> - if (!process)
>> - return ERR_PTR(-EINVAL);
>> -
>> - return process;
>> -}
>> -
>> static struct kfd_process *find_process_by_mm(const struct mm_struct *mm)
>> {
>> struct kfd_process *process;
[-- Attachment #2: Type: text/html, Size: 4189 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2025-08-08 7:48 UTC | newest]
Thread overview: 21+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-08-04 11:05 [PATCH V3 00/13] amdkfd: Implement kfd multiple contexts Zhu Lingshan
2025-08-04 11:05 ` [PATCH V3 01/13] amdkfd: enlarge the hashtable of kfd_process Zhu Lingshan
2025-08-04 11:05 ` [PATCH V3 02/13] amdkfd: mark the first kfd_process as the primary one Zhu Lingshan
2025-08-04 11:05 ` [PATCH V3 03/13] amdkfd: find_process_by_mm always return the primary context Zhu Lingshan
2025-08-04 11:05 ` [PATCH V3 04/13] amdkfd: Introduce kfd_create_process_sysfs as a separate function Zhu Lingshan
2025-08-04 11:05 ` [PATCH V3 05/13] amdkfd: destroy kfd secondary contexts through fd close Zhu Lingshan
2025-08-04 11:05 ` [PATCH V3 06/13] amdkfd: process svm ioctl only on the primary kfd process Zhu Lingshan
2025-08-04 11:05 ` [PATCH V3 07/13] amdkfd: process USERPTR allocation " Zhu Lingshan
2025-08-04 11:05 ` [PATCH V3 08/13] amdkfd: identify a secondary kfd process by its id Zhu Lingshan
2025-08-07 20:49 ` Felix Kuehling
2025-08-04 11:05 ` [PATCH V3 09/13] amdkfd: find kfd_process by filep->private_data in kfd_mmap Zhu Lingshan
2025-08-07 20:55 ` Felix Kuehling
2025-08-04 11:05 ` [PATCH V3 10/13] amdkfd: process pointer of a HIQ should be set to NULL Zhu Lingshan
2025-08-07 20:55 ` Felix Kuehling
2025-08-08 7:48 ` Zhu, Lingshan
2025-08-04 11:05 ` [PATCH V3 11/13] amdkfd: remove DIQ support Zhu Lingshan
2025-08-07 21:03 ` Felix Kuehling
2025-08-08 7:45 ` Zhu, Lingshan
2025-08-04 11:05 ` [PATCH V3 12/13] amdkfd: remove test_kq Zhu Lingshan
2025-08-07 21:06 ` Felix Kuehling
2025-08-04 11:05 ` [PATCH V3 13/13] amdkfd: introduce new ioctl AMDKFD_IOC_CREATE_PROCESS Zhu Lingshan
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).