[PATCH v2 00/13] drm/amdgpu: Add Peak Tops Limiter (PTL)

[PATCH v2 00/13] drm/amdgpu: Add Peak Tops Limiter (PTL)

[Perry Yuan]

This series adds support for Peak Tops Limiter (PTL) on AMD GFX 9.4.4
GPUs. PTL is a hardware feature that limits peak
computational throughput to stay within power and thermal constraints.

v1->v2:
 - Rework patch set based on review feedback (Alex)
    - Consolidate PTL structure and interface definitions
    - Drop patches that have been merged into earlier patches
  - Add explicit PSP response status check in psp_ptl_invoke()
  - Add GFX idle wait before PTL state transition to prevent CP hang
  - Unify refcount logic for PTL disable/enable
  - Rework profiler IOCTL interface based on review feedback (James)

Benjamin Welton (1):
  Add kfd_ioctl_profiler to contain profiler kernel driver changes

Perry Yuan (12):
  drm/amdgpu: add new performance monitor PSP interfaces
  drm/amdgpu: add psp interfaces for peak tops limiter driver
  drm/amdgpu: add PTL enable/query gfx control support for GC 9.4.4
  drm/amdkfd: add kgd control interface for ptl
  Documentation/amdgpu: Add documentation for Peak Tops Limiter (PTL)
    sysfs interface
  drm/amdgpu: add sysfs for Peak Tops Limiter (PTL)
  drm/amdkfd: Add PTL control IOCTL Option and unify refcount logic
  drm/amdkfd: suspend scheduler during PTL re-enabling
  drm/amdgpu: Track PTL disable requests by source
  drm/amdgpu: add amdgpu.ptl module parameter for PTL control
  drm/amdgpu: add new data types F8 and Vector for PTL
  drm/amdgpu: Wait for GFX idle before PTL state transition

 Documentation/gpu/amdgpu/index.rst            |   1 +
 Documentation/gpu/amdgpu/ptl.rst              |  94 +++++
 drivers/gpu/drm/amd/amdgpu/amdgpu.h           |   1 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h    |   2 +
 .../drm/amd/amdgpu/amdgpu_amdkfd_gc_9_4_3.c   |  13 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c    |   3 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c       |  13 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c       | 349 ++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h       |  27 +-
 drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c       |  62 ++++
 drivers/gpu/drm/amd/amdgpu/psp_gfx_if.h       |  17 +
 drivers/gpu/drm/amd/amdkfd/kfd_chardev.c      | 219 +++++++++++
 drivers/gpu/drm/amd/amdkfd/kfd_device.c       |   4 +
 .../drm/amd/amdkfd/kfd_device_queue_manager.c |  25 ++
 .../drm/amd/amdkfd/kfd_device_queue_manager.h |   2 +
 .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c  |  16 +-
 .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c  |  14 +-
 .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c  |   8 +-
 .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c   |  15 +-
 .../gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c   |  11 +
 drivers/gpu/drm/amd/amdkfd/kfd_priv.h         |  16 +
 drivers/gpu/drm/amd/amdkfd/kfd_process.c      |  15 +
 drivers/gpu/drm/amd/include/amdgpu_ptl.h      |  61 +++
 .../gpu/drm/amd/include/kgd_kfd_interface.h   |   6 +
 include/uapi/linux/kfd_ioctl.h                |  36 ++
 25 files changed, 1020 insertions(+), 10 deletions(-)
 create mode 100644 Documentation/gpu/amdgpu/ptl.rst
 create mode 100644 drivers/gpu/drm/amd/include/amdgpu_ptl.h

-- 
2.34.1

[PATCH v2 01/13] Add kfd_ioctl_profiler to contain profiler kernel driver changes

[Perry Yuan]

From: Benjamin Welton <bewelton@amd.com>

kfd_ioctl_profiler takes a similar approach to that of
kfd_ioctl_dbg_trap (which contains debugger related IOCTL
services) where kfd_ioctl_profiler will contain all profiler
related IOCTL services. The IOCTL is designed to be expanded
as needed to support additional profiler functionality.

The current functionality of the IOCTL is to allow for profilers
which need PMC counters from GPU devices to both signal to other
profilers that may be on the system that the device has active PMC
profiling taking place on it (multiple PMC profilers on the same
device can result in corrupted counter data) and to setup the device
to allow for the collection of SQ PMC data on all queues on the device.

For PMC data for the SQ block (such as SQ_WAVES) to be available
to a profiler, mmPERFCOUNT_ENABLE must be set on the queues. When
profiling a single process, the profiler can inject PM4 packets into
each queue to turn on PERFCOUNT_ENABLE. When profiling system wide,
the profiler does not have this option and must have a way to turn
on profiling for queues in which it cannot inject packets into directly.

Accomplishing this requires a few steps:

1. Checking if the user has the necessary permissions to profile system
   wide on the device. This check uses the same check that linux perf
   uses to determine if a user has the necessary permissions to profile
   at this scope (primarily if the process has CAP_SYS_PERFMON or is root).

2. Locking the device for profiling. This is done by setting a lock bit
   on the device struct and storing the process that locked the device.

3. Iterating all queues on the device and issuing an MQD Update to enable
   perfcounting on the queues.

4. Actions to cleanup if the process exits or releases the lock.

The IOCTL also contains a link to the existing PC Sampling IOCTL as well.
This is per a suggestion that we should potentially remove the PC Sampling
IOCTL to have it be a part of the profiler IOCTL. This is a future change.
In addition, we do expect to expand the profiler IOCTL to include
additional profiler functionality in the future (which necessitates the
use of a version number).

Signed-off-by: Benjamin Welton <benjamin.welton@amd.com>
Signed-off-by: Perry Yuan <perry.yuan@amd.com>
Acked-by: Kent Russell <kent.russell@amd.com>
Reviewed-by: Yifan Zhang <yifan1.zhang@amd.com>
---
 drivers/gpu/drm/amd/amdkfd/kfd_chardev.c      | 82 +++++++++++++++++++
 drivers/gpu/drm/amd/amdkfd/kfd_device.c       |  4 +
 .../drm/amd/amdkfd/kfd_device_queue_manager.c | 25 ++++++
 .../drm/amd/amdkfd/kfd_device_queue_manager.h |  2 +
 .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c  | 16 +++-
 .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c  | 14 +++-
 .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c  |  8 +-
 .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c   | 15 +++-
 .../gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c   | 11 +++
 drivers/gpu/drm/amd/amdkfd/kfd_priv.h         |  7 ++
 drivers/gpu/drm/amd/amdkfd/kfd_process.c      | 11 +++
 include/uapi/linux/kfd_ioctl.h                | 29 +++++++
 12 files changed, 216 insertions(+), 8 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index 732ad1224a61..dbb111a33678 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -21,6 +21,7 @@
  * OTHER DEALINGS IN THE SOFTWARE.
  */
 
+#include <linux/capability.h>
 #include <linux/device.h>
 #include <linux/err.h>
 #include <linux/fs.h>
@@ -3204,6 +3205,84 @@ static int kfd_ioctl_create_process(struct file *filep, struct kfd_process *p, v
 	return 0;
 }
 
+static inline uint32_t profile_lock_device(struct kfd_process *p,
+					   uint32_t gpu_id, uint32_t op)
+{
+	struct kfd_process_device *pdd;
+	struct kfd_dev *kfd;
+	int status = -EINVAL;
+
+	if (!p)
+		return -EINVAL;
+
+	mutex_lock(&p->mutex);
+	pdd = kfd_process_device_data_by_id(p, gpu_id);
+	mutex_unlock(&p->mutex);
+
+	if (!pdd || !pdd->dev || !pdd->dev->kfd)
+		return -EINVAL;
+
+	kfd = pdd->dev->kfd;
+
+	mutex_lock(&kfd->profiler_lock);
+	if (op == 1) {
+		if (!kfd->profiler_process) {
+			kfd->profiler_process = p;
+			status = 0;
+		} else if (kfd->profiler_process == p) {
+			status = -EALREADY;
+		} else {
+			status = -EBUSY;
+		}
+	} else if (op == 0 && kfd->profiler_process == p) {
+		kfd->profiler_process = NULL;
+		status = 0;
+	}
+	mutex_unlock(&kfd->profiler_lock);
+
+	return status;
+}
+
+static inline int kfd_profiler_pmc(struct kfd_process *p,
+				   struct kfd_ioctl_pmc_settings *args)
+{
+	struct kfd_process_device *pdd;
+	struct device_queue_manager *dqm;
+	int status;
+
+	/* Check if we have the correct permissions. */
+	if (!perfmon_capable())
+		return -EPERM;
+
+	/* Lock/Unlock the device based on the parameter given in OP */
+	status = profile_lock_device(p, args->gpu_id, args->lock);
+	if (status != 0)
+		return status;
+
+	/* Enable/disable perfcount if requested */
+	mutex_lock(&p->mutex);
+	pdd = kfd_process_device_data_by_id(p, args->gpu_id);
+	dqm = pdd->dev->dqm;
+	mutex_unlock(&p->mutex);
+
+	dqm->ops.set_perfcount(dqm, args->perfcount_enable);
+	return status;
+}
+
+static int kfd_ioctl_profiler(struct file *filep, struct kfd_process *p, void *data)
+{
+	struct kfd_ioctl_profiler_args *args = data;
+
+	switch (args->op) {
+	case KFD_IOC_PROFILER_VERSION:
+		args->version = KFD_IOC_PROFILER_VERSION_NUM;
+		return 0;
+	case KFD_IOC_PROFILER_PMC:
+		return kfd_profiler_pmc(p, &args->pmc);
+	}
+	return -EINVAL;
+}
+
 #define AMDKFD_IOCTL_DEF(ioctl, _func, _flags) \
 	[_IOC_NR(ioctl)] = {.cmd = ioctl, .func = _func, .flags = _flags, \
 			    .cmd_drv = 0, .name = #ioctl}
@@ -3325,6 +3404,9 @@ static const struct amdkfd_ioctl_desc amdkfd_ioctls[] = {
 
 	AMDKFD_IOCTL_DEF(AMDKFD_IOC_CREATE_PROCESS,
 			kfd_ioctl_create_process, 0),
+
+	AMDKFD_IOCTL_DEF(AMDKFD_IOC_PROFILER,
+			kfd_ioctl_profiler, 0),
 };
 
 #define AMDKFD_CORE_IOCTL_COUNT	ARRAY_SIZE(amdkfd_ioctls)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device.c b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
index 9a66ee661e57..f231e46e8528 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
@@ -936,6 +936,9 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd,
 
 	svm_range_set_max_pages(kfd->adev);
 
+	kfd->profiler_process = NULL;
+	mutex_init(&kfd->profiler_lock);
+
 	kfd->init_complete = true;
 	dev_info(kfd_device, "added device %x:%x\n", kfd->adev->pdev->vendor,
 		 kfd->adev->pdev->device);
@@ -971,6 +974,7 @@ void kgd2kfd_device_exit(struct kfd_dev *kfd)
 		ida_destroy(&kfd->doorbell_ida);
 		kfd_gtt_sa_fini(kfd);
 		amdgpu_amdkfd_free_kernel_mem(kfd->adev, &kfd->gtt_mem);
+		mutex_destroy(&kfd->profiler_lock);
 	}
 
 	kfree(kfd);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index 804851632c4c..4170a283db5b 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -305,6 +305,29 @@ static int remove_queue_mes(struct device_queue_manager *dqm, struct queue *q,
 	return r;
 }
 
+static void set_perfcount(struct device_queue_manager *dqm, int enable)
+{
+	struct device_process_node *cur;
+	struct qcm_process_device *qpd;
+	struct queue *q;
+	struct mqd_update_info minfo = { 0 };
+
+	if (!dqm)
+		return;
+
+	minfo.update_flag = (enable == 1 ? UPDATE_FLAG_PERFCOUNT_ENABLE :
+						 UPDATE_FLAG_PERFCOUNT_DISABLE);
+	dqm_lock(dqm);
+	list_for_each_entry(cur, &dqm->queues, list) {
+		qpd = cur->qpd;
+		list_for_each_entry(q, &qpd->queues_list, list) {
+			pqm_update_mqd(qpd->pqm, q->properties.queue_id,
+						&minfo);
+		}
+	}
+	dqm_unlock(dqm);
+}
+
 static int remove_all_kfd_queues_mes(struct device_queue_manager *dqm)
 {
 	struct device_process_node *cur;
@@ -2967,6 +2990,7 @@ struct device_queue_manager *device_queue_manager_init(struct kfd_node *dev)
 		dqm->ops.reset_queues = reset_queues_cpsch;
 		dqm->ops.get_queue_checkpoint_info = get_queue_checkpoint_info;
 		dqm->ops.checkpoint_mqd = checkpoint_mqd;
+		dqm->ops.set_perfcount = set_perfcount;
 		break;
 	case KFD_SCHED_POLICY_NO_HWS:
 		/* initialize dqm for no cp scheduling */
@@ -2987,6 +3011,7 @@ struct device_queue_manager *device_queue_manager_init(struct kfd_node *dev)
 		dqm->ops.get_wave_state = get_wave_state;
 		dqm->ops.get_queue_checkpoint_info = get_queue_checkpoint_info;
 		dqm->ops.checkpoint_mqd = checkpoint_mqd;
+		dqm->ops.set_perfcount = set_perfcount;
 		break;
 	default:
 		dev_err(dev->adev->dev, "Invalid scheduling policy %d\n", dqm->sched_policy);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h
index ef07e44916f8..74a3bcec3e4e 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h
@@ -200,6 +200,8 @@ struct device_queue_manager_ops {
 				  const struct queue *q,
 				  void *mqd,
 				  void *ctl_stack);
+	void	(*set_perfcount)(struct device_queue_manager *dqm,
+				  int enable);
 };
 
 struct device_queue_manager_asic_ops {
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c
index 97055f808d4a..993d60a24d50 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c
@@ -124,10 +124,9 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
 	 */
 	m->cp_hqd_hq_scheduler0 = 1 << 14;
 
-	if (q->format == KFD_QUEUE_FORMAT_AQL) {
+	if (q->format == KFD_QUEUE_FORMAT_AQL)
 		m->cp_hqd_aql_control =
 			1 << CP_HQD_AQL_CONTROL__CONTROL0__SHIFT;
-	}
 
 	if (mm->dev->kfd->cwsr_enabled) {
 		m->cp_hqd_persistent_state |=
@@ -142,6 +141,12 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
 		m->cp_hqd_wg_state_offset = q->ctl_stack_size;
 	}
 
+	mutex_lock(&mm->dev->kfd->profiler_lock);
+	if (mm->dev->kfd->profiler_process != NULL)
+		m->compute_perfcount_enable = 1;
+
+	mutex_unlock(&mm->dev->kfd->profiler_lock);
+
 	*mqd = m;
 	if (gart_addr)
 		*gart_addr = addr;
@@ -221,6 +226,13 @@ static void update_mqd(struct mqd_manager *mm, void *mqd,
 	if (mm->dev->kfd->cwsr_enabled)
 		m->cp_hqd_ctx_save_control = 0;
 
+	if (minfo) {
+		if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_ENABLE)
+			m->compute_perfcount_enable = 1;
+		else if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_DISABLE)
+			m->compute_perfcount_enable = 0;
+	}
+
 	update_cu_mask(mm, mqd, minfo);
 	set_priority(m, q);
 
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c
index 7e5a7ab6d0c0..4a574bbb5f37 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c
@@ -164,10 +164,9 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
 	if (amdgpu_amdkfd_have_atomics_support(mm->dev->adev))
 		m->cp_hqd_hq_status0 |= 1 << 29;
 
-	if (q->format == KFD_QUEUE_FORMAT_AQL) {
+	if (q->format == KFD_QUEUE_FORMAT_AQL)
 		m->cp_hqd_aql_control =
 			1 << CP_HQD_AQL_CONTROL__CONTROL0__SHIFT;
-	}
 
 	if (mm->dev->kfd->cwsr_enabled) {
 		m->cp_hqd_persistent_state |=
@@ -182,6 +181,11 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
 		m->cp_hqd_wg_state_offset = q->ctl_stack_size;
 	}
 
+	mutex_lock(&mm->dev->kfd->profiler_lock);
+	if (mm->dev->kfd->profiler_process != NULL)
+		m->compute_perfcount_enable = 1;
+	mutex_unlock(&mm->dev->kfd->profiler_lock);
+
 	*mqd = m;
 	if (gart_addr)
 		*gart_addr = addr;
@@ -259,6 +263,12 @@ static void update_mqd(struct mqd_manager *mm, void *mqd,
 	}
 	if (mm->dev->kfd->cwsr_enabled)
 		m->cp_hqd_ctx_save_control = 0;
+	if (minfo) {
+		if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_ENABLE)
+			m->compute_perfcount_enable = 1;
+		else if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_DISABLE)
+			m->compute_perfcount_enable = 0;
+	}
 
 	update_cu_mask(mm, mqd, minfo);
 	set_priority(m, q);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c
index a51f217329db..7173f6470e5e 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c
@@ -139,10 +139,9 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
 	if (amdgpu_amdkfd_have_atomics_support(mm->dev->adev))
 		m->cp_hqd_hq_status0 |= 1 << 29;
 
-	if (q->format == KFD_QUEUE_FORMAT_AQL) {
+	if (q->format == KFD_QUEUE_FORMAT_AQL)
 		m->cp_hqd_aql_control =
 			1 << CP_HQD_AQL_CONTROL__CONTROL0__SHIFT;
-	}
 
 	if (mm->dev->kfd->cwsr_enabled) {
 		m->cp_hqd_persistent_state |=
@@ -157,6 +156,11 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
 		m->cp_hqd_wg_state_offset = q->ctl_stack_size;
 	}
 
+	mutex_lock(&mm->dev->kfd->profiler_lock);
+	if (mm->dev->kfd->profiler_process != NULL)
+		m->compute_perfcount_enable = 1;
+	mutex_unlock(&mm->dev->kfd->profiler_lock);
+
 	*mqd = m;
 	if (gart_addr)
 		*gart_addr = addr;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
index dcf4bbfa641b..d4659a438be5 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
@@ -218,10 +218,9 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
 		m->cp_hqd_aql_control =
 			1 << CP_HQD_AQL_CONTROL__CONTROL0__SHIFT;
 
-	if (q->tba_addr) {
+	if (q->tba_addr)
 		m->compute_pgm_rsrc2 |=
 			(1 << COMPUTE_PGM_RSRC2__TRAP_PRESENT__SHIFT);
-	}
 
 	if (mm->dev->kfd->cwsr_enabled && q->ctx_save_restore_area_address) {
 		m->cp_hqd_persistent_state |=
@@ -236,6 +235,11 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
 		m->cp_hqd_wg_state_offset = q->ctl_stack_size;
 	}
 
+	mutex_lock(&mm->dev->kfd->profiler_lock);
+	if (mm->dev->kfd->profiler_process != NULL)
+		m->compute_perfcount_enable = 1;
+	mutex_unlock(&mm->dev->kfd->profiler_lock);
+
 	*mqd = m;
 	if (gart_addr)
 		*gart_addr = addr;
@@ -318,6 +322,13 @@ static void update_mqd(struct mqd_manager *mm, void *mqd,
 	if (mm->dev->kfd->cwsr_enabled && q->ctx_save_restore_area_address)
 		m->cp_hqd_ctx_save_control = 0;
 
+	if (minfo) {
+		if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_ENABLE)
+			m->compute_perfcount_enable = 1;
+		else if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_DISABLE)
+			m->compute_perfcount_enable = 0;
+	}
+
 	if (KFD_GC_VERSION(mm->dev) != IP_VERSION(9, 4, 3) &&
 	    KFD_GC_VERSION(mm->dev) != IP_VERSION(9, 4, 4) &&
 	    KFD_GC_VERSION(mm->dev) != IP_VERSION(9, 5, 0))
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
index 09483f0862d4..e8967f5e3892 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
@@ -149,6 +149,11 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
 		m->cp_hqd_wg_state_offset = q->ctl_stack_size;
 	}
 
+	mutex_lock(&mm->dev->kfd->profiler_lock);
+	if (mm->dev->kfd->profiler_process != NULL)
+		m->compute_perfcount_enable = 1;
+	mutex_unlock(&mm->dev->kfd->profiler_lock);
+
 	*mqd = m;
 	if (gart_addr)
 		*gart_addr = addr;
@@ -231,6 +236,12 @@ static void __update_mqd(struct mqd_manager *mm, void *mqd,
 		m->cp_hqd_ctx_save_control =
 			atc_bit << CP_HQD_CTX_SAVE_CONTROL__ATC__SHIFT |
 			mtype << CP_HQD_CTX_SAVE_CONTROL__MTYPE__SHIFT;
+	if (minfo) {
+		if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_ENABLE)
+			m->compute_perfcount_enable = 1;
+		else if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_DISABLE)
+			m->compute_perfcount_enable = 0;
+	}
 
 	update_cu_mask(mm, mqd, minfo);
 	set_priority(m, q);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
index 9849b54f54ba..8983065645fa 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
@@ -384,6 +384,11 @@ struct kfd_dev {
 	int kfd_dev_lock;
 
 	atomic_t kfd_processes_count;
+
+	/* Lock for profiler process */
+	struct mutex profiler_lock;
+	/* Process currently holding the lock */
+	struct kfd_process *profiler_process;
 };
 
 enum kfd_mempool {
@@ -556,6 +561,8 @@ enum mqd_update_flag {
 	UPDATE_FLAG_DBG_WA_ENABLE = 1,
 	UPDATE_FLAG_DBG_WA_DISABLE = 2,
 	UPDATE_FLAG_IS_GWS = 4, /* quirk for gfx9 IP */
+	UPDATE_FLAG_PERFCOUNT_ENABLE = 5,
+	UPDATE_FLAG_PERFCOUNT_DISABLE = 6,
 };
 
 struct mqd_update_info {
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index 8511fbebf327..deca19b478d0 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -1110,6 +1110,16 @@ static void kfd_process_free_outstanding_kfd_bos(struct kfd_process *p)
 		kfd_process_device_free_bos(p->pdds[i]);
 }
 
+static void kfd_process_profiler_release(struct kfd_process *p, struct kfd_process_device *pdd)
+{
+	mutex_lock(&pdd->dev->kfd->profiler_lock);
+	if (pdd->dev->kfd->profiler_process == p) {
+		pdd->qpd.dqm->ops.set_perfcount(pdd->qpd.dqm, 0);
+		pdd->dev->kfd->profiler_process = NULL;
+	}
+	mutex_unlock(&pdd->dev->kfd->profiler_lock);
+}
+
 static void kfd_process_destroy_pdds(struct kfd_process *p)
 {
 	int i;
@@ -1121,6 +1131,7 @@ static void kfd_process_destroy_pdds(struct kfd_process *p)
 
 		pr_debug("Releasing pdd (topology id %d, for pid %d)\n",
 			pdd->dev->id, p->lead_thread->pid);
+		kfd_process_profiler_release(p, pdd);
 		kfd_process_device_destroy_cwsr_dgpu(pdd);
 		kfd_process_device_destroy_ib_mem(pdd);
 
diff --git a/include/uapi/linux/kfd_ioctl.h b/include/uapi/linux/kfd_ioctl.h
index e72359370857..a8b2a18d07cf 100644
--- a/include/uapi/linux/kfd_ioctl.h
+++ b/include/uapi/linux/kfd_ioctl.h
@@ -1558,6 +1558,29 @@ struct kfd_ioctl_dbg_trap_args {
 	};
 };
 
+#define KFD_IOC_PROFILER_VERSION_NUM 1
+enum kfd_profiler_ops {
+	KFD_IOC_PROFILER_PMC = 0,
+	KFD_IOC_PROFILER_VERSION = 2,
+};
+
+/**
+ * Enables/Disables GPU Specific profiler settings
+ */
+struct kfd_ioctl_pmc_settings {
+	__u32 gpu_id;             /* This is the user_gpu_id */
+	__u32 lock;               /* Lock GPU for Profiling */
+	__u32 perfcount_enable;   /* Force Perfcount Enable for queues on GPU */
+};
+
+struct kfd_ioctl_profiler_args {
+	__u32 op;						/* kfd_profiler_op */
+	union {
+		struct kfd_ioctl_pmc_settings  pmc;
+		__u32 version;				/* KFD_IOC_PROFILER_VERSION_NUM */
+	};
+};
+
 #define AMDKFD_IOCTL_BASE 'K'
 #define AMDKFD_IO(nr)			_IO(AMDKFD_IOCTL_BASE, nr)
 #define AMDKFD_IOR(nr, type)		_IOR(AMDKFD_IOCTL_BASE, nr, type)
@@ -1684,4 +1707,10 @@ struct kfd_ioctl_dbg_trap_args {
 #define AMDKFD_COMMAND_START		0x01
 #define AMDKFD_COMMAND_END		0x28
 
+#define AMDKFD_IOC_PROFILER			\
+		AMDKFD_IOWR(0x86, struct kfd_ioctl_profiler_args)
+
+#define AMDKFD_COMMAND_START_2		0x80
+#define AMDKFD_COMMAND_END_2		0x87
+
 #endif
-- 
2.34.1

Re: [PATCH v2 01/13] Add kfd_ioctl_profiler to contain profiler kernel driver changes

[Alex Deucher]

On Sun, Feb 8, 2026 at 11:52 AM Perry Yuan <perry.yuan@amd.com> wrote:
>
> From: Benjamin Welton <bewelton@amd.com>
>
> kfd_ioctl_profiler takes a similar approach to that of
> kfd_ioctl_dbg_trap (which contains debugger related IOCTL
> services) where kfd_ioctl_profiler will contain all profiler
> related IOCTL services. The IOCTL is designed to be expanded
> as needed to support additional profiler functionality.
>
> The current functionality of the IOCTL is to allow for profilers
> which need PMC counters from GPU devices to both signal to other
> profilers that may be on the system that the device has active PMC
> profiling taking place on it (multiple PMC profilers on the same
> device can result in corrupted counter data) and to setup the device
> to allow for the collection of SQ PMC data on all queues on the device.
>
> For PMC data for the SQ block (such as SQ_WAVES) to be available
> to a profiler, mmPERFCOUNT_ENABLE must be set on the queues. When
> profiling a single process, the profiler can inject PM4 packets into
> each queue to turn on PERFCOUNT_ENABLE. When profiling system wide,
> the profiler does not have this option and must have a way to turn
> on profiling for queues in which it cannot inject packets into directly.
>
> Accomplishing this requires a few steps:
>
> 1. Checking if the user has the necessary permissions to profile system
>    wide on the device. This check uses the same check that linux perf
>    uses to determine if a user has the necessary permissions to profile
>    at this scope (primarily if the process has CAP_SYS_PERFMON or is root).
>
> 2. Locking the device for profiling. This is done by setting a lock bit
>    on the device struct and storing the process that locked the device.
>
> 3. Iterating all queues on the device and issuing an MQD Update to enable
>    perfcounting on the queues.
>
> 4. Actions to cleanup if the process exits or releases the lock.
>
> The IOCTL also contains a link to the existing PC Sampling IOCTL as well.
> This is per a suggestion that we should potentially remove the PC Sampling
> IOCTL to have it be a part of the profiler IOCTL. This is a future change.
> In addition, we do expect to expand the profiler IOCTL to include
> additional profiler functionality in the future (which necessitates the
> use of a version number).
>
> Signed-off-by: Benjamin Welton <benjamin.welton@amd.com>
> Signed-off-by: Perry Yuan <perry.yuan@amd.com>
> Acked-by: Kent Russell <kent.russell@amd.com>
> Reviewed-by: Yifan Zhang <yifan1.zhang@amd.com>

Please include the link to the usermode stuff that uses this new IOCTL.

Alex

> ---
>  drivers/gpu/drm/amd/amdkfd/kfd_chardev.c      | 82 +++++++++++++++++++
>  drivers/gpu/drm/amd/amdkfd/kfd_device.c       |  4 +
>  .../drm/amd/amdkfd/kfd_device_queue_manager.c | 25 ++++++
>  .../drm/amd/amdkfd/kfd_device_queue_manager.h |  2 +
>  .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c  | 16 +++-
>  .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c  | 14 +++-
>  .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c  |  8 +-
>  .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c   | 15 +++-
>  .../gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c   | 11 +++
>  drivers/gpu/drm/amd/amdkfd/kfd_priv.h         |  7 ++
>  drivers/gpu/drm/amd/amdkfd/kfd_process.c      | 11 +++
>  include/uapi/linux/kfd_ioctl.h                | 29 +++++++
>  12 files changed, 216 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
> index 732ad1224a61..dbb111a33678 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
> @@ -21,6 +21,7 @@
>   * OTHER DEALINGS IN THE SOFTWARE.
>   */
>
> +#include <linux/capability.h>
>  #include <linux/device.h>
>  #include <linux/err.h>
>  #include <linux/fs.h>
> @@ -3204,6 +3205,84 @@ static int kfd_ioctl_create_process(struct file *filep, struct kfd_process *p, v
>         return 0;
>  }
>
> +static inline uint32_t profile_lock_device(struct kfd_process *p,
> +                                          uint32_t gpu_id, uint32_t op)
> +{
> +       struct kfd_process_device *pdd;
> +       struct kfd_dev *kfd;
> +       int status = -EINVAL;
> +
> +       if (!p)
> +               return -EINVAL;
> +
> +       mutex_lock(&p->mutex);
> +       pdd = kfd_process_device_data_by_id(p, gpu_id);
> +       mutex_unlock(&p->mutex);
> +
> +       if (!pdd || !pdd->dev || !pdd->dev->kfd)
> +               return -EINVAL;
> +
> +       kfd = pdd->dev->kfd;
> +
> +       mutex_lock(&kfd->profiler_lock);
> +       if (op == 1) {
> +               if (!kfd->profiler_process) {
> +                       kfd->profiler_process = p;
> +                       status = 0;
> +               } else if (kfd->profiler_process == p) {
> +                       status = -EALREADY;
> +               } else {
> +                       status = -EBUSY;
> +               }
> +       } else if (op == 0 && kfd->profiler_process == p) {
> +               kfd->profiler_process = NULL;
> +               status = 0;
> +       }
> +       mutex_unlock(&kfd->profiler_lock);
> +
> +       return status;
> +}
> +
> +static inline int kfd_profiler_pmc(struct kfd_process *p,
> +                                  struct kfd_ioctl_pmc_settings *args)
> +{
> +       struct kfd_process_device *pdd;
> +       struct device_queue_manager *dqm;
> +       int status;
> +
> +       /* Check if we have the correct permissions. */
> +       if (!perfmon_capable())
> +               return -EPERM;
> +
> +       /* Lock/Unlock the device based on the parameter given in OP */
> +       status = profile_lock_device(p, args->gpu_id, args->lock);
> +       if (status != 0)
> +               return status;
> +
> +       /* Enable/disable perfcount if requested */
> +       mutex_lock(&p->mutex);
> +       pdd = kfd_process_device_data_by_id(p, args->gpu_id);
> +       dqm = pdd->dev->dqm;
> +       mutex_unlock(&p->mutex);
> +
> +       dqm->ops.set_perfcount(dqm, args->perfcount_enable);
> +       return status;
> +}
> +
> +static int kfd_ioctl_profiler(struct file *filep, struct kfd_process *p, void *data)
> +{
> +       struct kfd_ioctl_profiler_args *args = data;
> +
> +       switch (args->op) {
> +       case KFD_IOC_PROFILER_VERSION:
> +               args->version = KFD_IOC_PROFILER_VERSION_NUM;
> +               return 0;
> +       case KFD_IOC_PROFILER_PMC:
> +               return kfd_profiler_pmc(p, &args->pmc);
> +       }
> +       return -EINVAL;
> +}
> +
>  #define AMDKFD_IOCTL_DEF(ioctl, _func, _flags) \
>         [_IOC_NR(ioctl)] = {.cmd = ioctl, .func = _func, .flags = _flags, \
>                             .cmd_drv = 0, .name = #ioctl}
> @@ -3325,6 +3404,9 @@ static const struct amdkfd_ioctl_desc amdkfd_ioctls[] = {
>
>         AMDKFD_IOCTL_DEF(AMDKFD_IOC_CREATE_PROCESS,
>                         kfd_ioctl_create_process, 0),
> +
> +       AMDKFD_IOCTL_DEF(AMDKFD_IOC_PROFILER,
> +                       kfd_ioctl_profiler, 0),
>  };
>
>  #define AMDKFD_CORE_IOCTL_COUNT        ARRAY_SIZE(amdkfd_ioctls)
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device.c b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
> index 9a66ee661e57..f231e46e8528 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_device.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
> @@ -936,6 +936,9 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd,
>
>         svm_range_set_max_pages(kfd->adev);
>
> +       kfd->profiler_process = NULL;
> +       mutex_init(&kfd->profiler_lock);
> +
>         kfd->init_complete = true;
>         dev_info(kfd_device, "added device %x:%x\n", kfd->adev->pdev->vendor,
>                  kfd->adev->pdev->device);
> @@ -971,6 +974,7 @@ void kgd2kfd_device_exit(struct kfd_dev *kfd)
>                 ida_destroy(&kfd->doorbell_ida);
>                 kfd_gtt_sa_fini(kfd);
>                 amdgpu_amdkfd_free_kernel_mem(kfd->adev, &kfd->gtt_mem);
> +               mutex_destroy(&kfd->profiler_lock);
>         }
>
>         kfree(kfd);
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> index 804851632c4c..4170a283db5b 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> @@ -305,6 +305,29 @@ static int remove_queue_mes(struct device_queue_manager *dqm, struct queue *q,
>         return r;
>  }
>
> +static void set_perfcount(struct device_queue_manager *dqm, int enable)
> +{
> +       struct device_process_node *cur;
> +       struct qcm_process_device *qpd;
> +       struct queue *q;
> +       struct mqd_update_info minfo = { 0 };
> +
> +       if (!dqm)
> +               return;
> +
> +       minfo.update_flag = (enable == 1 ? UPDATE_FLAG_PERFCOUNT_ENABLE :
> +                                                UPDATE_FLAG_PERFCOUNT_DISABLE);
> +       dqm_lock(dqm);
> +       list_for_each_entry(cur, &dqm->queues, list) {
> +               qpd = cur->qpd;
> +               list_for_each_entry(q, &qpd->queues_list, list) {
> +                       pqm_update_mqd(qpd->pqm, q->properties.queue_id,
> +                                               &minfo);
> +               }
> +       }
> +       dqm_unlock(dqm);
> +}
> +
>  static int remove_all_kfd_queues_mes(struct device_queue_manager *dqm)
>  {
>         struct device_process_node *cur;
> @@ -2967,6 +2990,7 @@ struct device_queue_manager *device_queue_manager_init(struct kfd_node *dev)
>                 dqm->ops.reset_queues = reset_queues_cpsch;
>                 dqm->ops.get_queue_checkpoint_info = get_queue_checkpoint_info;
>                 dqm->ops.checkpoint_mqd = checkpoint_mqd;
> +               dqm->ops.set_perfcount = set_perfcount;
>                 break;
>         case KFD_SCHED_POLICY_NO_HWS:
>                 /* initialize dqm for no cp scheduling */
> @@ -2987,6 +3011,7 @@ struct device_queue_manager *device_queue_manager_init(struct kfd_node *dev)
>                 dqm->ops.get_wave_state = get_wave_state;
>                 dqm->ops.get_queue_checkpoint_info = get_queue_checkpoint_info;
>                 dqm->ops.checkpoint_mqd = checkpoint_mqd;
> +               dqm->ops.set_perfcount = set_perfcount;
>                 break;
>         default:
>                 dev_err(dev->adev->dev, "Invalid scheduling policy %d\n", dqm->sched_policy);
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h
> index ef07e44916f8..74a3bcec3e4e 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h
> @@ -200,6 +200,8 @@ struct device_queue_manager_ops {
>                                   const struct queue *q,
>                                   void *mqd,
>                                   void *ctl_stack);
> +       void    (*set_perfcount)(struct device_queue_manager *dqm,
> +                                 int enable);
>  };
>
>  struct device_queue_manager_asic_ops {
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c
> index 97055f808d4a..993d60a24d50 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c
> @@ -124,10 +124,9 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
>          */
>         m->cp_hqd_hq_scheduler0 = 1 << 14;
>
> -       if (q->format == KFD_QUEUE_FORMAT_AQL) {
> +       if (q->format == KFD_QUEUE_FORMAT_AQL)
>                 m->cp_hqd_aql_control =
>                         1 << CP_HQD_AQL_CONTROL__CONTROL0__SHIFT;
> -       }
>
>         if (mm->dev->kfd->cwsr_enabled) {
>                 m->cp_hqd_persistent_state |=
> @@ -142,6 +141,12 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
>                 m->cp_hqd_wg_state_offset = q->ctl_stack_size;
>         }
>
> +       mutex_lock(&mm->dev->kfd->profiler_lock);
> +       if (mm->dev->kfd->profiler_process != NULL)
> +               m->compute_perfcount_enable = 1;
> +
> +       mutex_unlock(&mm->dev->kfd->profiler_lock);
> +
>         *mqd = m;
>         if (gart_addr)
>                 *gart_addr = addr;
> @@ -221,6 +226,13 @@ static void update_mqd(struct mqd_manager *mm, void *mqd,
>         if (mm->dev->kfd->cwsr_enabled)
>                 m->cp_hqd_ctx_save_control = 0;
>
> +       if (minfo) {
> +               if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_ENABLE)
> +                       m->compute_perfcount_enable = 1;
> +               else if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_DISABLE)
> +                       m->compute_perfcount_enable = 0;
> +       }
> +
>         update_cu_mask(mm, mqd, minfo);
>         set_priority(m, q);
>
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c
> index 7e5a7ab6d0c0..4a574bbb5f37 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c
> @@ -164,10 +164,9 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
>         if (amdgpu_amdkfd_have_atomics_support(mm->dev->adev))
>                 m->cp_hqd_hq_status0 |= 1 << 29;
>
> -       if (q->format == KFD_QUEUE_FORMAT_AQL) {
> +       if (q->format == KFD_QUEUE_FORMAT_AQL)
>                 m->cp_hqd_aql_control =
>                         1 << CP_HQD_AQL_CONTROL__CONTROL0__SHIFT;
> -       }
>
>         if (mm->dev->kfd->cwsr_enabled) {
>                 m->cp_hqd_persistent_state |=
> @@ -182,6 +181,11 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
>                 m->cp_hqd_wg_state_offset = q->ctl_stack_size;
>         }
>
> +       mutex_lock(&mm->dev->kfd->profiler_lock);
> +       if (mm->dev->kfd->profiler_process != NULL)
> +               m->compute_perfcount_enable = 1;
> +       mutex_unlock(&mm->dev->kfd->profiler_lock);
> +
>         *mqd = m;
>         if (gart_addr)
>                 *gart_addr = addr;
> @@ -259,6 +263,12 @@ static void update_mqd(struct mqd_manager *mm, void *mqd,
>         }
>         if (mm->dev->kfd->cwsr_enabled)
>                 m->cp_hqd_ctx_save_control = 0;
> +       if (minfo) {
> +               if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_ENABLE)
> +                       m->compute_perfcount_enable = 1;
> +               else if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_DISABLE)
> +                       m->compute_perfcount_enable = 0;
> +       }
>
>         update_cu_mask(mm, mqd, minfo);
>         set_priority(m, q);
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c
> index a51f217329db..7173f6470e5e 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c
> @@ -139,10 +139,9 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
>         if (amdgpu_amdkfd_have_atomics_support(mm->dev->adev))
>                 m->cp_hqd_hq_status0 |= 1 << 29;
>
> -       if (q->format == KFD_QUEUE_FORMAT_AQL) {
> +       if (q->format == KFD_QUEUE_FORMAT_AQL)
>                 m->cp_hqd_aql_control =
>                         1 << CP_HQD_AQL_CONTROL__CONTROL0__SHIFT;
> -       }
>
>         if (mm->dev->kfd->cwsr_enabled) {
>                 m->cp_hqd_persistent_state |=
> @@ -157,6 +156,11 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
>                 m->cp_hqd_wg_state_offset = q->ctl_stack_size;
>         }
>
> +       mutex_lock(&mm->dev->kfd->profiler_lock);
> +       if (mm->dev->kfd->profiler_process != NULL)
> +               m->compute_perfcount_enable = 1;
> +       mutex_unlock(&mm->dev->kfd->profiler_lock);
> +
>         *mqd = m;
>         if (gart_addr)
>                 *gart_addr = addr;
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
> index dcf4bbfa641b..d4659a438be5 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
> @@ -218,10 +218,9 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
>                 m->cp_hqd_aql_control =
>                         1 << CP_HQD_AQL_CONTROL__CONTROL0__SHIFT;
>
> -       if (q->tba_addr) {
> +       if (q->tba_addr)
>                 m->compute_pgm_rsrc2 |=
>                         (1 << COMPUTE_PGM_RSRC2__TRAP_PRESENT__SHIFT);
> -       }
>
>         if (mm->dev->kfd->cwsr_enabled && q->ctx_save_restore_area_address) {
>                 m->cp_hqd_persistent_state |=
> @@ -236,6 +235,11 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
>                 m->cp_hqd_wg_state_offset = q->ctl_stack_size;
>         }
>
> +       mutex_lock(&mm->dev->kfd->profiler_lock);
> +       if (mm->dev->kfd->profiler_process != NULL)
> +               m->compute_perfcount_enable = 1;
> +       mutex_unlock(&mm->dev->kfd->profiler_lock);
> +
>         *mqd = m;
>         if (gart_addr)
>                 *gart_addr = addr;
> @@ -318,6 +322,13 @@ static void update_mqd(struct mqd_manager *mm, void *mqd,
>         if (mm->dev->kfd->cwsr_enabled && q->ctx_save_restore_area_address)
>                 m->cp_hqd_ctx_save_control = 0;
>
> +       if (minfo) {
> +               if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_ENABLE)
> +                       m->compute_perfcount_enable = 1;
> +               else if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_DISABLE)
> +                       m->compute_perfcount_enable = 0;
> +       }
> +
>         if (KFD_GC_VERSION(mm->dev) != IP_VERSION(9, 4, 3) &&
>             KFD_GC_VERSION(mm->dev) != IP_VERSION(9, 4, 4) &&
>             KFD_GC_VERSION(mm->dev) != IP_VERSION(9, 5, 0))
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
> index 09483f0862d4..e8967f5e3892 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
> @@ -149,6 +149,11 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
>                 m->cp_hqd_wg_state_offset = q->ctl_stack_size;
>         }
>
> +       mutex_lock(&mm->dev->kfd->profiler_lock);
> +       if (mm->dev->kfd->profiler_process != NULL)
> +               m->compute_perfcount_enable = 1;
> +       mutex_unlock(&mm->dev->kfd->profiler_lock);
> +
>         *mqd = m;
>         if (gart_addr)
>                 *gart_addr = addr;
> @@ -231,6 +236,12 @@ static void __update_mqd(struct mqd_manager *mm, void *mqd,
>                 m->cp_hqd_ctx_save_control =
>                         atc_bit << CP_HQD_CTX_SAVE_CONTROL__ATC__SHIFT |
>                         mtype << CP_HQD_CTX_SAVE_CONTROL__MTYPE__SHIFT;
> +       if (minfo) {
> +               if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_ENABLE)
> +                       m->compute_perfcount_enable = 1;
> +               else if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_DISABLE)
> +                       m->compute_perfcount_enable = 0;
> +       }
>
>         update_cu_mask(mm, mqd, minfo);
>         set_priority(m, q);
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> index 9849b54f54ba..8983065645fa 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> @@ -384,6 +384,11 @@ struct kfd_dev {
>         int kfd_dev_lock;
>
>         atomic_t kfd_processes_count;
> +
> +       /* Lock for profiler process */
> +       struct mutex profiler_lock;
> +       /* Process currently holding the lock */
> +       struct kfd_process *profiler_process;
>  };
>
>  enum kfd_mempool {
> @@ -556,6 +561,8 @@ enum mqd_update_flag {
>         UPDATE_FLAG_DBG_WA_ENABLE = 1,
>         UPDATE_FLAG_DBG_WA_DISABLE = 2,
>         UPDATE_FLAG_IS_GWS = 4, /* quirk for gfx9 IP */
> +       UPDATE_FLAG_PERFCOUNT_ENABLE = 5,
> +       UPDATE_FLAG_PERFCOUNT_DISABLE = 6,
>  };
>
>  struct mqd_update_info {
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> index 8511fbebf327..deca19b478d0 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> @@ -1110,6 +1110,16 @@ static void kfd_process_free_outstanding_kfd_bos(struct kfd_process *p)
>                 kfd_process_device_free_bos(p->pdds[i]);
>  }
>
> +static void kfd_process_profiler_release(struct kfd_process *p, struct kfd_process_device *pdd)
> +{
> +       mutex_lock(&pdd->dev->kfd->profiler_lock);
> +       if (pdd->dev->kfd->profiler_process == p) {
> +               pdd->qpd.dqm->ops.set_perfcount(pdd->qpd.dqm, 0);
> +               pdd->dev->kfd->profiler_process = NULL;
> +       }
> +       mutex_unlock(&pdd->dev->kfd->profiler_lock);
> +}
> +
>  static void kfd_process_destroy_pdds(struct kfd_process *p)
>  {
>         int i;
> @@ -1121,6 +1131,7 @@ static void kfd_process_destroy_pdds(struct kfd_process *p)
>
>                 pr_debug("Releasing pdd (topology id %d, for pid %d)\n",
>                         pdd->dev->id, p->lead_thread->pid);
> +               kfd_process_profiler_release(p, pdd);
>                 kfd_process_device_destroy_cwsr_dgpu(pdd);
>                 kfd_process_device_destroy_ib_mem(pdd);
>
> diff --git a/include/uapi/linux/kfd_ioctl.h b/include/uapi/linux/kfd_ioctl.h
> index e72359370857..a8b2a18d07cf 100644
> --- a/include/uapi/linux/kfd_ioctl.h
> +++ b/include/uapi/linux/kfd_ioctl.h
> @@ -1558,6 +1558,29 @@ struct kfd_ioctl_dbg_trap_args {
>         };
>  };
>
> +#define KFD_IOC_PROFILER_VERSION_NUM 1
> +enum kfd_profiler_ops {
> +       KFD_IOC_PROFILER_PMC = 0,
> +       KFD_IOC_PROFILER_VERSION = 2,
> +};
> +
> +/**
> + * Enables/Disables GPU Specific profiler settings
> + */
> +struct kfd_ioctl_pmc_settings {
> +       __u32 gpu_id;             /* This is the user_gpu_id */
> +       __u32 lock;               /* Lock GPU for Profiling */
> +       __u32 perfcount_enable;   /* Force Perfcount Enable for queues on GPU */
> +};
> +
> +struct kfd_ioctl_profiler_args {
> +       __u32 op;                                               /* kfd_profiler_op */
> +       union {
> +               struct kfd_ioctl_pmc_settings  pmc;
> +               __u32 version;                          /* KFD_IOC_PROFILER_VERSION_NUM */
> +       };
> +};
> +
>  #define AMDKFD_IOCTL_BASE 'K'
>  #define AMDKFD_IO(nr)                  _IO(AMDKFD_IOCTL_BASE, nr)
>  #define AMDKFD_IOR(nr, type)           _IOR(AMDKFD_IOCTL_BASE, nr, type)
> @@ -1684,4 +1707,10 @@ struct kfd_ioctl_dbg_trap_args {
>  #define AMDKFD_COMMAND_START           0x01
>  #define AMDKFD_COMMAND_END             0x28
>
> +#define AMDKFD_IOC_PROFILER                    \
> +               AMDKFD_IOWR(0x86, struct kfd_ioctl_profiler_args)
> +
> +#define AMDKFD_COMMAND_START_2         0x80
> +#define AMDKFD_COMMAND_END_2           0x87
> +
>  #endif
> --
> 2.34.1
>

Re: [PATCH v2 01/13] Add kfd_ioctl_profiler to contain profiler kernel driver changes

[Alex Deucher]

On Mon, Feb 9, 2026 at 9:47 AM Alex Deucher <alexdeucher@gmail.com> wrote:
>
> On Sun, Feb 8, 2026 at 11:52 AM Perry Yuan <perry.yuan@amd.com> wrote:
> >
> > From: Benjamin Welton <bewelton@amd.com>
> >
> > kfd_ioctl_profiler takes a similar approach to that of
> > kfd_ioctl_dbg_trap (which contains debugger related IOCTL
> > services) where kfd_ioctl_profiler will contain all profiler
> > related IOCTL services. The IOCTL is designed to be expanded
> > as needed to support additional profiler functionality.
> >
> > The current functionality of the IOCTL is to allow for profilers
> > which need PMC counters from GPU devices to both signal to other
> > profilers that may be on the system that the device has active PMC
> > profiling taking place on it (multiple PMC profilers on the same
> > device can result in corrupted counter data) and to setup the device
> > to allow for the collection of SQ PMC data on all queues on the device.
> >
> > For PMC data for the SQ block (such as SQ_WAVES) to be available
> > to a profiler, mmPERFCOUNT_ENABLE must be set on the queues. When
> > profiling a single process, the profiler can inject PM4 packets into
> > each queue to turn on PERFCOUNT_ENABLE. When profiling system wide,
> > the profiler does not have this option and must have a way to turn
> > on profiling for queues in which it cannot inject packets into directly.
> >
> > Accomplishing this requires a few steps:
> >
> > 1. Checking if the user has the necessary permissions to profile system
> >    wide on the device. This check uses the same check that linux perf
> >    uses to determine if a user has the necessary permissions to profile
> >    at this scope (primarily if the process has CAP_SYS_PERFMON or is root).
> >
> > 2. Locking the device for profiling. This is done by setting a lock bit
> >    on the device struct and storing the process that locked the device.
> >
> > 3. Iterating all queues on the device and issuing an MQD Update to enable
> >    perfcounting on the queues.
> >
> > 4. Actions to cleanup if the process exits or releases the lock.
> >
> > The IOCTL also contains a link to the existing PC Sampling IOCTL as well.
> > This is per a suggestion that we should potentially remove the PC Sampling
> > IOCTL to have it be a part of the profiler IOCTL. This is a future change.
> > In addition, we do expect to expand the profiler IOCTL to include
> > additional profiler functionality in the future (which necessitates the
> > use of a version number).
> >
> > Signed-off-by: Benjamin Welton <benjamin.welton@amd.com>
> > Signed-off-by: Perry Yuan <perry.yuan@amd.com>
> > Acked-by: Kent Russell <kent.russell@amd.com>
> > Reviewed-by: Yifan Zhang <yifan1.zhang@amd.com>
>
> Please include the link to the usermode stuff that uses this new IOCTL.

For all of the patches which add or modify IOCTLs, we need a link to
the userspace which uses the interface.  With that fixed, the series
is:
Acked-by: Alex Deucher <alexander.deucher@amd.com>

>
> Alex
>
> > ---
> >  drivers/gpu/drm/amd/amdkfd/kfd_chardev.c      | 82 +++++++++++++++++++
> >  drivers/gpu/drm/amd/amdkfd/kfd_device.c       |  4 +
> >  .../drm/amd/amdkfd/kfd_device_queue_manager.c | 25 ++++++
> >  .../drm/amd/amdkfd/kfd_device_queue_manager.h |  2 +
> >  .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c  | 16 +++-
> >  .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c  | 14 +++-
> >  .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c  |  8 +-
> >  .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c   | 15 +++-
> >  .../gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c   | 11 +++
> >  drivers/gpu/drm/amd/amdkfd/kfd_priv.h         |  7 ++
> >  drivers/gpu/drm/amd/amdkfd/kfd_process.c      | 11 +++
> >  include/uapi/linux/kfd_ioctl.h                | 29 +++++++
> >  12 files changed, 216 insertions(+), 8 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
> > index 732ad1224a61..dbb111a33678 100644
> > --- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
> > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
> > @@ -21,6 +21,7 @@
> >   * OTHER DEALINGS IN THE SOFTWARE.
> >   */
> >
> > +#include <linux/capability.h>
> >  #include <linux/device.h>
> >  #include <linux/err.h>
> >  #include <linux/fs.h>
> > @@ -3204,6 +3205,84 @@ static int kfd_ioctl_create_process(struct file *filep, struct kfd_process *p, v
> >         return 0;
> >  }
> >
> > +static inline uint32_t profile_lock_device(struct kfd_process *p,
> > +                                          uint32_t gpu_id, uint32_t op)
> > +{
> > +       struct kfd_process_device *pdd;
> > +       struct kfd_dev *kfd;
> > +       int status = -EINVAL;
> > +
> > +       if (!p)
> > +               return -EINVAL;
> > +
> > +       mutex_lock(&p->mutex);
> > +       pdd = kfd_process_device_data_by_id(p, gpu_id);
> > +       mutex_unlock(&p->mutex);
> > +
> > +       if (!pdd || !pdd->dev || !pdd->dev->kfd)
> > +               return -EINVAL;
> > +
> > +       kfd = pdd->dev->kfd;
> > +
> > +       mutex_lock(&kfd->profiler_lock);
> > +       if (op == 1) {
> > +               if (!kfd->profiler_process) {
> > +                       kfd->profiler_process = p;
> > +                       status = 0;
> > +               } else if (kfd->profiler_process == p) {
> > +                       status = -EALREADY;
> > +               } else {
> > +                       status = -EBUSY;
> > +               }
> > +       } else if (op == 0 && kfd->profiler_process == p) {
> > +               kfd->profiler_process = NULL;
> > +               status = 0;
> > +       }
> > +       mutex_unlock(&kfd->profiler_lock);
> > +
> > +       return status;
> > +}
> > +
> > +static inline int kfd_profiler_pmc(struct kfd_process *p,
> > +                                  struct kfd_ioctl_pmc_settings *args)
> > +{
> > +       struct kfd_process_device *pdd;
> > +       struct device_queue_manager *dqm;
> > +       int status;
> > +
> > +       /* Check if we have the correct permissions. */
> > +       if (!perfmon_capable())
> > +               return -EPERM;
> > +
> > +       /* Lock/Unlock the device based on the parameter given in OP */
> > +       status = profile_lock_device(p, args->gpu_id, args->lock);
> > +       if (status != 0)
> > +               return status;
> > +
> > +       /* Enable/disable perfcount if requested */
> > +       mutex_lock(&p->mutex);
> > +       pdd = kfd_process_device_data_by_id(p, args->gpu_id);
> > +       dqm = pdd->dev->dqm;
> > +       mutex_unlock(&p->mutex);
> > +
> > +       dqm->ops.set_perfcount(dqm, args->perfcount_enable);
> > +       return status;
> > +}
> > +
> > +static int kfd_ioctl_profiler(struct file *filep, struct kfd_process *p, void *data)
> > +{
> > +       struct kfd_ioctl_profiler_args *args = data;
> > +
> > +       switch (args->op) {
> > +       case KFD_IOC_PROFILER_VERSION:
> > +               args->version = KFD_IOC_PROFILER_VERSION_NUM;
> > +               return 0;
> > +       case KFD_IOC_PROFILER_PMC:
> > +               return kfd_profiler_pmc(p, &args->pmc);
> > +       }
> > +       return -EINVAL;
> > +}
> > +
> >  #define AMDKFD_IOCTL_DEF(ioctl, _func, _flags) \
> >         [_IOC_NR(ioctl)] = {.cmd = ioctl, .func = _func, .flags = _flags, \
> >                             .cmd_drv = 0, .name = #ioctl}
> > @@ -3325,6 +3404,9 @@ static const struct amdkfd_ioctl_desc amdkfd_ioctls[] = {
> >
> >         AMDKFD_IOCTL_DEF(AMDKFD_IOC_CREATE_PROCESS,
> >                         kfd_ioctl_create_process, 0),
> > +
> > +       AMDKFD_IOCTL_DEF(AMDKFD_IOC_PROFILER,
> > +                       kfd_ioctl_profiler, 0),
> >  };
> >
> >  #define AMDKFD_CORE_IOCTL_COUNT        ARRAY_SIZE(amdkfd_ioctls)
> > diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device.c b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
> > index 9a66ee661e57..f231e46e8528 100644
> > --- a/drivers/gpu/drm/amd/amdkfd/kfd_device.c
> > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
> > @@ -936,6 +936,9 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd,
> >
> >         svm_range_set_max_pages(kfd->adev);
> >
> > +       kfd->profiler_process = NULL;
> > +       mutex_init(&kfd->profiler_lock);
> > +
> >         kfd->init_complete = true;
> >         dev_info(kfd_device, "added device %x:%x\n", kfd->adev->pdev->vendor,
> >                  kfd->adev->pdev->device);
> > @@ -971,6 +974,7 @@ void kgd2kfd_device_exit(struct kfd_dev *kfd)
> >                 ida_destroy(&kfd->doorbell_ida);
> >                 kfd_gtt_sa_fini(kfd);
> >                 amdgpu_amdkfd_free_kernel_mem(kfd->adev, &kfd->gtt_mem);
> > +               mutex_destroy(&kfd->profiler_lock);
> >         }
> >
> >         kfree(kfd);
> > diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> > index 804851632c4c..4170a283db5b 100644
> > --- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> > @@ -305,6 +305,29 @@ static int remove_queue_mes(struct device_queue_manager *dqm, struct queue *q,
> >         return r;
> >  }
> >
> > +static void set_perfcount(struct device_queue_manager *dqm, int enable)
> > +{
> > +       struct device_process_node *cur;
> > +       struct qcm_process_device *qpd;
> > +       struct queue *q;
> > +       struct mqd_update_info minfo = { 0 };
> > +
> > +       if (!dqm)
> > +               return;
> > +
> > +       minfo.update_flag = (enable == 1 ? UPDATE_FLAG_PERFCOUNT_ENABLE :
> > +                                                UPDATE_FLAG_PERFCOUNT_DISABLE);
> > +       dqm_lock(dqm);
> > +       list_for_each_entry(cur, &dqm->queues, list) {
> > +               qpd = cur->qpd;
> > +               list_for_each_entry(q, &qpd->queues_list, list) {
> > +                       pqm_update_mqd(qpd->pqm, q->properties.queue_id,
> > +                                               &minfo);
> > +               }
> > +       }
> > +       dqm_unlock(dqm);
> > +}
> > +
> >  static int remove_all_kfd_queues_mes(struct device_queue_manager *dqm)
> >  {
> >         struct device_process_node *cur;
> > @@ -2967,6 +2990,7 @@ struct device_queue_manager *device_queue_manager_init(struct kfd_node *dev)
> >                 dqm->ops.reset_queues = reset_queues_cpsch;
> >                 dqm->ops.get_queue_checkpoint_info = get_queue_checkpoint_info;
> >                 dqm->ops.checkpoint_mqd = checkpoint_mqd;
> > +               dqm->ops.set_perfcount = set_perfcount;
> >                 break;
> >         case KFD_SCHED_POLICY_NO_HWS:
> >                 /* initialize dqm for no cp scheduling */
> > @@ -2987,6 +3011,7 @@ struct device_queue_manager *device_queue_manager_init(struct kfd_node *dev)
> >                 dqm->ops.get_wave_state = get_wave_state;
> >                 dqm->ops.get_queue_checkpoint_info = get_queue_checkpoint_info;
> >                 dqm->ops.checkpoint_mqd = checkpoint_mqd;
> > +               dqm->ops.set_perfcount = set_perfcount;
> >                 break;
> >         default:
> >                 dev_err(dev->adev->dev, "Invalid scheduling policy %d\n", dqm->sched_policy);
> > diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h
> > index ef07e44916f8..74a3bcec3e4e 100644
> > --- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h
> > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h
> > @@ -200,6 +200,8 @@ struct device_queue_manager_ops {
> >                                   const struct queue *q,
> >                                   void *mqd,
> >                                   void *ctl_stack);
> > +       void    (*set_perfcount)(struct device_queue_manager *dqm,
> > +                                 int enable);
> >  };
> >
> >  struct device_queue_manager_asic_ops {
> > diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c
> > index 97055f808d4a..993d60a24d50 100644
> > --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c
> > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c
> > @@ -124,10 +124,9 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
> >          */
> >         m->cp_hqd_hq_scheduler0 = 1 << 14;
> >
> > -       if (q->format == KFD_QUEUE_FORMAT_AQL) {
> > +       if (q->format == KFD_QUEUE_FORMAT_AQL)
> >                 m->cp_hqd_aql_control =
> >                         1 << CP_HQD_AQL_CONTROL__CONTROL0__SHIFT;
> > -       }
> >
> >         if (mm->dev->kfd->cwsr_enabled) {
> >                 m->cp_hqd_persistent_state |=
> > @@ -142,6 +141,12 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
> >                 m->cp_hqd_wg_state_offset = q->ctl_stack_size;
> >         }
> >
> > +       mutex_lock(&mm->dev->kfd->profiler_lock);
> > +       if (mm->dev->kfd->profiler_process != NULL)
> > +               m->compute_perfcount_enable = 1;
> > +
> > +       mutex_unlock(&mm->dev->kfd->profiler_lock);
> > +
> >         *mqd = m;
> >         if (gart_addr)
> >                 *gart_addr = addr;
> > @@ -221,6 +226,13 @@ static void update_mqd(struct mqd_manager *mm, void *mqd,
> >         if (mm->dev->kfd->cwsr_enabled)
> >                 m->cp_hqd_ctx_save_control = 0;
> >
> > +       if (minfo) {
> > +               if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_ENABLE)
> > +                       m->compute_perfcount_enable = 1;
> > +               else if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_DISABLE)
> > +                       m->compute_perfcount_enable = 0;
> > +       }
> > +
> >         update_cu_mask(mm, mqd, minfo);
> >         set_priority(m, q);
> >
> > diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c
> > index 7e5a7ab6d0c0..4a574bbb5f37 100644
> > --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c
> > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c
> > @@ -164,10 +164,9 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
> >         if (amdgpu_amdkfd_have_atomics_support(mm->dev->adev))
> >                 m->cp_hqd_hq_status0 |= 1 << 29;
> >
> > -       if (q->format == KFD_QUEUE_FORMAT_AQL) {
> > +       if (q->format == KFD_QUEUE_FORMAT_AQL)
> >                 m->cp_hqd_aql_control =
> >                         1 << CP_HQD_AQL_CONTROL__CONTROL0__SHIFT;
> > -       }
> >
> >         if (mm->dev->kfd->cwsr_enabled) {
> >                 m->cp_hqd_persistent_state |=
> > @@ -182,6 +181,11 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
> >                 m->cp_hqd_wg_state_offset = q->ctl_stack_size;
> >         }
> >
> > +       mutex_lock(&mm->dev->kfd->profiler_lock);
> > +       if (mm->dev->kfd->profiler_process != NULL)
> > +               m->compute_perfcount_enable = 1;
> > +       mutex_unlock(&mm->dev->kfd->profiler_lock);
> > +
> >         *mqd = m;
> >         if (gart_addr)
> >                 *gart_addr = addr;
> > @@ -259,6 +263,12 @@ static void update_mqd(struct mqd_manager *mm, void *mqd,
> >         }
> >         if (mm->dev->kfd->cwsr_enabled)
> >                 m->cp_hqd_ctx_save_control = 0;
> > +       if (minfo) {
> > +               if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_ENABLE)
> > +                       m->compute_perfcount_enable = 1;
> > +               else if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_DISABLE)
> > +                       m->compute_perfcount_enable = 0;
> > +       }
> >
> >         update_cu_mask(mm, mqd, minfo);
> >         set_priority(m, q);
> > diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c
> > index a51f217329db..7173f6470e5e 100644
> > --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c
> > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c
> > @@ -139,10 +139,9 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
> >         if (amdgpu_amdkfd_have_atomics_support(mm->dev->adev))
> >                 m->cp_hqd_hq_status0 |= 1 << 29;
> >
> > -       if (q->format == KFD_QUEUE_FORMAT_AQL) {
> > +       if (q->format == KFD_QUEUE_FORMAT_AQL)
> >                 m->cp_hqd_aql_control =
> >                         1 << CP_HQD_AQL_CONTROL__CONTROL0__SHIFT;
> > -       }
> >
> >         if (mm->dev->kfd->cwsr_enabled) {
> >                 m->cp_hqd_persistent_state |=
> > @@ -157,6 +156,11 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
> >                 m->cp_hqd_wg_state_offset = q->ctl_stack_size;
> >         }
> >
> > +       mutex_lock(&mm->dev->kfd->profiler_lock);
> > +       if (mm->dev->kfd->profiler_process != NULL)
> > +               m->compute_perfcount_enable = 1;
> > +       mutex_unlock(&mm->dev->kfd->profiler_lock);
> > +
> >         *mqd = m;
> >         if (gart_addr)
> >                 *gart_addr = addr;
> > diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
> > index dcf4bbfa641b..d4659a438be5 100644
> > --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
> > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
> > @@ -218,10 +218,9 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
> >                 m->cp_hqd_aql_control =
> >                         1 << CP_HQD_AQL_CONTROL__CONTROL0__SHIFT;
> >
> > -       if (q->tba_addr) {
> > +       if (q->tba_addr)
> >                 m->compute_pgm_rsrc2 |=
> >                         (1 << COMPUTE_PGM_RSRC2__TRAP_PRESENT__SHIFT);
> > -       }
> >
> >         if (mm->dev->kfd->cwsr_enabled && q->ctx_save_restore_area_address) {
> >                 m->cp_hqd_persistent_state |=
> > @@ -236,6 +235,11 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
> >                 m->cp_hqd_wg_state_offset = q->ctl_stack_size;
> >         }
> >
> > +       mutex_lock(&mm->dev->kfd->profiler_lock);
> > +       if (mm->dev->kfd->profiler_process != NULL)
> > +               m->compute_perfcount_enable = 1;
> > +       mutex_unlock(&mm->dev->kfd->profiler_lock);
> > +
> >         *mqd = m;
> >         if (gart_addr)
> >                 *gart_addr = addr;
> > @@ -318,6 +322,13 @@ static void update_mqd(struct mqd_manager *mm, void *mqd,
> >         if (mm->dev->kfd->cwsr_enabled && q->ctx_save_restore_area_address)
> >                 m->cp_hqd_ctx_save_control = 0;
> >
> > +       if (minfo) {
> > +               if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_ENABLE)
> > +                       m->compute_perfcount_enable = 1;
> > +               else if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_DISABLE)
> > +                       m->compute_perfcount_enable = 0;
> > +       }
> > +
> >         if (KFD_GC_VERSION(mm->dev) != IP_VERSION(9, 4, 3) &&
> >             KFD_GC_VERSION(mm->dev) != IP_VERSION(9, 4, 4) &&
> >             KFD_GC_VERSION(mm->dev) != IP_VERSION(9, 5, 0))
> > diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
> > index 09483f0862d4..e8967f5e3892 100644
> > --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
> > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
> > @@ -149,6 +149,11 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
> >                 m->cp_hqd_wg_state_offset = q->ctl_stack_size;
> >         }
> >
> > +       mutex_lock(&mm->dev->kfd->profiler_lock);
> > +       if (mm->dev->kfd->profiler_process != NULL)
> > +               m->compute_perfcount_enable = 1;
> > +       mutex_unlock(&mm->dev->kfd->profiler_lock);
> > +
> >         *mqd = m;
> >         if (gart_addr)
> >                 *gart_addr = addr;
> > @@ -231,6 +236,12 @@ static void __update_mqd(struct mqd_manager *mm, void *mqd,
> >                 m->cp_hqd_ctx_save_control =
> >                         atc_bit << CP_HQD_CTX_SAVE_CONTROL__ATC__SHIFT |
> >                         mtype << CP_HQD_CTX_SAVE_CONTROL__MTYPE__SHIFT;
> > +       if (minfo) {
> > +               if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_ENABLE)
> > +                       m->compute_perfcount_enable = 1;
> > +               else if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_DISABLE)
> > +                       m->compute_perfcount_enable = 0;
> > +       }
> >
> >         update_cu_mask(mm, mqd, minfo);
> >         set_priority(m, q);
> > diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> > index 9849b54f54ba..8983065645fa 100644
> > --- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> > @@ -384,6 +384,11 @@ struct kfd_dev {
> >         int kfd_dev_lock;
> >
> >         atomic_t kfd_processes_count;
> > +
> > +       /* Lock for profiler process */
> > +       struct mutex profiler_lock;
> > +       /* Process currently holding the lock */
> > +       struct kfd_process *profiler_process;
> >  };
> >
> >  enum kfd_mempool {
> > @@ -556,6 +561,8 @@ enum mqd_update_flag {
> >         UPDATE_FLAG_DBG_WA_ENABLE = 1,
> >         UPDATE_FLAG_DBG_WA_DISABLE = 2,
> >         UPDATE_FLAG_IS_GWS = 4, /* quirk for gfx9 IP */
> > +       UPDATE_FLAG_PERFCOUNT_ENABLE = 5,
> > +       UPDATE_FLAG_PERFCOUNT_DISABLE = 6,
> >  };
> >
> >  struct mqd_update_info {
> > diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> > index 8511fbebf327..deca19b478d0 100644
> > --- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> > @@ -1110,6 +1110,16 @@ static void kfd_process_free_outstanding_kfd_bos(struct kfd_process *p)
> >                 kfd_process_device_free_bos(p->pdds[i]);
> >  }
> >
> > +static void kfd_process_profiler_release(struct kfd_process *p, struct kfd_process_device *pdd)
> > +{
> > +       mutex_lock(&pdd->dev->kfd->profiler_lock);
> > +       if (pdd->dev->kfd->profiler_process == p) {
> > +               pdd->qpd.dqm->ops.set_perfcount(pdd->qpd.dqm, 0);
> > +               pdd->dev->kfd->profiler_process = NULL;
> > +       }
> > +       mutex_unlock(&pdd->dev->kfd->profiler_lock);
> > +}
> > +
> >  static void kfd_process_destroy_pdds(struct kfd_process *p)
> >  {
> >         int i;
> > @@ -1121,6 +1131,7 @@ static void kfd_process_destroy_pdds(struct kfd_process *p)
> >
> >                 pr_debug("Releasing pdd (topology id %d, for pid %d)\n",
> >                         pdd->dev->id, p->lead_thread->pid);
> > +               kfd_process_profiler_release(p, pdd);
> >                 kfd_process_device_destroy_cwsr_dgpu(pdd);
> >                 kfd_process_device_destroy_ib_mem(pdd);
> >
> > diff --git a/include/uapi/linux/kfd_ioctl.h b/include/uapi/linux/kfd_ioctl.h
> > index e72359370857..a8b2a18d07cf 100644
> > --- a/include/uapi/linux/kfd_ioctl.h
> > +++ b/include/uapi/linux/kfd_ioctl.h
> > @@ -1558,6 +1558,29 @@ struct kfd_ioctl_dbg_trap_args {
> >         };
> >  };
> >
> > +#define KFD_IOC_PROFILER_VERSION_NUM 1
> > +enum kfd_profiler_ops {
> > +       KFD_IOC_PROFILER_PMC = 0,
> > +       KFD_IOC_PROFILER_VERSION = 2,
> > +};
> > +
> > +/**
> > + * Enables/Disables GPU Specific profiler settings
> > + */
> > +struct kfd_ioctl_pmc_settings {
> > +       __u32 gpu_id;             /* This is the user_gpu_id */
> > +       __u32 lock;               /* Lock GPU for Profiling */
> > +       __u32 perfcount_enable;   /* Force Perfcount Enable for queues on GPU */
> > +};
> > +
> > +struct kfd_ioctl_profiler_args {
> > +       __u32 op;                                               /* kfd_profiler_op */
> > +       union {
> > +               struct kfd_ioctl_pmc_settings  pmc;
> > +               __u32 version;                          /* KFD_IOC_PROFILER_VERSION_NUM */
> > +       };
> > +};
> > +
> >  #define AMDKFD_IOCTL_BASE 'K'
> >  #define AMDKFD_IO(nr)                  _IO(AMDKFD_IOCTL_BASE, nr)
> >  #define AMDKFD_IOR(nr, type)           _IOR(AMDKFD_IOCTL_BASE, nr, type)
> > @@ -1684,4 +1707,10 @@ struct kfd_ioctl_dbg_trap_args {
> >  #define AMDKFD_COMMAND_START           0x01
> >  #define AMDKFD_COMMAND_END             0x28
> >
> > +#define AMDKFD_IOC_PROFILER                    \
> > +               AMDKFD_IOWR(0x86, struct kfd_ioctl_profiler_args)
> > +
> > +#define AMDKFD_COMMAND_START_2         0x80
> > +#define AMDKFD_COMMAND_END_2           0x87
> > +
> >  #endif
> > --
> > 2.34.1
> >

RE: [PATCH v2 01/13] Add kfd_ioctl_profiler to contain profiler kernel driver changes

[Yuan, Perry]

[AMD Official Use Only - AMD Internal Distribution Only]

Hi Alex,

> -----Original Message-----
> From: Alex Deucher <alexdeucher@gmail.com>
> Sent: Monday, February 9, 2026 11:56 PM
> To: Yuan, Perry <Perry.Yuan@amd.com>
> Cc: amd-gfx@lists.freedesktop.org; Deucher, Alexander
> <Alexander.Deucher@amd.com>; Zhang, Yifan <Yifan1.Zhang@amd.com>
> Subject: Re: [PATCH v2 01/13] Add kfd_ioctl_profiler to contain profiler kernel driver
> changes
>
> On Mon, Feb 9, 2026 at 9:47 AM Alex Deucher <alexdeucher@gmail.com> wrote:
> >
> > On Sun, Feb 8, 2026 at 11:52 AM Perry Yuan <perry.yuan@amd.com> wrote:
> > >
> > > From: Benjamin Welton <bewelton@amd.com>
> > >
> > > kfd_ioctl_profiler takes a similar approach to that of
> > > kfd_ioctl_dbg_trap (which contains debugger related IOCTL
> > > services) where kfd_ioctl_profiler will contain all profiler related
> > > IOCTL services. The IOCTL is designed to be expanded as needed to
> > > support additional profiler functionality.
> > >
> > > The current functionality of the IOCTL is to allow for profilers
> > > which need PMC counters from GPU devices to both signal to other
> > > profilers that may be on the system that the device has active PMC
> > > profiling taking place on it (multiple PMC profilers on the same
> > > device can result in corrupted counter data) and to setup the device
> > > to allow for the collection of SQ PMC data on all queues on the device.
> > >
> > > For PMC data for the SQ block (such as SQ_WAVES) to be available to
> > > a profiler, mmPERFCOUNT_ENABLE must be set on the queues. When
> > > profiling a single process, the profiler can inject PM4 packets into
> > > each queue to turn on PERFCOUNT_ENABLE. When profiling system wide,
> > > the profiler does not have this option and must have a way to turn
> > > on profiling for queues in which it cannot inject packets into directly.
> > >
> > > Accomplishing this requires a few steps:
> > >
> > > 1. Checking if the user has the necessary permissions to profile system
> > >    wide on the device. This check uses the same check that linux perf
> > >    uses to determine if a user has the necessary permissions to profile
> > >    at this scope (primarily if the process has CAP_SYS_PERFMON or is root).
> > >
> > > 2. Locking the device for profiling. This is done by setting a lock bit
> > >    on the device struct and storing the process that locked the device.
> > >
> > > 3. Iterating all queues on the device and issuing an MQD Update to enable
> > >    perfcounting on the queues.
> > >
> > > 4. Actions to cleanup if the process exits or releases the lock.
> > >
> > > The IOCTL also contains a link to the existing PC Sampling IOCTL as well.
> > > This is per a suggestion that we should potentially remove the PC
> > > Sampling IOCTL to have it be a part of the profiler IOCTL. This is a future
> change.
> > > In addition, we do expect to expand the profiler IOCTL to include
> > > additional profiler functionality in the future (which necessitates
> > > the use of a version number).
> > >
> > > Signed-off-by: Benjamin Welton <benjamin.welton@amd.com>
> > > Signed-off-by: Perry Yuan <perry.yuan@amd.com>
> > > Acked-by: Kent Russell <kent.russell@amd.com>
> > > Reviewed-by: Yifan Zhang <yifan1.zhang@amd.com>
> >
> > Please include the link to the usermode stuff that uses this new IOCTL.
>
> For all of the patches which add or modify IOCTLs, we need a link to the userspace
> which uses the interface.  With that fixed, the series
> is:
> Acked-by: Alex Deucher <alexander.deucher@amd.com>


Thank you,  will check and add it.

>
> >
> > Alex
> >
> > > ---
> > >  drivers/gpu/drm/amd/amdkfd/kfd_chardev.c      | 82 +++++++++++++++++++
> > >  drivers/gpu/drm/amd/amdkfd/kfd_device.c       |  4 +
> > >  .../drm/amd/amdkfd/kfd_device_queue_manager.c | 25 ++++++
> > > .../drm/amd/amdkfd/kfd_device_queue_manager.h |  2 +
> > > .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c  | 16 +++-
> > > .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c  | 14 +++-
> > > .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c  |  8 +-
> > >  .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c   | 15 +++-
> > >  .../gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c   | 11 +++
> > >  drivers/gpu/drm/amd/amdkfd/kfd_priv.h         |  7 ++
> > >  drivers/gpu/drm/amd/amdkfd/kfd_process.c      | 11 +++
> > >  include/uapi/linux/kfd_ioctl.h                | 29 +++++++
> > >  12 files changed, 216 insertions(+), 8 deletions(-)
> > >
> > > diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
> > > b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
> > > index 732ad1224a61..dbb111a33678 100644
> > > --- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
> > > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
> > > @@ -21,6 +21,7 @@
> > >   * OTHER DEALINGS IN THE SOFTWARE.
> > >   */
> > >
> > > +#include <linux/capability.h>
> > >  #include <linux/device.h>
> > >  #include <linux/err.h>
> > >  #include <linux/fs.h>
> > > @@ -3204,6 +3205,84 @@ static int kfd_ioctl_create_process(struct file *filep,
> struct kfd_process *p, v
> > >         return 0;
> > >  }
> > >
> > > +static inline uint32_t profile_lock_device(struct kfd_process *p,
> > > +                                          uint32_t gpu_id, uint32_t
> > > +op) {
> > > +       struct kfd_process_device *pdd;
> > > +       struct kfd_dev *kfd;
> > > +       int status = -EINVAL;
> > > +
> > > +       if (!p)
> > > +               return -EINVAL;
> > > +
> > > +       mutex_lock(&p->mutex);
> > > +       pdd = kfd_process_device_data_by_id(p, gpu_id);
> > > +       mutex_unlock(&p->mutex);
> > > +
> > > +       if (!pdd || !pdd->dev || !pdd->dev->kfd)
> > > +               return -EINVAL;
> > > +
> > > +       kfd = pdd->dev->kfd;
> > > +
> > > +       mutex_lock(&kfd->profiler_lock);
> > > +       if (op == 1) {
> > > +               if (!kfd->profiler_process) {
> > > +                       kfd->profiler_process = p;
> > > +                       status = 0;
> > > +               } else if (kfd->profiler_process == p) {
> > > +                       status = -EALREADY;
> > > +               } else {
> > > +                       status = -EBUSY;
> > > +               }
> > > +       } else if (op == 0 && kfd->profiler_process == p) {
> > > +               kfd->profiler_process = NULL;
> > > +               status = 0;
> > > +       }
> > > +       mutex_unlock(&kfd->profiler_lock);
> > > +
> > > +       return status;
> > > +}
> > > +
> > > +static inline int kfd_profiler_pmc(struct kfd_process *p,
> > > +                                  struct kfd_ioctl_pmc_settings
> > > +*args) {
> > > +       struct kfd_process_device *pdd;
> > > +       struct device_queue_manager *dqm;
> > > +       int status;
> > > +
> > > +       /* Check if we have the correct permissions. */
> > > +       if (!perfmon_capable())
> > > +               return -EPERM;
> > > +
> > > +       /* Lock/Unlock the device based on the parameter given in OP */
> > > +       status = profile_lock_device(p, args->gpu_id, args->lock);
> > > +       if (status != 0)
> > > +               return status;
> > > +
> > > +       /* Enable/disable perfcount if requested */
> > > +       mutex_lock(&p->mutex);
> > > +       pdd = kfd_process_device_data_by_id(p, args->gpu_id);
> > > +       dqm = pdd->dev->dqm;
> > > +       mutex_unlock(&p->mutex);
> > > +
> > > +       dqm->ops.set_perfcount(dqm, args->perfcount_enable);
> > > +       return status;
> > > +}
> > > +
> > > +static int kfd_ioctl_profiler(struct file *filep, struct
> > > +kfd_process *p, void *data) {
> > > +       struct kfd_ioctl_profiler_args *args = data;
> > > +
> > > +       switch (args->op) {
> > > +       case KFD_IOC_PROFILER_VERSION:
> > > +               args->version = KFD_IOC_PROFILER_VERSION_NUM;
> > > +               return 0;
> > > +       case KFD_IOC_PROFILER_PMC:
> > > +               return kfd_profiler_pmc(p, &args->pmc);
> > > +       }
> > > +       return -EINVAL;
> > > +}
> > > +
> > >  #define AMDKFD_IOCTL_DEF(ioctl, _func, _flags) \
> > >         [_IOC_NR(ioctl)] = {.cmd = ioctl, .func = _func, .flags = _flags, \
> > >                             .cmd_drv = 0, .name = #ioctl} @@ -3325,6
> > > +3404,9 @@ static const struct amdkfd_ioctl_desc amdkfd_ioctls[] = {
> > >
> > >         AMDKFD_IOCTL_DEF(AMDKFD_IOC_CREATE_PROCESS,
> > >                         kfd_ioctl_create_process, 0),
> > > +
> > > +       AMDKFD_IOCTL_DEF(AMDKFD_IOC_PROFILER,
> > > +                       kfd_ioctl_profiler, 0),
> > >  };
> > >
> > >  #define AMDKFD_CORE_IOCTL_COUNT        ARRAY_SIZE(amdkfd_ioctls)
> > > diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device.c
> > > b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
> > > index 9a66ee661e57..f231e46e8528 100644
> > > --- a/drivers/gpu/drm/amd/amdkfd/kfd_device.c
> > > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
> > > @@ -936,6 +936,9 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd,
> > >
> > >         svm_range_set_max_pages(kfd->adev);
> > >
> > > +       kfd->profiler_process = NULL;
> > > +       mutex_init(&kfd->profiler_lock);
> > > +
> > >         kfd->init_complete = true;
> > >         dev_info(kfd_device, "added device %x:%x\n", kfd->adev->pdev->vendor,
> > >                  kfd->adev->pdev->device); @@ -971,6 +974,7 @@ void
> > > kgd2kfd_device_exit(struct kfd_dev *kfd)
> > >                 ida_destroy(&kfd->doorbell_ida);
> > >                 kfd_gtt_sa_fini(kfd);
> > >                 amdgpu_amdkfd_free_kernel_mem(kfd->adev,
> > > &kfd->gtt_mem);
> > > +               mutex_destroy(&kfd->profiler_lock);
> > >         }
> > >
> > >         kfree(kfd);
> > > diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> > > b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> > > index 804851632c4c..4170a283db5b 100644
> > > --- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> > > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> > > @@ -305,6 +305,29 @@ static int remove_queue_mes(struct
> device_queue_manager *dqm, struct queue *q,
> > >         return r;
> > >  }
> > >
> > > +static void set_perfcount(struct device_queue_manager *dqm, int
> > > +enable) {
> > > +       struct device_process_node *cur;
> > > +       struct qcm_process_device *qpd;
> > > +       struct queue *q;
> > > +       struct mqd_update_info minfo = { 0 };
> > > +
> > > +       if (!dqm)
> > > +               return;
> > > +
> > > +       minfo.update_flag = (enable == 1 ?
> UPDATE_FLAG_PERFCOUNT_ENABLE :
> > > +                                                UPDATE_FLAG_PERFCOUNT_DISABLE);
> > > +       dqm_lock(dqm);
> > > +       list_for_each_entry(cur, &dqm->queues, list) {
> > > +               qpd = cur->qpd;
> > > +               list_for_each_entry(q, &qpd->queues_list, list) {
> > > +                       pqm_update_mqd(qpd->pqm, q->properties.queue_id,
> > > +                                               &minfo);
> > > +               }
> > > +       }
> > > +       dqm_unlock(dqm);
> > > +}
> > > +
> > >  static int remove_all_kfd_queues_mes(struct device_queue_manager
> > > *dqm)  {
> > >         struct device_process_node *cur; @@ -2967,6 +2990,7 @@
> > > struct device_queue_manager *device_queue_manager_init(struct kfd_node
> *dev)
> > >                 dqm->ops.reset_queues = reset_queues_cpsch;
> > >                 dqm->ops.get_queue_checkpoint_info = get_queue_checkpoint_info;
> > >                 dqm->ops.checkpoint_mqd = checkpoint_mqd;
> > > +               dqm->ops.set_perfcount = set_perfcount;
> > >                 break;
> > >         case KFD_SCHED_POLICY_NO_HWS:
> > >                 /* initialize dqm for no cp scheduling */ @@ -2987,6
> > > +3011,7 @@ struct device_queue_manager
> *device_queue_manager_init(struct kfd_node *dev)
> > >                 dqm->ops.get_wave_state = get_wave_state;
> > >                 dqm->ops.get_queue_checkpoint_info = get_queue_checkpoint_info;
> > >                 dqm->ops.checkpoint_mqd = checkpoint_mqd;
> > > +               dqm->ops.set_perfcount = set_perfcount;
> > >                 break;
> > >         default:
> > >                 dev_err(dev->adev->dev, "Invalid scheduling policy
> > > %d\n", dqm->sched_policy); diff --git
> > > a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h
> > > b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h
> > > index ef07e44916f8..74a3bcec3e4e 100644
> > > --- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h
> > > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h
> > > @@ -200,6 +200,8 @@ struct device_queue_manager_ops {
> > >                                   const struct queue *q,
> > >                                   void *mqd,
> > >                                   void *ctl_stack);
> > > +       void    (*set_perfcount)(struct device_queue_manager *dqm,
> > > +                                 int enable);
> > >  };
> > >
> > >  struct device_queue_manager_asic_ops { diff --git
> > > a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c
> > > b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c
> > > index 97055f808d4a..993d60a24d50 100644
> > > --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c
> > > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c
> > > @@ -124,10 +124,9 @@ static void init_mqd(struct mqd_manager *mm, void
> **mqd,
> > >          */
> > >         m->cp_hqd_hq_scheduler0 = 1 << 14;
> > >
> > > -       if (q->format == KFD_QUEUE_FORMAT_AQL) {
> > > +       if (q->format == KFD_QUEUE_FORMAT_AQL)
> > >                 m->cp_hqd_aql_control =
> > >                         1 << CP_HQD_AQL_CONTROL__CONTROL0__SHIFT;
> > > -       }
> > >
> > >         if (mm->dev->kfd->cwsr_enabled) {
> > >                 m->cp_hqd_persistent_state |= @@ -142,6 +141,12 @@
> > > static void init_mqd(struct mqd_manager *mm, void **mqd,
> > >                 m->cp_hqd_wg_state_offset = q->ctl_stack_size;
> > >         }
> > >
> > > +       mutex_lock(&mm->dev->kfd->profiler_lock);
> > > +       if (mm->dev->kfd->profiler_process != NULL)
> > > +               m->compute_perfcount_enable = 1;
> > > +
> > > +       mutex_unlock(&mm->dev->kfd->profiler_lock);
> > > +
> > >         *mqd = m;
> > >         if (gart_addr)
> > >                 *gart_addr = addr;
> > > @@ -221,6 +226,13 @@ static void update_mqd(struct mqd_manager *mm,
> void *mqd,
> > >         if (mm->dev->kfd->cwsr_enabled)
> > >                 m->cp_hqd_ctx_save_control = 0;
> > >
> > > +       if (minfo) {
> > > +               if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_ENABLE)
> > > +                       m->compute_perfcount_enable = 1;
> > > +               else if (minfo->update_flag ==
> UPDATE_FLAG_PERFCOUNT_DISABLE)
> > > +                       m->compute_perfcount_enable = 0;
> > > +       }
> > > +
> > >         update_cu_mask(mm, mqd, minfo);
> > >         set_priority(m, q);
> > >
> > > diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c
> > > b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c
> > > index 7e5a7ab6d0c0..4a574bbb5f37 100644
> > > --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c
> > > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c
> > > @@ -164,10 +164,9 @@ static void init_mqd(struct mqd_manager *mm, void
> **mqd,
> > >         if (amdgpu_amdkfd_have_atomics_support(mm->dev->adev))
> > >                 m->cp_hqd_hq_status0 |= 1 << 29;
> > >
> > > -       if (q->format == KFD_QUEUE_FORMAT_AQL) {
> > > +       if (q->format == KFD_QUEUE_FORMAT_AQL)
> > >                 m->cp_hqd_aql_control =
> > >                         1 << CP_HQD_AQL_CONTROL__CONTROL0__SHIFT;
> > > -       }
> > >
> > >         if (mm->dev->kfd->cwsr_enabled) {
> > >                 m->cp_hqd_persistent_state |= @@ -182,6 +181,11 @@
> > > static void init_mqd(struct mqd_manager *mm, void **mqd,
> > >                 m->cp_hqd_wg_state_offset = q->ctl_stack_size;
> > >         }
> > >
> > > +       mutex_lock(&mm->dev->kfd->profiler_lock);
> > > +       if (mm->dev->kfd->profiler_process != NULL)
> > > +               m->compute_perfcount_enable = 1;
> > > +       mutex_unlock(&mm->dev->kfd->profiler_lock);
> > > +
> > >         *mqd = m;
> > >         if (gart_addr)
> > >                 *gart_addr = addr;
> > > @@ -259,6 +263,12 @@ static void update_mqd(struct mqd_manager *mm,
> void *mqd,
> > >         }
> > >         if (mm->dev->kfd->cwsr_enabled)
> > >                 m->cp_hqd_ctx_save_control = 0;
> > > +       if (minfo) {
> > > +               if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_ENABLE)
> > > +                       m->compute_perfcount_enable = 1;
> > > +               else if (minfo->update_flag ==
> UPDATE_FLAG_PERFCOUNT_DISABLE)
> > > +                       m->compute_perfcount_enable = 0;
> > > +       }
> > >
> > >         update_cu_mask(mm, mqd, minfo);
> > >         set_priority(m, q);
> > > diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c
> > > b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c
> > > index a51f217329db..7173f6470e5e 100644
> > > --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c
> > > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c
> > > @@ -139,10 +139,9 @@ static void init_mqd(struct mqd_manager *mm, void
> **mqd,
> > >         if (amdgpu_amdkfd_have_atomics_support(mm->dev->adev))
> > >                 m->cp_hqd_hq_status0 |= 1 << 29;
> > >
> > > -       if (q->format == KFD_QUEUE_FORMAT_AQL) {
> > > +       if (q->format == KFD_QUEUE_FORMAT_AQL)
> > >                 m->cp_hqd_aql_control =
> > >                         1 << CP_HQD_AQL_CONTROL__CONTROL0__SHIFT;
> > > -       }
> > >
> > >         if (mm->dev->kfd->cwsr_enabled) {
> > >                 m->cp_hqd_persistent_state |= @@ -157,6 +156,11 @@
> > > static void init_mqd(struct mqd_manager *mm, void **mqd,
> > >                 m->cp_hqd_wg_state_offset = q->ctl_stack_size;
> > >         }
> > >
> > > +       mutex_lock(&mm->dev->kfd->profiler_lock);
> > > +       if (mm->dev->kfd->profiler_process != NULL)
> > > +               m->compute_perfcount_enable = 1;
> > > +       mutex_unlock(&mm->dev->kfd->profiler_lock);
> > > +
> > >         *mqd = m;
> > >         if (gart_addr)
> > >                 *gart_addr = addr;
> > > diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
> > > b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
> > > index dcf4bbfa641b..d4659a438be5 100644
> > > --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
> > > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
> > > @@ -218,10 +218,9 @@ static void init_mqd(struct mqd_manager *mm, void
> **mqd,
> > >                 m->cp_hqd_aql_control =
> > >                         1 << CP_HQD_AQL_CONTROL__CONTROL0__SHIFT;
> > >
> > > -       if (q->tba_addr) {
> > > +       if (q->tba_addr)
> > >                 m->compute_pgm_rsrc2 |=
> > >                         (1 << COMPUTE_PGM_RSRC2__TRAP_PRESENT__SHIFT);
> > > -       }
> > >
> > >         if (mm->dev->kfd->cwsr_enabled && q->ctx_save_restore_area_address)
> {
> > >                 m->cp_hqd_persistent_state |= @@ -236,6 +235,11 @@
> > > static void init_mqd(struct mqd_manager *mm, void **mqd,
> > >                 m->cp_hqd_wg_state_offset = q->ctl_stack_size;
> > >         }
> > >
> > > +       mutex_lock(&mm->dev->kfd->profiler_lock);
> > > +       if (mm->dev->kfd->profiler_process != NULL)
> > > +               m->compute_perfcount_enable = 1;
> > > +       mutex_unlock(&mm->dev->kfd->profiler_lock);
> > > +
> > >         *mqd = m;
> > >         if (gart_addr)
> > >                 *gart_addr = addr;
> > > @@ -318,6 +322,13 @@ static void update_mqd(struct mqd_manager *mm,
> void *mqd,
> > >         if (mm->dev->kfd->cwsr_enabled && q->ctx_save_restore_area_address)
> > >                 m->cp_hqd_ctx_save_control = 0;
> > >
> > > +       if (minfo) {
> > > +               if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_ENABLE)
> > > +                       m->compute_perfcount_enable = 1;
> > > +               else if (minfo->update_flag ==
> UPDATE_FLAG_PERFCOUNT_DISABLE)
> > > +                       m->compute_perfcount_enable = 0;
> > > +       }
> > > +
> > >         if (KFD_GC_VERSION(mm->dev) != IP_VERSION(9, 4, 3) &&
> > >             KFD_GC_VERSION(mm->dev) != IP_VERSION(9, 4, 4) &&
> > >             KFD_GC_VERSION(mm->dev) != IP_VERSION(9, 5, 0)) diff
> > > --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
> > > b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
> > > index 09483f0862d4..e8967f5e3892 100644
> > > --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
> > > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
> > > @@ -149,6 +149,11 @@ static void init_mqd(struct mqd_manager *mm, void
> **mqd,
> > >                 m->cp_hqd_wg_state_offset = q->ctl_stack_size;
> > >         }
> > >
> > > +       mutex_lock(&mm->dev->kfd->profiler_lock);
> > > +       if (mm->dev->kfd->profiler_process != NULL)
> > > +               m->compute_perfcount_enable = 1;
> > > +       mutex_unlock(&mm->dev->kfd->profiler_lock);
> > > +
> > >         *mqd = m;
> > >         if (gart_addr)
> > >                 *gart_addr = addr;
> > > @@ -231,6 +236,12 @@ static void __update_mqd(struct mqd_manager *mm,
> void *mqd,
> > >                 m->cp_hqd_ctx_save_control =
> > >                         atc_bit << CP_HQD_CTX_SAVE_CONTROL__ATC__SHIFT |
> > >                         mtype <<
> > > CP_HQD_CTX_SAVE_CONTROL__MTYPE__SHIFT;
> > > +       if (minfo) {
> > > +               if (minfo->update_flag == UPDATE_FLAG_PERFCOUNT_ENABLE)
> > > +                       m->compute_perfcount_enable = 1;
> > > +               else if (minfo->update_flag ==
> UPDATE_FLAG_PERFCOUNT_DISABLE)
> > > +                       m->compute_perfcount_enable = 0;
> > > +       }
> > >
> > >         update_cu_mask(mm, mqd, minfo);
> > >         set_priority(m, q);
> > > diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> > > b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> > > index 9849b54f54ba..8983065645fa 100644
> > > --- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> > > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
> > > @@ -384,6 +384,11 @@ struct kfd_dev {
> > >         int kfd_dev_lock;
> > >
> > >         atomic_t kfd_processes_count;
> > > +
> > > +       /* Lock for profiler process */
> > > +       struct mutex profiler_lock;
> > > +       /* Process currently holding the lock */
> > > +       struct kfd_process *profiler_process;
> > >  };
> > >
> > >  enum kfd_mempool {
> > > @@ -556,6 +561,8 @@ enum mqd_update_flag {
> > >         UPDATE_FLAG_DBG_WA_ENABLE = 1,
> > >         UPDATE_FLAG_DBG_WA_DISABLE = 2,
> > >         UPDATE_FLAG_IS_GWS = 4, /* quirk for gfx9 IP */
> > > +       UPDATE_FLAG_PERFCOUNT_ENABLE = 5,
> > > +       UPDATE_FLAG_PERFCOUNT_DISABLE = 6,
> > >  };
> > >
> > >  struct mqd_update_info {
> > > diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> > > b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> > > index 8511fbebf327..deca19b478d0 100644
> > > --- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> > > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
> > > @@ -1110,6 +1110,16 @@ static void
> kfd_process_free_outstanding_kfd_bos(struct kfd_process *p)
> > >                 kfd_process_device_free_bos(p->pdds[i]);
> > >  }
> > >
> > > +static void kfd_process_profiler_release(struct kfd_process *p,
> > > +struct kfd_process_device *pdd) {
> > > +       mutex_lock(&pdd->dev->kfd->profiler_lock);
> > > +       if (pdd->dev->kfd->profiler_process == p) {
> > > +               pdd->qpd.dqm->ops.set_perfcount(pdd->qpd.dqm, 0);
> > > +               pdd->dev->kfd->profiler_process = NULL;
> > > +       }
> > > +       mutex_unlock(&pdd->dev->kfd->profiler_lock);
> > > +}
> > > +
> > >  static void kfd_process_destroy_pdds(struct kfd_process *p)  {
> > >         int i;
> > > @@ -1121,6 +1131,7 @@ static void kfd_process_destroy_pdds(struct
> > > kfd_process *p)
> > >
> > >                 pr_debug("Releasing pdd (topology id %d, for pid %d)\n",
> > >                         pdd->dev->id, p->lead_thread->pid);
> > > +               kfd_process_profiler_release(p, pdd);
> > >                 kfd_process_device_destroy_cwsr_dgpu(pdd);
> > >                 kfd_process_device_destroy_ib_mem(pdd);
> > >
> > > diff --git a/include/uapi/linux/kfd_ioctl.h
> > > b/include/uapi/linux/kfd_ioctl.h index e72359370857..a8b2a18d07cf
> > > 100644
> > > --- a/include/uapi/linux/kfd_ioctl.h
> > > +++ b/include/uapi/linux/kfd_ioctl.h
> > > @@ -1558,6 +1558,29 @@ struct kfd_ioctl_dbg_trap_args {
> > >         };
> > >  };
> > >
> > > +#define KFD_IOC_PROFILER_VERSION_NUM 1 enum kfd_profiler_ops {
> > > +       KFD_IOC_PROFILER_PMC = 0,
> > > +       KFD_IOC_PROFILER_VERSION = 2, };
> > > +
> > > +/**
> > > + * Enables/Disables GPU Specific profiler settings  */ struct
> > > +kfd_ioctl_pmc_settings {
> > > +       __u32 gpu_id;             /* This is the user_gpu_id */
> > > +       __u32 lock;               /* Lock GPU for Profiling */
> > > +       __u32 perfcount_enable;   /* Force Perfcount Enable for queues on GPU
> */
> > > +};
> > > +
> > > +struct kfd_ioctl_profiler_args {
> > > +       __u32 op;                                               /* kfd_profiler_op */
> > > +       union {
> > > +               struct kfd_ioctl_pmc_settings  pmc;
> > > +               __u32 version;                          /*
> KFD_IOC_PROFILER_VERSION_NUM */
> > > +       };
> > > +};
> > > +
> > >  #define AMDKFD_IOCTL_BASE 'K'
> > >  #define AMDKFD_IO(nr)                  _IO(AMDKFD_IOCTL_BASE, nr)
> > >  #define AMDKFD_IOR(nr, type)           _IOR(AMDKFD_IOCTL_BASE, nr,
> type)
> > > @@ -1684,4 +1707,10 @@ struct kfd_ioctl_dbg_trap_args {
> > >  #define AMDKFD_COMMAND_START           0x01
> > >  #define AMDKFD_COMMAND_END             0x28
> > >
> > > +#define AMDKFD_IOC_PROFILER                    \
> > > +               AMDKFD_IOWR(0x86, struct kfd_ioctl_profiler_args)
> > > +
> > > +#define AMDKFD_COMMAND_START_2         0x80
> > > +#define AMDKFD_COMMAND_END_2           0x87
> > > +
> > >  #endif
> > > --
> > > 2.34.1
> > >

[PATCH v2 02/13] drm/amdgpu: add new performance monitor PSP interfaces

[Perry Yuan]

Introduce new psp interfaces and structures for performance
monitoring hardware control.

Signed-off-by: Perry Yuan <perry.yuan@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h | 21 +++++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/psp_gfx_if.h | 17 +++++++++++++++++
 2 files changed, 38 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
index 79a49cba8d40..58f691db2e97 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
@@ -359,6 +359,27 @@ struct spirom_bo {
 };
 #endif
 
+enum psp_ptl_cmd {
+	PSP_PTL_PERF_MON_QUERY = 0xA0000000,
+	PSP_PTL_PERF_MON_SET = 0xA0000001,
+};
+
+enum psp_ptl_format_type {
+	GFX_FTYPE_I8          = 0x00000000,
+	GFX_FTYPE_F16         = 0x00000001,
+	GFX_FTYPE_BF16        = 0x00000002,
+	GFX_FTYPE_F32         = 0x00000003,
+	GFX_FTYPE_F64         = 0x00000004,
+	GFX_FTYPE_INVALID     = 0xFFFFFFFF,
+};
+
+struct psp_ptl_perf_req {
+	enum psp_ptl_cmd req;
+	uint32_t ptl_state;
+	uint32_t pref_format1;
+	uint32_t pref_format2;
+};
+
 struct psp_context {
 	struct amdgpu_device		*adev;
 	struct psp_ring			km_ring;
diff --git a/drivers/gpu/drm/amd/amdgpu/psp_gfx_if.h b/drivers/gpu/drm/amd/amdgpu/psp_gfx_if.h
index e8f768638fd5..ac34bac3c839 100644
--- a/drivers/gpu/drm/amd/amdgpu/psp_gfx_if.h
+++ b/drivers/gpu/drm/amd/amdgpu/psp_gfx_if.h
@@ -107,6 +107,7 @@ enum psp_gfx_cmd_id
     GFX_CMD_ID_CONFIG_SQ_PERFMON  = 0x00000046,   /* Config CGTT_SQ_CLK_CTRL */
     /* Dynamic memory partitioninig (NPS mode change)*/
     GFX_CMD_ID_FB_NPS_MODE        = 0x00000048,  /* Configure memory partitioning mode */
+    GFX_CMD_ID_PERF_HW            = 0x0000004C,   /* performance monitor */
     GFX_CMD_ID_FB_FW_RESERV_ADDR  = 0x00000050,  /* Query FW reservation addr */
     GFX_CMD_ID_FB_FW_RESERV_EXT_ADDR = 0x00000051,  /* Query FW reservation extended addr */
 };
@@ -373,6 +374,13 @@ struct psp_gfx_cmd_fb_memory_part {
 	uint32_t resvd;
 };
 
+struct psp_gfx_cmd_req_perf_hw {
+	uint32_t req;
+	uint32_t ptl_state;
+	uint32_t pref_format1;
+	uint32_t pref_format2;
+};
+
 /* All GFX ring buffer commands. */
 union psp_gfx_commands
 {
@@ -389,6 +397,7 @@ union psp_gfx_commands
     struct psp_gfx_cmd_sriov_spatial_part cmd_spatial_part;
     struct psp_gfx_cmd_config_sq_perfmon config_sq_perfmon;
     struct psp_gfx_cmd_fb_memory_part cmd_memory_part;
+    struct psp_gfx_cmd_req_perf_hw     cmd_req_perf_hw;
 };
 
 struct psp_gfx_uresp_reserved
@@ -415,12 +424,20 @@ struct psp_gfx_uresp_fw_reserve_info {
     uint32_t reserve_size;
 };
 
+struct psp_gfx_uresp_perf_hw {
+	uint32_t resp;
+	uint32_t ptl_state;
+	uint32_t pref_format1;
+	uint32_t pref_format2;
+};
+
 /* Union of command-specific responses for GPCOM ring. */
 union psp_gfx_uresp {
 	struct psp_gfx_uresp_reserved		reserved;
 	struct psp_gfx_uresp_bootcfg		boot_cfg;
 	struct psp_gfx_uresp_fwar_db_info	fwar_db_info;
 	struct psp_gfx_uresp_fw_reserve_info	fw_reserve_info;
+	struct psp_gfx_uresp_perf_hw		perf_hw_info;
 };
 
 /* Structure of GFX Response buffer.
-- 
2.34.1

[PATCH v2 03/13] drm/amdgpu: add psp interfaces for peak tops limiter driver

[Perry Yuan]

Introduce a Peak Tops Limiter (PTL) driver that dynamically caps
engine frequency to ensure delivered TOPS never exceeds a defined
TOPS_limit. This initial implementation provides core data structures
and kernel-space interfaces (set/get, enable/disable) to manage PTL state.

PTL performs a firmware handshake to initialize its state and update
predefined format types. It supports updating these format types at
runtime while user-space tools automatically switch PTL state, and
also allows explicitly switching PTL state via newly added commands.

Signed-off-by: Perry Yuan <perry.yuan@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h    |   2 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c    |   1 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c       | 112 ++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h       |   3 +-
 drivers/gpu/drm/amd/amdkfd/kfd_chardev.c      |   1 +
 drivers/gpu/drm/amd/include/amdgpu_ptl.h      |  48 ++++++++
 .../gpu/drm/amd/include/kgd_kfd_interface.h   |   1 +
 7 files changed, 167 insertions(+), 1 deletion(-)
 create mode 100644 drivers/gpu/drm/amd/include/amdgpu_ptl.h

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
index cdbab7f8cee8..4fee011c2e26 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
@@ -38,6 +38,8 @@
 #include "amdgpu_vm.h"
 #include "amdgpu_xcp.h"
 #include "kfd_topology.h"
+#include "amdgpu_ptl.h"
+
 extern uint64_t amdgpu_amdkfd_total_mem_size;
 
 enum TLB_FLUSH_TYPE {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 5e73b9d67325..a07fe386d275 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -4440,6 +4440,7 @@ int amdgpu_device_init(struct amdgpu_device *adev,
 	mutex_init(&adev->virt.vf_errors.lock);
 	hash_init(adev->mn_hash);
 	mutex_init(&adev->psp.mutex);
+	mutex_init(&adev->psp.ptl.mutex);
 	mutex_init(&adev->notifier_lock);
 	mutex_init(&adev->pm.stable_pstate_ctx_lock);
 	mutex_init(&adev->benchmark_mutex);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
index b0540b009e84..19b81f515374 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
@@ -679,6 +679,8 @@ static const char *psp_gfx_cmd_name(enum psp_gfx_cmd_id cmd_id)
 		return "SPATIAL_PARTITION";
 	case GFX_CMD_ID_FB_NPS_MODE:
 		return "NPS_MODE_CHANGE";
+	case GFX_CMD_ID_PERF_HW:
+		return "PERF MONITORING HW";
 	default:
 		return "UNKNOWN CMD";
 	}
@@ -1197,6 +1199,116 @@ int psp_memory_partition(struct psp_context *psp, int mode)
 	return ret;
 }
 
+static int psp_ptl_fmt_verify(struct psp_context *psp, enum amdgpu_ptl_fmt fmt,
+						 uint32_t *ptl_fmt)
+{
+	struct amdgpu_device *adev = psp->adev;
+
+	if (amdgpu_ip_version(adev, GC_HWIP, 0) != IP_VERSION(9, 4, 4))
+		return -EINVAL;
+
+	switch (fmt) {
+	case AMDGPU_PTL_FMT_I8:
+		*ptl_fmt = GFX_FTYPE_I8;
+		break;
+	case AMDGPU_PTL_FMT_F16:
+		*ptl_fmt = GFX_FTYPE_F16;
+		break;
+	case AMDGPU_PTL_FMT_BF16:
+		*ptl_fmt = GFX_FTYPE_BF16;
+		break;
+	case AMDGPU_PTL_FMT_F32:
+		*ptl_fmt = GFX_FTYPE_F32;
+		break;
+	case AMDGPU_PTL_FMT_F64:
+		*ptl_fmt = GFX_FTYPE_F64;
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static int psp_ptl_invoke(struct psp_context *psp, u32 req_code,
+		uint32_t *ptl_state, uint32_t *fmt1, uint32_t *fmt2)
+{
+	struct psp_gfx_cmd_resp *cmd;
+	struct amdgpu_ptl *ptl = &psp->ptl;
+	int ret;
+
+	cmd = acquire_psp_cmd_buf(psp);
+
+	cmd->cmd_id                     = GFX_CMD_ID_PERF_HW;
+	cmd->cmd.cmd_req_perf_hw.req    = req_code;
+	cmd->cmd.cmd_req_perf_hw.ptl_state    = *ptl_state;
+	cmd->cmd.cmd_req_perf_hw.pref_format1 = *fmt1;
+	cmd->cmd.cmd_req_perf_hw.pref_format2 = *fmt2;
+
+	ret = psp_cmd_submit_buf(psp, NULL, cmd, psp->fence_buf_mc_addr);
+	if (ret)
+		goto out;
+
+	/* Parse response */
+	switch (req_code) {
+	case PSP_PTL_PERF_MON_QUERY:
+		*ptl_state = cmd->resp.uresp.perf_hw_info.ptl_state;
+		*fmt1      = cmd->resp.uresp.perf_hw_info.pref_format1;
+		*fmt2      = cmd->resp.uresp.perf_hw_info.pref_format2;
+		break;
+	case PSP_PTL_PERF_MON_SET:
+		/* Update cached state only on success */
+		ptl->enabled = *ptl_state;
+		ptl->fmt1    = *fmt1;
+		ptl->fmt2    = *fmt2;
+		break;
+	}
+
+out:
+	release_psp_cmd_buf(psp);
+	return ret;
+}
+
+int amdgpu_ptl_perf_monitor_ctrl(struct amdgpu_device *adev, u32 req_code,
+				uint32_t *ptl_state,
+				enum amdgpu_ptl_fmt *fmt1,
+				enum amdgpu_ptl_fmt *fmt2)
+{
+	uint32_t ptl_fmt1, ptl_fmt2;
+	struct psp_context *psp;
+	struct amdgpu_ptl *ptl;
+
+	if (!adev || !ptl_state || !fmt1 || !fmt2)
+		return -EINVAL;
+
+	if (amdgpu_sriov_vf(adev))
+		return 0;
+
+	psp = &adev->psp;
+	ptl = &psp->ptl;
+
+	if (amdgpu_ip_version(adev, GC_HWIP, 0) != IP_VERSION(9, 4, 4) ||
+			psp->sos.fw_version < 0x0036081a)
+		return -EOPNOTSUPP;
+
+	/* Verify formats */
+	if (psp_ptl_fmt_verify(psp, *fmt1, &ptl_fmt1) ||
+			psp_ptl_fmt_verify(psp, *fmt2, &ptl_fmt2))
+		return -EINVAL;
+
+	/*
+	 * Add check to skip if state and formats are identical to current ones
+	 */
+	if (req_code == PSP_PTL_PERF_MON_SET &&
+			ptl->enabled == *ptl_state &&
+			ptl->fmt1 == ptl_fmt1 &&
+			ptl->fmt2 == ptl_fmt2)
+		return 0;
+
+	return psp_ptl_invoke(psp, req_code, ptl_state, &ptl_fmt1, &ptl_fmt2);
+}
+}
+
 int psp_spatial_partition(struct psp_context *psp, int mode)
 {
 	struct psp_gfx_cmd_resp *cmd;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
index 58f691db2e97..332633f6f4be 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
@@ -31,6 +31,7 @@
 #include "ta_ras_if.h"
 #include "ta_rap_if.h"
 #include "ta_secureDisplay_if.h"
+#include "amdgpu_ptl.h"
 
 #define PSP_FENCE_BUFFER_SIZE	0x1000
 #define PSP_CMD_BUFFER_SIZE	0x1000
@@ -470,6 +471,7 @@ struct psp_context {
 #if defined(CONFIG_DEBUG_FS)
 	struct spirom_bo *spirom_dump_trip;
 #endif
+	struct amdgpu_ptl		ptl;
 };
 
 struct amdgpu_psp_funcs {
@@ -653,5 +655,4 @@ void amdgpu_psp_debugfs_init(struct amdgpu_device *adev);
 int amdgpu_psp_get_fw_type(struct amdgpu_firmware_info *ucode,
 			   enum psp_gfx_fw_type *type);
 
-
 #endif
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index dbb111a33678..9c37e8248540 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -44,6 +44,7 @@
 #include "kfd_smi_events.h"
 #include "amdgpu_dma_buf.h"
 #include "kfd_debug.h"
+#include "amdgpu_ptl.h"
 
 static long kfd_ioctl(struct file *, unsigned int, unsigned long);
 static int kfd_open(struct inode *, struct file *);
diff --git a/drivers/gpu/drm/amd/include/amdgpu_ptl.h b/drivers/gpu/drm/amd/include/amdgpu_ptl.h
new file mode 100644
index 000000000000..12c9e0b4645a
--- /dev/null
+++ b/drivers/gpu/drm/amd/include/amdgpu_ptl.h
@@ -0,0 +1,48 @@
+/*
+ * Copyright 2026 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+
+#ifndef __AMDGPU_PTL_H__
+#define __AMDGPU_PTL_H__
+
+enum amdgpu_ptl_fmt {
+	AMDGPU_PTL_FMT_I8   = 0,
+	AMDGPU_PTL_FMT_F16  = 1,
+	AMDGPU_PTL_FMT_BF16 = 2,
+	AMDGPU_PTL_FMT_F32  = 3,
+	AMDGPU_PTL_FMT_F64  = 4,
+	AMDGPU_PTL_FMT_F8   = 5,
+	AMDGPU_PTL_FMT_VECTOR  = 6,
+	AMDGPU_PTL_FMT_INVALID = 7,
+};
+
+struct amdgpu_ptl {
+	enum amdgpu_ptl_fmt		fmt1;
+	enum amdgpu_ptl_fmt		fmt2;
+	bool				enabled;
+	bool				hw_supported;
+	struct mutex			mutex;
+};
+
+int amdgpu_ptl_perf_monitor_ctrl(struct amdgpu_device *adev, u32 req_code,
+		u32 *ptl_state, u32 *fmt1, u32 *fmt2);
+
+#endif /* __AMDGPU_PTL_H__ */
diff --git a/drivers/gpu/drm/amd/include/kgd_kfd_interface.h b/drivers/gpu/drm/amd/include/kgd_kfd_interface.h
index 9aba8596faa7..6df5afb242ae 100644
--- a/drivers/gpu/drm/amd/include/kgd_kfd_interface.h
+++ b/drivers/gpu/drm/amd/include/kgd_kfd_interface.h
@@ -33,6 +33,7 @@
 #include <linux/dma-fence.h>
 #include "amdgpu_irq.h"
 #include "amdgpu_gfx.h"
+#include "amdgpu_ptl.h"
 
 struct pci_dev;
 struct amdgpu_device;
-- 
2.34.1

[PATCH v2 04/13] drm/amdgpu: add PTL enable/query gfx control support for GC 9.4.4

[Perry Yuan]

Introduce hardware detection, runtime state tracking and a
kgd->ptl_ctrl() callback to enable/disable/query PTL via the
PSP performance-monitor interface (commands 0xA0000000/1).
The driver now exposes PTL capability to KFD and keeps the
software state in sync with the hardware.

Signed-off-by: Perry Yuan <perry.yuan@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c | 34 +++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c b/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c
index ad4d442e7345..aa9307d88fde 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c
@@ -2370,11 +2370,43 @@ static int gfx_v9_4_3_hw_init(struct amdgpu_ip_block *ip_block)
 	return r;
 }
 
+static int gfx_v9_4_3_perf_monitor_ptl_init(struct amdgpu_device *adev, bool enable)
+{
+	struct amdgpu_ptl *ptl = &adev->psp.ptl;
+	uint32_t ptl_state = enable ? 1 : 0;
+	uint32_t fmt1, fmt2;
+	int r;
+
+	if (!adev->psp.funcs)
+		return -EOPNOTSUPP;
+
+	if (!ptl->hw_supported) {
+		fmt1 = GFX_FTYPE_I8;
+		fmt2 = GFX_FTYPE_BF16;
+	} else {
+		fmt1 = ptl->fmt1;
+		fmt2 = ptl->fmt2;
+	}
+
+	/* initialize PTL with default formats: GFX_FTYPE_I8 & GFX_FTYPE_BF16 */
+	r = amdgpu_ptl_perf_monitor_ctrl(adev, PSP_PTL_PERF_MON_SET, &ptl_state,
+							&fmt1, &fmt2);
+	if (r)
+		return r;
+
+	ptl->hw_supported = true;
+
+	return 0;
+}
+
 static int gfx_v9_4_3_hw_fini(struct amdgpu_ip_block *ip_block)
 {
 	struct amdgpu_device *adev = ip_block->adev;
 	int i, num_xcc;
 
+	if (adev->psp.ptl.hw_supported)
+		gfx_v9_4_3_perf_monitor_ptl_init(adev, false);
+
 	amdgpu_irq_put(adev, &adev->gfx.priv_reg_irq, 0);
 	amdgpu_irq_put(adev, &adev->gfx.priv_inst_irq, 0);
 	amdgpu_irq_put(adev, &adev->gfx.bad_op_irq, 0);
@@ -2549,6 +2581,8 @@ static int gfx_v9_4_3_late_init(struct amdgpu_ip_block *ip_block)
 	    adev->gfx.ras->enable_watchdog_timer)
 		adev->gfx.ras->enable_watchdog_timer(adev);
 
+	gfx_v9_4_3_perf_monitor_ptl_init(adev, true);
+
 	return 0;
 }
 
-- 
2.34.1

[PATCH v2 05/13] drm/amdkfd: add kgd control interface for ptl

[Perry Yuan]

Add kgd->ptl_ctrl() callback so KFD can query/enable/disable
PTL state through the PSP performance monitor interface.

Signed-off-by: Perry Yuan <perry.yuan@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gc_9_4_3.c | 13 ++++++++++++-
 drivers/gpu/drm/amd/include/kgd_kfd_interface.h     |  5 +++++
 2 files changed, 17 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gc_9_4_3.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gc_9_4_3.c
index 89a45a9218f3..74eb57332e07 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gc_9_4_3.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gc_9_4_3.c
@@ -520,6 +520,16 @@ static uint32_t kgd_gfx_v9_4_3_hqd_sdma_get_doorbell(struct amdgpu_device *adev,
 	return is_active ? doorbell_off >> 2 : 0;
 }
 
+static uint32_t kgd_v9_4_3_ptl_ctrl(struct amdgpu_device *adev,
+				uint32_t cmd,
+				uint32_t *ptl_state,
+				enum amdgpu_ptl_fmt *fmt1,
+				enum amdgpu_ptl_fmt *fmt2)
+{
+	return amdgpu_ptl_perf_monitor_ctrl(adev, cmd,
+			ptl_state, fmt1, fmt2);
+}
+
 const struct kfd2kgd_calls gc_9_4_3_kfd2kgd = {
 	.program_sh_mem_settings = kgd_gfx_v9_program_sh_mem_settings,
 	.set_pasid_vmid_mapping = kgd_gfx_v9_4_3_set_pasid_vmid_mapping,
@@ -555,5 +565,6 @@ const struct kfd2kgd_calls gc_9_4_3_kfd2kgd = {
 	.clear_address_watch = kgd_gfx_v9_4_3_clear_address_watch,
 	.hqd_get_pq_addr = kgd_gfx_v9_hqd_get_pq_addr,
 	.hqd_reset = kgd_gfx_v9_hqd_reset,
-	.hqd_sdma_get_doorbell = kgd_gfx_v9_4_3_hqd_sdma_get_doorbell
+	.hqd_sdma_get_doorbell = kgd_gfx_v9_4_3_hqd_sdma_get_doorbell,
+	.ptl_ctrl = kgd_v9_4_3_ptl_ctrl
 };
diff --git a/drivers/gpu/drm/amd/include/kgd_kfd_interface.h b/drivers/gpu/drm/amd/include/kgd_kfd_interface.h
index 6df5afb242ae..44e225e097d0 100644
--- a/drivers/gpu/drm/amd/include/kgd_kfd_interface.h
+++ b/drivers/gpu/drm/amd/include/kgd_kfd_interface.h
@@ -334,6 +334,11 @@ struct kfd2kgd_calls {
 			      uint32_t inst, unsigned int utimeout);
 	uint32_t (*hqd_sdma_get_doorbell)(struct amdgpu_device *adev,
 					  int engine, int queue);
+	uint32_t (*ptl_ctrl)(struct amdgpu_device *adev,
+			     uint32_t cmd,
+			     uint32_t *ptl_state,
+			     enum amdgpu_ptl_fmt *fmt1,
+			     enum amdgpu_ptl_fmt *fmt2);
 };
 
 #endif	/* KGD_KFD_INTERFACE_H_INCLUDED */
-- 
2.34.1

[PATCH v2 06/13] Documentation/amdgpu: Add documentation for Peak Tops Limiter (PTL) sysfs interface

[Perry Yuan]

The PTL (Peak Tops Limiter) feature exposes per-GPU sysfs files under
/sys/class/drm/cardX/device/ptl/ to allow users to enable or disable PTL,
configure preferred data formats, and query supported formats. The usage
of these sysfs files is not always obvious, so add documentation to
describe their purpose and provide concrete usage examples.

V3 changes:
 * format show will display preferred formats instead of N/A (Alex)

Signed-off-by: Perry Yuan <perry.yuan@amd.com>
Suggested-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Yifan Zhang <yifan1.zhang@amd.com>
---
 Documentation/gpu/amdgpu/index.rst |  1 +
 Documentation/gpu/amdgpu/ptl.rst   | 94 ++++++++++++++++++++++++++++++
 2 files changed, 95 insertions(+)
 create mode 100644 Documentation/gpu/amdgpu/ptl.rst

diff --git a/Documentation/gpu/amdgpu/index.rst b/Documentation/gpu/amdgpu/index.rst
index 8732084186a4..b2ab182236ef 100644
--- a/Documentation/gpu/amdgpu/index.rst
+++ b/Documentation/gpu/amdgpu/index.rst
@@ -23,3 +23,4 @@ Next (GCN), Radeon DNA (RDNA), and Compute DNA (CDNA) architectures.
    debugfs
    process-isolation
    amdgpu-glossary
+   ptl
diff --git a/Documentation/gpu/amdgpu/ptl.rst b/Documentation/gpu/amdgpu/ptl.rst
new file mode 100644
index 000000000000..c7f16dea7954
--- /dev/null
+++ b/Documentation/gpu/amdgpu/ptl.rst
@@ -0,0 +1,94 @@
+=======================================
+Peak Tops Limiter (PTL) sysfs Interface
+=======================================
+
+Overview
+--------
+The Peak Tops Limiter (PTL) sysfs interface enables users to control and
+configure the PTL feature for each GPU individually.  All PTL-related
+sysfs files are located under `/sys/class/drm/cardX/device/ptl/`, where
+`X` is the GPU index.  Through these files, users can enable or disable
+PTL, set preferred data formats, and query supported formats for each GPU.
+
+PTL sysfs files
+----------------
+The following files are available under `/sys/class/drm/cardX/device/ptl/`:
+
+- `ptl_enable`
+- `ptl_format`
+- `ptl_supported_formats`
+
+PTL Enable/Disable
+------------------
+File: `ptl_enable`
+Type: Read/Write (rw)
+
+Read: Returns the current PTL status as a string: `enabled` if PTL
+is active, or `disabled` if inactive.
+
+Write:
+
+- Write `1` or `enabled` to enable PTL
+- Write `0` or `disabled` to disable PTL
+
+Examples::
+
+    # Query PTL status
+    cat /sys/class/drm/card1/device/ptl/ptl_enable
+    # Output: enabled
+
+    # Enable PTL
+    sudo bash -c "echo 1 > /sys/class/drm/card1/device/ptl/ptl_enable"
+
+    # Disable PTL
+    sudo bash -c "echo 0 > /sys/class/drm/card1/device/ptl/ptl_enable"
+
+PTL Format (Preferred Data Formats)
+-----------------------------------
+File: `ptl_format`
+Type: Read/Write (rw)
+
+Read: Returns the two preferred formats, e.g. `I8,F32`.
+
+Write: Accepts two formats separated by a comma, e.g. `I8,F32`.
+
+- Both formats must be supported and different.
+- If an invalid format is provided (not supported, or both formats are the
+  same), the driver will return "write error: Invalid argument".
+
+Examples::
+
+    # Query PTL formats
+    cat /sys/class/drm/card1/device/ptl/ptl_format
+    # Output: I8,F32
+
+    # Set PTL formats
+    sudo bash -c "echo I8,F32 > /sys/class/drm/card1/device/ptl/ptl_format"
+
+Supported Formats
+-----------------
+File: `ptl_supported_formats`
+Type: Read-only (r)
+
+Read: Returns a comma-separated list of supported formats, e.g.
+`I8,F16,BF16,F32,F64`.
+
+Example::
+
+    # Check supported formats
+    cat /sys/class/drm/card1/device/ptl/ptl_supported_formats
+    # Output: I8,F16,BF16,F32,F64
+
+Behavioral Notes
+----------------
+- PTL formats can only be set when PTL is enabled.
+- If PTL is disabled, `ptl_format` returns `N/A`.
+- Only two formats can be set at a time, and they must be from the supported set and different..
+- All commands support per-GPU targeting.
+- Root permission is required to enable/disable PTL or change formats.
+- If the hardware does not support PTL, the PTL sysfs directory will not
+  be created.
+
+Implementation
+--------------
+The PTL sysfs nodes are implemented in `drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c`.
-- 
2.34.1

[PATCH v2 07/13] drm/amdgpu: add sysfs for Peak Tops Limiter (PTL)

[Perry Yuan]

Add per-GPU sysfs files under /sys/class/drm/cardX/device/ptl to
control the Peak Tops Limiter (PTL) feature. Exposes ptl_enable
(enable/disable PTL), ptl_format (set/query preferred formats),
and ptl_supported_formats (list supported formats)

Example usage
-------------
Query PTL status:
    `cat /sys/class/drm/card1/device/ptl/ptl_enable`

Enable PTL:
    `sudo bash -c "echo 1 > /sys/class/drm/card1/device/ptl/ptl_enable"`

Disable PTL:
    `sudo bash -c "echo 0 > /sys/class/drm/card1/device/ptl/ptl_enable"`

Set PTL preferred formats:
    `sudo bash -c "echo I8,F32 > /sys/class/drm/card1/device/ptl/ptl_format"`

Query supported formats:
    `cat /sys/class/drm/card1/device/ptl/ptl_supported_formats`

v3 changes:
 * move N/A to previous format in format show(Alex)
 * fix format check for format store(Alex)
 * drop the ptl declarations into amdgpu_ptl.h(Alex)

v2 changes:
 * add usage commands in commit info (Alex)
 * move amdgpu_ptl_fmt into kgd_kfd_interface.h (Alex)

Signed-off-by: Perry Yuan <perry.yuan@amd.com>
Reviewed-by: Yifan Zhang <yifan1.zhang@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c |   2 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c    | 191 +++++++++++++++++++++
 drivers/gpu/drm/amd/include/amdgpu_ptl.h   |   4 +
 3 files changed, 197 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index a07fe386d275..da7585d1c6e7 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -4337,6 +4337,7 @@ static int amdgpu_device_sys_interface_init(struct amdgpu_device *adev)
 	amdgpu_reg_state_sysfs_init(adev);
 	amdgpu_xcp_sysfs_init(adev);
 	amdgpu_uma_sysfs_init(adev);
+	amdgpu_ptl_sysfs_init(adev);
 
 	return r;
 }
@@ -4353,6 +4354,7 @@ static void amdgpu_device_sys_interface_fini(struct amdgpu_device *adev)
 	amdgpu_reg_state_sysfs_fini(adev);
 	amdgpu_xcp_sysfs_fini(adev);
 	amdgpu_uma_sysfs_fini(adev);
+	amdgpu_ptl_sysfs_fini(adev);
 }
 
 /**
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
index 19b81f515374..a1481ad465a4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
@@ -52,6 +52,15 @@ static int psp_load_smu_fw(struct psp_context *psp);
 static int psp_rap_terminate(struct psp_context *psp);
 static int psp_securedisplay_terminate(struct psp_context *psp);
 
+static const char * const amdgpu_ptl_fmt_str[] = {
+	[AMDGPU_PTL_FMT_I8]      = "I8",
+	[AMDGPU_PTL_FMT_F16]     = "F16",
+	[AMDGPU_PTL_FMT_BF16]    = "BF16",
+	[AMDGPU_PTL_FMT_F32]     = "F32",
+	[AMDGPU_PTL_FMT_F64]     = "F64",
+	[AMDGPU_PTL_FMT_INVALID] = "INVALID",
+};
+
 static int psp_ring_init(struct psp_context *psp,
 			 enum psp_ring_type ring_type)
 {
@@ -1307,6 +1316,163 @@ int amdgpu_ptl_perf_monitor_ctrl(struct amdgpu_device *adev, u32 req_code,
 
 	return psp_ptl_invoke(psp, req_code, ptl_state, &ptl_fmt1, &ptl_fmt2);
 }
+
+static enum amdgpu_ptl_fmt str_to_ptl_fmt(const char *str)
+{
+	int i;
+
+	for (i = 0; i < AMDGPU_PTL_FMT_INVALID; ++i) {
+		if (!strcmp(str, amdgpu_ptl_fmt_str[i]))
+			return (enum amdgpu_ptl_fmt)i;
+	}
+
+	return AMDGPU_PTL_FMT_INVALID;
+}
+
+static ssize_t ptl_supported_formats_show(struct device *dev,
+		struct device_attribute *attr, char *buf)
+{
+	ssize_t len = 0;
+
+	for (int i = 0; i < AMDGPU_PTL_FMT_INVALID; ++i) {
+		len += sysfs_emit_at(buf, len, "%s%s",
+				amdgpu_ptl_fmt_str[i],
+				(i < AMDGPU_PTL_FMT_INVALID - 1) ? "," : "\n");
+	}
+
+	return len;
+}
+
+static ssize_t ptl_enable_store(struct device *dev,
+		struct device_attribute *attr,
+		const char *buf, size_t count)
+{
+	struct drm_device *ddev = dev_get_drvdata(dev);
+	struct amdgpu_device *adev = drm_to_adev(ddev);
+	struct amdgpu_ptl *ptl = &adev->psp.ptl;
+	uint32_t ptl_state, fmt1, fmt2;
+	int ret;
+	bool enable;
+
+	mutex_lock(&ptl->mutex);
+	if (sysfs_streq(buf, "enabled") || sysfs_streq(buf, "1")) {
+		enable = true;
+	} else if (sysfs_streq(buf, "disabled") || sysfs_streq(buf, "0")) {
+		enable = false;
+	} else {
+		mutex_unlock(&ptl->mutex);
+		return -EINVAL;
+	}
+
+	fmt1 = ptl->fmt1;
+	fmt2 = ptl->fmt2;
+	ptl_state = enable ? 1 : 0;
+
+	ret = amdgpu_ptl_perf_monitor_ctrl(adev, PSP_PTL_PERF_MON_SET, &ptl_state, &fmt1, &fmt2);
+	if (ret) {
+		dev_err(adev->dev, "Failed to set PTL err = %d\n", ret);
+		mutex_unlock(&ptl->mutex);
+		return ret;
+	}
+
+	mutex_unlock(&ptl->mutex);
+	return count;
+}
+
+static ssize_t ptl_enable_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+	struct drm_device *ddev = dev_get_drvdata(dev);
+	struct amdgpu_device *adev = drm_to_adev(ddev);
+	struct amdgpu_ptl *ptl = &adev->psp.ptl;
+
+	return sysfs_emit(buf, "%s\n", ptl->enabled ? "enabled" : "disabled");
+}
+
+static ssize_t ptl_format_store(struct device *dev,
+		struct device_attribute *attr,
+		const char *buf, size_t count)
+{
+	struct drm_device *ddev = dev_get_drvdata(dev);
+	struct amdgpu_device *adev = drm_to_adev(ddev);
+	char fmt1_str[8], fmt2_str[8];
+	enum amdgpu_ptl_fmt fmt1_enum, fmt2_enum;
+	struct amdgpu_ptl *ptl = &adev->psp.ptl;
+	uint32_t ptl_state, fmt1, fmt2;
+	int ret;
+
+	/* Only allow format update when PTL is enabled */
+	if (!ptl->enabled)
+		return -EPERM;
+
+	mutex_lock(&ptl->mutex);
+	/* Parse input, expecting "FMT1,FMT2" */
+	if (sscanf(buf, "%7[^,],%7s", fmt1_str, fmt2_str) != 2) {
+		mutex_unlock(&ptl->mutex);
+		return -EINVAL;
+	}
+
+	fmt1_enum = str_to_ptl_fmt(fmt1_str);
+	fmt2_enum = str_to_ptl_fmt(fmt2_str);
+
+	if (fmt1_enum >= AMDGPU_PTL_FMT_INVALID ||
+			fmt2_enum >= AMDGPU_PTL_FMT_INVALID ||
+			fmt1_enum == fmt2_enum) {
+		mutex_unlock(&ptl->mutex);
+		return -EINVAL;
+	}
+
+	ptl_state = ptl->enabled;
+	fmt1 = fmt1_enum;
+	fmt2 = fmt2_enum;
+	ret = amdgpu_ptl_perf_monitor_ctrl(adev, PSP_PTL_PERF_MON_SET, &ptl_state, &fmt1, &fmt2);
+	if (ret) {
+		dev_err(adev->dev, "Failed to update PTL err = %d\n", ret);
+		mutex_unlock(&ptl->mutex);
+		return ret;
+	}
+	mutex_unlock(&ptl->mutex);
+
+	return count;
+}
+
+static ssize_t ptl_format_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+	struct drm_device *ddev = dev_get_drvdata(dev);
+	struct amdgpu_device *adev = drm_to_adev(ddev);
+	struct psp_context *psp = &adev->psp;
+
+	return sysfs_emit(buf, "%s,%s\n",
+			amdgpu_ptl_fmt_str[psp->ptl.fmt1],
+			amdgpu_ptl_fmt_str[psp->ptl.fmt2]);
+}
+
+static umode_t amdgpu_ptl_is_visible(struct kobject *kobj, struct attribute *attr, int idx)
+{
+	struct device *dev = kobj_to_dev(kobj);
+	struct drm_device *ddev = dev_get_drvdata(dev);
+	struct amdgpu_device *adev = drm_to_adev(ddev);
+
+	/* Only show PTL sysfs files if PTL hardware is supported */
+	if (!adev->psp.ptl.hw_supported)
+		return 0;
+
+	return attr->mode;
+}
+
+int amdgpu_ptl_sysfs_init(struct amdgpu_device *adev)
+{
+	if (!adev->psp.ptl.hw_supported)
+		return 0;
+
+	return sysfs_create_group(&adev->dev->kobj, &amdgpu_ptl_attr_group);
+}
+
+void amdgpu_ptl_sysfs_fini(struct amdgpu_device *adev)
+{
+	if (!adev->psp.ptl.hw_supported)
+		return;
+
+	sysfs_remove_group(&adev->dev->kobj, &amdgpu_ptl_attr_group);
 }
 
 int psp_spatial_partition(struct psp_context *psp, int mode)
@@ -4303,6 +4469,31 @@ void psp_copy_fw(struct psp_context *psp, uint8_t *start_addr, uint32_t bin_size
 static DEVICE_ATTR(usbc_pd_fw, 0644,
 		   psp_usbc_pd_fw_sysfs_read,
 		   psp_usbc_pd_fw_sysfs_write);
+/**
+ * DOC: PTL sysfs attributes
+ * These sysfs files under /sys/class/drm/cardX/device/ptl allow users to enable or disable
+ * the Peak Tops Limiter (PTL), configure preferred PTL data formats, and query supported
+ * formats for each GPU.
+ */
+static DEVICE_ATTR(ptl_enable, 0644,
+			ptl_enable_show, ptl_enable_store);
+static DEVICE_ATTR(ptl_format, 0644,
+			ptl_format_show, ptl_format_store);
+static DEVICE_ATTR(ptl_supported_formats, 0444,
+			ptl_supported_formats_show, NULL);
+
+static struct attribute *ptl_attrs[] = {
+	&dev_attr_ptl_enable.attr,
+	&dev_attr_ptl_format.attr,
+	&dev_attr_ptl_supported_formats.attr,
+	NULL,
+};
+
+const struct attribute_group amdgpu_ptl_attr_group = {
+	.name = "ptl",
+	.attrs = ptl_attrs,
+	.is_visible = amdgpu_ptl_is_visible,
+};
 
 int is_psp_fw_valid(struct psp_bin_desc bin)
 {
diff --git a/drivers/gpu/drm/amd/include/amdgpu_ptl.h b/drivers/gpu/drm/amd/include/amdgpu_ptl.h
index 12c9e0b4645a..e5ea1084bb09 100644
--- a/drivers/gpu/drm/amd/include/amdgpu_ptl.h
+++ b/drivers/gpu/drm/amd/include/amdgpu_ptl.h
@@ -45,4 +45,8 @@ struct amdgpu_ptl {
 int amdgpu_ptl_perf_monitor_ctrl(struct amdgpu_device *adev, u32 req_code,
 		u32 *ptl_state, u32 *fmt1, u32 *fmt2);
 
+int amdgpu_ptl_sysfs_init(struct amdgpu_device *adev);
+void amdgpu_ptl_sysfs_fini(struct amdgpu_device *adev);
+
+extern const struct attribute_group amdgpu_ptl_attr_group;
 #endif /* __AMDGPU_PTL_H__ */
-- 
2.34.1

[PATCH v2 08/13] drm/amdkfd: Add PTL control IOCTL Option and unify refcount logic

[Perry Yuan]

Introduce a new IOCTL option to allow userspace explicit control over
the Peak Tops Limiter (PTL) state for profiling

Link: https://github.com/ROCm/rocm-systems/tree/develop/projects/rocprofiler-sdk
Signed-off-by: Perry Yuan <perry.yuan@amd.com>
Reviewed-by: Yifan Zhang <yifan1.zhang@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c  |   2 +
 drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 102 +++++++++++++++++++++++
 drivers/gpu/drm/amd/amdkfd/kfd_priv.h    |   9 ++
 drivers/gpu/drm/amd/amdkfd/kfd_process.c |   4 +
 drivers/gpu/drm/amd/include/amdgpu_ptl.h |   2 +
 include/uapi/linux/kfd_ioctl.h           |   7 ++
 6 files changed, 126 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c b/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c
index aa9307d88fde..4e04eba9879a 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c
@@ -2396,6 +2396,8 @@ static int gfx_v9_4_3_perf_monitor_ptl_init(struct amdgpu_device *adev, bool ena
 
 	ptl->hw_supported = true;
 
+	atomic_set(&ptl->disable_ref, 0);
+
 	return 0;
 }
 
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index 9c37e8248540..9a23621542fa 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -1765,6 +1765,104 @@ static int kfd_ioctl_svm(struct file *filep, struct kfd_process *p, void *data)
 }
 #endif
 
+static int kfd_ptl_control(struct kfd_process_device *pdd, bool enable)
+{
+	struct amdgpu_device *adev = pdd->dev->adev;
+	struct amdgpu_ptl *ptl = &adev->psp.ptl;
+	enum amdgpu_ptl_fmt pref_format1 = ptl->fmt1;
+	enum amdgpu_ptl_fmt pref_format2 = ptl->fmt2;
+	uint32_t ptl_state = enable ? 1 : 0;
+	int ret;
+
+	if (!ptl->hw_supported)
+		return -EOPNOTSUPP;
+
+	if (!pdd->dev->kfd2kgd || !pdd->dev->kfd2kgd->ptl_ctrl)
+		return -EOPNOTSUPP;
+
+	ret = pdd->dev->kfd2kgd->ptl_ctrl(adev, PSP_PTL_PERF_MON_SET,
+					  &ptl_state,
+					  &pref_format1,
+					  &pref_format2);
+	return ret;
+}
+
+int kfd_ptl_disable_request(struct kfd_process_device *pdd,
+		struct kfd_process *p)
+{
+	struct amdgpu_device *adev = pdd->dev->adev;
+	struct amdgpu_ptl *ptl = &adev->psp.ptl;
+	int ret = 0;
+
+	mutex_lock(&ptl->mutex);
+
+	if (pdd->ptl_disable_req)
+		goto out;
+
+	if (atomic_inc_return(&ptl->disable_ref) == 1) {
+		ret = kfd_ptl_control(pdd, false);
+		if (ret) {
+			atomic_dec(&ptl->disable_ref);
+			dev_warn(pdd->dev->adev->dev,
+					"failed to disable PTL\n");
+			goto out;
+		}
+	}
+	pdd->ptl_disable_req = true;
+
+out:
+	mutex_unlock(&ptl->mutex);
+	return ret;
+}
+
+int kfd_ptl_disable_release(struct kfd_process_device *pdd,
+		struct kfd_process *p)
+{
+	struct amdgpu_device *adev = pdd->dev->adev;
+	struct amdgpu_ptl *ptl = &adev->psp.ptl;
+	int ret = 0;
+
+	mutex_lock(&ptl->mutex);
+
+	if (!pdd->ptl_disable_req)
+		goto out;
+
+	if (atomic_dec_return(&ptl->disable_ref) == 0) {
+		ret = kfd_ptl_control(pdd, true);
+		if (ret) {
+			atomic_inc(&ptl->disable_ref);
+			dev_warn(adev->dev, "Failed to enable PTL on release: %d\n", ret);
+			goto out;
+		}
+	}
+	pdd->ptl_disable_req = false;
+
+out:
+	mutex_unlock(&ptl->mutex);
+	return ret;
+}
+
+static int kfd_profiler_ptl_control(struct kfd_process *p,
+		struct kfd_ioctl_ptl_control *args)
+{
+	struct kfd_process_device *pdd;
+	int ret;
+
+	mutex_lock(&p->mutex);
+	pdd = kfd_process_device_data_by_id(p, args->gpu_id);
+	mutex_unlock(&p->mutex);
+
+	if (!pdd || !pdd->dev || !pdd->dev->kfd)
+		return -EINVAL;
+
+	if (args->enable == 0)
+		ret = kfd_ptl_disable_request(pdd, p);
+	else
+		ret = kfd_ptl_disable_release(pdd, p);
+
+	return ret;
+}
+
 static int criu_checkpoint_process(struct kfd_process *p,
 			     uint8_t __user *user_priv_data,
 			     uint64_t *priv_offset)
@@ -3230,6 +3328,7 @@ static inline uint32_t profile_lock_device(struct kfd_process *p,
 		if (!kfd->profiler_process) {
 			kfd->profiler_process = p;
 			status = 0;
+			kfd_ptl_disable_request(pdd, p);
 		} else if (kfd->profiler_process == p) {
 			status = -EALREADY;
 		} else {
@@ -3238,6 +3337,7 @@ static inline uint32_t profile_lock_device(struct kfd_process *p,
 	} else if (op == 0 && kfd->profiler_process == p) {
 		kfd->profiler_process = NULL;
 		status = 0;
+		kfd_ptl_disable_release(pdd, p);
 	}
 	mutex_unlock(&kfd->profiler_lock);
 
@@ -3280,6 +3380,8 @@ static int kfd_ioctl_profiler(struct file *filep, struct kfd_process *p, void *d
 		return 0;
 	case KFD_IOC_PROFILER_PMC:
 		return kfd_profiler_pmc(p, &args->pmc);
+	case KFD_IOC_PROFILER_PTL_CONTROL:
+		return kfd_profiler_ptl_control(p, &args->ptl);
 	}
 	return -EINVAL;
 }
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
index 8983065645fa..48347065b9cd 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
@@ -872,6 +872,8 @@ struct kfd_process_device {
 	bool has_reset_queue;
 
 	u32 pasid;
+	/* Indicates this process has requested PTL stay disabled */
+	bool ptl_disable_req;
 };
 
 #define qpd_to_pdd(x) container_of(x, struct kfd_process_device, qpd)
@@ -1607,6 +1609,13 @@ static inline bool kfd_is_first_node(struct kfd_node *node)
 	return (node == node->kfd->nodes[0]);
 }
 
+/* PTL support */
+int kfd_ptl_control(struct kfd_process_device *pdd, bool enable);
+int kfd_ptl_disable_request(struct kfd_process_device *pdd,
+		struct kfd_process *p);
+int kfd_ptl_disable_release(struct kfd_process_device *pdd,
+		struct kfd_process *p);
+
 /* Debugfs */
 #if defined(CONFIG_DEBUG_FS)
 
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index deca19b478d0..882080dc4925 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -1132,6 +1132,10 @@ static void kfd_process_destroy_pdds(struct kfd_process *p)
 		pr_debug("Releasing pdd (topology id %d, for pid %d)\n",
 			pdd->dev->id, p->lead_thread->pid);
 		kfd_process_profiler_release(p, pdd);
+
+		if (pdd->ptl_disable_req)
+			kfd_ptl_disable_release(pdd, p);
+
 		kfd_process_device_destroy_cwsr_dgpu(pdd);
 		kfd_process_device_destroy_ib_mem(pdd);
 
diff --git a/drivers/gpu/drm/amd/include/amdgpu_ptl.h b/drivers/gpu/drm/amd/include/amdgpu_ptl.h
index e5ea1084bb09..f944ab45d1ea 100644
--- a/drivers/gpu/drm/amd/include/amdgpu_ptl.h
+++ b/drivers/gpu/drm/amd/include/amdgpu_ptl.h
@@ -39,6 +39,8 @@ struct amdgpu_ptl {
 	enum amdgpu_ptl_fmt		fmt2;
 	bool				enabled;
 	bool				hw_supported;
+	/* PTL disable reference counting */
+	atomic_t			disable_ref;
 	struct mutex			mutex;
 };
 
diff --git a/include/uapi/linux/kfd_ioctl.h b/include/uapi/linux/kfd_ioctl.h
index a8b2a18d07cf..da93daa3283c 100644
--- a/include/uapi/linux/kfd_ioctl.h
+++ b/include/uapi/linux/kfd_ioctl.h
@@ -1562,6 +1562,7 @@ struct kfd_ioctl_dbg_trap_args {
 enum kfd_profiler_ops {
 	KFD_IOC_PROFILER_PMC = 0,
 	KFD_IOC_PROFILER_VERSION = 2,
+	KFD_IOC_PROFILER_PTL_CONTROL = 3,
 };
 
 /**
@@ -1573,10 +1574,16 @@ struct kfd_ioctl_pmc_settings {
 	__u32 perfcount_enable;   /* Force Perfcount Enable for queues on GPU */
 };
 
+struct kfd_ioctl_ptl_control {
+	__u32 gpu_id; /* user_gpu_id */
+	__u32 enable; /* set 1 to enable PTL, set 0 to disable PTL */
+};
+
 struct kfd_ioctl_profiler_args {
 	__u32 op;						/* kfd_profiler_op */
 	union {
 		struct kfd_ioctl_pmc_settings  pmc;
+		struct kfd_ioctl_ptl_control   ptl;
 		__u32 version;				/* KFD_IOC_PROFILER_VERSION_NUM */
 	};
 };
-- 
2.34.1

[PATCH v2 09/13] drm/amdkfd: suspend scheduler during PTL re-enabling

[Perry Yuan]

Stop the scheduler before releasing the PTL disable request to ensure
the GPU is quiescent during the PTL state transition. This prevents
potential queue preemption failures and GPU resets caused by modifying
PTL state while waves are executing

v1->v2:
only stop/start the scheduler when the PTL state actually needs to transition(Yifan)

Signed-off-by: Perry Yuan <perry.yuan@amd.com>
Reviewed-by: Yifan Zhang <yifan1.zhang@amd.com>
---
 drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 30 ++++++++++++++++++++++--
 1 file changed, 28 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index 9a23621542fa..d699d0354bda 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -1780,10 +1780,17 @@ static int kfd_ptl_control(struct kfd_process_device *pdd, bool enable)
 	if (!pdd->dev->kfd2kgd || !pdd->dev->kfd2kgd->ptl_ctrl)
 		return -EOPNOTSUPP;
 
+	if (adev->kfd.init_complete)
+		amdgpu_amdkfd_stop_sched(adev, pdd->dev->node_id);
+
 	ret = pdd->dev->kfd2kgd->ptl_ctrl(adev, PSP_PTL_PERF_MON_SET,
 					  &ptl_state,
 					  &pref_format1,
 					  &pref_format2);
+
+	if (adev->kfd.init_complete)
+		amdgpu_amdkfd_start_sched(adev, pdd->dev->node_id);
+
 	return ret;
 }
 
@@ -3310,6 +3317,7 @@ static inline uint32_t profile_lock_device(struct kfd_process *p,
 	struct kfd_process_device *pdd;
 	struct kfd_dev *kfd;
 	int status = -EINVAL;
+	struct amdgpu_ptl *ptl;
 
 	if (!p)
 		return -EINVAL;
@@ -3322,13 +3330,22 @@ static inline uint32_t profile_lock_device(struct kfd_process *p,
 		return -EINVAL;
 
 	kfd = pdd->dev->kfd;
+	ptl = &pdd->dev->adev->psp.ptl;
 
 	mutex_lock(&kfd->profiler_lock);
 	if (op == 1) {
 		if (!kfd->profiler_process) {
 			kfd->profiler_process = p;
 			status = 0;
-			kfd_ptl_disable_request(pdd, p);
+			mutex_unlock(&kfd->profiler_lock);
+			if (ptl->hw_supported) {
+				status = kfd_ptl_disable_request(pdd, p);
+				if (status != 0)
+					dev_err(kfd_device,
+						"Failed to lock device %d for profiling, error %d\n",
+						gpu_id, status);
+			}
+			return status;
 		} else if (kfd->profiler_process == p) {
 			status = -EALREADY;
 		} else {
@@ -3337,7 +3354,16 @@ static inline uint32_t profile_lock_device(struct kfd_process *p,
 	} else if (op == 0 && kfd->profiler_process == p) {
 		kfd->profiler_process = NULL;
 		status = 0;
-		kfd_ptl_disable_release(pdd, p);
+		mutex_unlock(&kfd->profiler_lock);
+
+		if (ptl->hw_supported) {
+			status = kfd_ptl_disable_release(pdd, p);
+			if (status)
+				dev_err(kfd_device,
+						"Failed to unlock device %d for profiling, error %d\n",
+						gpu_id, status);
+		}
+		return status;
 	}
 	mutex_unlock(&kfd->profiler_lock);
 
-- 
2.34.1

[PATCH v2 10/13] drm/amdgpu: Track PTL disable requests by source

[Perry Yuan]

Use a bitmap to track PTL disable requests from sysfs and profiler.
PTL is only re-enabled once all sources have released their disable
requests, avoiding premature enablement.

Signed-off-by: Perry Yuan <perry.yuan@amd.com>
Reviewed-by: Yifan Zhang <yifan1.zhang@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c  | 26 ++++++++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h  |  1 +
 drivers/gpu/drm/amd/amdkfd/kfd_chardev.c |  3 +++
 drivers/gpu/drm/amd/include/amdgpu_ptl.h |  6 ++++++
 4 files changed, 36 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
index a1481ad465a4..413a7e44048c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
@@ -1314,6 +1314,21 @@ int amdgpu_ptl_perf_monitor_ctrl(struct amdgpu_device *adev, u32 req_code,
 			ptl->fmt2 == ptl_fmt2)
 		return 0;
 
+	/* If enabling PTL, check disable bitmap */
+	if (req_code == PSP_PTL_PERF_MON_SET && *ptl_state == 1) {
+		if (!bitmap_empty(ptl->disable_bitmap,
+					AMDGPU_PTL_DISABLE_MAX)) {
+			dev_dbg(adev->dev,
+					"PTL enable blocked: SYSFS=%d, PROFILER=%d (ref=%d)\n",
+					test_bit(AMDGPU_PTL_DISABLE_SYSFS,
+						ptl->disable_bitmap),
+					test_bit(AMDGPU_PTL_DISABLE_PROFILER,
+						ptl->disable_bitmap),
+					atomic_read(&ptl->disable_ref));
+			return 0;
+		}
+	}
+
 	return psp_ptl_invoke(psp, req_code, ptl_state, &ptl_fmt1, &ptl_fmt2);
 }
 
@@ -1353,6 +1368,7 @@ static ssize_t ptl_enable_store(struct device *dev,
 	uint32_t ptl_state, fmt1, fmt2;
 	int ret;
 	bool enable;
+	bool bit_changed = false;
 
 	mutex_lock(&ptl->mutex);
 	if (sysfs_streq(buf, "enabled") || sysfs_streq(buf, "1")) {
@@ -1368,14 +1384,24 @@ static ssize_t ptl_enable_store(struct device *dev,
 	fmt2 = ptl->fmt2;
 	ptl_state = enable ? 1 : 0;
 
+	if (enable)
+		bit_changed = test_and_clear_bit(AMDGPU_PTL_DISABLE_SYSFS,
+				ptl->disable_bitmap);
+
 	ret = amdgpu_ptl_perf_monitor_ctrl(adev, PSP_PTL_PERF_MON_SET, &ptl_state, &fmt1, &fmt2);
 	if (ret) {
 		dev_err(adev->dev, "Failed to set PTL err = %d\n", ret);
+		if (enable && bit_changed)
+			set_bit(AMDGPU_PTL_DISABLE_SYSFS, ptl->disable_bitmap);
 		mutex_unlock(&ptl->mutex);
 		return ret;
 	}
 
+	if (!enable)
+		set_bit(AMDGPU_PTL_DISABLE_SYSFS, ptl->disable_bitmap);
+
 	mutex_unlock(&ptl->mutex);
+
 	return count;
 }
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
index 332633f6f4be..11fafe332bff 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
@@ -31,6 +31,7 @@
 #include "ta_ras_if.h"
 #include "ta_rap_if.h"
 #include "ta_secureDisplay_if.h"
+#include <linux/bitops.h>
 #include "amdgpu_ptl.h"
 
 #define PSP_FENCE_BUFFER_SIZE	0x1000
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index d699d0354bda..4498de4aa925 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -1815,6 +1815,7 @@ int kfd_ptl_disable_request(struct kfd_process_device *pdd,
 			goto out;
 		}
 	}
+	set_bit(AMDGPU_PTL_DISABLE_PROFILER, ptl->disable_bitmap);
 	pdd->ptl_disable_req = true;
 
 out:
@@ -1835,9 +1836,11 @@ int kfd_ptl_disable_release(struct kfd_process_device *pdd,
 		goto out;
 
 	if (atomic_dec_return(&ptl->disable_ref) == 0) {
+		clear_bit(AMDGPU_PTL_DISABLE_PROFILER, ptl->disable_bitmap);
 		ret = kfd_ptl_control(pdd, true);
 		if (ret) {
 			atomic_inc(&ptl->disable_ref);
+			set_bit(AMDGPU_PTL_DISABLE_PROFILER, ptl->disable_bitmap);
 			dev_warn(adev->dev, "Failed to enable PTL on release: %d\n", ret);
 			goto out;
 		}
diff --git a/drivers/gpu/drm/amd/include/amdgpu_ptl.h b/drivers/gpu/drm/amd/include/amdgpu_ptl.h
index f944ab45d1ea..875277312d30 100644
--- a/drivers/gpu/drm/amd/include/amdgpu_ptl.h
+++ b/drivers/gpu/drm/amd/include/amdgpu_ptl.h
@@ -34,6 +34,11 @@ enum amdgpu_ptl_fmt {
 	AMDGPU_PTL_FMT_INVALID = 7,
 };
 
+enum amdgpu_ptl_disable_source {
+	AMDGPU_PTL_DISABLE_SYSFS = 0,
+	AMDGPU_PTL_DISABLE_PROFILER,
+	AMDGPU_PTL_DISABLE_MAX,
+};
 struct amdgpu_ptl {
 	enum amdgpu_ptl_fmt		fmt1;
 	enum amdgpu_ptl_fmt		fmt2;
@@ -42,6 +47,7 @@ struct amdgpu_ptl {
 	/* PTL disable reference counting */
 	atomic_t			disable_ref;
 	struct mutex			mutex;
+	DECLARE_BITMAP(disable_bitmap, AMDGPU_PTL_DISABLE_MAX);
 };
 
 int amdgpu_ptl_perf_monitor_ctrl(struct amdgpu_device *adev, u32 req_code,
-- 
2.34.1

[PATCH v2 11/13] drm/amdgpu: add amdgpu.ptl module parameter for PTL control

[Perry Yuan]

Add a new kernel module parameter 'amdgpu.ptl' to allow
users to enable or disable PTL feature at driver loading time.

Parameter values:
  *) 0 or -1: disable PTL (default)
  *) 1: enable PTL
  *) 2: permanently disable PTL

Signed-off-by: Perry Yuan <perry.yuan@amd.com>
Reviewed-by: Yifan Zhang <yifan1.zhang@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu.h      |  1 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c  | 13 +++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c  | 12 ++++++++++
 drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c  | 28 +++++++++++++++++++++++-
 drivers/gpu/drm/amd/include/amdgpu_ptl.h |  1 +
 5 files changed, 54 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index af4042387f3b..23c58361b4d8 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -268,6 +268,7 @@ extern int amdgpu_rebar;
 
 extern int amdgpu_wbrf;
 extern int amdgpu_user_queue;
+extern int amdgpu_ptl;
 
 extern uint amdgpu_hdmi_hpd_debounce_delay_ms;
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index 95d26f086d54..482fa222292e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -248,6 +248,7 @@ int amdgpu_umsch_mm_fwlog;
 int amdgpu_rebar = -1; /* auto */
 int amdgpu_user_queue = -1;
 uint amdgpu_hdmi_hpd_debounce_delay_ms;
+int amdgpu_ptl = -1; /* auto */
 
 DECLARE_DYNDBG_CLASSMAP(drm_debug_classes, DD_CLASS_TYPE_DISJOINT_BITS, 0,
 			"DRM_UT_CORE",
@@ -1134,6 +1135,18 @@ module_param_named(user_queue, amdgpu_user_queue, int, 0444);
 MODULE_PARM_DESC(hdmi_hpd_debounce_delay_ms, "HDMI HPD disconnect debounce delay in milliseconds (0 to disable (by default), 1500 is common)");
 module_param_named(hdmi_hpd_debounce_delay_ms, amdgpu_hdmi_hpd_debounce_delay_ms, uint, 0644);
 
+/**
+ * DOC: ptl (int)
+ * Enable PTL feature at boot time. Possible values:
+ *
+ * - -1 = auto (ASIC specific default)
+ * -  0 = disable PTL (default)
+ * -  1 = enable PTL
+ * -  2 = permanently disable PTL (cannot be re-enabled at runtime)
+ */
+MODULE_PARM_DESC(ptl, "Enable PTL (-1 = auto, 0 = disable (default), 1 = enable, 2 = permanently disable)");
+module_param_named(ptl, amdgpu_ptl, int, 0444);
+
 /* These devices are not supported by amdgpu.
  * They are supported by the mach64, r128, radeon drivers
  */
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
index 413a7e44048c..dfda694aefe4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
@@ -1296,6 +1296,9 @@ int amdgpu_ptl_perf_monitor_ctrl(struct amdgpu_device *adev, u32 req_code,
 	psp = &adev->psp;
 	ptl = &psp->ptl;
 
+	if (ptl->permanently_disabled && *ptl_state == 1)
+		return 0;
+
 	if (amdgpu_ip_version(adev, GC_HWIP, 0) != IP_VERSION(9, 4, 4) ||
 			psp->sos.fw_version < 0x0036081a)
 		return -EOPNOTSUPP;
@@ -1380,6 +1383,12 @@ static ssize_t ptl_enable_store(struct device *dev,
 		return -EINVAL;
 	}
 
+	/* Block enable when permanently disabled */
+	if (ptl->permanently_disabled) {
+		mutex_unlock(&ptl->mutex);
+		return -EPERM;
+	}
+
 	fmt1 = ptl->fmt1;
 	fmt2 = ptl->fmt2;
 	ptl_state = enable ? 1 : 0;
@@ -1411,6 +1420,9 @@ static ssize_t ptl_enable_show(struct device *dev, struct device_attribute *attr
 	struct amdgpu_device *adev = drm_to_adev(ddev);
 	struct amdgpu_ptl *ptl = &adev->psp.ptl;
 
+	if (ptl->permanently_disabled)
+		return sysfs_emit(buf, "permanently disabled\n");
+
 	return sysfs_emit(buf, "%s\n", ptl->enabled ? "enabled" : "disabled");
 }
 
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c b/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c
index 4e04eba9879a..93bd2e06fa14 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c
@@ -2401,6 +2401,32 @@ static int gfx_v9_4_3_perf_monitor_ptl_init(struct amdgpu_device *adev, bool ena
 	return 0;
 }
 
+static int gfx_v9_4_3_ptl_hw_init(struct amdgpu_device *adev)
+{
+	struct amdgpu_ptl *ptl = &adev->psp.ptl;
+	bool enable;
+
+	switch (amdgpu_ptl) {
+	case 1:
+		enable = true;
+		break;
+	case 2:
+		/* Permanently disabled - cannot be re-enabled */
+		enable = false;
+		ptl->permanently_disabled = true;
+		break;
+	case -1:
+	case 0:
+	default:
+		enable = false;
+		break;
+	}
+
+	gfx_v9_4_3_perf_monitor_ptl_init(adev, enable ? 1 : 0);
+
+	return 0;
+}
+
 static int gfx_v9_4_3_hw_fini(struct amdgpu_ip_block *ip_block)
 {
 	struct amdgpu_device *adev = ip_block->adev;
@@ -2583,7 +2609,7 @@ static int gfx_v9_4_3_late_init(struct amdgpu_ip_block *ip_block)
 	    adev->gfx.ras->enable_watchdog_timer)
 		adev->gfx.ras->enable_watchdog_timer(adev);
 
-	gfx_v9_4_3_perf_monitor_ptl_init(adev, true);
+	gfx_v9_4_3_ptl_hw_init(adev);
 
 	return 0;
 }
diff --git a/drivers/gpu/drm/amd/include/amdgpu_ptl.h b/drivers/gpu/drm/amd/include/amdgpu_ptl.h
index 875277312d30..609b48b24367 100644
--- a/drivers/gpu/drm/amd/include/amdgpu_ptl.h
+++ b/drivers/gpu/drm/amd/include/amdgpu_ptl.h
@@ -44,6 +44,7 @@ struct amdgpu_ptl {
 	enum amdgpu_ptl_fmt		fmt2;
 	bool				enabled;
 	bool				hw_supported;
+	bool				permanently_disabled;
 	/* PTL disable reference counting */
 	atomic_t			disable_ref;
 	struct mutex			mutex;
-- 
2.34.1

[PATCH v2 12/13] drm/amdgpu: add new data types F8 and Vector for PTL

[Perry Yuan]

Add F8 and VECTOR to amdgpu_ptl_fmt and PSP format mapping.
Update PTL format strings and GFX format enum to keep PSP/KFD in sync.

Signed-off-by: Perry Yuan <perry.yuan@amd.com>
Reviewed-by: Yifan Zhang <yifan1.zhang@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c | 8 ++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h | 2 ++
 drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c | 4 ++--
 3 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
index dfda694aefe4..d033b4227f3c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
@@ -58,6 +58,8 @@ static const char * const amdgpu_ptl_fmt_str[] = {
 	[AMDGPU_PTL_FMT_BF16]    = "BF16",
 	[AMDGPU_PTL_FMT_F32]     = "F32",
 	[AMDGPU_PTL_FMT_F64]     = "F64",
+	[AMDGPU_PTL_FMT_F8]      = "F8",
+	[AMDGPU_PTL_FMT_VECTOR]  = "VECTOR",
 	[AMDGPU_PTL_FMT_INVALID] = "INVALID",
 };
 
@@ -1232,6 +1234,12 @@ static int psp_ptl_fmt_verify(struct psp_context *psp, enum amdgpu_ptl_fmt fmt,
 	case AMDGPU_PTL_FMT_F64:
 		*ptl_fmt = GFX_FTYPE_F64;
 		break;
+	case AMDGPU_PTL_FMT_F8:
+		*ptl_fmt = GFX_FTYPE_F8;
+		break;
+	case AMDGPU_PTL_FMT_VECTOR:
+		*ptl_fmt = GFX_FTYPE_VECTOR;
+		break;
 	default:
 		return -EINVAL;
 	}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
index 11fafe332bff..4909e0ef328d 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
@@ -372,6 +372,8 @@ enum psp_ptl_format_type {
 	GFX_FTYPE_BF16        = 0x00000002,
 	GFX_FTYPE_F32         = 0x00000003,
 	GFX_FTYPE_F64         = 0x00000004,
+	GFX_FTYPE_F8          = 0x00000005,
+	GFX_FTYPE_VECTOR      = 0x00000006,
 	GFX_FTYPE_INVALID     = 0xFFFFFFFF,
 };
 
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c b/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c
index 93bd2e06fa14..003c166b54f3 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c
@@ -2381,8 +2381,8 @@ static int gfx_v9_4_3_perf_monitor_ptl_init(struct amdgpu_device *adev, bool ena
 		return -EOPNOTSUPP;
 
 	if (!ptl->hw_supported) {
-		fmt1 = GFX_FTYPE_I8;
-		fmt2 = GFX_FTYPE_BF16;
+		fmt1 = GFX_FTYPE_VECTOR;
+		fmt2 = GFX_FTYPE_F8;
 	} else {
 		fmt1 = ptl->fmt1;
 		fmt2 = ptl->fmt2;
-- 
2.34.1

[PATCH v2 13/13] drm/amdgpu: Wait for GFX idle before PTL state transition

[Perry Yuan]

Ensure GFX engine is idle before switching PTL state to prevent
register access violations and CP hang. This addresses the race
condition where in-flight GPU commands could conflict with PTL
state changes.

Signed-off-by: Perry Yuan <perry.yuan@amd.com>
Reviewed-by: Yifan Zhang <yifan1.zhang@amd.com>
---
 drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index 4498de4aa925..775afa1b4989 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -1783,6 +1783,11 @@ static int kfd_ptl_control(struct kfd_process_device *pdd, bool enable)
 	if (adev->kfd.init_complete)
 		amdgpu_amdkfd_stop_sched(adev, pdd->dev->node_id);
 
+	/* Wait for GFX to be idle before PTL operation */
+	ret = amdgpu_device_ip_wait_for_idle(adev, AMD_IP_BLOCK_TYPE_GFX);
+	if (ret)
+		return -ETIMEDOUT;
+
 	ret = pdd->dev->kfd2kgd->ptl_ctrl(adev, PSP_PTL_PERF_MON_SET,
 					  &ptl_state,
 					  &pref_format1,
-- 
2.34.1