* [PATCH 0/3] drm/xe/pmu: PMU interface for Xe
@ 2024-09-25 23:21 Vinay Belgaumkar
2024-09-25 23:21 ` [PATCH 1/3] drm/xe/pmu: Enable PMU interface Vinay Belgaumkar
` (5 more replies)
0 siblings, 6 replies; 19+ messages in thread
From: Vinay Belgaumkar @ 2024-09-25 23:21 UTC (permalink / raw)
To: intel-xe; +Cc: Vinay Belgaumkar, Rui Zhang, Rodrigo Vivi
Port i915 PMU event capture capability to Xe driver. This
has been requested by users of tools like gputop and turbostat
where PMU events are used to monitor GT freq and residencies.
Cc: Rui Zhang <rui.zhang@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
Aravind Iddamsetty (1):
drm/xe/pmu: Enable PMU interface
Vinay Belgaumkar (2):
drm/xe/pmu: Add GT C6 events
drm/xe/pmu: Add GT frequency events
drivers/gpu/drm/xe/Makefile | 2 +
drivers/gpu/drm/xe/xe_device.c | 6 +
drivers/gpu/drm/xe/xe_device_types.h | 4 +
drivers/gpu/drm/xe/xe_gt.c | 4 +
drivers/gpu/drm/xe/xe_gt_idle.c | 20 +-
drivers/gpu/drm/xe/xe_gt_idle.h | 1 +
drivers/gpu/drm/xe/xe_module.c | 5 +
drivers/gpu/drm/xe/xe_pmu.c | 960 +++++++++++++++++++++++++++
drivers/gpu/drm/xe/xe_pmu.h | 30 +
drivers/gpu/drm/xe/xe_pmu_types.h | 143 ++++
include/uapi/drm/xe_drm.h | 17 +
11 files changed, 1186 insertions(+), 6 deletions(-)
create mode 100644 drivers/gpu/drm/xe/xe_pmu.c
create mode 100644 drivers/gpu/drm/xe/xe_pmu.h
create mode 100644 drivers/gpu/drm/xe/xe_pmu_types.h
--
2.38.1
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH 1/3] drm/xe/pmu: Enable PMU interface
2024-09-25 23:21 [PATCH 0/3] drm/xe/pmu: PMU interface for Xe Vinay Belgaumkar
@ 2024-09-25 23:21 ` Vinay Belgaumkar
2024-09-26 9:25 ` Jani Nikula
2024-09-25 23:21 ` [PATCH 2/3] drm/xe/pmu: Add GT C6 events Vinay Belgaumkar
` (4 subsequent siblings)
5 siblings, 1 reply; 19+ messages in thread
From: Vinay Belgaumkar @ 2024-09-25 23:21 UTC (permalink / raw)
To: intel-xe
Cc: Aravind Iddamsetty, Bommu Krishnaiah, Riana Tauro, Rodrigo Vivi,
Vinay Belgaumkar
From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
Basic PMU enabling patch. Setup the basic framework
for adding events/timers. This patch was previously
reviewed here -
https://patchwork.freedesktop.org/series/119504/
The pmu base implementation is still from the
i915 driver.
v2: Review comments(Rodrigo) and do not init pmu for VFs
as they don't have access to freq and c6 residency anyways.
Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
Signed-off-by: Riana Tauro <riana.tauro@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
---
drivers/gpu/drm/xe/Makefile | 2 +
drivers/gpu/drm/xe/xe_device.c | 6 +
drivers/gpu/drm/xe/xe_device_types.h | 4 +
drivers/gpu/drm/xe/xe_module.c | 5 +
drivers/gpu/drm/xe/xe_pmu.c | 570 +++++++++++++++++++++++++++
drivers/gpu/drm/xe/xe_pmu.h | 26 ++
drivers/gpu/drm/xe/xe_pmu_types.h | 63 +++
include/uapi/drm/xe_drm.h | 8 +
8 files changed, 684 insertions(+)
create mode 100644 drivers/gpu/drm/xe/xe_pmu.c
create mode 100644 drivers/gpu/drm/xe/xe_pmu.h
create mode 100644 drivers/gpu/drm/xe/xe_pmu_types.h
diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
index ae245fbd91ee..bddb93e8b949 100644
--- a/drivers/gpu/drm/xe/Makefile
+++ b/drivers/gpu/drm/xe/Makefile
@@ -266,6 +266,8 @@ xe-$(CONFIG_DRM_XE_DISPLAY) += \
i915-display/skl_universal_plane.o \
i915-display/skl_watermark.o
+xe-$(CONFIG_PERF_EVENTS) += xe_pmu.o
+
ifeq ($(CONFIG_ACPI),y)
xe-$(CONFIG_DRM_XE_DISPLAY) += \
i915-display/intel_acpi.o \
diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
index cb5a9fd820cf..027a796d7431 100644
--- a/drivers/gpu/drm/xe/xe_device.c
+++ b/drivers/gpu/drm/xe/xe_device.c
@@ -762,6 +762,9 @@ int xe_device_probe(struct xe_device *xe)
for_each_gt(gt, xe, id)
xe_gt_sanitize_freq(gt);
+ if (!IS_SRIOV_VF(xe))
+ xe_pmu_register(&xe->pmu);
+
return devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
err_fini_display:
@@ -806,6 +809,9 @@ void xe_device_remove(struct xe_device *xe)
xe_heci_gsc_fini(xe);
+ if (!IS_SRIOV_VF(xe))
+ xe_pmu_unregister(&xe->pmu);
+
for_each_gt(gt, xe, id)
xe_gt_remove(gt);
}
diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
index 5ad96d283a71..51485d2eadcf 100644
--- a/drivers/gpu/drm/xe/xe_device_types.h
+++ b/drivers/gpu/drm/xe/xe_device_types.h
@@ -18,6 +18,7 @@
#include "xe_memirq_types.h"
#include "xe_oa.h"
#include "xe_platform_types.h"
+#include "xe_pmu.h"
#include "xe_pt_types.h"
#include "xe_sriov_types.h"
#include "xe_step_types.h"
@@ -513,6 +514,9 @@ struct xe_device {
int mode;
} wedged;
+ /** @pmu: performance monitoring unit */
+ struct xe_pmu pmu;
+
#ifdef TEST_VM_OPS_ERROR
/**
* @vm_inject_error_position: inject errors at different places in VM
diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
index 77ce9f9ca7a5..1bf2bf8447c0 100644
--- a/drivers/gpu/drm/xe/xe_module.c
+++ b/drivers/gpu/drm/xe/xe_module.c
@@ -14,6 +14,7 @@
#include "xe_hw_fence.h"
#include "xe_pci.h"
#include "xe_pm.h"
+#include "xe_pmu.h"
#include "xe_observation.h"
#include "xe_sched_job.h"
@@ -96,6 +97,10 @@ static const struct init_funcs init_funcs[] = {
.init = xe_sched_job_module_init,
.exit = xe_sched_job_module_exit,
},
+ {
+ .init = xe_pmu_init,
+ .exit = xe_pmu_exit,
+ },
{
.init = xe_register_pci_driver,
.exit = xe_unregister_pci_driver,
diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
new file mode 100644
index 000000000000..bdaea9ca1065
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_pmu.c
@@ -0,0 +1,570 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2024 Intel Corporation
+ */
+
+#include <drm/drm_drv.h>
+#include <drm/drm_managed.h>
+#include <drm/xe_drm.h>
+
+#include "regs/xe_gt_regs.h"
+#include "xe_device.h"
+#include "xe_force_wake.h"
+#include "xe_gt_clock.h"
+#include "xe_mmio.h"
+#include "xe_macros.h"
+#include "xe_pm.h"
+
+static cpumask_t xe_pmu_cpumask;
+static unsigned int xe_pmu_target_cpu = -1;
+
+/**
+ * DOC: Xe PMU (Performance Monitoring Unit)
+ *
+ * Expose events/counters like C6 residency and GT frequency to user land.
+ * Perf tool can be used to list and record these counters from the command
+ * line.
+ *
+ * Example commands to list/record supported perf events-
+ *
+ * >> perf list
+ * >> ls /sys/bus/event_source/devices/xe_0000_00_02.0/events/
+ *
+ * To record events-
+ * >> perf stat -e <event_name> -I <interval>
+ *
+ */
+
+static unsigned int config_gt_id(const u64 config)
+{
+ return config >> __XE_PMU_GT_SHIFT;
+}
+
+static u64 config_counter(const u64 config)
+{
+ return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
+}
+
+static void xe_pmu_event_destroy(struct perf_event *event)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+
+ drm_WARN_ON(&xe->drm, event->parent);
+
+ drm_dev_put(&xe->drm);
+}
+
+static int
+config_status(struct xe_device *xe, u64 config)
+{
+ unsigned int gt_id = config_gt_id(config);
+
+ if (gt_id >= XE_PMU_MAX_GT)
+ return -ENOENT;
+
+ switch (config_counter(config)) {
+ default:
+ return -ENOENT;
+ }
+
+ return 0;
+}
+
+static int xe_pmu_event_init(struct perf_event *event)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct xe_pmu *pmu = &xe->pmu;
+ int ret;
+
+ if (pmu->closed)
+ return -ENODEV;
+
+ if (event->attr.type != event->pmu->type)
+ return -ENOENT;
+
+ /* unsupported modes and filters */
+ if (event->attr.sample_period) /* no sampling */
+ return -EINVAL;
+
+ if (has_branch_stack(event))
+ return -EOPNOTSUPP;
+
+ if (event->cpu < 0)
+ return -EINVAL;
+
+ /* only allow running on one cpu at a time */
+ if (!cpumask_test_cpu(event->cpu, &xe_pmu_cpumask))
+ return -EINVAL;
+
+ ret = config_status(xe, event->attr.config);
+ if (ret)
+ return ret;
+
+ if (!event->parent) {
+ drm_dev_get(&xe->drm);
+ event->destroy = xe_pmu_event_destroy;
+ }
+
+ return 0;
+}
+
+static u64 __xe_pmu_event_read(struct perf_event *event)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ const unsigned int gt_id = config_gt_id(event->attr.config);
+ const u64 config = event->attr.config;
+ struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
+ u64 val = 0;
+
+ switch (config_counter(config)) {
+ default:
+ drm_warn(>->tile->xe->drm, "unknown pmu event\n");
+ }
+
+ return val;
+}
+
+static void xe_pmu_event_read(struct perf_event *event)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct hw_perf_event *hwc = &event->hw;
+ struct xe_pmu *pmu = &xe->pmu;
+ u64 prev, new;
+
+ if (pmu->closed) {
+ event->hw.state = PERF_HES_STOPPED;
+ return;
+ }
+again:
+ prev = local64_read(&hwc->prev_count);
+ new = __xe_pmu_event_read(event);
+
+ if (local64_cmpxchg(&hwc->prev_count, prev, new) != prev)
+ goto again;
+
+ local64_add(new - prev, &event->count);
+}
+
+static void xe_pmu_enable(struct perf_event *event)
+{
+ /*
+ * Store the current counter value so we can report the correct delta
+ * for all listeners. Even when the event was already enabled and has
+ * an existing non-zero value.
+ */
+ local64_set(&event->hw.prev_count, __xe_pmu_event_read(event));
+}
+
+static void xe_pmu_event_start(struct perf_event *event, int flags)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct xe_pmu *pmu = &xe->pmu;
+
+ if (pmu->closed)
+ return;
+
+ xe_pmu_enable(event);
+ event->hw.state = 0;
+}
+
+static void xe_pmu_event_stop(struct perf_event *event, int flags)
+{
+ if (flags & PERF_EF_UPDATE)
+ xe_pmu_event_read(event);
+
+ event->hw.state = PERF_HES_STOPPED;
+}
+
+static int xe_pmu_event_add(struct perf_event *event, int flags)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct xe_pmu *pmu = &xe->pmu;
+
+ if (pmu->closed)
+ return -ENODEV;
+
+ if (flags & PERF_EF_START)
+ xe_pmu_event_start(event, flags);
+
+ return 0;
+}
+
+static void xe_pmu_event_del(struct perf_event *event, int flags)
+{
+ xe_pmu_event_stop(event, PERF_EF_UPDATE);
+}
+
+static int xe_pmu_event_event_idx(struct perf_event *event)
+{
+ return 0;
+}
+
+struct xe_ext_attribute {
+ struct device_attribute attr;
+ unsigned long val;
+};
+
+static ssize_t xe_pmu_event_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct xe_ext_attribute *eattr;
+
+ eattr = container_of(attr, struct xe_ext_attribute, attr);
+ return sprintf(buf, "config=0x%lx\n", eattr->val);
+}
+
+static ssize_t cpumask_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ return cpumap_print_to_pagebuf(true, buf, &xe_pmu_cpumask);
+}
+
+static DEVICE_ATTR_RO(cpumask);
+
+static struct attribute *xe_cpumask_attrs[] = {
+ &dev_attr_cpumask.attr,
+ NULL,
+};
+
+static const struct attribute_group xe_pmu_cpumask_attr_group = {
+ .attrs = xe_cpumask_attrs,
+};
+
+#define __event(__counter, __name, __unit) \
+{ \
+ .counter = (__counter), \
+ .name = (__name), \
+ .unit = (__unit), \
+}
+
+static struct xe_ext_attribute *
+add_xe_attr(struct xe_ext_attribute *attr, const char *name, u64 config)
+{
+ sysfs_attr_init(&attr->attr.attr);
+ attr->attr.attr.name = name;
+ attr->attr.attr.mode = 0444;
+ attr->attr.show = xe_pmu_event_show;
+ attr->val = config;
+
+ return ++attr;
+}
+
+static struct perf_pmu_events_attr *
+add_pmu_attr(struct perf_pmu_events_attr *attr, const char *name,
+ const char *str)
+{
+ sysfs_attr_init(&attr->attr.attr);
+ attr->attr.attr.name = name;
+ attr->attr.attr.mode = 0444;
+ attr->attr.show = perf_event_sysfs_show;
+ attr->event_str = str;
+
+ return ++attr;
+}
+
+static struct attribute **
+create_event_attributes(struct xe_pmu *pmu)
+{
+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
+ static const struct {
+ unsigned int counter;
+ const char *name;
+ const char *unit;
+ } events[] = {
+ };
+
+ struct perf_pmu_events_attr *pmu_attr = NULL, *pmu_iter;
+ struct xe_ext_attribute *xe_attr = NULL, *xe_iter;
+ struct attribute **attr = NULL, **attr_iter;
+ unsigned int count = 0;
+ unsigned int i, j;
+ struct xe_gt *gt;
+
+ /* Count how many counters we will be exposing. */
+ for_each_gt(gt, xe, j) {
+ for (i = 0; i < ARRAY_SIZE(events); i++) {
+ u64 config = ___XE_PMU_OTHER(j, events[i].counter);
+
+ if (!config_status(xe, config))
+ count++;
+ }
+ }
+
+ /* Allocate attribute objects and table. */
+ xe_attr = kcalloc(count, sizeof(*xe_attr), GFP_KERNEL);
+ if (!xe_attr)
+ goto err_alloc;
+
+ pmu_attr = kcalloc(count, sizeof(*pmu_attr), GFP_KERNEL);
+ if (!pmu_attr)
+ goto err_alloc;
+
+ /* Max one pointer of each attribute type plus a termination entry. */
+ attr = kcalloc(count * 2 + 1, sizeof(*attr), GFP_KERNEL);
+ if (!attr)
+ goto err_alloc;
+
+ xe_iter = xe_attr;
+ pmu_iter = pmu_attr;
+ attr_iter = attr;
+
+ for_each_gt(gt, xe, j) {
+ for (i = 0; i < ARRAY_SIZE(events); i++) {
+ u64 config = ___XE_PMU_OTHER(j, events[i].counter);
+ char *str;
+
+ if (config_status(xe, config))
+ continue;
+
+ str = kasprintf(GFP_KERNEL, "%s-gt%u",
+ events[i].name, j);
+ if (!str)
+ goto err;
+
+ *attr_iter++ = &xe_iter->attr.attr;
+ xe_iter = add_xe_attr(xe_iter, str, config);
+
+ if (events[i].unit) {
+ str = kasprintf(GFP_KERNEL, "%s-gt%u.unit",
+ events[i].name, j);
+ if (!str)
+ goto err;
+
+ *attr_iter++ = &pmu_iter->attr.attr;
+ pmu_iter = add_pmu_attr(pmu_iter, str,
+ events[i].unit);
+ }
+ }
+ }
+
+ pmu->xe_attr = xe_attr;
+ pmu->pmu_attr = pmu_attr;
+
+ return attr;
+
+err:
+ for (attr_iter = attr; *attr_iter; attr_iter++)
+ kfree((*attr_iter)->name);
+
+err_alloc:
+ kfree(attr);
+ kfree(xe_attr);
+ kfree(pmu_attr);
+
+ return NULL;
+}
+
+static void free_event_attributes(struct xe_pmu *pmu)
+{
+ struct attribute **attr_iter = pmu->events_attr_group.attrs;
+
+ for (; *attr_iter; attr_iter++)
+ kfree((*attr_iter)->name);
+
+ kfree(pmu->events_attr_group.attrs);
+ kfree(pmu->xe_attr);
+ kfree(pmu->pmu_attr);
+
+ pmu->events_attr_group.attrs = NULL;
+ pmu->xe_attr = NULL;
+ pmu->pmu_attr = NULL;
+}
+
+static int xe_pmu_cpu_online(unsigned int cpu, struct hlist_node *node)
+{
+ struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
+
+ XE_WARN_ON(!pmu->base.event_init);
+
+ /* Select the first online CPU as a designated reader. */
+ if (cpumask_empty(&xe_pmu_cpumask))
+ cpumask_set_cpu(cpu, &xe_pmu_cpumask);
+
+ return 0;
+}
+
+static int xe_pmu_cpu_offline(unsigned int cpu, struct hlist_node *node)
+{
+ struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
+ unsigned int target = xe_pmu_target_cpu;
+
+ /*
+ * Unregistering an instance generates a CPU offline event which we must
+ * ignore to avoid incorrectly modifying the shared xe_pmu_cpumask.
+ */
+ if (pmu->closed)
+ return 0;
+
+ if (cpumask_test_and_clear_cpu(cpu, &xe_pmu_cpumask)) {
+ target = cpumask_any_but(topology_sibling_cpumask(cpu), cpu);
+
+ /* Migrate events if there is a valid target */
+ if (target < nr_cpu_ids) {
+ cpumask_set_cpu(target, &xe_pmu_cpumask);
+ xe_pmu_target_cpu = target;
+ }
+ }
+
+ if (target < nr_cpu_ids && target != pmu->cpuhp.cpu) {
+ perf_pmu_migrate_context(&pmu->base, cpu, target);
+ pmu->cpuhp.cpu = target;
+ }
+
+ return 0;
+}
+
+static enum cpuhp_state cpuhp_slot = CPUHP_INVALID;
+
+/**
+ * xe_pmu_init() - Setup CPU hotplug state/callbacks for Xe PMU
+ *
+ * Returns: 0 if successful, else error code
+ */
+int xe_pmu_init(void)
+{
+ int ret;
+
+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
+ "perf/x86/intel/xe:online",
+ xe_pmu_cpu_online,
+ xe_pmu_cpu_offline);
+ if (ret < 0)
+ pr_notice("Failed to setup cpuhp state for xe PMU! (%d)\n",
+ ret);
+ else
+ cpuhp_slot = ret;
+
+ return 0;
+}
+
+/**
+ * xe_pmu_exit() - Remove CPU hotplug state/callbacks for Xe PMU
+ */
+void xe_pmu_exit(void)
+{
+ if (cpuhp_slot != CPUHP_INVALID)
+ cpuhp_remove_multi_state(cpuhp_slot);
+}
+
+static int xe_pmu_register_cpuhp_state(struct xe_pmu *pmu)
+{
+ if (cpuhp_slot == CPUHP_INVALID)
+ return -EINVAL;
+
+ return cpuhp_state_add_instance(cpuhp_slot, &pmu->cpuhp.node);
+}
+
+static void xe_pmu_unregister_cpuhp_state(struct xe_pmu *pmu)
+{
+ cpuhp_state_remove_instance(cpuhp_slot, &pmu->cpuhp.node);
+}
+
+/**
+ * xe_pmu_unregister() - Remove/cleanup PMU registration
+ */
+void xe_pmu_unregister(void *arg)
+{
+ struct xe_pmu *pmu = arg;
+
+ if (!pmu->base.event_init)
+ return;
+
+ /*
+ * "Disconnect" the PMU callbacks - since all are atomic synchronize_rcu
+ * ensures all currently executing ones will have exited before we
+ * proceed with unregistration.
+ */
+ pmu->closed = true;
+ synchronize_rcu();
+
+ xe_pmu_unregister_cpuhp_state(pmu);
+
+ perf_pmu_unregister(&pmu->base);
+ pmu->base.event_init = NULL;
+ kfree(pmu->base.attr_groups);
+ kfree(pmu->name);
+ free_event_attributes(pmu);
+}
+
+/**
+ * xe_pmu_register() - Define basic PMU properties for Xe and add event callbacks.
+ *
+ */
+void xe_pmu_register(struct xe_pmu *pmu)
+{
+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
+ const struct attribute_group *attr_groups[] = {
+ &pmu->events_attr_group,
+ &xe_pmu_cpumask_attr_group,
+ NULL
+ };
+
+ int ret = -ENOMEM;
+
+ spin_lock_init(&pmu->lock);
+ pmu->cpuhp.cpu = -1;
+
+ pmu->name = kasprintf(GFP_KERNEL,
+ "xe_%s",
+ dev_name(xe->drm.dev));
+ if (pmu->name)
+ /* tools/perf reserves colons as special. */
+ strreplace((char *)pmu->name, ':', '_');
+
+ if (!pmu->name)
+ goto err;
+
+ pmu->events_attr_group.name = "events";
+ pmu->events_attr_group.attrs = create_event_attributes(pmu);
+ if (!pmu->events_attr_group.attrs)
+ goto err_name;
+
+ pmu->base.attr_groups = kmemdup(attr_groups, sizeof(attr_groups),
+ GFP_KERNEL);
+ if (!pmu->base.attr_groups)
+ goto err_attr;
+
+ pmu->base.module = THIS_MODULE;
+ pmu->base.task_ctx_nr = perf_invalid_context;
+ pmu->base.event_init = xe_pmu_event_init;
+ pmu->base.add = xe_pmu_event_add;
+ pmu->base.del = xe_pmu_event_del;
+ pmu->base.start = xe_pmu_event_start;
+ pmu->base.stop = xe_pmu_event_stop;
+ pmu->base.read = xe_pmu_event_read;
+ pmu->base.event_idx = xe_pmu_event_event_idx;
+
+ ret = perf_pmu_register(&pmu->base, pmu->name, -1);
+ if (ret)
+ goto err_groups;
+
+ ret = xe_pmu_register_cpuhp_state(pmu);
+ if (ret)
+ goto err_unreg;
+
+ ret = devm_add_action_or_reset(xe->drm.dev, xe_pmu_unregister, pmu);
+ if (ret)
+ goto err_cpuhp;
+
+ return;
+
+err_cpuhp:
+ xe_pmu_unregister_cpuhp_state(pmu);
+err_unreg:
+ perf_pmu_unregister(&pmu->base);
+err_groups:
+ kfree(pmu->base.attr_groups);
+err_attr:
+ pmu->base.event_init = NULL;
+ free_event_attributes(pmu);
+err_name:
+ kfree(pmu->name);
+err:
+ drm_notice(&xe->drm, "Failed to register PMU!\n");
+}
diff --git a/drivers/gpu/drm/xe/xe_pmu.h b/drivers/gpu/drm/xe/xe_pmu.h
new file mode 100644
index 000000000000..af8108ebef87
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_pmu.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2024 Intel Corporation
+ */
+
+#ifndef _XE_PMU_H_
+#define _XE_PMU_H_
+
+#include "xe_pmu_types.h"
+
+struct xe_gt;
+
+#if IS_ENABLED(CONFIG_PERF_EVENTS)
+int xe_pmu_init(void);
+void xe_pmu_exit(void);
+void xe_pmu_register(struct xe_pmu *pmu);
+void xe_pmu_unregister(void *arg);
+#else
+static inline int xe_pmu_init(void) { return 0; }
+static inline void xe_pmu_exit(void) {}
+static inline void xe_pmu_register(struct xe_pmu *pmu) {}
+static void xe_pmu_unregister(void *arg) {}
+#endif
+
+#endif
+
diff --git a/drivers/gpu/drm/xe/xe_pmu_types.h b/drivers/gpu/drm/xe/xe_pmu_types.h
new file mode 100644
index 000000000000..ca0e7cbe2081
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_pmu_types.h
@@ -0,0 +1,63 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2024 Intel Corporation
+ */
+
+#ifndef _XE_PMU_TYPES_H_
+#define _XE_PMU_TYPES_H_
+
+#include <linux/perf_event.h>
+#include <linux/spinlock_types.h>
+#include <uapi/drm/xe_drm.h>
+
+enum {
+ __XE_NUM_PMU_SAMPLERS
+};
+
+#define XE_PMU_MAX_GT 2
+
+struct xe_pmu {
+ /**
+ * @cpuhp: Struct used for CPU hotplug handling.
+ */
+ struct {
+ struct hlist_node node;
+ unsigned int cpu;
+ } cpuhp;
+ /**
+ * @base: PMU base.
+ */
+ struct pmu base;
+ /**
+ * @closed: xe is unregistering.
+ */
+ bool closed;
+ /**
+ * @name: Name as registered with perf core.
+ */
+ const char *name;
+ /**
+ * @lock: Lock protecting enable mask and ref count handling.
+ */
+ spinlock_t lock;
+ /**
+ * @sample: Current and previous (raw) counters.
+ *
+ * These counters are updated when the device is awake.
+ */
+ u64 sample[XE_PMU_MAX_GT][__XE_NUM_PMU_SAMPLERS];
+ /**
+ * @events_attr_group: Device events attribute group.
+ */
+ struct attribute_group events_attr_group;
+ /**
+ * @xe_attr: Memory block holding device attributes.
+ */
+ void *xe_attr;
+ /**
+ * @pmu_attr: Memory block holding device attributes.
+ */
+ void *pmu_attr;
+};
+
+#endif
diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
index b6fbe4988f2e..2c5f258eee3a 100644
--- a/include/uapi/drm/xe_drm.h
+++ b/include/uapi/drm/xe_drm.h
@@ -1389,6 +1389,14 @@ struct drm_xe_wait_user_fence {
__u64 reserved[2];
};
+/*
+ * Top bits of every counter are GT id.
+ */
+#define __XE_PMU_GT_SHIFT (56)
+
+#define ___XE_PMU_OTHER(gt, x) \
+ (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
+
/**
* enum drm_xe_observation_type - Observation stream types
*/
--
2.38.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 2/3] drm/xe/pmu: Add GT C6 events
2024-09-25 23:21 [PATCH 0/3] drm/xe/pmu: PMU interface for Xe Vinay Belgaumkar
2024-09-25 23:21 ` [PATCH 1/3] drm/xe/pmu: Enable PMU interface Vinay Belgaumkar
@ 2024-09-25 23:21 ` Vinay Belgaumkar
2024-09-25 23:21 ` [PATCH 3/3] drm/xe/pmu: Add GT frequency events Vinay Belgaumkar
` (3 subsequent siblings)
5 siblings, 0 replies; 19+ messages in thread
From: Vinay Belgaumkar @ 2024-09-25 23:21 UTC (permalink / raw)
To: intel-xe; +Cc: Vinay Belgaumkar, Rodrigo Vivi
Provide a PMU interface for GT C6 residency counters. The implementation
is ported over from the i915 PMU code. Residency is provided in units of
ms(similar to sysfs entry - /sys/class/drm/card0/device/tile0/gt0/gtidle).
Following PMU events are being added-
>> perf list | grep rc6
xe_0000_00_02.0/rc6-residency-gt0/ [Kernel PMU event]
xe_0000_00_02.0/rc6-residency-gt1/ [Kernel PMU event]
v2: Checkpatch fix, move timer code to next patch
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
---
drivers/gpu/drm/xe/xe_gt.c | 2 +
drivers/gpu/drm/xe/xe_gt_idle.c | 20 ++--
drivers/gpu/drm/xe/xe_gt_idle.h | 1 +
drivers/gpu/drm/xe/xe_pmu.c | 172 ++++++++++++++++++++++++++++++
drivers/gpu/drm/xe/xe_pmu.h | 2 +
drivers/gpu/drm/xe/xe_pmu_types.h | 58 ++++++++++
include/uapi/drm/xe_drm.h | 4 +
7 files changed, 253 insertions(+), 6 deletions(-)
diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c
index 9b0218109647..7a190c49c573 100644
--- a/drivers/gpu/drm/xe/xe_gt.c
+++ b/drivers/gpu/drm/xe/xe_gt.c
@@ -872,6 +872,8 @@ int xe_gt_suspend(struct xe_gt *gt)
xe_gt_idle_disable_pg(gt);
+ xe_pmu_suspend(gt);
+
xe_gt_disable_host_l2_vram(gt);
XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FORCEWAKE_ALL));
diff --git a/drivers/gpu/drm/xe/xe_gt_idle.c b/drivers/gpu/drm/xe/xe_gt_idle.c
index 531924b6c0a1..e0a12ac7387c 100644
--- a/drivers/gpu/drm/xe/xe_gt_idle.c
+++ b/drivers/gpu/drm/xe/xe_gt_idle.c
@@ -277,18 +277,26 @@ static ssize_t idle_status_show(struct device *dev,
}
static DEVICE_ATTR_RO(idle_status);
-static ssize_t idle_residency_ms_show(struct device *dev,
- struct device_attribute *attr, char *buff)
+u64 xe_gt_idle_residency(struct xe_gt *gt)
{
- struct xe_gt_idle *gtidle = dev_to_gtidle(dev);
+ struct xe_device *xe = gt_to_xe(gt);
+ struct xe_gt_idle *gtidle = >->gtidle;
struct xe_guc_pc *pc = gtidle_to_pc(gtidle);
u64 residency;
- xe_pm_runtime_get(pc_to_xe(pc));
+ xe_pm_runtime_get(xe);
residency = gtidle->idle_residency(pc);
- xe_pm_runtime_put(pc_to_xe(pc));
+ xe_pm_runtime_put(xe);
+
+ return get_residency_ms(gtidle, residency);
+}
+
+static ssize_t idle_residency_ms_show(struct device *dev,
+ struct device_attribute *attr, char *buff)
+{
+ struct xe_gt_idle *gtidle = dev_to_gtidle(dev);
- return sysfs_emit(buff, "%llu\n", get_residency_ms(gtidle, residency));
+ return sysfs_emit(buff, "%llu\n", xe_gt_idle_residency(gtidle_to_gt(gtidle)));
}
static DEVICE_ATTR_RO(idle_residency_ms);
diff --git a/drivers/gpu/drm/xe/xe_gt_idle.h b/drivers/gpu/drm/xe/xe_gt_idle.h
index 4455a6501cb0..887791f653ac 100644
--- a/drivers/gpu/drm/xe/xe_gt_idle.h
+++ b/drivers/gpu/drm/xe/xe_gt_idle.h
@@ -17,5 +17,6 @@ void xe_gt_idle_disable_c6(struct xe_gt *gt);
void xe_gt_idle_enable_pg(struct xe_gt *gt);
void xe_gt_idle_disable_pg(struct xe_gt *gt);
int xe_gt_idle_pg_print(struct xe_gt *gt, struct drm_printer *p);
+u64 xe_gt_idle_residency(struct xe_gt *gt);
#endif /* _XE_GT_IDLE_H_ */
diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
index bdaea9ca1065..b1b38d245e00 100644
--- a/drivers/gpu/drm/xe/xe_pmu.c
+++ b/drivers/gpu/drm/xe/xe_pmu.c
@@ -11,10 +11,15 @@
#include "xe_device.h"
#include "xe_force_wake.h"
#include "xe_gt_clock.h"
+#include "xe_gt_idle.h"
+#include "xe_guc_pc.h"
#include "xe_mmio.h"
#include "xe_macros.h"
+#include "xe_module.h"
#include "xe_pm.h"
+#define FREQUENCY 200
+
static cpumask_t xe_pmu_cpumask;
static unsigned int xe_pmu_target_cpu = -1;
@@ -35,6 +40,11 @@ static unsigned int xe_pmu_target_cpu = -1;
*
*/
+static struct xe_pmu *event_to_pmu(struct perf_event *event)
+{
+ return container_of(event->pmu, struct xe_pmu, base);
+}
+
static unsigned int config_gt_id(const u64 config)
{
return config >> __XE_PMU_GT_SHIFT;
@@ -45,6 +55,35 @@ static u64 config_counter(const u64 config)
return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
}
+static unsigned int other_bit(const u64 config)
+{
+ unsigned int val;
+
+ switch (config_counter(config)) {
+ case XE_PMU_RC6_RESIDENCY:
+ val = __XE_PMU_RC6_RESIDENCY_ENABLED;
+ break;
+ default:
+ /*
+ * Events that do not require sampling, or tracking state
+ * transitions between enabled and disabled can be ignored.
+ */
+ return -1;
+ }
+
+ return config_gt_id(config) * __XE_PMU_TRACKED_EVENT_COUNT + val;
+}
+
+static unsigned int config_bit(const u64 config)
+{
+ return other_bit(config);
+}
+
+static unsigned int event_bit(struct perf_event *event)
+{
+ return config_bit(event->attr.config);
+}
+
static void xe_pmu_event_destroy(struct perf_event *event)
{
struct xe_device *xe =
@@ -64,6 +103,10 @@ config_status(struct xe_device *xe, u64 config)
return -ENOENT;
switch (config_counter(config)) {
+ case XE_PMU_RC6_RESIDENCY:
+ if (xe->info.skip_guc_pc)
+ return -ENODEV;
+ break;
default:
return -ENOENT;
}
@@ -110,6 +153,63 @@ static int xe_pmu_event_init(struct perf_event *event)
return 0;
}
+static inline s64 ktime_since_raw(const ktime_t kt)
+{
+ return ktime_to_ns(ktime_sub(ktime_get_raw(), kt));
+}
+
+static u64 read_sample(struct xe_pmu *pmu, unsigned int gt_id, int sample)
+{
+ return pmu->event_sample[gt_id][sample].cur;
+}
+
+static void
+store_sample(struct xe_pmu *pmu, unsigned int gt_id, int sample, u64 val)
+{
+ pmu->event_sample[gt_id][sample].cur = val;
+}
+
+static u64 get_rc6(struct xe_gt *gt)
+{
+ struct xe_device *xe = gt_to_xe(gt);
+ const unsigned int gt_id = gt->info.id;
+ struct xe_pmu *pmu = &xe->pmu;
+ bool device_awake;
+ unsigned long flags;
+ u64 val;
+
+ device_awake = xe_pm_runtime_get_if_active(xe);
+ if (device_awake) {
+ val = xe_gt_idle_residency(gt);
+ xe_pm_runtime_put(xe);
+ }
+
+ spin_lock_irqsave(&pmu->lock, flags);
+
+ if (device_awake) {
+ store_sample(pmu, gt_id, __XE_SAMPLE_RC6, val);
+ } else {
+ /*
+ * We think we are runtime suspended.
+ *
+ * Report the delta from when the device was suspended to now,
+ * on top of the last known real value, as the approximated RC6
+ * counter value.
+ */
+ val = ktime_since_raw(pmu->sleep_last[gt_id]);
+ val += read_sample(pmu, gt_id, __XE_SAMPLE_RC6);
+ }
+
+ if (val < read_sample(pmu, gt_id, __XE_SAMPLE_RC6_LAST_REPORTED))
+ val = read_sample(pmu, gt_id, __XE_SAMPLE_RC6_LAST_REPORTED);
+ else
+ store_sample(pmu, gt_id, __XE_SAMPLE_RC6_LAST_REPORTED, val);
+
+ spin_unlock_irqrestore(&pmu->lock, flags);
+
+ return val;
+}
+
static u64 __xe_pmu_event_read(struct perf_event *event)
{
struct xe_device *xe =
@@ -120,6 +220,9 @@ static u64 __xe_pmu_event_read(struct perf_event *event)
u64 val = 0;
switch (config_counter(config)) {
+ case XE_PMU_RC6_RESIDENCY:
+ val = get_rc6(gt);
+ break;
default:
drm_warn(>->tile->xe->drm, "unknown pmu event\n");
}
@@ -151,6 +254,28 @@ static void xe_pmu_event_read(struct perf_event *event)
static void xe_pmu_enable(struct perf_event *event)
{
+ struct xe_pmu *pmu = event_to_pmu(event);
+ const unsigned int bit = event_bit(event);
+ unsigned long flags;
+
+ if (bit == -1)
+ goto update;
+
+ spin_lock_irqsave(&pmu->lock, flags);
+
+ /*
+ * Update the bitmask of enabled events and increment
+ * the event reference counter.
+ */
+ BUILD_BUG_ON(ARRAY_SIZE(pmu->enable_count) != XE_PMU_MASK_BITS);
+ XE_WARN_ON(bit >= ARRAY_SIZE(pmu->enable_count));
+ XE_WARN_ON(pmu->enable_count[bit] == ~0);
+
+ pmu->enable |= BIT(bit);
+ pmu->enable_count[bit]++;
+
+ spin_unlock_irqrestore(&pmu->lock, flags);
+update:
/*
* Store the current counter value so we can report the correct delta
* for all listeners. Even when the event was already enabled and has
@@ -277,6 +402,7 @@ create_event_attributes(struct xe_pmu *pmu)
const char *name;
const char *unit;
} events[] = {
+ __event(0, "rc6-residency", "ms"),
};
struct perf_pmu_events_attr *pmu_attr = NULL, *pmu_iter;
@@ -465,6 +591,32 @@ static void xe_pmu_unregister_cpuhp_state(struct xe_pmu *pmu)
cpuhp_state_remove_instance(cpuhp_slot, &pmu->cpuhp.node);
}
+static void store_rc6_residency(struct xe_gt *gt)
+{
+ struct xe_device *xe = gt_to_xe(gt);
+ struct xe_pmu *pmu = &xe->pmu;
+
+ store_sample(pmu, gt->info.id, __XE_SAMPLE_RC6,
+ xe_gt_idle_residency(gt));
+ pmu->sleep_last[gt->info.id] = ktime_get_raw();
+}
+
+/**
+ * xe_pmu_suspend() - Save residency count before suspend
+ */
+void xe_pmu_suspend(struct xe_gt *gt)
+{
+ struct xe_device *xe = gt_to_xe(gt);
+ struct xe_pmu *pmu = &xe->pmu;
+
+ if (!pmu->base.event_init)
+ return;
+
+ spin_lock_irq(&pmu->lock);
+ store_rc6_residency(gt);
+ spin_unlock_irq(&pmu->lock);
+}
+
/**
* xe_pmu_unregister() - Remove/cleanup PMU registration
*/
@@ -492,6 +644,24 @@ void xe_pmu_unregister(void *arg)
free_event_attributes(pmu);
}
+static void init_rc6(struct xe_pmu *pmu)
+{
+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
+ struct xe_gt *gt;
+ unsigned int j;
+
+ for_each_gt(gt, xe, j) {
+ xe_pm_runtime_get(xe);
+ u64 val = xe_gt_idle_residency(gt);
+
+ store_sample(pmu, j, __XE_SAMPLE_RC6, val);
+ store_sample(pmu, j, __XE_SAMPLE_RC6_LAST_REPORTED,
+ val);
+ pmu->sleep_last[j] = ktime_get_raw();
+ xe_pm_runtime_put(xe);
+ }
+}
+
/**
* xe_pmu_register() - Define basic PMU properties for Xe and add event callbacks.
*
@@ -525,6 +695,8 @@ void xe_pmu_register(struct xe_pmu *pmu)
if (!pmu->events_attr_group.attrs)
goto err_name;
+ init_rc6(pmu);
+
pmu->base.attr_groups = kmemdup(attr_groups, sizeof(attr_groups),
GFP_KERNEL);
if (!pmu->base.attr_groups)
diff --git a/drivers/gpu/drm/xe/xe_pmu.h b/drivers/gpu/drm/xe/xe_pmu.h
index af8108ebef87..733f2c33039c 100644
--- a/drivers/gpu/drm/xe/xe_pmu.h
+++ b/drivers/gpu/drm/xe/xe_pmu.h
@@ -15,11 +15,13 @@ int xe_pmu_init(void);
void xe_pmu_exit(void);
void xe_pmu_register(struct xe_pmu *pmu);
void xe_pmu_unregister(void *arg);
+void xe_pmu_suspend(struct xe_gt *gt);
#else
static inline int xe_pmu_init(void) { return 0; }
static inline void xe_pmu_exit(void) {}
static inline void xe_pmu_register(struct xe_pmu *pmu) {}
static void xe_pmu_unregister(void *arg) {}
+void xe_pmu_suspend(struct xe_gt *gt) {}
#endif
#endif
diff --git a/drivers/gpu/drm/xe/xe_pmu_types.h b/drivers/gpu/drm/xe/xe_pmu_types.h
index ca0e7cbe2081..1213d2a73492 100644
--- a/drivers/gpu/drm/xe/xe_pmu_types.h
+++ b/drivers/gpu/drm/xe/xe_pmu_types.h
@@ -11,11 +11,34 @@
#include <uapi/drm/xe_drm.h>
enum {
+ __XE_SAMPLE_RC6,
+ __XE_SAMPLE_RC6_LAST_REPORTED,
__XE_NUM_PMU_SAMPLERS
};
#define XE_PMU_MAX_GT 2
+/*
+ * Non-engine events that we need to track enabled-disabled transition and
+ * current state.
+ */
+enum xe_pmu_tracked_events {
+ __XE_PMU_RC6_RESIDENCY_ENABLED,
+ __XE_PMU_TRACKED_EVENT_COUNT, /* count marker */
+};
+
+/*
+ * How many different events we track in the global PMU mask.
+ *
+ * It is also used to know to needed number of event reference counters.
+ */
+#define XE_PMU_MASK_BITS \
+ (XE_PMU_MAX_GT * __XE_PMU_TRACKED_EVENT_COUNT)
+
+struct xe_pmu_sample {
+ u64 cur;
+};
+
struct xe_pmu {
/**
* @cpuhp: Struct used for CPU hotplug handling.
@@ -58,6 +81,41 @@ struct xe_pmu {
* @pmu_attr: Memory block holding device attributes.
*/
void *pmu_attr;
+
+ /**
+ * @enable: Bitmask of specific enabled events.
+ *
+ * For some events we need to track their state and do some internal
+ * house keeping.
+ *
+ * Each engine event sampler type and event listed in enum
+ * i915_pmu_tracked_events gets a bit in this field.
+ *
+ * Low bits are engine samplers and other events continue from there.
+ */
+ u32 enable;
+
+ /**
+ * @enable_count: Reference counts for the enabled events.
+ *
+ * Array indices are mapped in the same way as bits in the @enable field
+ * and they are used to control sampling on/off when multiple clients
+ * are using the PMU API.
+ */
+ unsigned int enable_count[XE_PMU_MASK_BITS];
+ /**
+ * @sample: Current and previous (raw) counters for sampling events.
+ *
+ * These counters are updated from the i915 PMU sampling timer.
+ *
+ * Only global counters are held here, while the per-engine ones are in
+ * struct intel_engine_cs.
+ */
+ struct xe_pmu_sample event_sample[XE_PMU_MAX_GT][__XE_NUM_PMU_SAMPLERS];
+ /**
+ * @sleep_last: Last time GT parked for RC6 estimation.
+ */
+ ktime_t sleep_last[XE_PMU_MAX_GT];
};
#endif
diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
index 2c5f258eee3a..3ef3926551dd 100644
--- a/include/uapi/drm/xe_drm.h
+++ b/include/uapi/drm/xe_drm.h
@@ -1396,6 +1396,10 @@ struct drm_xe_wait_user_fence {
#define ___XE_PMU_OTHER(gt, x) \
(((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
+#define __XE_PMU_OTHER(x) ___XE_PMU_OTHER(0, x)
+
+#define XE_PMU_RC6_RESIDENCY __XE_PMU_OTHER(0)
+#define __XE_PMU_RC6_RESIDENCY(gt) ___XE_PMU_OTHER(gt, 0)
/**
* enum drm_xe_observation_type - Observation stream types
--
2.38.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 3/3] drm/xe/pmu: Add GT frequency events
2024-09-25 23:21 [PATCH 0/3] drm/xe/pmu: PMU interface for Xe Vinay Belgaumkar
2024-09-25 23:21 ` [PATCH 1/3] drm/xe/pmu: Enable PMU interface Vinay Belgaumkar
2024-09-25 23:21 ` [PATCH 2/3] drm/xe/pmu: Add GT C6 events Vinay Belgaumkar
@ 2024-09-25 23:21 ` Vinay Belgaumkar
2024-09-26 16:57 ` ✓ CI.Patch_applied: success for drm/xe/pmu: PMU interface for Xe Patchwork
` (2 subsequent siblings)
5 siblings, 0 replies; 19+ messages in thread
From: Vinay Belgaumkar @ 2024-09-25 23:21 UTC (permalink / raw)
To: intel-xe; +Cc: Vinay Belgaumkar, Rodrigo Vivi
Define PMU events for GT frequency (actual and requested). This is
a port from the i915 driver implementation.
Following PMU events are being added-
xe_0000_00_02.0/actual-frequency-gt0/ [Kernel PMU event]
xe_0000_00_02.0/actual-frequency-gt1/ [Kernel PMU event]
xe_0000_00_02.0/requested-frequency-gt0/ [Kernel PMU event]
xe_0000_00_02.0/requested-frequency-gt1/ [Kernel PMU event]
v2: Checkpatch fix, moved timer code to this patch
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
---
drivers/gpu/drm/xe/xe_gt.c | 2 +
drivers/gpu/drm/xe/xe_pmu.c | 218 ++++++++++++++++++++++++++++++
drivers/gpu/drm/xe/xe_pmu.h | 2 +
drivers/gpu/drm/xe/xe_pmu_types.h | 22 +++
include/uapi/drm/xe_drm.h | 5 +
5 files changed, 249 insertions(+)
diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c
index 7a190c49c573..796b330ffd62 100644
--- a/drivers/gpu/drm/xe/xe_gt.c
+++ b/drivers/gpu/drm/xe/xe_gt.c
@@ -929,6 +929,8 @@ int xe_gt_resume(struct xe_gt *gt)
xe_gt_idle_enable_pg(gt);
+ xe_pmu_resume(gt);
+
XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FORCEWAKE_ALL));
xe_gt_dbg(gt, "resumed\n");
diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
index b1b38d245e00..39bae25a92ed 100644
--- a/drivers/gpu/drm/xe/xe_pmu.c
+++ b/drivers/gpu/drm/xe/xe_pmu.c
@@ -63,6 +63,12 @@ static unsigned int other_bit(const u64 config)
case XE_PMU_RC6_RESIDENCY:
val = __XE_PMU_RC6_RESIDENCY_ENABLED;
break;
+ case XE_PMU_ACTUAL_FREQUENCY:
+ val = __XE_PMU_ACTUAL_FREQUENCY_ENABLED;
+ break;
+ case XE_PMU_REQUESTED_FREQUENCY:
+ val = __XE_PMU_REQUESTED_FREQUENCY_ENABLED;
+ break;
default:
/*
* Events that do not require sampling, or tracking state
@@ -79,6 +85,22 @@ static unsigned int config_bit(const u64 config)
return other_bit(config);
}
+static u32 config_mask(const u64 config)
+{
+ unsigned int bit = config_bit(config);
+
+ if (__builtin_constant_p(config))
+ BUILD_BUG_ON(bit >
+ BITS_PER_TYPE(typeof_member(struct xe_pmu,
+ enable)) - 1);
+ else
+ WARN_ON_ONCE(bit >
+ BITS_PER_TYPE(typeof_member(struct xe_pmu,
+ enable)) - 1);
+
+ return BIT(config_bit(config));
+}
+
static unsigned int event_bit(struct perf_event *event)
{
return config_bit(event->attr.config);
@@ -107,6 +129,10 @@ config_status(struct xe_device *xe, u64 config)
if (xe->info.skip_guc_pc)
return -ENODEV;
break;
+ case XE_PMU_ACTUAL_FREQUENCY:
+ fallthrough;
+ case XE_PMU_REQUESTED_FREQUENCY:
+ break;
default:
return -ENOENT;
}
@@ -169,6 +195,12 @@ store_sample(struct xe_pmu *pmu, unsigned int gt_id, int sample, u64 val)
pmu->event_sample[gt_id][sample].cur = val;
}
+static void
+add_sample_mult(struct xe_pmu *pmu, unsigned int gt_id, int sample, u32 val, u32 mul)
+{
+ pmu->event_sample[gt_id][sample].cur += mul_u32_u32(val, mul);
+}
+
static u64 get_rc6(struct xe_gt *gt)
{
struct xe_device *xe = gt_to_xe(gt);
@@ -214,6 +246,7 @@ static u64 __xe_pmu_event_read(struct perf_event *event)
{
struct xe_device *xe =
container_of(event->pmu, typeof(*xe), pmu.base);
+ struct xe_pmu *pmu = &xe->pmu;
const unsigned int gt_id = config_gt_id(event->attr.config);
const u64 config = event->attr.config;
struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
@@ -223,6 +256,18 @@ static u64 __xe_pmu_event_read(struct perf_event *event)
case XE_PMU_RC6_RESIDENCY:
val = get_rc6(gt);
break;
+ case XE_PMU_ACTUAL_FREQUENCY:
+ val =
+ div_u64(read_sample(pmu, gt_id,
+ __XE_SAMPLE_FREQ_ACT),
+ USEC_PER_SEC /* to MHz */);
+ break;
+ case XE_PMU_REQUESTED_FREQUENCY:
+ val =
+ div_u64(read_sample(pmu, gt_id,
+ __XE_SAMPLE_FREQ_REQ),
+ USEC_PER_SEC /* to MHz */);
+ break;
default:
drm_warn(>->tile->xe->drm, "unknown pmu event\n");
}
@@ -252,6 +297,144 @@ static void xe_pmu_event_read(struct perf_event *event)
local64_add(new - prev, &event->count);
}
+static u32 frequency_enabled_mask(void)
+{
+ unsigned int i;
+ u32 mask = 0;
+
+ for (i = 0; i < XE_PMU_MAX_GT; i++)
+ mask |= config_mask(__XE_PMU_ACTUAL_FREQUENCY(i)) |
+ config_mask(__XE_PMU_REQUESTED_FREQUENCY(i));
+
+ return mask;
+}
+
+static bool
+frequency_sampling_enabled(struct xe_pmu *pmu, unsigned int gt)
+{
+ return pmu->enable &
+ (config_mask(__XE_PMU_ACTUAL_FREQUENCY(gt)) |
+ config_mask(__XE_PMU_REQUESTED_FREQUENCY(gt)));
+}
+
+static void
+frequency_sample(struct xe_gt *gt, unsigned int period_ns)
+{
+ struct xe_device *xe = gt_to_xe(gt);
+ const unsigned int gt_id = gt->info.id;
+ struct xe_pmu *pmu = &xe->pmu;
+ bool device_awake;
+ int ret;
+ u32 cur_freq;
+
+ if (!frequency_sampling_enabled(pmu, gt_id))
+ return;
+
+ /* Report 0/0 (actual/requested) frequency while GT is suspended. */
+ device_awake = xe_pm_runtime_get_if_active(xe);
+ if (!device_awake)
+ return;
+
+ if (pmu->enable & config_mask(__XE_PMU_ACTUAL_FREQUENCY(gt_id))) {
+ u32 val;
+
+ /*
+ * We take a quick peek here without using forcewake
+ * so that we don't perturb the system under observation
+ * (forcewake => !rc6 => increased power use). We expect
+ * that if the read fails because it is outside of the
+ * mmio power well, then it will return 0 -- in which
+ * case we assume the system is running at the intended
+ * frequency. Fortunately, the read should rarely fail!
+ */
+ val = xe_guc_pc_get_act_freq(>->uc.guc.pc);
+
+ add_sample_mult(pmu, gt_id, __XE_SAMPLE_FREQ_ACT,
+ val, period_ns / 1000);
+ }
+
+ if (pmu->enable & config_mask(__XE_PMU_REQUESTED_FREQUENCY(gt_id))) {
+ ret = xe_guc_pc_get_cur_freq(>->uc.guc.pc, &cur_freq);
+ if (!ret)
+ add_sample_mult(pmu, gt_id, __XE_SAMPLE_FREQ_REQ,
+ cur_freq,
+ period_ns / 1000);
+ }
+
+ xe_pm_runtime_put(xe);
+}
+
+static enum hrtimer_restart xe_sample(struct hrtimer *hrtimer)
+{
+ struct xe_pmu *pmu = container_of(hrtimer, struct xe_pmu, timer);
+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
+ u64 period = max_t(u64, 10000, NSEC_PER_SEC / FREQUENCY);
+ unsigned int period_ns;
+ struct xe_gt *gt;
+ unsigned int i;
+ ktime_t now;
+
+ if (!READ_ONCE(pmu->timer_enabled))
+ return HRTIMER_NORESTART;
+
+ now = ktime_get();
+ period_ns = ktime_to_ns(ktime_sub(now, pmu->timer_last));
+ pmu->timer_last = now;
+
+ /*
+ * Strictly speaking the passed in period may not be 100% accurate for
+ * all internal calculation, since some amount of time can be spent on
+ * grabbing the forcewake. However the potential error from timer call-
+ * back delay greatly dominates this so we keep it simple.
+ */
+
+ for_each_gt(gt, xe, i) {
+ if (!(pmu->active_gts & BIT(i)))
+ continue;
+ frequency_sample(gt, period_ns);
+ }
+
+ hrtimer_forward(hrtimer, now, ns_to_ktime(period));
+
+ return HRTIMER_RESTART;
+}
+
+static bool pmu_needs_timer(struct xe_pmu *pmu)
+{
+ u32 enable;
+
+ /*
+ * Only some counters need the sampling timer.
+ *
+ * We start with a bitmask of all currently enabled events.
+ */
+ enable = pmu->enable;
+
+ /*
+ * Mask out all the ones which do not need the timer, or in
+ * other words keep all the ones that could need the timer.
+ */
+ enable &= frequency_enabled_mask();
+
+ /*
+ * If some bits remain it means we need the sampling timer running.
+ */
+ return enable;
+}
+
+static void __xe_pmu_maybe_start_timer(struct xe_pmu *pmu)
+{
+ u64 period = max_t(u64, 10000, NSEC_PER_SEC / FREQUENCY);
+
+ if (!pmu->timer_enabled && pmu_needs_timer(pmu)) {
+ pmu->timer_enabled = true;
+ pmu->timer_last = ktime_get();
+ hrtimer_start_range_ns(&pmu->timer,
+ ns_to_ktime(period), 0,
+ HRTIMER_MODE_REL_PINNED);
+ }
+}
+
static void xe_pmu_enable(struct perf_event *event)
{
struct xe_pmu *pmu = event_to_pmu(event);
@@ -263,6 +446,11 @@ static void xe_pmu_enable(struct perf_event *event)
spin_lock_irqsave(&pmu->lock, flags);
+ /*
+ * Start the sampling timer if needed and not already enabled.
+ */
+ __xe_pmu_maybe_start_timer(pmu);
+
/*
* Update the bitmask of enabled events and increment
* the event reference counter.
@@ -403,6 +591,8 @@ create_event_attributes(struct xe_pmu *pmu)
const char *unit;
} events[] = {
__event(0, "rc6-residency", "ms"),
+ __event(1, "actual-frequency", "M"),
+ __event(2, "requested-frequency", "M"),
};
struct perf_pmu_events_attr *pmu_attr = NULL, *pmu_iter;
@@ -617,6 +807,30 @@ void xe_pmu_suspend(struct xe_gt *gt)
spin_unlock_irq(&pmu->lock);
}
+/**
+ * xe_pmu_resume() - Restart the timer if needed
+ */
+void xe_pmu_resume(struct xe_gt *gt)
+{
+ struct xe_device *xe = gt_to_xe(gt);
+ struct xe_pmu *pmu = &xe->pmu;
+
+ if (!pmu->base.event_init)
+ return;
+
+ spin_lock_irq(&pmu->lock);
+
+ /*
+ * Re-enable sampling timer when GPU goes active.
+ */
+ if (pmu->active_gts == 0)
+ __xe_pmu_maybe_start_timer(pmu);
+
+ pmu->active_gts |= BIT(gt->info.id);
+
+ spin_unlock_irq(&pmu->lock);
+}
+
/**
* xe_pmu_unregister() - Remove/cleanup PMU registration
*/
@@ -635,6 +849,8 @@ void xe_pmu_unregister(void *arg)
pmu->closed = true;
synchronize_rcu();
+ hrtimer_cancel(&pmu->timer);
+
xe_pmu_unregister_cpuhp_state(pmu);
perf_pmu_unregister(&pmu->base);
@@ -678,6 +894,8 @@ void xe_pmu_register(struct xe_pmu *pmu)
int ret = -ENOMEM;
spin_lock_init(&pmu->lock);
+ hrtimer_init(&pmu->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+ pmu->timer.function = xe_sample;
pmu->cpuhp.cpu = -1;
pmu->name = kasprintf(GFP_KERNEL,
diff --git a/drivers/gpu/drm/xe/xe_pmu.h b/drivers/gpu/drm/xe/xe_pmu.h
index 733f2c33039c..47d7f0a8e4bb 100644
--- a/drivers/gpu/drm/xe/xe_pmu.h
+++ b/drivers/gpu/drm/xe/xe_pmu.h
@@ -16,12 +16,14 @@ void xe_pmu_exit(void);
void xe_pmu_register(struct xe_pmu *pmu);
void xe_pmu_unregister(void *arg);
void xe_pmu_suspend(struct xe_gt *gt);
+void xe_pmu_resume(struct xe_gt *gt);
#else
static inline int xe_pmu_init(void) { return 0; }
static inline void xe_pmu_exit(void) {}
static inline void xe_pmu_register(struct xe_pmu *pmu) {}
static void xe_pmu_unregister(void *arg) {}
void xe_pmu_suspend(struct xe_gt *gt) {}
+void xe_pmu_resume(struct xe_gt *gt) {}
#endif
#endif
diff --git a/drivers/gpu/drm/xe/xe_pmu_types.h b/drivers/gpu/drm/xe/xe_pmu_types.h
index 1213d2a73492..f4cf85d72aac 100644
--- a/drivers/gpu/drm/xe/xe_pmu_types.h
+++ b/drivers/gpu/drm/xe/xe_pmu_types.h
@@ -13,6 +13,8 @@
enum {
__XE_SAMPLE_RC6,
__XE_SAMPLE_RC6_LAST_REPORTED,
+ __XE_SAMPLE_FREQ_ACT,
+ __XE_SAMPLE_FREQ_REQ,
__XE_NUM_PMU_SAMPLERS
};
@@ -23,6 +25,8 @@ enum {
* current state.
*/
enum xe_pmu_tracked_events {
+ __XE_PMU_ACTUAL_FREQUENCY_ENABLED,
+ __XE_PMU_REQUESTED_FREQUENCY_ENABLED,
__XE_PMU_RC6_RESIDENCY_ENABLED,
__XE_PMU_TRACKED_EVENT_COUNT, /* count marker */
};
@@ -116,6 +120,24 @@ struct xe_pmu {
* @sleep_last: Last time GT parked for RC6 estimation.
*/
ktime_t sleep_last[XE_PMU_MAX_GT];
+ /**
+ * @timer: Timer for internal Xe PMU sampling.
+ */
+ struct hrtimer timer;
+ /**
+ * @timer_last:
+ *
+ * Timestmap of the previous timer invocation.
+ */
+ ktime_t timer_last;
+ /**
+ * @timer_enabled: Should the internal sampling timer be running.
+ */
+ bool timer_enabled;
+ /**
+ * @active_gts: GT active mask.
+ */
+ unsigned int active_gts;
};
#endif
diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
index 3ef3926551dd..1f0ddbd77838 100644
--- a/include/uapi/drm/xe_drm.h
+++ b/include/uapi/drm/xe_drm.h
@@ -1399,7 +1399,12 @@ struct drm_xe_wait_user_fence {
#define __XE_PMU_OTHER(x) ___XE_PMU_OTHER(0, x)
#define XE_PMU_RC6_RESIDENCY __XE_PMU_OTHER(0)
+#define XE_PMU_ACTUAL_FREQUENCY __XE_PMU_OTHER(1)
+#define XE_PMU_REQUESTED_FREQUENCY __XE_PMU_OTHER(2)
+
#define __XE_PMU_RC6_RESIDENCY(gt) ___XE_PMU_OTHER(gt, 0)
+#define __XE_PMU_ACTUAL_FREQUENCY(gt) ___XE_PMU_OTHER(gt, 1)
+#define __XE_PMU_REQUESTED_FREQUENCY(gt) ___XE_PMU_OTHER(gt, 2)
/**
* enum drm_xe_observation_type - Observation stream types
--
2.38.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PATCH 1/3] drm/xe/pmu: Enable PMU interface
2024-09-25 23:21 ` [PATCH 1/3] drm/xe/pmu: Enable PMU interface Vinay Belgaumkar
@ 2024-09-26 9:25 ` Jani Nikula
0 siblings, 0 replies; 19+ messages in thread
From: Jani Nikula @ 2024-09-26 9:25 UTC (permalink / raw)
To: Vinay Belgaumkar, intel-xe
Cc: Aravind Iddamsetty, Bommu Krishnaiah, Riana Tauro, Rodrigo Vivi,
Vinay Belgaumkar
On Wed, 25 Sep 2024, Vinay Belgaumkar <vinay.belgaumkar@intel.com> wrote:
> diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
> index ae245fbd91ee..bddb93e8b949 100644
> --- a/drivers/gpu/drm/xe/Makefile
> +++ b/drivers/gpu/drm/xe/Makefile
> @@ -266,6 +266,8 @@ xe-$(CONFIG_DRM_XE_DISPLAY) += \
> i915-display/skl_universal_plane.o \
> i915-display/skl_watermark.o
>
> +xe-$(CONFIG_PERF_EVENTS) += xe_pmu.o
> +
Why add perf events stuff *between* two blocks of display objects?
> ifeq ($(CONFIG_ACPI),y)
> xe-$(CONFIG_DRM_XE_DISPLAY) += \
> i915-display/intel_acpi.o \
--
Jani Nikula, Intel
^ permalink raw reply [flat|nested] 19+ messages in thread
* ✓ CI.Patch_applied: success for drm/xe/pmu: PMU interface for Xe
2024-09-25 23:21 [PATCH 0/3] drm/xe/pmu: PMU interface for Xe Vinay Belgaumkar
` (2 preceding siblings ...)
2024-09-25 23:21 ` [PATCH 3/3] drm/xe/pmu: Add GT frequency events Vinay Belgaumkar
@ 2024-09-26 16:57 ` Patchwork
2024-09-26 16:57 ` ✗ CI.checkpatch: warning " Patchwork
2024-09-26 16:58 ` ✗ CI.KUnit: failure " Patchwork
5 siblings, 0 replies; 19+ messages in thread
From: Patchwork @ 2024-09-26 16:57 UTC (permalink / raw)
To: Vinay Belgaumkar; +Cc: intel-xe
== Series Details ==
Series: drm/xe/pmu: PMU interface for Xe
URL : https://patchwork.freedesktop.org/series/139121/
State : success
== Summary ==
=== Applying kernel patches on branch 'drm-tip' with base: ===
Base commit: ec87b409d3a9 drm-tip: 2024y-09m-26d-13h-48m-52s UTC integration manifest
=== git am output follows ===
.git/rebase-apply/patch:704: new blank line at EOF.
+
warning: 1 line adds whitespace errors.
Applying: drm/xe/pmu: Enable PMU interface
Applying: drm/xe/pmu: Add GT C6 events
Applying: drm/xe/pmu: Add GT frequency events
^ permalink raw reply [flat|nested] 19+ messages in thread
* ✗ CI.checkpatch: warning for drm/xe/pmu: PMU interface for Xe
2024-09-25 23:21 [PATCH 0/3] drm/xe/pmu: PMU interface for Xe Vinay Belgaumkar
` (3 preceding siblings ...)
2024-09-26 16:57 ` ✓ CI.Patch_applied: success for drm/xe/pmu: PMU interface for Xe Patchwork
@ 2024-09-26 16:57 ` Patchwork
2024-09-26 16:58 ` ✗ CI.KUnit: failure " Patchwork
5 siblings, 0 replies; 19+ messages in thread
From: Patchwork @ 2024-09-26 16:57 UTC (permalink / raw)
To: Vinay Belgaumkar; +Cc: intel-xe
== Series Details ==
Series: drm/xe/pmu: PMU interface for Xe
URL : https://patchwork.freedesktop.org/series/139121/
State : warning
== Summary ==
+ KERNEL=/kernel
+ git clone https://gitlab.freedesktop.org/drm/maintainer-tools mt
Cloning into 'mt'...
warning: redirecting to https://gitlab.freedesktop.org/drm/maintainer-tools.git/
+ git -C mt rev-list -n1 origin/master
30ab6715fc09baee6cc14cb3c89ad8858688d474
+ cd /kernel
+ git config --global --add safe.directory /kernel
+ git log -n1
commit 46e86fb616b4ed19df1aa1fca442e4efe8e13fdf
Author: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
Date: Wed Sep 25 16:21:59 2024 -0700
drm/xe/pmu: Add GT frequency events
Define PMU events for GT frequency (actual and requested). This is
a port from the i915 driver implementation.
Following PMU events are being added-
xe_0000_00_02.0/actual-frequency-gt0/ [Kernel PMU event]
xe_0000_00_02.0/actual-frequency-gt1/ [Kernel PMU event]
xe_0000_00_02.0/requested-frequency-gt0/ [Kernel PMU event]
xe_0000_00_02.0/requested-frequency-gt1/ [Kernel PMU event]
v2: Checkpatch fix, moved timer code to this patch
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
+ /mt/dim checkpatch ec87b409d3a9358c9a079186d528fad3f7d64334 drm-intel
f7ad341df96e drm/xe/pmu: Enable PMU interface
-:107: WARNING:FILE_PATH_CHANGES: added, moved or deleted file(s), does MAINTAINERS need updating?
#107:
new file mode 100644
total: 0 errors, 1 warnings, 0 checks, 732 lines checked
e389fe0f889d drm/xe/pmu: Add GT C6 events
46e86fb616b4 drm/xe/pmu: Add GT frequency events
-:62: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#62: FILE: drivers/gpu/drm/xe/xe_pmu.c:95:
+ BITS_PER_TYPE(typeof_member(struct xe_pmu,
+ enable)) - 1);
total: 0 errors, 0 warnings, 1 checks, 364 lines checked
^ permalink raw reply [flat|nested] 19+ messages in thread
* ✗ CI.KUnit: failure for drm/xe/pmu: PMU interface for Xe
2024-09-25 23:21 [PATCH 0/3] drm/xe/pmu: PMU interface for Xe Vinay Belgaumkar
` (4 preceding siblings ...)
2024-09-26 16:57 ` ✗ CI.checkpatch: warning " Patchwork
@ 2024-09-26 16:58 ` Patchwork
5 siblings, 0 replies; 19+ messages in thread
From: Patchwork @ 2024-09-26 16:58 UTC (permalink / raw)
To: Vinay Belgaumkar; +Cc: intel-xe
== Series Details ==
Series: drm/xe/pmu: PMU interface for Xe
URL : https://patchwork.freedesktop.org/series/139121/
State : failure
== Summary ==
+ trap cleanup EXIT
+ /kernel/tools/testing/kunit/kunit.py run --kunitconfig /kernel/drivers/gpu/drm/xe/.kunitconfig
[16:57:34] Configuring KUnit Kernel ...
Generating .config ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
[16:57:38] Building KUnit Kernel ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
Building with:
$ make ARCH=um O=.kunit --jobs=48
ERROR:root:In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/tests/xe_pci_test.c:13:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_bb.c:9:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/tests/xe_rtp_test.c:16:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/tests/xe_wa_test.c:11:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_bo.c:6:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_bo_evict.c:8:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_devcoredump.c:14:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_device.c:6:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_device_sysfs.c:12:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_dma_buf.c:17:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_drm_client.c:13:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_exec.c:14:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_execlist.c:14:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_exec_queue.h:10,
from ../drivers/gpu/drm/xe/xe_exec_queue.c:6:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_force_wake.h:9,
from ../drivers/gpu/drm/xe/xe_force_wake.c:6:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_ggtt.c:19:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_gsc.c:17:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_gsc_debugfs.c:11:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_gsc_proxy.c:17:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_gsc_submit.c:11:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_gt.h:11,
from ../drivers/gpu/drm/xe/xe_gt.c:6:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_gt_ccs_mode.c:9:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_gt_clock.c:12:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_gt_freq.c:16:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_force_wake.h:9,
from ../drivers/gpu/drm/xe/xe_gt_idle.c:8:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_gt_topology.h:9,
from ../drivers/gpu/drm/xe/xe_gt_mcr.h:10,
from ../drivers/gpu/drm/xe/xe_gt_mcr.c:6:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_gt_pagefault.c:16:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_gt.h:11,
from ../drivers/gpu/drm/xe/xe_gt_sysfs.c:13:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c:9:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_gt_throttle.c:9:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_gt_topology.h:9,
from ../drivers/gpu/drm/xe/xe_gt_topology.c:6:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_gt.h:11,
from ../drivers/gpu/drm/xe/xe_guc.h:9,
from ../drivers/gpu/drm/xe/xe_guc.c:6:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_guc_ads.c:16:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_guc_db_mgr.c:13:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_guc_ct.c:19:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_guc_hwconfig.c:12:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_guc_id_mgr.c:11:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_guc_log.c:10:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_guc_pc.c:16:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_guc_submit.c:20:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_heci_gsc.c:11:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_hw_engine.c:15:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_hw_engine_group.c:8:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c:10:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_hw_fence.c:11:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_huc.c:15:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/display/xe_display.h:9,
from ../drivers/gpu/drm/xe/xe_irq.c:12:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_lrc.c:18:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_mmio.h:9,
from ../drivers/gpu/drm/xe/xe_mmio.c:6:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_migrate.c:21:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_module.c:17:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_oa.c:21:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_mocs.c:9:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_pat.c:13:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/display/xe_display.h:9,
from ../drivers/gpu/drm/xe/xe_pci.c:18:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_pcode.c:13:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/display/xe_display.h:9,
from ../drivers/gpu/drm/xe/xe_pm.c:13:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_exec_queue.h:10,
from ../drivers/gpu/drm/xe/xe_preempt_fence.c:10:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_pt.c:11:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_query.c:16:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_reg_sr.c:18:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_reg_whitelist.c:12:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_gt.h:11,
from ../drivers/gpu/drm/xe/xe_rtp.c:12:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_gt.h:11,
from ../drivers/gpu/drm/xe/xe_ring_ops.c:16:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_sa.c:12:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_sched_job.c:12:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_step.c:10:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_sync.c:17:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_tile.c:8:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_tile.h:9,
from ../drivers/gpu/drm/xe/xe_tile_sysfs.c:11:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_trace.h:18,
from ../drivers/gpu/drm/xe/xe_trace.c:8:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_trace_guc.h:15,
from ../drivers/gpu/drm/xe/xe_trace_guc.c:8:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_trace_bo.h:15,
from ../drivers/gpu/drm/xe/xe_trace_bo.c:8:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_ttm_sys_mgr.c:15:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_ttm_stolen_mgr.c:18:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_ttm_vram_mgr.c:12:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_tuning.c:11:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_uc.c:8:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_vm.h:9,
from ../drivers/gpu/drm/xe/xe_vm.c:6:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_vram.c:14:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_uc_fw.c:12:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_vram_freq.c:8:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_wa.h:9,
from ../drivers/gpu/drm/xe/xe_wa.c:6:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_wait_user_fence.c:13:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_wopcm.c:9:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_hwmon.c:14:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_vm.h:9,
from ../drivers/gpu/drm/xe/xe_hmm.c:14:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_gt_sriov_vf.c:19:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../lib/iomap.c:156:5: warning: no previous prototype for ‘ioread64_lo_hi’ [-Wmissing-prototypes]
156 | u64 ioread64_lo_hi(const void __iomem *addr)
| ^~~~~~~~~~~~~~
../lib/iomap.c:163:5: warning: no previous prototype for ‘ioread64_hi_lo’ [-Wmissing-prototypes]
163 | u64 ioread64_hi_lo(const void __iomem *addr)
| ^~~~~~~~~~~~~~
../lib/iomap.c:170:5: warning: no previous prototype for ‘ioread64be_lo_hi’ [-Wmissing-prototypes]
170 | u64 ioread64be_lo_hi(const void __iomem *addr)
| ^~~~~~~~~~~~~~~~
../lib/iomap.c:178:5: warning: no previous prototype for ‘ioread64be_hi_lo’ [-Wmissing-prototypes]
178 | u64 ioread64be_hi_lo(const void __iomem *addr)
| ^~~~~~~~~~~~~~~~
../lib/iomap.c:264:6: warning: no previous prototype for ‘iowrite64_lo_hi’ [-Wmissing-prototypes]
264 | void iowrite64_lo_hi(u64 val, void __iomem *addr)
| ^~~~~~~~~~~~~~~
../lib/iomap.c:272:6: warning: no previous prototype for ‘iowrite64_hi_lo’ [-Wmissing-prototypes]
272 | void iowrite64_hi_lo(u64 val, void __iomem *addr)
| ^~~~~~~~~~~~~~~
../lib/iomap.c:280:6: warning: no previous prototype for ‘iowrite64be_lo_hi’ [-Wmissing-prototypes]
280 | void iowrite64be_lo_hi(u64 val, void __iomem *addr)
| ^~~~~~~~~~~~~~~~~
../lib/iomap.c:288:6: warning: no previous prototype for ‘iowrite64be_hi_lo’ [-Wmissing-prototypes]
288 | void iowrite64be_hi_lo(u64 val, void __iomem *addr)
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_guc_relay.c:18:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_memirq.c:12:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_gt.h:11,
from ../drivers/gpu/drm/xe/xe_gt_sriov_pf.c:11:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_sriov.c:10:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_gt_sriov_pf_migration.c:9:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_gt_sriov_pf_helpers.h:9,
from ../drivers/gpu/drm/xe/xe_gt_sriov_pf_monitor.c:10:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_mmio.h:9,
from ../drivers/gpu/drm/xe/xe_gt_sriov_pf_service.c:15:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_gt_sriov_pf_config.c:14:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_gt_sriov_pf_control.c:10:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_pci_sriov.c:6:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_gt_sriov_pf_policy.c:8:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_lmtt.c:12:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_assert.h:13,
from ../drivers/gpu/drm/xe/xe_sriov_pf.c:8:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_debugfs.c:13:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_huc_debugfs.c:11:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_vm_types.h:16,
from ../drivers/gpu/drm/xe/xe_bo.h:13,
from ../drivers/gpu/drm/xe/xe_gt_sriov_pf_debugfs.c:11:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/tests/xe_kunit_helpers.c:15:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_gt_types.h:9,
from ../drivers/gpu/drm/xe/xe_gt_sriov_vf_debugfs.c:13:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_gt_debugfs.c:13:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_gt.h:11,
from ../drivers/gpu/drm/xe/xe_gt_stats.c:10:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_guc_debugfs.c:11:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/xe_device_types.h:21,
from ../drivers/gpu/drm/xe/xe_device.h:11,
from ../drivers/gpu/drm/xe/xe_gt.h:11,
from ../drivers/gpu/drm/xe/xe_uc_debugfs.c:10:
../drivers/gpu/drm/xe/xe_pmu.h:25:6: warning: no previous prototype for ‘xe_pmu_suspend’ [-Wmissing-prototypes]
25 | void xe_pmu_suspend(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:26:6: warning: no previous prototype for ‘xe_pmu_resume’ [-Wmissing-prototypes]
26 | void xe_pmu_resume(struct xe_gt *gt) {}
| ^~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_pmu.h:24:13: warning: ‘xe_pmu_unregister’ defined but not used [-Wunused-function]
24 | static void xe_pmu_unregister(void *arg) {}
| ^~~~~~~~~~~~~~~~~
ld: drivers/gpu/drm/xe/xe_bo.o: in function `xe_pmu_suspend':
xe_bo.c:(.text+0x1320): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_bo.o: in function `xe_pmu_resume':
xe_bo.c:(.text+0x1330): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_bo_evict.o: in function `xe_pmu_suspend':
xe_bo_evict.c:(.text+0x0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_bo_evict.o: in function `xe_pmu_resume':
xe_bo_evict.c:(.text+0x10): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_devcoredump.o: in function `xe_pmu_suspend':
xe_devcoredump.c:(.text+0x4e0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_devcoredump.o: in function `xe_pmu_resume':
xe_devcoredump.c:(.text+0x4f0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_device.o: in function `xe_pmu_suspend':
xe_device.c:(.text+0xe40): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_device.o: in function `xe_pmu_resume':
xe_device.c:(.text+0xe50): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_device_sysfs.o: in function `xe_pmu_suspend':
xe_device_sysfs.c:(.text+0x160): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_device_sysfs.o: in function `xe_pmu_resume':
xe_device_sysfs.c:(.text+0x170): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_dma_buf.o: in function `xe_pmu_suspend':
xe_dma_buf.c:(.text+0x490): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_dma_buf.o: in function `xe_pmu_resume':
xe_dma_buf.c:(.text+0x4a0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_drm_client.o: in function `xe_pmu_suspend':
xe_drm_client.c:(.text+0x7c0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_drm_client.o: in function `xe_pmu_resume':
xe_drm_client.c:(.text+0x7d0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_exec.o: in function `xe_pmu_suspend':
xe_exec.c:(.text+0x20): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_exec.o: in function `xe_pmu_resume':
xe_exec.c:(.text+0x30): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_execlist.o: in function `xe_pmu_suspend':
xe_execlist.c:(.text+0xc10): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_execlist.o: in function `xe_pmu_resume':
xe_execlist.c:(.text+0xc20): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_exec_queue.o: in function `xe_pmu_suspend':
xe_exec_queue.c:(.text+0x6a0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_exec_queue.o: in function `xe_pmu_resume':
xe_exec_queue.c:(.text+0x6b0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_force_wake.o: in function `xe_pmu_suspend':
xe_force_wake.c:(.text+0xb0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_force_wake.o: in function `xe_pmu_resume':
xe_force_wake.c:(.text+0xc0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_ggtt.o: in function `xe_pmu_suspend':
xe_ggtt.c:(.text+0x4f0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_ggtt.o: in function `xe_pmu_resume':
xe_ggtt.c:(.text+0x500): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gsc.o: in function `xe_pmu_suspend':
xe_gsc.c:(.text+0xc40): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gsc.o: in function `xe_pmu_resume':
xe_gsc.c:(.text+0xc50): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gsc_debugfs.o: in function `xe_pmu_suspend':
xe_gsc_debugfs.c:(.text+0xb0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gsc_debugfs.o: in function `xe_pmu_resume':
xe_gsc_debugfs.c:(.text+0xc0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gsc_proxy.o: in function `xe_pmu_suspend':
xe_gsc_proxy.c:(.text+0x7e0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gsc_proxy.o: in function `xe_pmu_resume':
xe_gsc_proxy.c:(.text+0x7f0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gsc_submit.o: in function `xe_pmu_suspend':
xe_gsc_submit.c:(.text+0x0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gsc_submit.o: in function `xe_pmu_resume':
xe_gsc_submit.c:(.text+0x10): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gt.o: in function `xe_pmu_suspend':
xe_gt.c:(.text+0x450): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gt.o: in function `xe_pmu_resume':
xe_gt.c:(.text+0x460): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gt_ccs_mode.o: in function `xe_pmu_suspend':
xe_gt_ccs_mode.c:(.text+0x270): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gt_ccs_mode.o: in function `xe_pmu_resume':
xe_gt_ccs_mode.c:(.text+0x280): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gt_clock.o: in function `xe_pmu_suspend':
xe_gt_clock.c:(.text+0x0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gt_clock.o: in function `xe_pmu_resume':
xe_gt_clock.c:(.text+0x10): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gt_freq.o: in function `xe_pmu_suspend':
xe_gt_freq.c:(.text+0x540): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gt_freq.o: in function `xe_pmu_resume':
xe_gt_freq.c:(.text+0x550): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gt_idle.o: in function `xe_pmu_suspend':
xe_gt_idle.c:(.text+0x110): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gt_idle.o: in function `xe_pmu_resume':
xe_gt_idle.c:(.text+0x120): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gt_mcr.o: in function `xe_pmu_suspend':
xe_gt_mcr.c:(.text+0x520): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gt_mcr.o: in function `xe_pmu_resume':
xe_gt_mcr.c:(.text+0x530): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gt_pagefault.o: in function `xe_pmu_suspend':
xe_gt_pagefault.c:(.text+0xe00): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gt_pagefault.o: in function `xe_pmu_resume':
xe_gt_pagefault.c:(.text+0xe10): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gt_sysfs.o: in function `xe_pmu_suspend':
xe_gt_sysfs.c:(.text+0x30): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gt_sysfs.o: in function `xe_pmu_resume':
xe_gt_sysfs.c:(.text+0x40): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gt_throttle.o: in function `xe_pmu_suspend':
xe_gt_throttle.c:(.text+0x30): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gt_throttle.o: in function `xe_pmu_resume':
xe_gt_throttle.c:(.text+0x40): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gt_tlb_invalidation.o: in function `xe_pmu_suspend':
xe_gt_tlb_invalidation.c:(.text+0x540): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gt_tlb_invalidation.o: in function `xe_pmu_resume':
xe_gt_tlb_invalidation.c:(.text+0x550): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gt_topology.o: in function `xe_pmu_suspend':
xe_gt_topology.c:(.text+0x460): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gt_topology.o: in function `xe_pmu_resume':
xe_gt_topology.c:(.text+0x470): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_guc.o: in function `xe_pmu_suspend':
xe_guc.c:(.text+0x7d0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_guc.o: in function `xe_pmu_resume':
xe_guc.c:(.text+0x7e0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_guc_ads.o: in function `xe_pmu_suspend':
xe_guc_ads.c:(.text+0x7b0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_guc_ads.o: in function `xe_pmu_resume':
xe_guc_ads.c:(.text+0x7c0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_guc_ct.o: in function `xe_pmu_suspend':
xe_guc_ct.c:(.text+0x11e0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_guc_ct.o: in function `xe_pmu_resume':
xe_guc_ct.c:(.text+0x11f0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_guc_db_mgr.o: in function `xe_pmu_suspend':
xe_guc_db_mgr.c:(.text+0x590): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_guc_db_mgr.o: in function `xe_pmu_resume':
xe_guc_db_mgr.c:(.text+0x5a0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_guc_hwconfig.o: in function `xe_pmu_suspend':
xe_guc_hwconfig.c:(.text+0x0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_guc_hwconfig.o: in function `xe_pmu_resume':
xe_guc_hwconfig.c:(.text+0x10): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_guc_id_mgr.o: in function `xe_pmu_suspend':
xe_guc_id_mgr.c:(.text+0x5a0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_guc_id_mgr.o: in function `xe_pmu_resume':
xe_guc_id_mgr.c:(.text+0x5b0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_guc_log.o: in function `xe_pmu_suspend':
xe_guc_log.c:(.text+0x0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_guc_log.o: in function `xe_pmu_resume':
xe_guc_log.c:(.text+0x10): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_guc_pc.o: in function `xe_pmu_suspend':
xe_guc_pc.c:(.text+0x590): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_guc_pc.o: in function `xe_pmu_resume':
xe_guc_pc.c:(.text+0x5a0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_guc_submit.o: in function `xe_pmu_suspend':
xe_guc_submit.c:(.text+0x3430): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_guc_submit.o: in function `xe_pmu_resume':
xe_guc_submit.c:(.text+0x3440): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_heci_gsc.o: in function `xe_pmu_suspend':
xe_heci_gsc.c:(.text+0x30): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_heci_gsc.o: in function `xe_pmu_resume':
xe_heci_gsc.c:(.text+0x40): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_hw_engine.o: in function `xe_pmu_suspend':
xe_hw_engine.c:(.text+0x3f0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_hw_engine.o: in function `xe_pmu_resume':
xe_hw_engine.c:(.text+0x400): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.o: in function `xe_pmu_suspend':
xe_hw_engine_class_sysfs.c:(.text+0xa70): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.o: in function `xe_pmu_resume':
xe_hw_engine_class_sysfs.c:(.text+0xa80): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_hw_engine_group.o: in function `xe_pmu_suspend':
xe_hw_engine_group.c:(.text+0x170): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_hw_engine_group.o: in function `xe_pmu_resume':
xe_hw_engine_group.c:(.text+0x180): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_hw_fence.o: in function `xe_pmu_suspend':
xe_hw_fence.c:(.text+0x5c0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_hw_fence.o: in function `xe_pmu_resume':
xe_hw_fence.c:(.text+0x5d0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_huc.o: in function `xe_pmu_suspend':
xe_huc.c:(.text+0x0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_huc.o: in function `xe_pmu_resume':
xe_huc.c:(.text+0x10): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_irq.o: in function `xe_pmu_suspend':
xe_irq.c:(.text+0x1290): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_irq.o: in function `xe_pmu_resume':
xe_irq.c:(.text+0x12a0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_lrc.o: in function `xe_pmu_suspend':
xe_lrc.c:(.text+0x230): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_lrc.o: in function `xe_pmu_resume':
xe_lrc.c:(.text+0x240): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_migrate.o: in function `xe_pmu_suspend':
xe_migrate.c:(.text+0x22f0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_migrate.o: in function `xe_pmu_resume':
xe_migrate.c:(.text+0x2300): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_mmio.o: in function `xe_pmu_suspend':
xe_mmio.c:(.text+0x80): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_mmio.o: in function `xe_pmu_resume':
xe_mmio.c:(.text+0x90): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_mocs.o: in function `xe_pmu_suspend':
xe_mocs.c:(.text+0x1aa0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_mocs.o: in function `xe_pmu_resume':
xe_mocs.c:(.text+0x1ab0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_module.o: in function `xe_pmu_suspend':
xe_module.c:(.text+0x50): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_module.o: in function `xe_pmu_resume':
xe_module.c:(.text+0x60): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_oa.o: in function `xe_pmu_suspend':
xe_oa.c:(.text+0x3430): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_oa.o: in function `xe_pmu_resume':
xe_oa.c:(.text+0x3440): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_pat.o: in function `xe_pmu_suspend':
xe_pat.c:(.text+0xa10): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_pat.o: in function `xe_pmu_resume':
xe_pat.c:(.text+0xa20): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_pci.o: in function `xe_pmu_suspend':
xe_pci.c:(.text+0x1820): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_pci.o: in function `xe_pmu_resume':
xe_pci.c:(.text+0x1830): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_pcode.o: in function `xe_pmu_suspend':
xe_pcode.c:(.text+0x3a0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_pcode.o: in function `xe_pmu_resume':
xe_pcode.c:(.text+0x3b0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_pm.o: in function `xe_pmu_suspend':
xe_pm.c:(.text+0x280): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_pm.o: in function `xe_pmu_resume':
xe_pm.c:(.text+0x290): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_preempt_fence.o: in function `xe_pmu_suspend':
xe_preempt_fence.c:(.text+0x250): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_preempt_fence.o: in function `xe_pmu_resume':
xe_preempt_fence.c:(.text+0x260): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_pt.o: in function `xe_pmu_suspend':
xe_pt.c:(.text+0x2d90): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_pt.o: in function `xe_pmu_resume':
xe_pt.c:(.text+0x2da0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_query.o: in function `xe_pmu_suspend':
xe_query.c:(.text+0x1d40): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_query.o: in function `xe_pmu_resume':
xe_query.c:(.text+0x1d50): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_reg_sr.o: in function `xe_pmu_suspend':
xe_reg_sr.c:(.text+0xd0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_reg_sr.o: in function `xe_pmu_resume':
xe_reg_sr.c:(.text+0xe0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_reg_whitelist.o: in function `xe_pmu_suspend':
xe_reg_whitelist.c:(.text+0x10): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_reg_whitelist.o: in function `xe_pmu_resume':
xe_reg_whitelist.c:(.text+0x20): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_rtp.o: in function `xe_pmu_suspend':
xe_rtp.c:(.text+0x880): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_rtp.o: in function `xe_pmu_resume':
xe_rtp.c:(.text+0x890): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_ring_ops.o: in function `xe_pmu_suspend':
xe_ring_ops.c:(.text+0xd90): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_ring_ops.o: in function `xe_pmu_resume':
xe_ring_ops.c:(.text+0xda0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_sa.o: in function `xe_pmu_suspend':
xe_sa.c:(.text+0x60): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_sa.o: in function `xe_pmu_resume':
xe_sa.c:(.text+0x70): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_sched_job.o: in function `xe_pmu_suspend':
xe_sched_job.c:(.text+0x130): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_sched_job.o: in function `xe_pmu_resume':
xe_sched_job.c:(.text+0x140): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_step.o: in function `xe_pmu_suspend':
xe_step.c:(.text+0x0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_step.o: in function `xe_pmu_resume':
xe_step.c:(.text+0x10): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_sync.o: in function `xe_pmu_suspend':
xe_sync.c:(.text+0x450): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_sync.o: in function `xe_pmu_resume':
xe_sync.c:(.text+0x460): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_tile.o: in function `xe_pmu_suspend':
xe_tile.c:(.text+0x0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_tile.o: in function `xe_pmu_resume':
xe_tile.c:(.text+0x10): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_tile_sysfs.o: in function `xe_pmu_suspend':
xe_tile_sysfs.c:(.text+0x30): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_tile_sysfs.o: in function `xe_pmu_resume':
xe_tile_sysfs.c:(.text+0x40): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_trace.o: in function `xe_pmu_suspend':
xe_trace.c:(.text+0x1eb0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_trace.o: in function `xe_pmu_resume':
xe_trace.c:(.text+0x1ec0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_trace_bo.o: in function `xe_pmu_suspend':
xe_trace_bo.c:(.text+0x10d0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_trace_bo.o: in function `xe_pmu_resume':
xe_trace_bo.c:(.text+0x10e0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_trace_guc.o: in function `xe_pmu_suspend':
xe_trace_guc.c:(.text+0x8f0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_trace_guc.o: in function `xe_pmu_resume':
xe_trace_guc.c:(.text+0x900): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_ttm_sys_mgr.o: in function `xe_pmu_suspend':
xe_ttm_sys_mgr.c:(.text+0x270): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_ttm_sys_mgr.o: in function `xe_pmu_resume':
xe_ttm_sys_mgr.c:(.text+0x280): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_ttm_stolen_mgr.o: in function `xe_pmu_suspend':
xe_ttm_stolen_mgr.c:(.text+0x0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_ttm_stolen_mgr.o: in function `xe_pmu_resume':
xe_ttm_stolen_mgr.c:(.text+0x10): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_ttm_vram_mgr.o: in function `xe_pmu_suspend':
xe_ttm_vram_mgr.c:(.text+0xa20): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_ttm_vram_mgr.o: in function `xe_pmu_resume':
xe_ttm_vram_mgr.c:(.text+0xa30): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_tuning.o: in function `xe_pmu_suspend':
xe_tuning.c:(.text+0xa0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_tuning.o: in function `xe_pmu_resume':
xe_tuning.c:(.text+0xb0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_uc.o: in function `xe_pmu_suspend':
xe_uc.c:(.text+0x0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_uc.o: in function `xe_pmu_resume':
xe_uc.c:(.text+0x10): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_uc_fw.o: in function `xe_pmu_suspend':
xe_uc_fw.c:(.text+0x3f0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_uc_fw.o: in function `xe_pmu_resume':
xe_uc_fw.c:(.text+0x400): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_vm.o: in function `xe_pmu_suspend':
xe_vm.c:(.text+0x1d10): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_vm.o: in function `xe_pmu_resume':
xe_vm.c:(.text+0x1d20): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_vram.o: in function `xe_pmu_suspend':
xe_vram.c:(.text+0x2f0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_vram.o: in function `xe_pmu_resume':
xe_vram.c:(.text+0x300): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_vram_freq.o: in function `xe_pmu_suspend':
xe_vram_freq.c:(.text+0x110): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_vram_freq.o: in function `xe_pmu_resume':
xe_vram_freq.c:(.text+0x120): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_wait_user_fence.o: in function `xe_pmu_suspend':
xe_wait_user_fence.c:(.text+0x0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_wait_user_fence.o: in function `xe_pmu_resume':
xe_wait_user_fence.c:(.text+0x10): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_wa.o: in function `xe_pmu_suspend':
xe_wa.c:(.text+0x70): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_wa.o: in function `xe_pmu_resume':
xe_wa.c:(.text+0x80): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_wopcm.o: in function `xe_pmu_suspend':
xe_wopcm.c:(.text+0x0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_wopcm.o: in function `xe_pmu_resume':
xe_wopcm.c:(.text+0x10): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_hmm.o: in function `xe_pmu_suspend':
xe_hmm.c:(.text+0x0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_hmm.o: in function `xe_pmu_resume':
xe_hmm.c:(.text+0x10): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_hwmon.o: in function `xe_pmu_suspend':
xe_hwmon.c:(.text+0xec0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_hwmon.o: in function `xe_pmu_resume':
xe_hwmon.c:(.text+0xed0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gt_sriov_vf.o: in function `xe_pmu_suspend':
xe_gt_sriov_vf.c:(.text+0x350): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gt_sriov_vf.o: in function `xe_pmu_resume':
xe_gt_sriov_vf.c:(.text+0x360): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_guc_relay.o: in function `xe_pmu_suspend':
xe_guc_relay.c:(.text+0x2a10): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_guc_relay.o: in function `xe_pmu_resume':
xe_guc_relay.c:(.text+0x2a20): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_memirq.o: in function `xe_pmu_suspend':
xe_memirq.c:(.text+0x130): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_memirq.o: in function `xe_pmu_resume':
xe_memirq.c:(.text+0x140): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_sriov.o: in function `xe_pmu_suspend':
xe_sriov.c:(.text+0x30): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_sriov.o: in function `xe_pmu_resume':
xe_sriov.c:(.text+0x40): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gt_sriov_pf.o: in function `xe_pmu_suspend':
xe_gt_sriov_pf.c:(.text+0x0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gt_sriov_pf.o: in function `xe_pmu_resume':
xe_gt_sriov_pf.c:(.text+0x10): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gt_sriov_pf_config.o: in function `xe_pmu_suspend':
xe_gt_sriov_pf_config.c:(.text+0x1a80): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gt_sriov_pf_config.o: in function `xe_pmu_resume':
xe_gt_sriov_pf_config.c:(.text+0x1a90): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gt_sriov_pf_control.o: in function `xe_pmu_suspend':
xe_gt_sriov_pf_control.c:(.text+0xeb0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gt_sriov_pf_control.o: in function `xe_pmu_resume':
xe_gt_sriov_pf_control.c:(.text+0xec0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gt_sriov_pf_migration.o: in function `xe_pmu_suspend':
xe_gt_sriov_pf_migration.c:(.text+0x130): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gt_sriov_pf_migration.o: in function `xe_pmu_resume':
xe_gt_sriov_pf_migration.c:(.text+0x140): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gt_sriov_pf_monitor.o: in function `xe_pmu_suspend':
xe_gt_sriov_pf_monitor.c:(.text+0x0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gt_sriov_pf_monitor.o: in function `xe_pmu_resume':
xe_gt_sriov_pf_monitor.c:(.text+0x10): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gt_sriov_pf_policy.o: in function `xe_pmu_suspend':
xe_gt_sriov_pf_policy.c:(.text+0x7d0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gt_sriov_pf_policy.o: in function `xe_pmu_resume':
xe_gt_sriov_pf_policy.c:(.text+0x7e0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gt_sriov_pf_service.o: in function `xe_pmu_suspend':
xe_gt_sriov_pf_service.c:(.text+0x1790): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gt_sriov_pf_service.o: in function `xe_pmu_resume':
xe_gt_sriov_pf_service.c:(.text+0x17a0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_lmtt.o: in function `xe_pmu_suspend':
xe_lmtt.c:(.text+0xd10): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_lmtt.o: in function `xe_pmu_resume':
xe_lmtt.c:(.text+0xd20): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_pci_sriov.o: in function `xe_pmu_suspend':
xe_pci_sriov.c:(.text+0x280): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_pci_sriov.o: in function `xe_pmu_resume':
xe_pci_sriov.c:(.text+0x290): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_sriov_pf.o: in function `xe_pmu_suspend':
xe_sriov_pf.c:(.text+0x40): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_sriov_pf.o: in function `xe_pmu_resume':
xe_sriov_pf.c:(.text+0x50): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/tests/xe_kunit_helpers.o: in function `xe_pmu_suspend':
xe_kunit_helpers.c:(.text+0x430): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/tests/xe_kunit_helpers.o: in function `xe_pmu_resume':
xe_kunit_helpers.c:(.text+0x440): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_debugfs.o: in function `xe_pmu_suspend':
xe_debugfs.c:(.text+0xd90): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_debugfs.o: in function `xe_pmu_resume':
xe_debugfs.c:(.text+0xda0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gt_debugfs.o: in function `xe_pmu_suspend':
xe_gt_debugfs.c:(.text+0x8a0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gt_debugfs.o: in function `xe_pmu_resume':
xe_gt_debugfs.c:(.text+0x8b0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gt_sriov_vf_debugfs.o: in function `xe_pmu_suspend':
xe_gt_sriov_vf_debugfs.c:(.text+0x0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gt_sriov_vf_debugfs.o: in function `xe_pmu_resume':
xe_gt_sriov_vf_debugfs.c:(.text+0x10): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gt_stats.o: in function `xe_pmu_suspend':
xe_gt_stats.c:(.text+0x0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gt_stats.o: in function `xe_pmu_resume':
xe_gt_stats.c:(.text+0x10): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_guc_debugfs.o: in function `xe_pmu_suspend':
xe_guc_debugfs.c:(.text+0x160): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_guc_debugfs.o: in function `xe_pmu_resume':
xe_guc_debugfs.c:(.text+0x170): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_huc_debugfs.o: in function `xe_pmu_suspend':
xe_huc_debugfs.c:(.text+0xb0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_huc_debugfs.o: in function `xe_pmu_resume':
xe_huc_debugfs.c:(.text+0xc0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_uc_debugfs.o: in function `xe_pmu_suspend':
xe_uc_debugfs.c:(.text+0x0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_uc_debugfs.o: in function `xe_pmu_resume':
xe_uc_debugfs.c:(.text+0x10): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/xe_gt_sriov_pf_debugfs.o: in function `xe_pmu_suspend':
xe_gt_sriov_pf_debugfs.c:(.text+0x1320): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/xe_gt_sriov_pf_debugfs.o: in function `xe_pmu_resume':
xe_gt_sriov_pf_debugfs.c:(.text+0x1330): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/tests/xe_pci_test.o: in function `xe_pmu_suspend':
xe_pci_test.c:(.text+0x1c0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/tests/xe_pci_test.o: in function `xe_pmu_resume':
xe_pci_test.c:(.text+0x1d0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/tests/xe_rtp_test.o: in function `xe_pmu_suspend':
xe_rtp_test.c:(.text+0x8c0): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/tests/xe_rtp_test.o: in function `xe_pmu_resume':
xe_rtp_test.c:(.text+0x8d0): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
ld: drivers/gpu/drm/xe/tests/xe_wa_test.o: in function `xe_pmu_suspend':
xe_wa_test.c:(.text+0x640): multiple definition of `xe_pmu_suspend'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x60): first defined here
ld: drivers/gpu/drm/xe/tests/xe_wa_test.o: in function `xe_pmu_resume':
xe_wa_test.c:(.text+0x650): multiple definition of `xe_pmu_resume'; drivers/gpu/drm/xe/xe_bb.o:xe_bb.c:(.text+0x70): first defined here
make[3]: *** [../scripts/Makefile.vmlinux_o:62: vmlinux.o] Error 1
make[2]: *** [/kernel/Makefile:1138: vmlinux_o] Error 2
make[1]: *** [/kernel/Makefile:224: __sub-make] Error 2
make: *** [Makefile:224: __sub-make] Error 2
+ cleanup
++ stat -c %u:%g /kernel
+ chown -R 1003:1003 /kernel
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH 1/3] drm/xe/pmu: Enable PMU interface
2024-09-27 0:23 [PATCH 0/3] " Vinay Belgaumkar
@ 2024-09-27 0:23 ` Vinay Belgaumkar
2024-09-27 18:26 ` Rodrigo Vivi
2024-10-01 21:32 ` Lucas De Marchi
0 siblings, 2 replies; 19+ messages in thread
From: Vinay Belgaumkar @ 2024-09-27 0:23 UTC (permalink / raw)
To: intel-xe
Cc: Aravind Iddamsetty, Bommu Krishnaiah, Riana Tauro, Rodrigo Vivi,
Vinay Belgaumkar
From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
Basic PMU enabling patch. Setup the basic framework
for adding events/timers. This patch was previously
reviewed here -
https://patchwork.freedesktop.org/series/119504/
The pmu base implementation is still from the
i915 driver.
v2: Review comments(Rodrigo) and do not init pmu for VFs
as they don't have access to freq and c6 residency anyways.
v3: Fix kunit issue, move xe_pmu entry in Makefile (Jani)
Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
Signed-off-by: Riana Tauro <riana.tauro@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
---
drivers/gpu/drm/xe/Makefile | 2 +
drivers/gpu/drm/xe/xe_device.c | 6 +
drivers/gpu/drm/xe/xe_device_types.h | 4 +
drivers/gpu/drm/xe/xe_module.c | 5 +
drivers/gpu/drm/xe/xe_pmu.c | 570 +++++++++++++++++++++++++++
drivers/gpu/drm/xe/xe_pmu.h | 26 ++
drivers/gpu/drm/xe/xe_pmu_types.h | 63 +++
include/uapi/drm/xe_drm.h | 8 +
8 files changed, 684 insertions(+)
create mode 100644 drivers/gpu/drm/xe/xe_pmu.c
create mode 100644 drivers/gpu/drm/xe/xe_pmu.h
create mode 100644 drivers/gpu/drm/xe/xe_pmu_types.h
diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
index ae245fbd91ee..3a3eabc0e0a3 100644
--- a/drivers/gpu/drm/xe/Makefile
+++ b/drivers/gpu/drm/xe/Makefile
@@ -293,6 +293,8 @@ ifeq ($(CONFIG_DEBUG_FS),y)
i915-display/intel_pipe_crc.o
endif
+xe-$(CONFIG_PERF_EVENTS) += xe_pmu.o
+
obj-$(CONFIG_DRM_XE) += xe.o
obj-$(CONFIG_DRM_XE_KUNIT_TEST) += tests/
diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
index cb5a9fd820cf..027a796d7431 100644
--- a/drivers/gpu/drm/xe/xe_device.c
+++ b/drivers/gpu/drm/xe/xe_device.c
@@ -762,6 +762,9 @@ int xe_device_probe(struct xe_device *xe)
for_each_gt(gt, xe, id)
xe_gt_sanitize_freq(gt);
+ if (!IS_SRIOV_VF(xe))
+ xe_pmu_register(&xe->pmu);
+
return devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
err_fini_display:
@@ -806,6 +809,9 @@ void xe_device_remove(struct xe_device *xe)
xe_heci_gsc_fini(xe);
+ if (!IS_SRIOV_VF(xe))
+ xe_pmu_unregister(&xe->pmu);
+
for_each_gt(gt, xe, id)
xe_gt_remove(gt);
}
diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
index 5ad96d283a71..51485d2eadcf 100644
--- a/drivers/gpu/drm/xe/xe_device_types.h
+++ b/drivers/gpu/drm/xe/xe_device_types.h
@@ -18,6 +18,7 @@
#include "xe_memirq_types.h"
#include "xe_oa.h"
#include "xe_platform_types.h"
+#include "xe_pmu.h"
#include "xe_pt_types.h"
#include "xe_sriov_types.h"
#include "xe_step_types.h"
@@ -513,6 +514,9 @@ struct xe_device {
int mode;
} wedged;
+ /** @pmu: performance monitoring unit */
+ struct xe_pmu pmu;
+
#ifdef TEST_VM_OPS_ERROR
/**
* @vm_inject_error_position: inject errors at different places in VM
diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
index 77ce9f9ca7a5..1bf2bf8447c0 100644
--- a/drivers/gpu/drm/xe/xe_module.c
+++ b/drivers/gpu/drm/xe/xe_module.c
@@ -14,6 +14,7 @@
#include "xe_hw_fence.h"
#include "xe_pci.h"
#include "xe_pm.h"
+#include "xe_pmu.h"
#include "xe_observation.h"
#include "xe_sched_job.h"
@@ -96,6 +97,10 @@ static const struct init_funcs init_funcs[] = {
.init = xe_sched_job_module_init,
.exit = xe_sched_job_module_exit,
},
+ {
+ .init = xe_pmu_init,
+ .exit = xe_pmu_exit,
+ },
{
.init = xe_register_pci_driver,
.exit = xe_unregister_pci_driver,
diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
new file mode 100644
index 000000000000..bdaea9ca1065
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_pmu.c
@@ -0,0 +1,570 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2024 Intel Corporation
+ */
+
+#include <drm/drm_drv.h>
+#include <drm/drm_managed.h>
+#include <drm/xe_drm.h>
+
+#include "regs/xe_gt_regs.h"
+#include "xe_device.h"
+#include "xe_force_wake.h"
+#include "xe_gt_clock.h"
+#include "xe_mmio.h"
+#include "xe_macros.h"
+#include "xe_pm.h"
+
+static cpumask_t xe_pmu_cpumask;
+static unsigned int xe_pmu_target_cpu = -1;
+
+/**
+ * DOC: Xe PMU (Performance Monitoring Unit)
+ *
+ * Expose events/counters like C6 residency and GT frequency to user land.
+ * Perf tool can be used to list and record these counters from the command
+ * line.
+ *
+ * Example commands to list/record supported perf events-
+ *
+ * >> perf list
+ * >> ls /sys/bus/event_source/devices/xe_0000_00_02.0/events/
+ *
+ * To record events-
+ * >> perf stat -e <event_name> -I <interval>
+ *
+ */
+
+static unsigned int config_gt_id(const u64 config)
+{
+ return config >> __XE_PMU_GT_SHIFT;
+}
+
+static u64 config_counter(const u64 config)
+{
+ return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
+}
+
+static void xe_pmu_event_destroy(struct perf_event *event)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+
+ drm_WARN_ON(&xe->drm, event->parent);
+
+ drm_dev_put(&xe->drm);
+}
+
+static int
+config_status(struct xe_device *xe, u64 config)
+{
+ unsigned int gt_id = config_gt_id(config);
+
+ if (gt_id >= XE_PMU_MAX_GT)
+ return -ENOENT;
+
+ switch (config_counter(config)) {
+ default:
+ return -ENOENT;
+ }
+
+ return 0;
+}
+
+static int xe_pmu_event_init(struct perf_event *event)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct xe_pmu *pmu = &xe->pmu;
+ int ret;
+
+ if (pmu->closed)
+ return -ENODEV;
+
+ if (event->attr.type != event->pmu->type)
+ return -ENOENT;
+
+ /* unsupported modes and filters */
+ if (event->attr.sample_period) /* no sampling */
+ return -EINVAL;
+
+ if (has_branch_stack(event))
+ return -EOPNOTSUPP;
+
+ if (event->cpu < 0)
+ return -EINVAL;
+
+ /* only allow running on one cpu at a time */
+ if (!cpumask_test_cpu(event->cpu, &xe_pmu_cpumask))
+ return -EINVAL;
+
+ ret = config_status(xe, event->attr.config);
+ if (ret)
+ return ret;
+
+ if (!event->parent) {
+ drm_dev_get(&xe->drm);
+ event->destroy = xe_pmu_event_destroy;
+ }
+
+ return 0;
+}
+
+static u64 __xe_pmu_event_read(struct perf_event *event)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ const unsigned int gt_id = config_gt_id(event->attr.config);
+ const u64 config = event->attr.config;
+ struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
+ u64 val = 0;
+
+ switch (config_counter(config)) {
+ default:
+ drm_warn(>->tile->xe->drm, "unknown pmu event\n");
+ }
+
+ return val;
+}
+
+static void xe_pmu_event_read(struct perf_event *event)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct hw_perf_event *hwc = &event->hw;
+ struct xe_pmu *pmu = &xe->pmu;
+ u64 prev, new;
+
+ if (pmu->closed) {
+ event->hw.state = PERF_HES_STOPPED;
+ return;
+ }
+again:
+ prev = local64_read(&hwc->prev_count);
+ new = __xe_pmu_event_read(event);
+
+ if (local64_cmpxchg(&hwc->prev_count, prev, new) != prev)
+ goto again;
+
+ local64_add(new - prev, &event->count);
+}
+
+static void xe_pmu_enable(struct perf_event *event)
+{
+ /*
+ * Store the current counter value so we can report the correct delta
+ * for all listeners. Even when the event was already enabled and has
+ * an existing non-zero value.
+ */
+ local64_set(&event->hw.prev_count, __xe_pmu_event_read(event));
+}
+
+static void xe_pmu_event_start(struct perf_event *event, int flags)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct xe_pmu *pmu = &xe->pmu;
+
+ if (pmu->closed)
+ return;
+
+ xe_pmu_enable(event);
+ event->hw.state = 0;
+}
+
+static void xe_pmu_event_stop(struct perf_event *event, int flags)
+{
+ if (flags & PERF_EF_UPDATE)
+ xe_pmu_event_read(event);
+
+ event->hw.state = PERF_HES_STOPPED;
+}
+
+static int xe_pmu_event_add(struct perf_event *event, int flags)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct xe_pmu *pmu = &xe->pmu;
+
+ if (pmu->closed)
+ return -ENODEV;
+
+ if (flags & PERF_EF_START)
+ xe_pmu_event_start(event, flags);
+
+ return 0;
+}
+
+static void xe_pmu_event_del(struct perf_event *event, int flags)
+{
+ xe_pmu_event_stop(event, PERF_EF_UPDATE);
+}
+
+static int xe_pmu_event_event_idx(struct perf_event *event)
+{
+ return 0;
+}
+
+struct xe_ext_attribute {
+ struct device_attribute attr;
+ unsigned long val;
+};
+
+static ssize_t xe_pmu_event_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct xe_ext_attribute *eattr;
+
+ eattr = container_of(attr, struct xe_ext_attribute, attr);
+ return sprintf(buf, "config=0x%lx\n", eattr->val);
+}
+
+static ssize_t cpumask_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ return cpumap_print_to_pagebuf(true, buf, &xe_pmu_cpumask);
+}
+
+static DEVICE_ATTR_RO(cpumask);
+
+static struct attribute *xe_cpumask_attrs[] = {
+ &dev_attr_cpumask.attr,
+ NULL,
+};
+
+static const struct attribute_group xe_pmu_cpumask_attr_group = {
+ .attrs = xe_cpumask_attrs,
+};
+
+#define __event(__counter, __name, __unit) \
+{ \
+ .counter = (__counter), \
+ .name = (__name), \
+ .unit = (__unit), \
+}
+
+static struct xe_ext_attribute *
+add_xe_attr(struct xe_ext_attribute *attr, const char *name, u64 config)
+{
+ sysfs_attr_init(&attr->attr.attr);
+ attr->attr.attr.name = name;
+ attr->attr.attr.mode = 0444;
+ attr->attr.show = xe_pmu_event_show;
+ attr->val = config;
+
+ return ++attr;
+}
+
+static struct perf_pmu_events_attr *
+add_pmu_attr(struct perf_pmu_events_attr *attr, const char *name,
+ const char *str)
+{
+ sysfs_attr_init(&attr->attr.attr);
+ attr->attr.attr.name = name;
+ attr->attr.attr.mode = 0444;
+ attr->attr.show = perf_event_sysfs_show;
+ attr->event_str = str;
+
+ return ++attr;
+}
+
+static struct attribute **
+create_event_attributes(struct xe_pmu *pmu)
+{
+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
+ static const struct {
+ unsigned int counter;
+ const char *name;
+ const char *unit;
+ } events[] = {
+ };
+
+ struct perf_pmu_events_attr *pmu_attr = NULL, *pmu_iter;
+ struct xe_ext_attribute *xe_attr = NULL, *xe_iter;
+ struct attribute **attr = NULL, **attr_iter;
+ unsigned int count = 0;
+ unsigned int i, j;
+ struct xe_gt *gt;
+
+ /* Count how many counters we will be exposing. */
+ for_each_gt(gt, xe, j) {
+ for (i = 0; i < ARRAY_SIZE(events); i++) {
+ u64 config = ___XE_PMU_OTHER(j, events[i].counter);
+
+ if (!config_status(xe, config))
+ count++;
+ }
+ }
+
+ /* Allocate attribute objects and table. */
+ xe_attr = kcalloc(count, sizeof(*xe_attr), GFP_KERNEL);
+ if (!xe_attr)
+ goto err_alloc;
+
+ pmu_attr = kcalloc(count, sizeof(*pmu_attr), GFP_KERNEL);
+ if (!pmu_attr)
+ goto err_alloc;
+
+ /* Max one pointer of each attribute type plus a termination entry. */
+ attr = kcalloc(count * 2 + 1, sizeof(*attr), GFP_KERNEL);
+ if (!attr)
+ goto err_alloc;
+
+ xe_iter = xe_attr;
+ pmu_iter = pmu_attr;
+ attr_iter = attr;
+
+ for_each_gt(gt, xe, j) {
+ for (i = 0; i < ARRAY_SIZE(events); i++) {
+ u64 config = ___XE_PMU_OTHER(j, events[i].counter);
+ char *str;
+
+ if (config_status(xe, config))
+ continue;
+
+ str = kasprintf(GFP_KERNEL, "%s-gt%u",
+ events[i].name, j);
+ if (!str)
+ goto err;
+
+ *attr_iter++ = &xe_iter->attr.attr;
+ xe_iter = add_xe_attr(xe_iter, str, config);
+
+ if (events[i].unit) {
+ str = kasprintf(GFP_KERNEL, "%s-gt%u.unit",
+ events[i].name, j);
+ if (!str)
+ goto err;
+
+ *attr_iter++ = &pmu_iter->attr.attr;
+ pmu_iter = add_pmu_attr(pmu_iter, str,
+ events[i].unit);
+ }
+ }
+ }
+
+ pmu->xe_attr = xe_attr;
+ pmu->pmu_attr = pmu_attr;
+
+ return attr;
+
+err:
+ for (attr_iter = attr; *attr_iter; attr_iter++)
+ kfree((*attr_iter)->name);
+
+err_alloc:
+ kfree(attr);
+ kfree(xe_attr);
+ kfree(pmu_attr);
+
+ return NULL;
+}
+
+static void free_event_attributes(struct xe_pmu *pmu)
+{
+ struct attribute **attr_iter = pmu->events_attr_group.attrs;
+
+ for (; *attr_iter; attr_iter++)
+ kfree((*attr_iter)->name);
+
+ kfree(pmu->events_attr_group.attrs);
+ kfree(pmu->xe_attr);
+ kfree(pmu->pmu_attr);
+
+ pmu->events_attr_group.attrs = NULL;
+ pmu->xe_attr = NULL;
+ pmu->pmu_attr = NULL;
+}
+
+static int xe_pmu_cpu_online(unsigned int cpu, struct hlist_node *node)
+{
+ struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
+
+ XE_WARN_ON(!pmu->base.event_init);
+
+ /* Select the first online CPU as a designated reader. */
+ if (cpumask_empty(&xe_pmu_cpumask))
+ cpumask_set_cpu(cpu, &xe_pmu_cpumask);
+
+ return 0;
+}
+
+static int xe_pmu_cpu_offline(unsigned int cpu, struct hlist_node *node)
+{
+ struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
+ unsigned int target = xe_pmu_target_cpu;
+
+ /*
+ * Unregistering an instance generates a CPU offline event which we must
+ * ignore to avoid incorrectly modifying the shared xe_pmu_cpumask.
+ */
+ if (pmu->closed)
+ return 0;
+
+ if (cpumask_test_and_clear_cpu(cpu, &xe_pmu_cpumask)) {
+ target = cpumask_any_but(topology_sibling_cpumask(cpu), cpu);
+
+ /* Migrate events if there is a valid target */
+ if (target < nr_cpu_ids) {
+ cpumask_set_cpu(target, &xe_pmu_cpumask);
+ xe_pmu_target_cpu = target;
+ }
+ }
+
+ if (target < nr_cpu_ids && target != pmu->cpuhp.cpu) {
+ perf_pmu_migrate_context(&pmu->base, cpu, target);
+ pmu->cpuhp.cpu = target;
+ }
+
+ return 0;
+}
+
+static enum cpuhp_state cpuhp_slot = CPUHP_INVALID;
+
+/**
+ * xe_pmu_init() - Setup CPU hotplug state/callbacks for Xe PMU
+ *
+ * Returns: 0 if successful, else error code
+ */
+int xe_pmu_init(void)
+{
+ int ret;
+
+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
+ "perf/x86/intel/xe:online",
+ xe_pmu_cpu_online,
+ xe_pmu_cpu_offline);
+ if (ret < 0)
+ pr_notice("Failed to setup cpuhp state for xe PMU! (%d)\n",
+ ret);
+ else
+ cpuhp_slot = ret;
+
+ return 0;
+}
+
+/**
+ * xe_pmu_exit() - Remove CPU hotplug state/callbacks for Xe PMU
+ */
+void xe_pmu_exit(void)
+{
+ if (cpuhp_slot != CPUHP_INVALID)
+ cpuhp_remove_multi_state(cpuhp_slot);
+}
+
+static int xe_pmu_register_cpuhp_state(struct xe_pmu *pmu)
+{
+ if (cpuhp_slot == CPUHP_INVALID)
+ return -EINVAL;
+
+ return cpuhp_state_add_instance(cpuhp_slot, &pmu->cpuhp.node);
+}
+
+static void xe_pmu_unregister_cpuhp_state(struct xe_pmu *pmu)
+{
+ cpuhp_state_remove_instance(cpuhp_slot, &pmu->cpuhp.node);
+}
+
+/**
+ * xe_pmu_unregister() - Remove/cleanup PMU registration
+ */
+void xe_pmu_unregister(void *arg)
+{
+ struct xe_pmu *pmu = arg;
+
+ if (!pmu->base.event_init)
+ return;
+
+ /*
+ * "Disconnect" the PMU callbacks - since all are atomic synchronize_rcu
+ * ensures all currently executing ones will have exited before we
+ * proceed with unregistration.
+ */
+ pmu->closed = true;
+ synchronize_rcu();
+
+ xe_pmu_unregister_cpuhp_state(pmu);
+
+ perf_pmu_unregister(&pmu->base);
+ pmu->base.event_init = NULL;
+ kfree(pmu->base.attr_groups);
+ kfree(pmu->name);
+ free_event_attributes(pmu);
+}
+
+/**
+ * xe_pmu_register() - Define basic PMU properties for Xe and add event callbacks.
+ *
+ */
+void xe_pmu_register(struct xe_pmu *pmu)
+{
+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
+ const struct attribute_group *attr_groups[] = {
+ &pmu->events_attr_group,
+ &xe_pmu_cpumask_attr_group,
+ NULL
+ };
+
+ int ret = -ENOMEM;
+
+ spin_lock_init(&pmu->lock);
+ pmu->cpuhp.cpu = -1;
+
+ pmu->name = kasprintf(GFP_KERNEL,
+ "xe_%s",
+ dev_name(xe->drm.dev));
+ if (pmu->name)
+ /* tools/perf reserves colons as special. */
+ strreplace((char *)pmu->name, ':', '_');
+
+ if (!pmu->name)
+ goto err;
+
+ pmu->events_attr_group.name = "events";
+ pmu->events_attr_group.attrs = create_event_attributes(pmu);
+ if (!pmu->events_attr_group.attrs)
+ goto err_name;
+
+ pmu->base.attr_groups = kmemdup(attr_groups, sizeof(attr_groups),
+ GFP_KERNEL);
+ if (!pmu->base.attr_groups)
+ goto err_attr;
+
+ pmu->base.module = THIS_MODULE;
+ pmu->base.task_ctx_nr = perf_invalid_context;
+ pmu->base.event_init = xe_pmu_event_init;
+ pmu->base.add = xe_pmu_event_add;
+ pmu->base.del = xe_pmu_event_del;
+ pmu->base.start = xe_pmu_event_start;
+ pmu->base.stop = xe_pmu_event_stop;
+ pmu->base.read = xe_pmu_event_read;
+ pmu->base.event_idx = xe_pmu_event_event_idx;
+
+ ret = perf_pmu_register(&pmu->base, pmu->name, -1);
+ if (ret)
+ goto err_groups;
+
+ ret = xe_pmu_register_cpuhp_state(pmu);
+ if (ret)
+ goto err_unreg;
+
+ ret = devm_add_action_or_reset(xe->drm.dev, xe_pmu_unregister, pmu);
+ if (ret)
+ goto err_cpuhp;
+
+ return;
+
+err_cpuhp:
+ xe_pmu_unregister_cpuhp_state(pmu);
+err_unreg:
+ perf_pmu_unregister(&pmu->base);
+err_groups:
+ kfree(pmu->base.attr_groups);
+err_attr:
+ pmu->base.event_init = NULL;
+ free_event_attributes(pmu);
+err_name:
+ kfree(pmu->name);
+err:
+ drm_notice(&xe->drm, "Failed to register PMU!\n");
+}
diff --git a/drivers/gpu/drm/xe/xe_pmu.h b/drivers/gpu/drm/xe/xe_pmu.h
new file mode 100644
index 000000000000..d07e5dfdfec0
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_pmu.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2024 Intel Corporation
+ */
+
+#ifndef _XE_PMU_H_
+#define _XE_PMU_H_
+
+#include "xe_pmu_types.h"
+
+struct xe_gt;
+
+#if IS_ENABLED(CONFIG_PERF_EVENTS)
+int xe_pmu_init(void);
+void xe_pmu_exit(void);
+void xe_pmu_register(struct xe_pmu *pmu);
+void xe_pmu_unregister(void *arg);
+#else
+static inline int xe_pmu_init(void) { return 0; }
+static inline void xe_pmu_exit(void) {}
+static inline void xe_pmu_register(struct xe_pmu *pmu) {}
+static inline void xe_pmu_unregister(void *arg) {}
+#endif
+
+#endif
+
diff --git a/drivers/gpu/drm/xe/xe_pmu_types.h b/drivers/gpu/drm/xe/xe_pmu_types.h
new file mode 100644
index 000000000000..ca0e7cbe2081
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_pmu_types.h
@@ -0,0 +1,63 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2024 Intel Corporation
+ */
+
+#ifndef _XE_PMU_TYPES_H_
+#define _XE_PMU_TYPES_H_
+
+#include <linux/perf_event.h>
+#include <linux/spinlock_types.h>
+#include <uapi/drm/xe_drm.h>
+
+enum {
+ __XE_NUM_PMU_SAMPLERS
+};
+
+#define XE_PMU_MAX_GT 2
+
+struct xe_pmu {
+ /**
+ * @cpuhp: Struct used for CPU hotplug handling.
+ */
+ struct {
+ struct hlist_node node;
+ unsigned int cpu;
+ } cpuhp;
+ /**
+ * @base: PMU base.
+ */
+ struct pmu base;
+ /**
+ * @closed: xe is unregistering.
+ */
+ bool closed;
+ /**
+ * @name: Name as registered with perf core.
+ */
+ const char *name;
+ /**
+ * @lock: Lock protecting enable mask and ref count handling.
+ */
+ spinlock_t lock;
+ /**
+ * @sample: Current and previous (raw) counters.
+ *
+ * These counters are updated when the device is awake.
+ */
+ u64 sample[XE_PMU_MAX_GT][__XE_NUM_PMU_SAMPLERS];
+ /**
+ * @events_attr_group: Device events attribute group.
+ */
+ struct attribute_group events_attr_group;
+ /**
+ * @xe_attr: Memory block holding device attributes.
+ */
+ void *xe_attr;
+ /**
+ * @pmu_attr: Memory block holding device attributes.
+ */
+ void *pmu_attr;
+};
+
+#endif
diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
index b6fbe4988f2e..2c5f258eee3a 100644
--- a/include/uapi/drm/xe_drm.h
+++ b/include/uapi/drm/xe_drm.h
@@ -1389,6 +1389,14 @@ struct drm_xe_wait_user_fence {
__u64 reserved[2];
};
+/*
+ * Top bits of every counter are GT id.
+ */
+#define __XE_PMU_GT_SHIFT (56)
+
+#define ___XE_PMU_OTHER(gt, x) \
+ (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
+
/**
* enum drm_xe_observation_type - Observation stream types
*/
--
2.38.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PATCH 1/3] drm/xe/pmu: Enable PMU interface
2024-09-27 0:23 ` [PATCH 1/3] drm/xe/pmu: Enable PMU interface Vinay Belgaumkar
@ 2024-09-27 18:26 ` Rodrigo Vivi
2024-09-27 19:45 ` Belgaumkar, Vinay
2024-10-24 22:58 ` Belgaumkar, Vinay
2024-10-01 21:32 ` Lucas De Marchi
1 sibling, 2 replies; 19+ messages in thread
From: Rodrigo Vivi @ 2024-09-27 18:26 UTC (permalink / raw)
To: Vinay Belgaumkar
Cc: intel-xe, Aravind Iddamsetty, Bommu Krishnaiah, Riana Tauro
On Thu, Sep 26, 2024 at 05:23:42PM -0700, Vinay Belgaumkar wrote:
> From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>
> Basic PMU enabling patch. Setup the basic framework
> for adding events/timers. This patch was previously
> reviewed here -
> https://patchwork.freedesktop.org/series/119504/
>
> The pmu base implementation is still from the
> i915 driver.
>
> v2: Review comments(Rodrigo) and do not init pmu for VFs
> as they don't have access to freq and c6 residency anyways.
sorry for the slow review and for not going through it all
in a single pass.
>
> v3: Fix kunit issue, move xe_pmu entry in Makefile (Jani)
>
> Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
> Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
> Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
> Signed-off-by: Riana Tauro <riana.tauro@intel.com>
> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> Signed-off-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
> ---
> drivers/gpu/drm/xe/Makefile | 2 +
> drivers/gpu/drm/xe/xe_device.c | 6 +
> drivers/gpu/drm/xe/xe_device_types.h | 4 +
> drivers/gpu/drm/xe/xe_module.c | 5 +
> drivers/gpu/drm/xe/xe_pmu.c | 570 +++++++++++++++++++++++++++
> drivers/gpu/drm/xe/xe_pmu.h | 26 ++
> drivers/gpu/drm/xe/xe_pmu_types.h | 63 +++
> include/uapi/drm/xe_drm.h | 8 +
> 8 files changed, 684 insertions(+)
> create mode 100644 drivers/gpu/drm/xe/xe_pmu.c
> create mode 100644 drivers/gpu/drm/xe/xe_pmu.h
> create mode 100644 drivers/gpu/drm/xe/xe_pmu_types.h
>
> diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
> index ae245fbd91ee..3a3eabc0e0a3 100644
> --- a/drivers/gpu/drm/xe/Makefile
> +++ b/drivers/gpu/drm/xe/Makefile
> @@ -293,6 +293,8 @@ ifeq ($(CONFIG_DEBUG_FS),y)
> i915-display/intel_pipe_crc.o
> endif
>
> +xe-$(CONFIG_PERF_EVENTS) += xe_pmu.o
> +
> obj-$(CONFIG_DRM_XE) += xe.o
> obj-$(CONFIG_DRM_XE_KUNIT_TEST) += tests/
>
> diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
> index cb5a9fd820cf..027a796d7431 100644
> --- a/drivers/gpu/drm/xe/xe_device.c
> +++ b/drivers/gpu/drm/xe/xe_device.c
> @@ -762,6 +762,9 @@ int xe_device_probe(struct xe_device *xe)
> for_each_gt(gt, xe, id)
> xe_gt_sanitize_freq(gt);
>
> + if (!IS_SRIOV_VF(xe))
> + xe_pmu_register(&xe->pmu);
> +
> return devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
>
> err_fini_display:
> @@ -806,6 +809,9 @@ void xe_device_remove(struct xe_device *xe)
>
> xe_heci_gsc_fini(xe);
>
> + if (!IS_SRIOV_VF(xe))
> + xe_pmu_unregister(&xe->pmu);
> +
> for_each_gt(gt, xe, id)
> xe_gt_remove(gt);
> }
> diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
> index 5ad96d283a71..51485d2eadcf 100644
> --- a/drivers/gpu/drm/xe/xe_device_types.h
> +++ b/drivers/gpu/drm/xe/xe_device_types.h
> @@ -18,6 +18,7 @@
> #include "xe_memirq_types.h"
> #include "xe_oa.h"
> #include "xe_platform_types.h"
> +#include "xe_pmu.h"
> #include "xe_pt_types.h"
> #include "xe_sriov_types.h"
> #include "xe_step_types.h"
> @@ -513,6 +514,9 @@ struct xe_device {
> int mode;
> } wedged;
>
> + /** @pmu: performance monitoring unit */
> + struct xe_pmu pmu;
> +
> #ifdef TEST_VM_OPS_ERROR
> /**
> * @vm_inject_error_position: inject errors at different places in VM
> diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
> index 77ce9f9ca7a5..1bf2bf8447c0 100644
> --- a/drivers/gpu/drm/xe/xe_module.c
> +++ b/drivers/gpu/drm/xe/xe_module.c
> @@ -14,6 +14,7 @@
> #include "xe_hw_fence.h"
> #include "xe_pci.h"
> #include "xe_pm.h"
> +#include "xe_pmu.h"
> #include "xe_observation.h"
> #include "xe_sched_job.h"
>
> @@ -96,6 +97,10 @@ static const struct init_funcs init_funcs[] = {
> .init = xe_sched_job_module_init,
> .exit = xe_sched_job_module_exit,
> },
> + {
> + .init = xe_pmu_init,
> + .exit = xe_pmu_exit,
> + },
> {
> .init = xe_register_pci_driver,
> .exit = xe_unregister_pci_driver,
> diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
> new file mode 100644
> index 000000000000..bdaea9ca1065
> --- /dev/null
> +++ b/drivers/gpu/drm/xe/xe_pmu.c
> @@ -0,0 +1,570 @@
> +// SPDX-License-Identifier: MIT
> +/*
> + * Copyright © 2024 Intel Corporation
> + */
> +
> +#include <drm/drm_drv.h>
> +#include <drm/drm_managed.h>
> +#include <drm/xe_drm.h>
> +
> +#include "regs/xe_gt_regs.h"
> +#include "xe_device.h"
> +#include "xe_force_wake.h"
> +#include "xe_gt_clock.h"
> +#include "xe_mmio.h"
> +#include "xe_macros.h"
> +#include "xe_pm.h"
> +
> +static cpumask_t xe_pmu_cpumask;
> +static unsigned int xe_pmu_target_cpu = -1;
Please no global variables. Think the case where we have
more then 1 device in the same cpu.
> +
> +/**
> + * DOC: Xe PMU (Performance Monitoring Unit)
> + *
> + * Expose events/counters like C6 residency and GT frequency to user land.
> + * Perf tool can be used to list and record these counters from the command
> + * line.
> + *
> + * Example commands to list/record supported perf events-
> + *
> + * >> perf list
perhas '$' instead of '>>' ?
> + * >> ls /sys/bus/event_source/devices/xe_0000_00_02.0/events/
perhaps ls -ld /sys/bus/event_source/devices/xe_*
to first know which devices are available?
> + *
> + * To record events-
> + * >> perf stat -e <event_name> -I <interval>
Could you please document here on how to ensure that those small incremental
values should be read?
How can I use this perf command and ensure that we get exactly the
same value as our other sysfs entries?
I'd like to do the math back to cross-check at least, but also
this information should be useful to ensure that users of this
are really well informed on the flow.
> + *
> + */
> +
> +static unsigned int config_gt_id(const u64 config)
> +{
> + return config >> __XE_PMU_GT_SHIFT;
> +}
> +
> +static u64 config_counter(const u64 config)
> +{
> + return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
> +}
> +
> +static void xe_pmu_event_destroy(struct perf_event *event)
> +{
> + struct xe_device *xe =
> + container_of(event->pmu, typeof(*xe), pmu.base);
> +
> + drm_WARN_ON(&xe->drm, event->parent);
> +
> + drm_dev_put(&xe->drm);
> +}
> +
> +static int
> +config_status(struct xe_device *xe, u64 config)
> +{
> + unsigned int gt_id = config_gt_id(config);
> +
> + if (gt_id >= XE_PMU_MAX_GT)
why do we have new GT defines instead of using the regular GT
numbering and entries?
> + return -ENOENT;
> +
> + switch (config_counter(config)) {
> + default:
> + return -ENOENT;
> + }
> +
> + return 0;
> +}
> +
> +static int xe_pmu_event_init(struct perf_event *event)
> +{
> + struct xe_device *xe =
> + container_of(event->pmu, typeof(*xe), pmu.base);
> + struct xe_pmu *pmu = &xe->pmu;
> + int ret;
> +
> + if (pmu->closed)
> + return -ENODEV;
> +
> + if (event->attr.type != event->pmu->type)
> + return -ENOENT;
> +
> + /* unsupported modes and filters */
> + if (event->attr.sample_period) /* no sampling */
> + return -EINVAL;
> +
> + if (has_branch_stack(event))
> + return -EOPNOTSUPP;
> +
> + if (event->cpu < 0)
> + return -EINVAL;
> +
> + /* only allow running on one cpu at a time */
> + if (!cpumask_test_cpu(event->cpu, &xe_pmu_cpumask))
> + return -EINVAL;
> +
> + ret = config_status(xe, event->attr.config);
> + if (ret)
> + return ret;
> +
> + if (!event->parent) {
> + drm_dev_get(&xe->drm);
> + event->destroy = xe_pmu_event_destroy;
> + }
> +
> + return 0;
> +}
> +
> +static u64 __xe_pmu_event_read(struct perf_event *event)
> +{
> + struct xe_device *xe =
> + container_of(event->pmu, typeof(*xe), pmu.base);
> + const unsigned int gt_id = config_gt_id(event->attr.config);
> + const u64 config = event->attr.config;
> + struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
> + u64 val = 0;
> +
> + switch (config_counter(config)) {
> + default:
> + drm_warn(>->tile->xe->drm, "unknown pmu event\n");
> + }
> +
> + return val;
> +}
> +
> +static void xe_pmu_event_read(struct perf_event *event)
> +{
> + struct xe_device *xe =
> + container_of(event->pmu, typeof(*xe), pmu.base);
> + struct hw_perf_event *hwc = &event->hw;
> + struct xe_pmu *pmu = &xe->pmu;
> + u64 prev, new;
> +
> + if (pmu->closed) {
> + event->hw.state = PERF_HES_STOPPED;
> + return;
> + }
> +again:
> + prev = local64_read(&hwc->prev_count);
> + new = __xe_pmu_event_read(event);
> +
> + if (local64_cmpxchg(&hwc->prev_count, prev, new) != prev)
> + goto again;
> +
> + local64_add(new - prev, &event->count);
> +}
> +
> +static void xe_pmu_enable(struct perf_event *event)
> +{
> + /*
> + * Store the current counter value so we can report the correct delta
> + * for all listeners. Even when the event was already enabled and has
> + * an existing non-zero value.
> + */
> + local64_set(&event->hw.prev_count, __xe_pmu_event_read(event));
> +}
> +
> +static void xe_pmu_event_start(struct perf_event *event, int flags)
> +{
> + struct xe_device *xe =
> + container_of(event->pmu, typeof(*xe), pmu.base);
> + struct xe_pmu *pmu = &xe->pmu;
> +
> + if (pmu->closed)
> + return;
> +
> + xe_pmu_enable(event);
> + event->hw.state = 0;
> +}
> +
> +static void xe_pmu_event_stop(struct perf_event *event, int flags)
> +{
> + if (flags & PERF_EF_UPDATE)
> + xe_pmu_event_read(event);
> +
> + event->hw.state = PERF_HES_STOPPED;
> +}
> +
> +static int xe_pmu_event_add(struct perf_event *event, int flags)
> +{
> + struct xe_device *xe =
> + container_of(event->pmu, typeof(*xe), pmu.base);
> + struct xe_pmu *pmu = &xe->pmu;
> +
> + if (pmu->closed)
> + return -ENODEV;
> +
> + if (flags & PERF_EF_START)
> + xe_pmu_event_start(event, flags);
> +
> + return 0;
> +}
> +
> +static void xe_pmu_event_del(struct perf_event *event, int flags)
> +{
> + xe_pmu_event_stop(event, PERF_EF_UPDATE);
> +}
> +
> +static int xe_pmu_event_event_idx(struct perf_event *event)
> +{
> + return 0;
> +}
> +
> +struct xe_ext_attribute {
> + struct device_attribute attr;
> + unsigned long val;
> +};
> +
> +static ssize_t xe_pmu_event_show(struct device *dev,
> + struct device_attribute *attr, char *buf)
> +{
> + struct xe_ext_attribute *eattr;
> +
> + eattr = container_of(attr, struct xe_ext_attribute, attr);
> + return sprintf(buf, "config=0x%lx\n", eattr->val);
> +}
> +
> +static ssize_t cpumask_show(struct device *dev,
> + struct device_attribute *attr, char *buf)
> +{
> + return cpumap_print_to_pagebuf(true, buf, &xe_pmu_cpumask);
> +}
> +
> +static DEVICE_ATTR_RO(cpumask);
> +
> +static struct attribute *xe_cpumask_attrs[] = {
> + &dev_attr_cpumask.attr,
> + NULL,
> +};
> +
> +static const struct attribute_group xe_pmu_cpumask_attr_group = {
> + .attrs = xe_cpumask_attrs,
> +};
> +
> +#define __event(__counter, __name, __unit) \
> +{ \
> + .counter = (__counter), \
> + .name = (__name), \
> + .unit = (__unit), \
> +}
> +
> +static struct xe_ext_attribute *
> +add_xe_attr(struct xe_ext_attribute *attr, const char *name, u64 config)
> +{
> + sysfs_attr_init(&attr->attr.attr);
> + attr->attr.attr.name = name;
> + attr->attr.attr.mode = 0444;
> + attr->attr.show = xe_pmu_event_show;
> + attr->val = config;
> +
> + return ++attr;
> +}
> +
> +static struct perf_pmu_events_attr *
> +add_pmu_attr(struct perf_pmu_events_attr *attr, const char *name,
> + const char *str)
> +{
> + sysfs_attr_init(&attr->attr.attr);
> + attr->attr.attr.name = name;
> + attr->attr.attr.mode = 0444;
> + attr->attr.show = perf_event_sysfs_show;
> + attr->event_str = str;
> +
> + return ++attr;
> +}
> +
> +static struct attribute **
> +create_event_attributes(struct xe_pmu *pmu)
> +{
> + struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
> + static const struct {
> + unsigned int counter;
> + const char *name;
> + const char *unit;
> + } events[] = {
> + };
> +
> + struct perf_pmu_events_attr *pmu_attr = NULL, *pmu_iter;
> + struct xe_ext_attribute *xe_attr = NULL, *xe_iter;
> + struct attribute **attr = NULL, **attr_iter;
> + unsigned int count = 0;
> + unsigned int i, j;
> + struct xe_gt *gt;
> +
> + /* Count how many counters we will be exposing. */
> + for_each_gt(gt, xe, j) {
> + for (i = 0; i < ARRAY_SIZE(events); i++) {
> + u64 config = ___XE_PMU_OTHER(j, events[i].counter);
> +
> + if (!config_status(xe, config))
> + count++;
> + }
> + }
> +
> + /* Allocate attribute objects and table. */
> + xe_attr = kcalloc(count, sizeof(*xe_attr), GFP_KERNEL);
> + if (!xe_attr)
> + goto err_alloc;
> +
> + pmu_attr = kcalloc(count, sizeof(*pmu_attr), GFP_KERNEL);
> + if (!pmu_attr)
> + goto err_alloc;
> +
> + /* Max one pointer of each attribute type plus a termination entry. */
> + attr = kcalloc(count * 2 + 1, sizeof(*attr), GFP_KERNEL);
> + if (!attr)
> + goto err_alloc;
> +
> + xe_iter = xe_attr;
> + pmu_iter = pmu_attr;
> + attr_iter = attr;
> +
> + for_each_gt(gt, xe, j) {
> + for (i = 0; i < ARRAY_SIZE(events); i++) {
> + u64 config = ___XE_PMU_OTHER(j, events[i].counter);
> + char *str;
> +
> + if (config_status(xe, config))
> + continue;
> +
> + str = kasprintf(GFP_KERNEL, "%s-gt%u",
> + events[i].name, j);
> + if (!str)
> + goto err;
> +
> + *attr_iter++ = &xe_iter->attr.attr;
> + xe_iter = add_xe_attr(xe_iter, str, config);
> +
> + if (events[i].unit) {
> + str = kasprintf(GFP_KERNEL, "%s-gt%u.unit",
> + events[i].name, j);
> + if (!str)
> + goto err;
> +
> + *attr_iter++ = &pmu_iter->attr.attr;
> + pmu_iter = add_pmu_attr(pmu_iter, str,
> + events[i].unit);
> + }
> + }
> + }
> +
> + pmu->xe_attr = xe_attr;
> + pmu->pmu_attr = pmu_attr;
> +
> + return attr;
> +
> +err:
> + for (attr_iter = attr; *attr_iter; attr_iter++)
> + kfree((*attr_iter)->name);
> +
> +err_alloc:
> + kfree(attr);
> + kfree(xe_attr);
> + kfree(pmu_attr);
> +
> + return NULL;
> +}
> +
> +static void free_event_attributes(struct xe_pmu *pmu)
> +{
> + struct attribute **attr_iter = pmu->events_attr_group.attrs;
> +
> + for (; *attr_iter; attr_iter++)
> + kfree((*attr_iter)->name);
> +
> + kfree(pmu->events_attr_group.attrs);
> + kfree(pmu->xe_attr);
> + kfree(pmu->pmu_attr);
> +
> + pmu->events_attr_group.attrs = NULL;
> + pmu->xe_attr = NULL;
> + pmu->pmu_attr = NULL;
> +}
> +
> +static int xe_pmu_cpu_online(unsigned int cpu, struct hlist_node *node)
> +{
> + struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
> +
> + XE_WARN_ON(!pmu->base.event_init);
> +
> + /* Select the first online CPU as a designated reader. */
> + if (cpumask_empty(&xe_pmu_cpumask))
> + cpumask_set_cpu(cpu, &xe_pmu_cpumask);
> +
> + return 0;
> +}
> +
> +static int xe_pmu_cpu_offline(unsigned int cpu, struct hlist_node *node)
> +{
> + struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
> + unsigned int target = xe_pmu_target_cpu;
> +
> + /*
> + * Unregistering an instance generates a CPU offline event which we must
> + * ignore to avoid incorrectly modifying the shared xe_pmu_cpumask.
> + */
> + if (pmu->closed)
> + return 0;
> +
> + if (cpumask_test_and_clear_cpu(cpu, &xe_pmu_cpumask)) {
> + target = cpumask_any_but(topology_sibling_cpumask(cpu), cpu);
> +
> + /* Migrate events if there is a valid target */
> + if (target < nr_cpu_ids) {
> + cpumask_set_cpu(target, &xe_pmu_cpumask);
> + xe_pmu_target_cpu = target;
> + }
> + }
> +
> + if (target < nr_cpu_ids && target != pmu->cpuhp.cpu) {
> + perf_pmu_migrate_context(&pmu->base, cpu, target);
> + pmu->cpuhp.cpu = target;
> + }
> +
> + return 0;
> +}
> +
> +static enum cpuhp_state cpuhp_slot = CPUHP_INVALID;
> +
> +/**
> + * xe_pmu_init() - Setup CPU hotplug state/callbacks for Xe PMU
> + *
> + * Returns: 0 if successful, else error code
> + */
> +int xe_pmu_init(void)
> +{
> + int ret;
> +
> + ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
> + "perf/x86/intel/xe:online",
> + xe_pmu_cpu_online,
> + xe_pmu_cpu_offline);
> + if (ret < 0)
> + pr_notice("Failed to setup cpuhp state for xe PMU! (%d)\n",
> + ret);
> + else
> + cpuhp_slot = ret;
> +
> + return 0;
> +}
> +
> +/**
> + * xe_pmu_exit() - Remove CPU hotplug state/callbacks for Xe PMU
> + */
> +void xe_pmu_exit(void)
> +{
> + if (cpuhp_slot != CPUHP_INVALID)
> + cpuhp_remove_multi_state(cpuhp_slot);
> +}
> +
> +static int xe_pmu_register_cpuhp_state(struct xe_pmu *pmu)
> +{
> + if (cpuhp_slot == CPUHP_INVALID)
> + return -EINVAL;
> +
> + return cpuhp_state_add_instance(cpuhp_slot, &pmu->cpuhp.node);
> +}
> +
> +static void xe_pmu_unregister_cpuhp_state(struct xe_pmu *pmu)
> +{
> + cpuhp_state_remove_instance(cpuhp_slot, &pmu->cpuhp.node);
> +}
> +
> +/**
> + * xe_pmu_unregister() - Remove/cleanup PMU registration
> + */
> +void xe_pmu_unregister(void *arg)
> +{
> + struct xe_pmu *pmu = arg;
> +
> + if (!pmu->base.event_init)
> + return;
> +
> + /*
> + * "Disconnect" the PMU callbacks - since all are atomic synchronize_rcu
> + * ensures all currently executing ones will have exited before we
> + * proceed with unregistration.
> + */
> + pmu->closed = true;
> + synchronize_rcu();
> +
> + xe_pmu_unregister_cpuhp_state(pmu);
> +
> + perf_pmu_unregister(&pmu->base);
> + pmu->base.event_init = NULL;
> + kfree(pmu->base.attr_groups);
> + kfree(pmu->name);
> + free_event_attributes(pmu);
> +}
> +
> +/**
> + * xe_pmu_register() - Define basic PMU properties for Xe and add event callbacks.
> + *
> + */
> +void xe_pmu_register(struct xe_pmu *pmu)
> +{
> + struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
> + const struct attribute_group *attr_groups[] = {
> + &pmu->events_attr_group,
> + &xe_pmu_cpumask_attr_group,
> + NULL
> + };
> +
> + int ret = -ENOMEM;
> +
> + spin_lock_init(&pmu->lock);
> + pmu->cpuhp.cpu = -1;
> +
> + pmu->name = kasprintf(GFP_KERNEL,
> + "xe_%s",
> + dev_name(xe->drm.dev));
> + if (pmu->name)
> + /* tools/perf reserves colons as special. */
> + strreplace((char *)pmu->name, ':', '_');
> +
> + if (!pmu->name)
> + goto err;
> +
> + pmu->events_attr_group.name = "events";
> + pmu->events_attr_group.attrs = create_event_attributes(pmu);
> + if (!pmu->events_attr_group.attrs)
> + goto err_name;
> +
> + pmu->base.attr_groups = kmemdup(attr_groups, sizeof(attr_groups),
> + GFP_KERNEL);
> + if (!pmu->base.attr_groups)
> + goto err_attr;
> +
> + pmu->base.module = THIS_MODULE;
> + pmu->base.task_ctx_nr = perf_invalid_context;
> + pmu->base.event_init = xe_pmu_event_init;
> + pmu->base.add = xe_pmu_event_add;
> + pmu->base.del = xe_pmu_event_del;
> + pmu->base.start = xe_pmu_event_start;
> + pmu->base.stop = xe_pmu_event_stop;
> + pmu->base.read = xe_pmu_event_read;
> + pmu->base.event_idx = xe_pmu_event_event_idx;
> +
> + ret = perf_pmu_register(&pmu->base, pmu->name, -1);
> + if (ret)
> + goto err_groups;
> +
> + ret = xe_pmu_register_cpuhp_state(pmu);
> + if (ret)
> + goto err_unreg;
> +
> + ret = devm_add_action_or_reset(xe->drm.dev, xe_pmu_unregister, pmu);
> + if (ret)
> + goto err_cpuhp;
> +
> + return;
> +
> +err_cpuhp:
> + xe_pmu_unregister_cpuhp_state(pmu);
> +err_unreg:
> + perf_pmu_unregister(&pmu->base);
> +err_groups:
> + kfree(pmu->base.attr_groups);
> +err_attr:
> + pmu->base.event_init = NULL;
> + free_event_attributes(pmu);
> +err_name:
> + kfree(pmu->name);
> +err:
> + drm_notice(&xe->drm, "Failed to register PMU!\n");
> +}
> diff --git a/drivers/gpu/drm/xe/xe_pmu.h b/drivers/gpu/drm/xe/xe_pmu.h
> new file mode 100644
> index 000000000000..d07e5dfdfec0
> --- /dev/null
> +++ b/drivers/gpu/drm/xe/xe_pmu.h
> @@ -0,0 +1,26 @@
> +/* SPDX-License-Identifier: MIT */
> +/*
> + * Copyright © 2024 Intel Corporation
> + */
> +
> +#ifndef _XE_PMU_H_
> +#define _XE_PMU_H_
> +
> +#include "xe_pmu_types.h"
> +
> +struct xe_gt;
> +
> +#if IS_ENABLED(CONFIG_PERF_EVENTS)
> +int xe_pmu_init(void);
> +void xe_pmu_exit(void);
> +void xe_pmu_register(struct xe_pmu *pmu);
> +void xe_pmu_unregister(void *arg);
> +#else
> +static inline int xe_pmu_init(void) { return 0; }
> +static inline void xe_pmu_exit(void) {}
> +static inline void xe_pmu_register(struct xe_pmu *pmu) {}
> +static inline void xe_pmu_unregister(void *arg) {}
> +#endif
> +
> +#endif
> +
> diff --git a/drivers/gpu/drm/xe/xe_pmu_types.h b/drivers/gpu/drm/xe/xe_pmu_types.h
> new file mode 100644
> index 000000000000..ca0e7cbe2081
> --- /dev/null
> +++ b/drivers/gpu/drm/xe/xe_pmu_types.h
> @@ -0,0 +1,63 @@
> +/* SPDX-License-Identifier: MIT */
> +/*
> + * Copyright © 2024 Intel Corporation
> + */
> +
> +#ifndef _XE_PMU_TYPES_H_
> +#define _XE_PMU_TYPES_H_
> +
> +#include <linux/perf_event.h>
> +#include <linux/spinlock_types.h>
> +#include <uapi/drm/xe_drm.h>
> +
> +enum {
> + __XE_NUM_PMU_SAMPLERS
> +};
> +
> +#define XE_PMU_MAX_GT 2
> +
> +struct xe_pmu {
> + /**
> + * @cpuhp: Struct used for CPU hotplug handling.
> + */
> + struct {
> + struct hlist_node node;
> + unsigned int cpu;
> + } cpuhp;
> + /**
> + * @base: PMU base.
> + */
> + struct pmu base;
> + /**
> + * @closed: xe is unregistering.
> + */
> + bool closed;
> + /**
> + * @name: Name as registered with perf core.
> + */
> + const char *name;
> + /**
> + * @lock: Lock protecting enable mask and ref count handling.
> + */
> + spinlock_t lock;
> + /**
> + * @sample: Current and previous (raw) counters.
> + *
> + * These counters are updated when the device is awake.
> + */
> + u64 sample[XE_PMU_MAX_GT][__XE_NUM_PMU_SAMPLERS];
> + /**
> + * @events_attr_group: Device events attribute group.
> + */
> + struct attribute_group events_attr_group;
> + /**
> + * @xe_attr: Memory block holding device attributes.
> + */
> + void *xe_attr;
> + /**
> + * @pmu_attr: Memory block holding device attributes.
> + */
> + void *pmu_attr;
> +};
> +
> +#endif
> diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
> index b6fbe4988f2e..2c5f258eee3a 100644
> --- a/include/uapi/drm/xe_drm.h
> +++ b/include/uapi/drm/xe_drm.h
> @@ -1389,6 +1389,14 @@ struct drm_xe_wait_user_fence {
> __u64 reserved[2];
> };
>
> +/*
> + * Top bits of every counter are GT id.
> + */
> +#define __XE_PMU_GT_SHIFT (56)
> +
> +#define ___XE_PMU_OTHER(gt, x) \
> + (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
> +
> /**
> * enum drm_xe_observation_type - Observation stream types
> */
> --
> 2.38.1
>
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 1/3] drm/xe/pmu: Enable PMU interface
2024-09-27 18:26 ` Rodrigo Vivi
@ 2024-09-27 19:45 ` Belgaumkar, Vinay
2024-10-24 22:58 ` Belgaumkar, Vinay
1 sibling, 0 replies; 19+ messages in thread
From: Belgaumkar, Vinay @ 2024-09-27 19:45 UTC (permalink / raw)
To: Rodrigo Vivi; +Cc: intel-xe, Aravind Iddamsetty, Bommu Krishnaiah, Riana Tauro
On 9/27/2024 11:26 AM, Rodrigo Vivi wrote:
> On Thu, Sep 26, 2024 at 05:23:42PM -0700, Vinay Belgaumkar wrote:
>> From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>>
>> Basic PMU enabling patch. Setup the basic framework
>> for adding events/timers. This patch was previously
>> reviewed here -
>> https://patchwork.freedesktop.org/series/119504/
>>
>> The pmu base implementation is still from the
>> i915 driver.
>>
>> v2: Review comments(Rodrigo) and do not init pmu for VFs
>> as they don't have access to freq and c6 residency anyways.
> sorry for the slow review and for not going through it all
> in a single pass.
>
>> v3: Fix kunit issue, move xe_pmu entry in Makefile (Jani)
>>
>> Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>> Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>> Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>> Signed-off-by: Riana Tauro <riana.tauro@intel.com>
>> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
>> Signed-off-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
>> ---
>> drivers/gpu/drm/xe/Makefile | 2 +
>> drivers/gpu/drm/xe/xe_device.c | 6 +
>> drivers/gpu/drm/xe/xe_device_types.h | 4 +
>> drivers/gpu/drm/xe/xe_module.c | 5 +
>> drivers/gpu/drm/xe/xe_pmu.c | 570 +++++++++++++++++++++++++++
>> drivers/gpu/drm/xe/xe_pmu.h | 26 ++
>> drivers/gpu/drm/xe/xe_pmu_types.h | 63 +++
>> include/uapi/drm/xe_drm.h | 8 +
>> 8 files changed, 684 insertions(+)
>> create mode 100644 drivers/gpu/drm/xe/xe_pmu.c
>> create mode 100644 drivers/gpu/drm/xe/xe_pmu.h
>> create mode 100644 drivers/gpu/drm/xe/xe_pmu_types.h
>>
>> diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
>> index ae245fbd91ee..3a3eabc0e0a3 100644
>> --- a/drivers/gpu/drm/xe/Makefile
>> +++ b/drivers/gpu/drm/xe/Makefile
>> @@ -293,6 +293,8 @@ ifeq ($(CONFIG_DEBUG_FS),y)
>> i915-display/intel_pipe_crc.o
>> endif
>>
>> +xe-$(CONFIG_PERF_EVENTS) += xe_pmu.o
>> +
>> obj-$(CONFIG_DRM_XE) += xe.o
>> obj-$(CONFIG_DRM_XE_KUNIT_TEST) += tests/
>>
>> diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
>> index cb5a9fd820cf..027a796d7431 100644
>> --- a/drivers/gpu/drm/xe/xe_device.c
>> +++ b/drivers/gpu/drm/xe/xe_device.c
>> @@ -762,6 +762,9 @@ int xe_device_probe(struct xe_device *xe)
>> for_each_gt(gt, xe, id)
>> xe_gt_sanitize_freq(gt);
>>
>> + if (!IS_SRIOV_VF(xe))
>> + xe_pmu_register(&xe->pmu);
>> +
>> return devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
>>
>> err_fini_display:
>> @@ -806,6 +809,9 @@ void xe_device_remove(struct xe_device *xe)
>>
>> xe_heci_gsc_fini(xe);
>>
>> + if (!IS_SRIOV_VF(xe))
>> + xe_pmu_unregister(&xe->pmu);
>> +
>> for_each_gt(gt, xe, id)
>> xe_gt_remove(gt);
>> }
>> diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
>> index 5ad96d283a71..51485d2eadcf 100644
>> --- a/drivers/gpu/drm/xe/xe_device_types.h
>> +++ b/drivers/gpu/drm/xe/xe_device_types.h
>> @@ -18,6 +18,7 @@
>> #include "xe_memirq_types.h"
>> #include "xe_oa.h"
>> #include "xe_platform_types.h"
>> +#include "xe_pmu.h"
>> #include "xe_pt_types.h"
>> #include "xe_sriov_types.h"
>> #include "xe_step_types.h"
>> @@ -513,6 +514,9 @@ struct xe_device {
>> int mode;
>> } wedged;
>>
>> + /** @pmu: performance monitoring unit */
>> + struct xe_pmu pmu;
>> +
>> #ifdef TEST_VM_OPS_ERROR
>> /**
>> * @vm_inject_error_position: inject errors at different places in VM
>> diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
>> index 77ce9f9ca7a5..1bf2bf8447c0 100644
>> --- a/drivers/gpu/drm/xe/xe_module.c
>> +++ b/drivers/gpu/drm/xe/xe_module.c
>> @@ -14,6 +14,7 @@
>> #include "xe_hw_fence.h"
>> #include "xe_pci.h"
>> #include "xe_pm.h"
>> +#include "xe_pmu.h"
>> #include "xe_observation.h"
>> #include "xe_sched_job.h"
>>
>> @@ -96,6 +97,10 @@ static const struct init_funcs init_funcs[] = {
>> .init = xe_sched_job_module_init,
>> .exit = xe_sched_job_module_exit,
>> },
>> + {
>> + .init = xe_pmu_init,
>> + .exit = xe_pmu_exit,
>> + },
>> {
>> .init = xe_register_pci_driver,
>> .exit = xe_unregister_pci_driver,
>> diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
>> new file mode 100644
>> index 000000000000..bdaea9ca1065
>> --- /dev/null
>> +++ b/drivers/gpu/drm/xe/xe_pmu.c
>> @@ -0,0 +1,570 @@
>> +// SPDX-License-Identifier: MIT
>> +/*
>> + * Copyright © 2024 Intel Corporation
>> + */
>> +
>> +#include <drm/drm_drv.h>
>> +#include <drm/drm_managed.h>
>> +#include <drm/xe_drm.h>
>> +
>> +#include "regs/xe_gt_regs.h"
>> +#include "xe_device.h"
>> +#include "xe_force_wake.h"
>> +#include "xe_gt_clock.h"
>> +#include "xe_mmio.h"
>> +#include "xe_macros.h"
>> +#include "xe_pm.h"
>> +
>> +static cpumask_t xe_pmu_cpumask;
>> +static unsigned int xe_pmu_target_cpu = -1;
> Please no global variables. Think the case where we have
> more then 1 device in the same cpu.
>
>> +
>> +/**
>> + * DOC: Xe PMU (Performance Monitoring Unit)
>> + *
>> + * Expose events/counters like C6 residency and GT frequency to user land.
>> + * Perf tool can be used to list and record these counters from the command
>> + * line.
>> + *
>> + * Example commands to list/record supported perf events-
>> + *
>> + * >> perf list
> perhas '$' instead of '>>' ?
>
>> + * >> ls /sys/bus/event_source/devices/xe_0000_00_02.0/events/
> perhaps ls -ld /sys/bus/event_source/devices/xe_*
>
> to first know which devices are available?
>
>> + *
>> + * To record events-
>> + * >> perf stat -e <event_name> -I <interval>
> Could you please document here on how to ensure that those small incremental
> values should be read?
> How can I use this perf command and ensure that we get exactly the
> same value as our other sysfs entries?
> I'd like to do the math back to cross-check at least, but also
> this information should be useful to ensure that users of this
> are really well informed on the flow.
Sure, something like this should help?
root@DUT147LNL:/home/gta# ./perf stat -e xe_0000_00_02.0/rc6-residency-gt0/
Performance counter stats for 'system wide':
1840175545 ms xe_0000_00_02.0/rc6-residency-gt0/
1.840228260 seconds time elapsed
This shows CPU elapsed time of 1.8s and GPU time in the similar order.
Issue that needs fixing is looks like we are displaying ns and calling
it ms!
Thanks,
Vinay.
>
>> + *
>> + */
>> +
>> +static unsigned int config_gt_id(const u64 config)
>> +{
>> + return config >> __XE_PMU_GT_SHIFT;
>> +}
>> +
>> +static u64 config_counter(const u64 config)
>> +{
>> + return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
>> +}
>> +
>> +static void xe_pmu_event_destroy(struct perf_event *event)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> +
>> + drm_WARN_ON(&xe->drm, event->parent);
>> +
>> + drm_dev_put(&xe->drm);
>> +}
>> +
>> +static int
>> +config_status(struct xe_device *xe, u64 config)
>> +{
>> + unsigned int gt_id = config_gt_id(config);
>> +
>> + if (gt_id >= XE_PMU_MAX_GT)
> why do we have new GT defines instead of using the regular GT
> numbering and entries?
>
>> + return -ENOENT;
>> +
>> + switch (config_counter(config)) {
>> + default:
>> + return -ENOENT;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static int xe_pmu_event_init(struct perf_event *event)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> + struct xe_pmu *pmu = &xe->pmu;
>> + int ret;
>> +
>> + if (pmu->closed)
>> + return -ENODEV;
>> +
>> + if (event->attr.type != event->pmu->type)
>> + return -ENOENT;
>> +
>> + /* unsupported modes and filters */
>> + if (event->attr.sample_period) /* no sampling */
>> + return -EINVAL;
>> +
>> + if (has_branch_stack(event))
>> + return -EOPNOTSUPP;
>> +
>> + if (event->cpu < 0)
>> + return -EINVAL;
>> +
>> + /* only allow running on one cpu at a time */
>> + if (!cpumask_test_cpu(event->cpu, &xe_pmu_cpumask))
>> + return -EINVAL;
>> +
>> + ret = config_status(xe, event->attr.config);
>> + if (ret)
>> + return ret;
>> +
>> + if (!event->parent) {
>> + drm_dev_get(&xe->drm);
>> + event->destroy = xe_pmu_event_destroy;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static u64 __xe_pmu_event_read(struct perf_event *event)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> + const unsigned int gt_id = config_gt_id(event->attr.config);
>> + const u64 config = event->attr.config;
>> + struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>> + u64 val = 0;
>> +
>> + switch (config_counter(config)) {
>> + default:
>> + drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>> + }
>> +
>> + return val;
>> +}
>> +
>> +static void xe_pmu_event_read(struct perf_event *event)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> + struct hw_perf_event *hwc = &event->hw;
>> + struct xe_pmu *pmu = &xe->pmu;
>> + u64 prev, new;
>> +
>> + if (pmu->closed) {
>> + event->hw.state = PERF_HES_STOPPED;
>> + return;
>> + }
>> +again:
>> + prev = local64_read(&hwc->prev_count);
>> + new = __xe_pmu_event_read(event);
>> +
>> + if (local64_cmpxchg(&hwc->prev_count, prev, new) != prev)
>> + goto again;
>> +
>> + local64_add(new - prev, &event->count);
>> +}
>> +
>> +static void xe_pmu_enable(struct perf_event *event)
>> +{
>> + /*
>> + * Store the current counter value so we can report the correct delta
>> + * for all listeners. Even when the event was already enabled and has
>> + * an existing non-zero value.
>> + */
>> + local64_set(&event->hw.prev_count, __xe_pmu_event_read(event));
>> +}
>> +
>> +static void xe_pmu_event_start(struct perf_event *event, int flags)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> + struct xe_pmu *pmu = &xe->pmu;
>> +
>> + if (pmu->closed)
>> + return;
>> +
>> + xe_pmu_enable(event);
>> + event->hw.state = 0;
>> +}
>> +
>> +static void xe_pmu_event_stop(struct perf_event *event, int flags)
>> +{
>> + if (flags & PERF_EF_UPDATE)
>> + xe_pmu_event_read(event);
>> +
>> + event->hw.state = PERF_HES_STOPPED;
>> +}
>> +
>> +static int xe_pmu_event_add(struct perf_event *event, int flags)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> + struct xe_pmu *pmu = &xe->pmu;
>> +
>> + if (pmu->closed)
>> + return -ENODEV;
>> +
>> + if (flags & PERF_EF_START)
>> + xe_pmu_event_start(event, flags);
>> +
>> + return 0;
>> +}
>> +
>> +static void xe_pmu_event_del(struct perf_event *event, int flags)
>> +{
>> + xe_pmu_event_stop(event, PERF_EF_UPDATE);
>> +}
>> +
>> +static int xe_pmu_event_event_idx(struct perf_event *event)
>> +{
>> + return 0;
>> +}
>> +
>> +struct xe_ext_attribute {
>> + struct device_attribute attr;
>> + unsigned long val;
>> +};
>> +
>> +static ssize_t xe_pmu_event_show(struct device *dev,
>> + struct device_attribute *attr, char *buf)
>> +{
>> + struct xe_ext_attribute *eattr;
>> +
>> + eattr = container_of(attr, struct xe_ext_attribute, attr);
>> + return sprintf(buf, "config=0x%lx\n", eattr->val);
>> +}
>> +
>> +static ssize_t cpumask_show(struct device *dev,
>> + struct device_attribute *attr, char *buf)
>> +{
>> + return cpumap_print_to_pagebuf(true, buf, &xe_pmu_cpumask);
>> +}
>> +
>> +static DEVICE_ATTR_RO(cpumask);
>> +
>> +static struct attribute *xe_cpumask_attrs[] = {
>> + &dev_attr_cpumask.attr,
>> + NULL,
>> +};
>> +
>> +static const struct attribute_group xe_pmu_cpumask_attr_group = {
>> + .attrs = xe_cpumask_attrs,
>> +};
>> +
>> +#define __event(__counter, __name, __unit) \
>> +{ \
>> + .counter = (__counter), \
>> + .name = (__name), \
>> + .unit = (__unit), \
>> +}
>> +
>> +static struct xe_ext_attribute *
>> +add_xe_attr(struct xe_ext_attribute *attr, const char *name, u64 config)
>> +{
>> + sysfs_attr_init(&attr->attr.attr);
>> + attr->attr.attr.name = name;
>> + attr->attr.attr.mode = 0444;
>> + attr->attr.show = xe_pmu_event_show;
>> + attr->val = config;
>> +
>> + return ++attr;
>> +}
>> +
>> +static struct perf_pmu_events_attr *
>> +add_pmu_attr(struct perf_pmu_events_attr *attr, const char *name,
>> + const char *str)
>> +{
>> + sysfs_attr_init(&attr->attr.attr);
>> + attr->attr.attr.name = name;
>> + attr->attr.attr.mode = 0444;
>> + attr->attr.show = perf_event_sysfs_show;
>> + attr->event_str = str;
>> +
>> + return ++attr;
>> +}
>> +
>> +static struct attribute **
>> +create_event_attributes(struct xe_pmu *pmu)
>> +{
>> + struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>> + static const struct {
>> + unsigned int counter;
>> + const char *name;
>> + const char *unit;
>> + } events[] = {
>> + };
>> +
>> + struct perf_pmu_events_attr *pmu_attr = NULL, *pmu_iter;
>> + struct xe_ext_attribute *xe_attr = NULL, *xe_iter;
>> + struct attribute **attr = NULL, **attr_iter;
>> + unsigned int count = 0;
>> + unsigned int i, j;
>> + struct xe_gt *gt;
>> +
>> + /* Count how many counters we will be exposing. */
>> + for_each_gt(gt, xe, j) {
>> + for (i = 0; i < ARRAY_SIZE(events); i++) {
>> + u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>> +
>> + if (!config_status(xe, config))
>> + count++;
>> + }
>> + }
>> +
>> + /* Allocate attribute objects and table. */
>> + xe_attr = kcalloc(count, sizeof(*xe_attr), GFP_KERNEL);
>> + if (!xe_attr)
>> + goto err_alloc;
>> +
>> + pmu_attr = kcalloc(count, sizeof(*pmu_attr), GFP_KERNEL);
>> + if (!pmu_attr)
>> + goto err_alloc;
>> +
>> + /* Max one pointer of each attribute type plus a termination entry. */
>> + attr = kcalloc(count * 2 + 1, sizeof(*attr), GFP_KERNEL);
>> + if (!attr)
>> + goto err_alloc;
>> +
>> + xe_iter = xe_attr;
>> + pmu_iter = pmu_attr;
>> + attr_iter = attr;
>> +
>> + for_each_gt(gt, xe, j) {
>> + for (i = 0; i < ARRAY_SIZE(events); i++) {
>> + u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>> + char *str;
>> +
>> + if (config_status(xe, config))
>> + continue;
>> +
>> + str = kasprintf(GFP_KERNEL, "%s-gt%u",
>> + events[i].name, j);
>> + if (!str)
>> + goto err;
>> +
>> + *attr_iter++ = &xe_iter->attr.attr;
>> + xe_iter = add_xe_attr(xe_iter, str, config);
>> +
>> + if (events[i].unit) {
>> + str = kasprintf(GFP_KERNEL, "%s-gt%u.unit",
>> + events[i].name, j);
>> + if (!str)
>> + goto err;
>> +
>> + *attr_iter++ = &pmu_iter->attr.attr;
>> + pmu_iter = add_pmu_attr(pmu_iter, str,
>> + events[i].unit);
>> + }
>> + }
>> + }
>> +
>> + pmu->xe_attr = xe_attr;
>> + pmu->pmu_attr = pmu_attr;
>> +
>> + return attr;
>> +
>> +err:
>> + for (attr_iter = attr; *attr_iter; attr_iter++)
>> + kfree((*attr_iter)->name);
>> +
>> +err_alloc:
>> + kfree(attr);
>> + kfree(xe_attr);
>> + kfree(pmu_attr);
>> +
>> + return NULL;
>> +}
>> +
>> +static void free_event_attributes(struct xe_pmu *pmu)
>> +{
>> + struct attribute **attr_iter = pmu->events_attr_group.attrs;
>> +
>> + for (; *attr_iter; attr_iter++)
>> + kfree((*attr_iter)->name);
>> +
>> + kfree(pmu->events_attr_group.attrs);
>> + kfree(pmu->xe_attr);
>> + kfree(pmu->pmu_attr);
>> +
>> + pmu->events_attr_group.attrs = NULL;
>> + pmu->xe_attr = NULL;
>> + pmu->pmu_attr = NULL;
>> +}
>> +
>> +static int xe_pmu_cpu_online(unsigned int cpu, struct hlist_node *node)
>> +{
>> + struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
>> +
>> + XE_WARN_ON(!pmu->base.event_init);
>> +
>> + /* Select the first online CPU as a designated reader. */
>> + if (cpumask_empty(&xe_pmu_cpumask))
>> + cpumask_set_cpu(cpu, &xe_pmu_cpumask);
>> +
>> + return 0;
>> +}
>> +
>> +static int xe_pmu_cpu_offline(unsigned int cpu, struct hlist_node *node)
>> +{
>> + struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
>> + unsigned int target = xe_pmu_target_cpu;
>> +
>> + /*
>> + * Unregistering an instance generates a CPU offline event which we must
>> + * ignore to avoid incorrectly modifying the shared xe_pmu_cpumask.
>> + */
>> + if (pmu->closed)
>> + return 0;
>> +
>> + if (cpumask_test_and_clear_cpu(cpu, &xe_pmu_cpumask)) {
>> + target = cpumask_any_but(topology_sibling_cpumask(cpu), cpu);
>> +
>> + /* Migrate events if there is a valid target */
>> + if (target < nr_cpu_ids) {
>> + cpumask_set_cpu(target, &xe_pmu_cpumask);
>> + xe_pmu_target_cpu = target;
>> + }
>> + }
>> +
>> + if (target < nr_cpu_ids && target != pmu->cpuhp.cpu) {
>> + perf_pmu_migrate_context(&pmu->base, cpu, target);
>> + pmu->cpuhp.cpu = target;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static enum cpuhp_state cpuhp_slot = CPUHP_INVALID;
>> +
>> +/**
>> + * xe_pmu_init() - Setup CPU hotplug state/callbacks for Xe PMU
>> + *
>> + * Returns: 0 if successful, else error code
>> + */
>> +int xe_pmu_init(void)
>> +{
>> + int ret;
>> +
>> + ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
>> + "perf/x86/intel/xe:online",
>> + xe_pmu_cpu_online,
>> + xe_pmu_cpu_offline);
>> + if (ret < 0)
>> + pr_notice("Failed to setup cpuhp state for xe PMU! (%d)\n",
>> + ret);
>> + else
>> + cpuhp_slot = ret;
>> +
>> + return 0;
>> +}
>> +
>> +/**
>> + * xe_pmu_exit() - Remove CPU hotplug state/callbacks for Xe PMU
>> + */
>> +void xe_pmu_exit(void)
>> +{
>> + if (cpuhp_slot != CPUHP_INVALID)
>> + cpuhp_remove_multi_state(cpuhp_slot);
>> +}
>> +
>> +static int xe_pmu_register_cpuhp_state(struct xe_pmu *pmu)
>> +{
>> + if (cpuhp_slot == CPUHP_INVALID)
>> + return -EINVAL;
>> +
>> + return cpuhp_state_add_instance(cpuhp_slot, &pmu->cpuhp.node);
>> +}
>> +
>> +static void xe_pmu_unregister_cpuhp_state(struct xe_pmu *pmu)
>> +{
>> + cpuhp_state_remove_instance(cpuhp_slot, &pmu->cpuhp.node);
>> +}
>> +
>> +/**
>> + * xe_pmu_unregister() - Remove/cleanup PMU registration
>> + */
>> +void xe_pmu_unregister(void *arg)
>> +{
>> + struct xe_pmu *pmu = arg;
>> +
>> + if (!pmu->base.event_init)
>> + return;
>> +
>> + /*
>> + * "Disconnect" the PMU callbacks - since all are atomic synchronize_rcu
>> + * ensures all currently executing ones will have exited before we
>> + * proceed with unregistration.
>> + */
>> + pmu->closed = true;
>> + synchronize_rcu();
>> +
>> + xe_pmu_unregister_cpuhp_state(pmu);
>> +
>> + perf_pmu_unregister(&pmu->base);
>> + pmu->base.event_init = NULL;
>> + kfree(pmu->base.attr_groups);
>> + kfree(pmu->name);
>> + free_event_attributes(pmu);
>> +}
>> +
>> +/**
>> + * xe_pmu_register() - Define basic PMU properties for Xe and add event callbacks.
>> + *
>> + */
>> +void xe_pmu_register(struct xe_pmu *pmu)
>> +{
>> + struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>> + const struct attribute_group *attr_groups[] = {
>> + &pmu->events_attr_group,
>> + &xe_pmu_cpumask_attr_group,
>> + NULL
>> + };
>> +
>> + int ret = -ENOMEM;
>> +
>> + spin_lock_init(&pmu->lock);
>> + pmu->cpuhp.cpu = -1;
>> +
>> + pmu->name = kasprintf(GFP_KERNEL,
>> + "xe_%s",
>> + dev_name(xe->drm.dev));
>> + if (pmu->name)
>> + /* tools/perf reserves colons as special. */
>> + strreplace((char *)pmu->name, ':', '_');
>> +
>> + if (!pmu->name)
>> + goto err;
>> +
>> + pmu->events_attr_group.name = "events";
>> + pmu->events_attr_group.attrs = create_event_attributes(pmu);
>> + if (!pmu->events_attr_group.attrs)
>> + goto err_name;
>> +
>> + pmu->base.attr_groups = kmemdup(attr_groups, sizeof(attr_groups),
>> + GFP_KERNEL);
>> + if (!pmu->base.attr_groups)
>> + goto err_attr;
>> +
>> + pmu->base.module = THIS_MODULE;
>> + pmu->base.task_ctx_nr = perf_invalid_context;
>> + pmu->base.event_init = xe_pmu_event_init;
>> + pmu->base.add = xe_pmu_event_add;
>> + pmu->base.del = xe_pmu_event_del;
>> + pmu->base.start = xe_pmu_event_start;
>> + pmu->base.stop = xe_pmu_event_stop;
>> + pmu->base.read = xe_pmu_event_read;
>> + pmu->base.event_idx = xe_pmu_event_event_idx;
>> +
>> + ret = perf_pmu_register(&pmu->base, pmu->name, -1);
>> + if (ret)
>> + goto err_groups;
>> +
>> + ret = xe_pmu_register_cpuhp_state(pmu);
>> + if (ret)
>> + goto err_unreg;
>> +
>> + ret = devm_add_action_or_reset(xe->drm.dev, xe_pmu_unregister, pmu);
>> + if (ret)
>> + goto err_cpuhp;
>> +
>> + return;
>> +
>> +err_cpuhp:
>> + xe_pmu_unregister_cpuhp_state(pmu);
>> +err_unreg:
>> + perf_pmu_unregister(&pmu->base);
>> +err_groups:
>> + kfree(pmu->base.attr_groups);
>> +err_attr:
>> + pmu->base.event_init = NULL;
>> + free_event_attributes(pmu);
>> +err_name:
>> + kfree(pmu->name);
>> +err:
>> + drm_notice(&xe->drm, "Failed to register PMU!\n");
>> +}
>> diff --git a/drivers/gpu/drm/xe/xe_pmu.h b/drivers/gpu/drm/xe/xe_pmu.h
>> new file mode 100644
>> index 000000000000..d07e5dfdfec0
>> --- /dev/null
>> +++ b/drivers/gpu/drm/xe/xe_pmu.h
>> @@ -0,0 +1,26 @@
>> +/* SPDX-License-Identifier: MIT */
>> +/*
>> + * Copyright © 2024 Intel Corporation
>> + */
>> +
>> +#ifndef _XE_PMU_H_
>> +#define _XE_PMU_H_
>> +
>> +#include "xe_pmu_types.h"
>> +
>> +struct xe_gt;
>> +
>> +#if IS_ENABLED(CONFIG_PERF_EVENTS)
>> +int xe_pmu_init(void);
>> +void xe_pmu_exit(void);
>> +void xe_pmu_register(struct xe_pmu *pmu);
>> +void xe_pmu_unregister(void *arg);
>> +#else
>> +static inline int xe_pmu_init(void) { return 0; }
>> +static inline void xe_pmu_exit(void) {}
>> +static inline void xe_pmu_register(struct xe_pmu *pmu) {}
>> +static inline void xe_pmu_unregister(void *arg) {}
>> +#endif
>> +
>> +#endif
>> +
>> diff --git a/drivers/gpu/drm/xe/xe_pmu_types.h b/drivers/gpu/drm/xe/xe_pmu_types.h
>> new file mode 100644
>> index 000000000000..ca0e7cbe2081
>> --- /dev/null
>> +++ b/drivers/gpu/drm/xe/xe_pmu_types.h
>> @@ -0,0 +1,63 @@
>> +/* SPDX-License-Identifier: MIT */
>> +/*
>> + * Copyright © 2024 Intel Corporation
>> + */
>> +
>> +#ifndef _XE_PMU_TYPES_H_
>> +#define _XE_PMU_TYPES_H_
>> +
>> +#include <linux/perf_event.h>
>> +#include <linux/spinlock_types.h>
>> +#include <uapi/drm/xe_drm.h>
>> +
>> +enum {
>> + __XE_NUM_PMU_SAMPLERS
>> +};
>> +
>> +#define XE_PMU_MAX_GT 2
>> +
>> +struct xe_pmu {
>> + /**
>> + * @cpuhp: Struct used for CPU hotplug handling.
>> + */
>> + struct {
>> + struct hlist_node node;
>> + unsigned int cpu;
>> + } cpuhp;
>> + /**
>> + * @base: PMU base.
>> + */
>> + struct pmu base;
>> + /**
>> + * @closed: xe is unregistering.
>> + */
>> + bool closed;
>> + /**
>> + * @name: Name as registered with perf core.
>> + */
>> + const char *name;
>> + /**
>> + * @lock: Lock protecting enable mask and ref count handling.
>> + */
>> + spinlock_t lock;
>> + /**
>> + * @sample: Current and previous (raw) counters.
>> + *
>> + * These counters are updated when the device is awake.
>> + */
>> + u64 sample[XE_PMU_MAX_GT][__XE_NUM_PMU_SAMPLERS];
>> + /**
>> + * @events_attr_group: Device events attribute group.
>> + */
>> + struct attribute_group events_attr_group;
>> + /**
>> + * @xe_attr: Memory block holding device attributes.
>> + */
>> + void *xe_attr;
>> + /**
>> + * @pmu_attr: Memory block holding device attributes.
>> + */
>> + void *pmu_attr;
>> +};
>> +
>> +#endif
>> diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
>> index b6fbe4988f2e..2c5f258eee3a 100644
>> --- a/include/uapi/drm/xe_drm.h
>> +++ b/include/uapi/drm/xe_drm.h
>> @@ -1389,6 +1389,14 @@ struct drm_xe_wait_user_fence {
>> __u64 reserved[2];
>> };
>>
>> +/*
>> + * Top bits of every counter are GT id.
>> + */
>> +#define __XE_PMU_GT_SHIFT (56)
>> +
>> +#define ___XE_PMU_OTHER(gt, x) \
>> + (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
>> +
>> /**
>> * enum drm_xe_observation_type - Observation stream types
>> */
>> --
>> 2.38.1
>>
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 1/3] drm/xe/pmu: Enable PMU interface
2024-09-27 0:23 ` [PATCH 1/3] drm/xe/pmu: Enable PMU interface Vinay Belgaumkar
2024-09-27 18:26 ` Rodrigo Vivi
@ 2024-10-01 21:32 ` Lucas De Marchi
2024-10-01 22:08 ` Belgaumkar, Vinay
1 sibling, 1 reply; 19+ messages in thread
From: Lucas De Marchi @ 2024-10-01 21:32 UTC (permalink / raw)
To: Vinay Belgaumkar
Cc: intel-xe, Aravind Iddamsetty, Bommu Krishnaiah, Riana Tauro,
Rodrigo Vivi
On Thu, Sep 26, 2024 at 05:23:42PM GMT, Vinay Belgaumkar wrote:
>diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
>index b6fbe4988f2e..2c5f258eee3a 100644
>--- a/include/uapi/drm/xe_drm.h
>+++ b/include/uapi/drm/xe_drm.h
>@@ -1389,6 +1389,14 @@ struct drm_xe_wait_user_fence {
> __u64 reserved[2];
> };
>
>+/*
>+ * Top bits of every counter are GT id.
>+ */
>+#define __XE_PMU_GT_SHIFT (56)
>+
>+#define ___XE_PMU_OTHER(gt, x) \
>+ (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
>+
why are you touching this?
I thought we agreed it's not needed at all on previous version of this
patch: https://lore.kernel.org/intel-xe/602bffcd-d66f-4b49-b3b4-abb934b00f3a@intel.com/
what changed?
Lucas De Marchi
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 1/3] drm/xe/pmu: Enable PMU interface
2024-10-01 21:32 ` Lucas De Marchi
@ 2024-10-01 22:08 ` Belgaumkar, Vinay
2024-10-02 14:51 ` Lucas De Marchi
0 siblings, 1 reply; 19+ messages in thread
From: Belgaumkar, Vinay @ 2024-10-01 22:08 UTC (permalink / raw)
To: Lucas De Marchi
Cc: intel-xe, Aravind Iddamsetty, Bommu Krishnaiah, Riana Tauro,
Rodrigo Vivi
On 10/1/2024 2:32 PM, Lucas De Marchi wrote:
> On Thu, Sep 26, 2024 at 05:23:42PM GMT, Vinay Belgaumkar wrote:
>> diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
>> index b6fbe4988f2e..2c5f258eee3a 100644
>> --- a/include/uapi/drm/xe_drm.h
>> +++ b/include/uapi/drm/xe_drm.h
>> @@ -1389,6 +1389,14 @@ struct drm_xe_wait_user_fence {
>> __u64 reserved[2];
>> };
>>
>> +/*
>> + * Top bits of every counter are GT id.
>> + */
>> +#define __XE_PMU_GT_SHIFT (56)
>> +
>> +#define ___XE_PMU_OTHER(gt, x) \
>> + (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
>> +
>
> why are you touching this?
>
> I thought we agreed it's not needed at all on previous version of this
> patch:
> https://lore.kernel.org/intel-xe/602bffcd-d66f-4b49-b3b4-abb934b00f3a@intel.com/
>
> what changed?
IGT tests like perf_pmu use this interface to open specific PMU files,
for example-
fd[pmus] = perf_i915_open_group(gem_fd,
__I915_PMU_RC6_RESIDENCY(gt_),
fd[0]);
If we define these inside non-header files, these kinds of tests and any
other user programs that may be doing this, will need to look at kernel
code to determine the PMU config? I assumed we needed these bare minimum
header definitions.
Thanks,
Vinay.
>
>
> Lucas De Marchi
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 1/3] drm/xe/pmu: Enable PMU interface
2024-10-01 22:08 ` Belgaumkar, Vinay
@ 2024-10-02 14:51 ` Lucas De Marchi
0 siblings, 0 replies; 19+ messages in thread
From: Lucas De Marchi @ 2024-10-02 14:51 UTC (permalink / raw)
To: Belgaumkar, Vinay
Cc: intel-xe, Aravind Iddamsetty, Bommu Krishnaiah, Riana Tauro,
Rodrigo Vivi
On Tue, Oct 01, 2024 at 03:08:03PM GMT, Belgaumkar, Vinay wrote:
>
>On 10/1/2024 2:32 PM, Lucas De Marchi wrote:
>>On Thu, Sep 26, 2024 at 05:23:42PM GMT, Vinay Belgaumkar wrote:
>>>diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
>>>index b6fbe4988f2e..2c5f258eee3a 100644
>>>--- a/include/uapi/drm/xe_drm.h
>>>+++ b/include/uapi/drm/xe_drm.h
>>>@@ -1389,6 +1389,14 @@ struct drm_xe_wait_user_fence {
>>> __u64 reserved[2];
>>>};
>>>
>>>+/*
>>>+ * Top bits of every counter are GT id.
>>>+ */
>>>+#define __XE_PMU_GT_SHIFT (56)
>>>+
>>>+#define ___XE_PMU_OTHER(gt, x) \
>>>+ (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
>>>+
>>
>>why are you touching this?
>>
>>I thought we agreed it's not needed at all on previous version of this
>>patch: https://lore.kernel.org/intel-xe/602bffcd-d66f-4b49-b3b4-abb934b00f3a@intel.com/
>>
>>what changed?
>
>IGT tests like perf_pmu use this interface to open specific PMU files,
>for example-
>
> fd[pmus] = perf_i915_open_group(gem_fd,
>__I915_PMU_RC6_RESIDENCY(gt_),
> fd[0]);
>If we define these inside non-header files, these kinds of tests and
>any other user programs that may be doing this, will need to look at
>kernel code to determine the PMU config? I assumed we needed these
>bare minimum header definitions.
no, they should look in sysfs and that should be sufficient. This is how
the `perf` tool works and you shouldn't need to change userspace tools
to support additional events
Lucas De Marchi
>
>Thanks,
>
>Vinay.
>
>>
>>
>>Lucas De Marchi
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 1/3] drm/xe/pmu: Enable PMU interface
2024-09-27 18:26 ` Rodrigo Vivi
2024-09-27 19:45 ` Belgaumkar, Vinay
@ 2024-10-24 22:58 ` Belgaumkar, Vinay
1 sibling, 0 replies; 19+ messages in thread
From: Belgaumkar, Vinay @ 2024-10-24 22:58 UTC (permalink / raw)
To: Rodrigo Vivi; +Cc: intel-xe, Aravind Iddamsetty, Bommu Krishnaiah, Riana Tauro
On 9/27/2024 11:26 AM, Rodrigo Vivi wrote:
> On Thu, Sep 26, 2024 at 05:23:42PM -0700, Vinay Belgaumkar wrote:
>> From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>>
>> Basic PMU enabling patch. Setup the basic framework
>> for adding events/timers. This patch was previously
>> reviewed here -
>> https://patchwork.freedesktop.org/series/119504/
>>
>> The pmu base implementation is still from the
>> i915 driver.
>>
>> v2: Review comments(Rodrigo) and do not init pmu for VFs
>> as they don't have access to freq and c6 residency anyways.
> sorry for the slow review and for not going through it all
> in a single pass.
>
>> v3: Fix kunit issue, move xe_pmu entry in Makefile (Jani)
>>
>> Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>> Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
>> Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>> Signed-off-by: Riana Tauro <riana.tauro@intel.com>
>> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
>> Signed-off-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
>> ---
>> drivers/gpu/drm/xe/Makefile | 2 +
>> drivers/gpu/drm/xe/xe_device.c | 6 +
>> drivers/gpu/drm/xe/xe_device_types.h | 4 +
>> drivers/gpu/drm/xe/xe_module.c | 5 +
>> drivers/gpu/drm/xe/xe_pmu.c | 570 +++++++++++++++++++++++++++
>> drivers/gpu/drm/xe/xe_pmu.h | 26 ++
>> drivers/gpu/drm/xe/xe_pmu_types.h | 63 +++
>> include/uapi/drm/xe_drm.h | 8 +
>> 8 files changed, 684 insertions(+)
>> create mode 100644 drivers/gpu/drm/xe/xe_pmu.c
>> create mode 100644 drivers/gpu/drm/xe/xe_pmu.h
>> create mode 100644 drivers/gpu/drm/xe/xe_pmu_types.h
>>
>> diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
>> index ae245fbd91ee..3a3eabc0e0a3 100644
>> --- a/drivers/gpu/drm/xe/Makefile
>> +++ b/drivers/gpu/drm/xe/Makefile
>> @@ -293,6 +293,8 @@ ifeq ($(CONFIG_DEBUG_FS),y)
>> i915-display/intel_pipe_crc.o
>> endif
>>
>> +xe-$(CONFIG_PERF_EVENTS) += xe_pmu.o
>> +
>> obj-$(CONFIG_DRM_XE) += xe.o
>> obj-$(CONFIG_DRM_XE_KUNIT_TEST) += tests/
>>
>> diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
>> index cb5a9fd820cf..027a796d7431 100644
>> --- a/drivers/gpu/drm/xe/xe_device.c
>> +++ b/drivers/gpu/drm/xe/xe_device.c
>> @@ -762,6 +762,9 @@ int xe_device_probe(struct xe_device *xe)
>> for_each_gt(gt, xe, id)
>> xe_gt_sanitize_freq(gt);
>>
>> + if (!IS_SRIOV_VF(xe))
>> + xe_pmu_register(&xe->pmu);
>> +
>> return devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
>>
>> err_fini_display:
>> @@ -806,6 +809,9 @@ void xe_device_remove(struct xe_device *xe)
>>
>> xe_heci_gsc_fini(xe);
>>
>> + if (!IS_SRIOV_VF(xe))
>> + xe_pmu_unregister(&xe->pmu);
>> +
>> for_each_gt(gt, xe, id)
>> xe_gt_remove(gt);
>> }
>> diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
>> index 5ad96d283a71..51485d2eadcf 100644
>> --- a/drivers/gpu/drm/xe/xe_device_types.h
>> +++ b/drivers/gpu/drm/xe/xe_device_types.h
>> @@ -18,6 +18,7 @@
>> #include "xe_memirq_types.h"
>> #include "xe_oa.h"
>> #include "xe_platform_types.h"
>> +#include "xe_pmu.h"
>> #include "xe_pt_types.h"
>> #include "xe_sriov_types.h"
>> #include "xe_step_types.h"
>> @@ -513,6 +514,9 @@ struct xe_device {
>> int mode;
>> } wedged;
>>
>> + /** @pmu: performance monitoring unit */
>> + struct xe_pmu pmu;
>> +
>> #ifdef TEST_VM_OPS_ERROR
>> /**
>> * @vm_inject_error_position: inject errors at different places in VM
>> diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
>> index 77ce9f9ca7a5..1bf2bf8447c0 100644
>> --- a/drivers/gpu/drm/xe/xe_module.c
>> +++ b/drivers/gpu/drm/xe/xe_module.c
>> @@ -14,6 +14,7 @@
>> #include "xe_hw_fence.h"
>> #include "xe_pci.h"
>> #include "xe_pm.h"
>> +#include "xe_pmu.h"
>> #include "xe_observation.h"
>> #include "xe_sched_job.h"
>>
>> @@ -96,6 +97,10 @@ static const struct init_funcs init_funcs[] = {
>> .init = xe_sched_job_module_init,
>> .exit = xe_sched_job_module_exit,
>> },
>> + {
>> + .init = xe_pmu_init,
>> + .exit = xe_pmu_exit,
>> + },
>> {
>> .init = xe_register_pci_driver,
>> .exit = xe_unregister_pci_driver,
>> diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
>> new file mode 100644
>> index 000000000000..bdaea9ca1065
>> --- /dev/null
>> +++ b/drivers/gpu/drm/xe/xe_pmu.c
>> @@ -0,0 +1,570 @@
>> +// SPDX-License-Identifier: MIT
>> +/*
>> + * Copyright © 2024 Intel Corporation
>> + */
>> +
>> +#include <drm/drm_drv.h>
>> +#include <drm/drm_managed.h>
>> +#include <drm/xe_drm.h>
>> +
>> +#include "regs/xe_gt_regs.h"
>> +#include "xe_device.h"
>> +#include "xe_force_wake.h"
>> +#include "xe_gt_clock.h"
>> +#include "xe_mmio.h"
>> +#include "xe_macros.h"
>> +#include "xe_pm.h"
>> +
>> +static cpumask_t xe_pmu_cpumask;
>> +static unsigned int xe_pmu_target_cpu = -1;
> Please no global variables. Think the case where we have
> more then 1 device in the same cpu.
As per offline discussion, will retain this with a meaningful comment
about it.
>
>> +
>> +/**
>> + * DOC: Xe PMU (Performance Monitoring Unit)
>> + *
>> + * Expose events/counters like C6 residency and GT frequency to user land.
>> + * Perf tool can be used to list and record these counters from the command
>> + * line.
>> + *
>> + * Example commands to list/record supported perf events-
>> + *
>> + * >> perf list
> perhas '$' instead of '>>' ?
ok.
>
>> + * >> ls /sys/bus/event_source/devices/xe_0000_00_02.0/events/
> perhaps ls -ld /sys/bus/event_source/devices/xe_*
>
> to first know which devices are available?
ok.
>
>> + *
>> + * To record events-
>> + * >> perf stat -e <event_name> -I <interval>
> Could you please document here on how to ensure that those small incremental
> values should be read?
> How can I use this perf command and ensure that we get exactly the
> same value as our other sysfs entries?
> I'd like to do the math back to cross-check at least, but also
> this information should be useful to ensure that users of this
> are really well informed on the flow.
Sure. Will add to that to the doc in the following patches where I add
the RC6 and freq events.
>
>> + *
>> + */
>> +
>> +static unsigned int config_gt_id(const u64 config)
>> +{
>> + return config >> __XE_PMU_GT_SHIFT;
>> +}
>> +
>> +static u64 config_counter(const u64 config)
>> +{
>> + return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
>> +}
>> +
>> +static void xe_pmu_event_destroy(struct perf_event *event)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> +
>> + drm_WARN_ON(&xe->drm, event->parent);
>> +
>> + drm_dev_put(&xe->drm);
>> +}
>> +
>> +static int
>> +config_status(struct xe_device *xe, u64 config)
>> +{
>> + unsigned int gt_id = config_gt_id(config);
>> +
>> + if (gt_id >= XE_PMU_MAX_GT)
> why do we have new GT defines instead of using the regular GT
> numbering and entries?
I cannot include xe_device.h in xe_pmu_types.h as it leads to circular
definition. I am suspecting that's why this was redefined in the
original patch.
Thanks,
Vinay.
>
>> + return -ENOENT;
>> +
>> + switch (config_counter(config)) {
>> + default:
>> + return -ENOENT;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static int xe_pmu_event_init(struct perf_event *event)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> + struct xe_pmu *pmu = &xe->pmu;
>> + int ret;
>> +
>> + if (pmu->closed)
>> + return -ENODEV;
>> +
>> + if (event->attr.type != event->pmu->type)
>> + return -ENOENT;
>> +
>> + /* unsupported modes and filters */
>> + if (event->attr.sample_period) /* no sampling */
>> + return -EINVAL;
>> +
>> + if (has_branch_stack(event))
>> + return -EOPNOTSUPP;
>> +
>> + if (event->cpu < 0)
>> + return -EINVAL;
>> +
>> + /* only allow running on one cpu at a time */
>> + if (!cpumask_test_cpu(event->cpu, &xe_pmu_cpumask))
>> + return -EINVAL;
>> +
>> + ret = config_status(xe, event->attr.config);
>> + if (ret)
>> + return ret;
>> +
>> + if (!event->parent) {
>> + drm_dev_get(&xe->drm);
>> + event->destroy = xe_pmu_event_destroy;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static u64 __xe_pmu_event_read(struct perf_event *event)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> + const unsigned int gt_id = config_gt_id(event->attr.config);
>> + const u64 config = event->attr.config;
>> + struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
>> + u64 val = 0;
>> +
>> + switch (config_counter(config)) {
>> + default:
>> + drm_warn(>->tile->xe->drm, "unknown pmu event\n");
>> + }
>> +
>> + return val;
>> +}
>> +
>> +static void xe_pmu_event_read(struct perf_event *event)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> + struct hw_perf_event *hwc = &event->hw;
>> + struct xe_pmu *pmu = &xe->pmu;
>> + u64 prev, new;
>> +
>> + if (pmu->closed) {
>> + event->hw.state = PERF_HES_STOPPED;
>> + return;
>> + }
>> +again:
>> + prev = local64_read(&hwc->prev_count);
>> + new = __xe_pmu_event_read(event);
>> +
>> + if (local64_cmpxchg(&hwc->prev_count, prev, new) != prev)
>> + goto again;
>> +
>> + local64_add(new - prev, &event->count);
>> +}
>> +
>> +static void xe_pmu_enable(struct perf_event *event)
>> +{
>> + /*
>> + * Store the current counter value so we can report the correct delta
>> + * for all listeners. Even when the event was already enabled and has
>> + * an existing non-zero value.
>> + */
>> + local64_set(&event->hw.prev_count, __xe_pmu_event_read(event));
>> +}
>> +
>> +static void xe_pmu_event_start(struct perf_event *event, int flags)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> + struct xe_pmu *pmu = &xe->pmu;
>> +
>> + if (pmu->closed)
>> + return;
>> +
>> + xe_pmu_enable(event);
>> + event->hw.state = 0;
>> +}
>> +
>> +static void xe_pmu_event_stop(struct perf_event *event, int flags)
>> +{
>> + if (flags & PERF_EF_UPDATE)
>> + xe_pmu_event_read(event);
>> +
>> + event->hw.state = PERF_HES_STOPPED;
>> +}
>> +
>> +static int xe_pmu_event_add(struct perf_event *event, int flags)
>> +{
>> + struct xe_device *xe =
>> + container_of(event->pmu, typeof(*xe), pmu.base);
>> + struct xe_pmu *pmu = &xe->pmu;
>> +
>> + if (pmu->closed)
>> + return -ENODEV;
>> +
>> + if (flags & PERF_EF_START)
>> + xe_pmu_event_start(event, flags);
>> +
>> + return 0;
>> +}
>> +
>> +static void xe_pmu_event_del(struct perf_event *event, int flags)
>> +{
>> + xe_pmu_event_stop(event, PERF_EF_UPDATE);
>> +}
>> +
>> +static int xe_pmu_event_event_idx(struct perf_event *event)
>> +{
>> + return 0;
>> +}
>> +
>> +struct xe_ext_attribute {
>> + struct device_attribute attr;
>> + unsigned long val;
>> +};
>> +
>> +static ssize_t xe_pmu_event_show(struct device *dev,
>> + struct device_attribute *attr, char *buf)
>> +{
>> + struct xe_ext_attribute *eattr;
>> +
>> + eattr = container_of(attr, struct xe_ext_attribute, attr);
>> + return sprintf(buf, "config=0x%lx\n", eattr->val);
>> +}
>> +
>> +static ssize_t cpumask_show(struct device *dev,
>> + struct device_attribute *attr, char *buf)
>> +{
>> + return cpumap_print_to_pagebuf(true, buf, &xe_pmu_cpumask);
>> +}
>> +
>> +static DEVICE_ATTR_RO(cpumask);
>> +
>> +static struct attribute *xe_cpumask_attrs[] = {
>> + &dev_attr_cpumask.attr,
>> + NULL,
>> +};
>> +
>> +static const struct attribute_group xe_pmu_cpumask_attr_group = {
>> + .attrs = xe_cpumask_attrs,
>> +};
>> +
>> +#define __event(__counter, __name, __unit) \
>> +{ \
>> + .counter = (__counter), \
>> + .name = (__name), \
>> + .unit = (__unit), \
>> +}
>> +
>> +static struct xe_ext_attribute *
>> +add_xe_attr(struct xe_ext_attribute *attr, const char *name, u64 config)
>> +{
>> + sysfs_attr_init(&attr->attr.attr);
>> + attr->attr.attr.name = name;
>> + attr->attr.attr.mode = 0444;
>> + attr->attr.show = xe_pmu_event_show;
>> + attr->val = config;
>> +
>> + return ++attr;
>> +}
>> +
>> +static struct perf_pmu_events_attr *
>> +add_pmu_attr(struct perf_pmu_events_attr *attr, const char *name,
>> + const char *str)
>> +{
>> + sysfs_attr_init(&attr->attr.attr);
>> + attr->attr.attr.name = name;
>> + attr->attr.attr.mode = 0444;
>> + attr->attr.show = perf_event_sysfs_show;
>> + attr->event_str = str;
>> +
>> + return ++attr;
>> +}
>> +
>> +static struct attribute **
>> +create_event_attributes(struct xe_pmu *pmu)
>> +{
>> + struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>> + static const struct {
>> + unsigned int counter;
>> + const char *name;
>> + const char *unit;
>> + } events[] = {
>> + };
>> +
>> + struct perf_pmu_events_attr *pmu_attr = NULL, *pmu_iter;
>> + struct xe_ext_attribute *xe_attr = NULL, *xe_iter;
>> + struct attribute **attr = NULL, **attr_iter;
>> + unsigned int count = 0;
>> + unsigned int i, j;
>> + struct xe_gt *gt;
>> +
>> + /* Count how many counters we will be exposing. */
>> + for_each_gt(gt, xe, j) {
>> + for (i = 0; i < ARRAY_SIZE(events); i++) {
>> + u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>> +
>> + if (!config_status(xe, config))
>> + count++;
>> + }
>> + }
>> +
>> + /* Allocate attribute objects and table. */
>> + xe_attr = kcalloc(count, sizeof(*xe_attr), GFP_KERNEL);
>> + if (!xe_attr)
>> + goto err_alloc;
>> +
>> + pmu_attr = kcalloc(count, sizeof(*pmu_attr), GFP_KERNEL);
>> + if (!pmu_attr)
>> + goto err_alloc;
>> +
>> + /* Max one pointer of each attribute type plus a termination entry. */
>> + attr = kcalloc(count * 2 + 1, sizeof(*attr), GFP_KERNEL);
>> + if (!attr)
>> + goto err_alloc;
>> +
>> + xe_iter = xe_attr;
>> + pmu_iter = pmu_attr;
>> + attr_iter = attr;
>> +
>> + for_each_gt(gt, xe, j) {
>> + for (i = 0; i < ARRAY_SIZE(events); i++) {
>> + u64 config = ___XE_PMU_OTHER(j, events[i].counter);
>> + char *str;
>> +
>> + if (config_status(xe, config))
>> + continue;
>> +
>> + str = kasprintf(GFP_KERNEL, "%s-gt%u",
>> + events[i].name, j);
>> + if (!str)
>> + goto err;
>> +
>> + *attr_iter++ = &xe_iter->attr.attr;
>> + xe_iter = add_xe_attr(xe_iter, str, config);
>> +
>> + if (events[i].unit) {
>> + str = kasprintf(GFP_KERNEL, "%s-gt%u.unit",
>> + events[i].name, j);
>> + if (!str)
>> + goto err;
>> +
>> + *attr_iter++ = &pmu_iter->attr.attr;
>> + pmu_iter = add_pmu_attr(pmu_iter, str,
>> + events[i].unit);
>> + }
>> + }
>> + }
>> +
>> + pmu->xe_attr = xe_attr;
>> + pmu->pmu_attr = pmu_attr;
>> +
>> + return attr;
>> +
>> +err:
>> + for (attr_iter = attr; *attr_iter; attr_iter++)
>> + kfree((*attr_iter)->name);
>> +
>> +err_alloc:
>> + kfree(attr);
>> + kfree(xe_attr);
>> + kfree(pmu_attr);
>> +
>> + return NULL;
>> +}
>> +
>> +static void free_event_attributes(struct xe_pmu *pmu)
>> +{
>> + struct attribute **attr_iter = pmu->events_attr_group.attrs;
>> +
>> + for (; *attr_iter; attr_iter++)
>> + kfree((*attr_iter)->name);
>> +
>> + kfree(pmu->events_attr_group.attrs);
>> + kfree(pmu->xe_attr);
>> + kfree(pmu->pmu_attr);
>> +
>> + pmu->events_attr_group.attrs = NULL;
>> + pmu->xe_attr = NULL;
>> + pmu->pmu_attr = NULL;
>> +}
>> +
>> +static int xe_pmu_cpu_online(unsigned int cpu, struct hlist_node *node)
>> +{
>> + struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
>> +
>> + XE_WARN_ON(!pmu->base.event_init);
>> +
>> + /* Select the first online CPU as a designated reader. */
>> + if (cpumask_empty(&xe_pmu_cpumask))
>> + cpumask_set_cpu(cpu, &xe_pmu_cpumask);
>> +
>> + return 0;
>> +}
>> +
>> +static int xe_pmu_cpu_offline(unsigned int cpu, struct hlist_node *node)
>> +{
>> + struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
>> + unsigned int target = xe_pmu_target_cpu;
>> +
>> + /*
>> + * Unregistering an instance generates a CPU offline event which we must
>> + * ignore to avoid incorrectly modifying the shared xe_pmu_cpumask.
>> + */
>> + if (pmu->closed)
>> + return 0;
>> +
>> + if (cpumask_test_and_clear_cpu(cpu, &xe_pmu_cpumask)) {
>> + target = cpumask_any_but(topology_sibling_cpumask(cpu), cpu);
>> +
>> + /* Migrate events if there is a valid target */
>> + if (target < nr_cpu_ids) {
>> + cpumask_set_cpu(target, &xe_pmu_cpumask);
>> + xe_pmu_target_cpu = target;
>> + }
>> + }
>> +
>> + if (target < nr_cpu_ids && target != pmu->cpuhp.cpu) {
>> + perf_pmu_migrate_context(&pmu->base, cpu, target);
>> + pmu->cpuhp.cpu = target;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static enum cpuhp_state cpuhp_slot = CPUHP_INVALID;
>> +
>> +/**
>> + * xe_pmu_init() - Setup CPU hotplug state/callbacks for Xe PMU
>> + *
>> + * Returns: 0 if successful, else error code
>> + */
>> +int xe_pmu_init(void)
>> +{
>> + int ret;
>> +
>> + ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
>> + "perf/x86/intel/xe:online",
>> + xe_pmu_cpu_online,
>> + xe_pmu_cpu_offline);
>> + if (ret < 0)
>> + pr_notice("Failed to setup cpuhp state for xe PMU! (%d)\n",
>> + ret);
>> + else
>> + cpuhp_slot = ret;
>> +
>> + return 0;
>> +}
>> +
>> +/**
>> + * xe_pmu_exit() - Remove CPU hotplug state/callbacks for Xe PMU
>> + */
>> +void xe_pmu_exit(void)
>> +{
>> + if (cpuhp_slot != CPUHP_INVALID)
>> + cpuhp_remove_multi_state(cpuhp_slot);
>> +}
>> +
>> +static int xe_pmu_register_cpuhp_state(struct xe_pmu *pmu)
>> +{
>> + if (cpuhp_slot == CPUHP_INVALID)
>> + return -EINVAL;
>> +
>> + return cpuhp_state_add_instance(cpuhp_slot, &pmu->cpuhp.node);
>> +}
>> +
>> +static void xe_pmu_unregister_cpuhp_state(struct xe_pmu *pmu)
>> +{
>> + cpuhp_state_remove_instance(cpuhp_slot, &pmu->cpuhp.node);
>> +}
>> +
>> +/**
>> + * xe_pmu_unregister() - Remove/cleanup PMU registration
>> + */
>> +void xe_pmu_unregister(void *arg)
>> +{
>> + struct xe_pmu *pmu = arg;
>> +
>> + if (!pmu->base.event_init)
>> + return;
>> +
>> + /*
>> + * "Disconnect" the PMU callbacks - since all are atomic synchronize_rcu
>> + * ensures all currently executing ones will have exited before we
>> + * proceed with unregistration.
>> + */
>> + pmu->closed = true;
>> + synchronize_rcu();
>> +
>> + xe_pmu_unregister_cpuhp_state(pmu);
>> +
>> + perf_pmu_unregister(&pmu->base);
>> + pmu->base.event_init = NULL;
>> + kfree(pmu->base.attr_groups);
>> + kfree(pmu->name);
>> + free_event_attributes(pmu);
>> +}
>> +
>> +/**
>> + * xe_pmu_register() - Define basic PMU properties for Xe and add event callbacks.
>> + *
>> + */
>> +void xe_pmu_register(struct xe_pmu *pmu)
>> +{
>> + struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
>> + const struct attribute_group *attr_groups[] = {
>> + &pmu->events_attr_group,
>> + &xe_pmu_cpumask_attr_group,
>> + NULL
>> + };
>> +
>> + int ret = -ENOMEM;
>> +
>> + spin_lock_init(&pmu->lock);
>> + pmu->cpuhp.cpu = -1;
>> +
>> + pmu->name = kasprintf(GFP_KERNEL,
>> + "xe_%s",
>> + dev_name(xe->drm.dev));
>> + if (pmu->name)
>> + /* tools/perf reserves colons as special. */
>> + strreplace((char *)pmu->name, ':', '_');
>> +
>> + if (!pmu->name)
>> + goto err;
>> +
>> + pmu->events_attr_group.name = "events";
>> + pmu->events_attr_group.attrs = create_event_attributes(pmu);
>> + if (!pmu->events_attr_group.attrs)
>> + goto err_name;
>> +
>> + pmu->base.attr_groups = kmemdup(attr_groups, sizeof(attr_groups),
>> + GFP_KERNEL);
>> + if (!pmu->base.attr_groups)
>> + goto err_attr;
>> +
>> + pmu->base.module = THIS_MODULE;
>> + pmu->base.task_ctx_nr = perf_invalid_context;
>> + pmu->base.event_init = xe_pmu_event_init;
>> + pmu->base.add = xe_pmu_event_add;
>> + pmu->base.del = xe_pmu_event_del;
>> + pmu->base.start = xe_pmu_event_start;
>> + pmu->base.stop = xe_pmu_event_stop;
>> + pmu->base.read = xe_pmu_event_read;
>> + pmu->base.event_idx = xe_pmu_event_event_idx;
>> +
>> + ret = perf_pmu_register(&pmu->base, pmu->name, -1);
>> + if (ret)
>> + goto err_groups;
>> +
>> + ret = xe_pmu_register_cpuhp_state(pmu);
>> + if (ret)
>> + goto err_unreg;
>> +
>> + ret = devm_add_action_or_reset(xe->drm.dev, xe_pmu_unregister, pmu);
>> + if (ret)
>> + goto err_cpuhp;
>> +
>> + return;
>> +
>> +err_cpuhp:
>> + xe_pmu_unregister_cpuhp_state(pmu);
>> +err_unreg:
>> + perf_pmu_unregister(&pmu->base);
>> +err_groups:
>> + kfree(pmu->base.attr_groups);
>> +err_attr:
>> + pmu->base.event_init = NULL;
>> + free_event_attributes(pmu);
>> +err_name:
>> + kfree(pmu->name);
>> +err:
>> + drm_notice(&xe->drm, "Failed to register PMU!\n");
>> +}
>> diff --git a/drivers/gpu/drm/xe/xe_pmu.h b/drivers/gpu/drm/xe/xe_pmu.h
>> new file mode 100644
>> index 000000000000..d07e5dfdfec0
>> --- /dev/null
>> +++ b/drivers/gpu/drm/xe/xe_pmu.h
>> @@ -0,0 +1,26 @@
>> +/* SPDX-License-Identifier: MIT */
>> +/*
>> + * Copyright © 2024 Intel Corporation
>> + */
>> +
>> +#ifndef _XE_PMU_H_
>> +#define _XE_PMU_H_
>> +
>> +#include "xe_pmu_types.h"
>> +
>> +struct xe_gt;
>> +
>> +#if IS_ENABLED(CONFIG_PERF_EVENTS)
>> +int xe_pmu_init(void);
>> +void xe_pmu_exit(void);
>> +void xe_pmu_register(struct xe_pmu *pmu);
>> +void xe_pmu_unregister(void *arg);
>> +#else
>> +static inline int xe_pmu_init(void) { return 0; }
>> +static inline void xe_pmu_exit(void) {}
>> +static inline void xe_pmu_register(struct xe_pmu *pmu) {}
>> +static inline void xe_pmu_unregister(void *arg) {}
>> +#endif
>> +
>> +#endif
>> +
>> diff --git a/drivers/gpu/drm/xe/xe_pmu_types.h b/drivers/gpu/drm/xe/xe_pmu_types.h
>> new file mode 100644
>> index 000000000000..ca0e7cbe2081
>> --- /dev/null
>> +++ b/drivers/gpu/drm/xe/xe_pmu_types.h
>> @@ -0,0 +1,63 @@
>> +/* SPDX-License-Identifier: MIT */
>> +/*
>> + * Copyright © 2024 Intel Corporation
>> + */
>> +
>> +#ifndef _XE_PMU_TYPES_H_
>> +#define _XE_PMU_TYPES_H_
>> +
>> +#include <linux/perf_event.h>
>> +#include <linux/spinlock_types.h>
>> +#include <uapi/drm/xe_drm.h>
>> +
>> +enum {
>> + __XE_NUM_PMU_SAMPLERS
>> +};
>> +
>> +#define XE_PMU_MAX_GT 2
>> +
>> +struct xe_pmu {
>> + /**
>> + * @cpuhp: Struct used for CPU hotplug handling.
>> + */
>> + struct {
>> + struct hlist_node node;
>> + unsigned int cpu;
>> + } cpuhp;
>> + /**
>> + * @base: PMU base.
>> + */
>> + struct pmu base;
>> + /**
>> + * @closed: xe is unregistering.
>> + */
>> + bool closed;
>> + /**
>> + * @name: Name as registered with perf core.
>> + */
>> + const char *name;
>> + /**
>> + * @lock: Lock protecting enable mask and ref count handling.
>> + */
>> + spinlock_t lock;
>> + /**
>> + * @sample: Current and previous (raw) counters.
>> + *
>> + * These counters are updated when the device is awake.
>> + */
>> + u64 sample[XE_PMU_MAX_GT][__XE_NUM_PMU_SAMPLERS];
>> + /**
>> + * @events_attr_group: Device events attribute group.
>> + */
>> + struct attribute_group events_attr_group;
>> + /**
>> + * @xe_attr: Memory block holding device attributes.
>> + */
>> + void *xe_attr;
>> + /**
>> + * @pmu_attr: Memory block holding device attributes.
>> + */
>> + void *pmu_attr;
>> +};
>> +
>> +#endif
>> diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
>> index b6fbe4988f2e..2c5f258eee3a 100644
>> --- a/include/uapi/drm/xe_drm.h
>> +++ b/include/uapi/drm/xe_drm.h
>> @@ -1389,6 +1389,14 @@ struct drm_xe_wait_user_fence {
>> __u64 reserved[2];
>> };
>>
>> +/*
>> + * Top bits of every counter are GT id.
>> + */
>> +#define __XE_PMU_GT_SHIFT (56)
>> +
>> +#define ___XE_PMU_OTHER(gt, x) \
>> + (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
>> +
>> /**
>> * enum drm_xe_observation_type - Observation stream types
>> */
>> --
>> 2.38.1
>>
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH 1/3] drm/xe/pmu: Enable PMU interface
2024-10-28 19:23 [PATCH 0/3] drm/xe/pmu: PMU interface for Xe Vinay Belgaumkar
@ 2024-10-28 19:24 ` Vinay Belgaumkar
0 siblings, 0 replies; 19+ messages in thread
From: Vinay Belgaumkar @ 2024-10-28 19:24 UTC (permalink / raw)
To: intel-xe
Cc: Aravind Iddamsetty, Bommu Krishnaiah, Riana Tauro, Rodrigo Vivi,
Vinay Belgaumkar
From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
Basic PMU enabling patch. Setup the basic framework
for adding events/timers. This patch was previously
reviewed here -
https://patchwork.freedesktop.org/series/119504/
The pmu base implementation is still from the
i915 driver.
v2: Review comments(Rodrigo) and do not init pmu for VFs
as they don't have access to freq and c6 residency anyways.
v3: Fix kunit issue, move xe_pmu entry in Makefile (Jani) and
move drm uapi definitions (Lucas)
Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
Signed-off-by: Riana Tauro <riana.tauro@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
---
drivers/gpu/drm/xe/Makefile | 2 +
drivers/gpu/drm/xe/xe_device.c | 6 +
drivers/gpu/drm/xe/xe_device_types.h | 4 +
drivers/gpu/drm/xe/xe_module.c | 5 +
drivers/gpu/drm/xe/xe_pmu.c | 577 +++++++++++++++++++++++++++
drivers/gpu/drm/xe/xe_pmu.h | 26 ++
drivers/gpu/drm/xe/xe_pmu_types.h | 70 ++++
7 files changed, 690 insertions(+)
create mode 100644 drivers/gpu/drm/xe/xe_pmu.c
create mode 100644 drivers/gpu/drm/xe/xe_pmu.h
create mode 100644 drivers/gpu/drm/xe/xe_pmu_types.h
diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
index bc7a04ce69fd..01db05da43f8 100644
--- a/drivers/gpu/drm/xe/Makefile
+++ b/drivers/gpu/drm/xe/Makefile
@@ -297,6 +297,8 @@ ifeq ($(CONFIG_DEBUG_FS),y)
i915-display/intel_pipe_crc.o
endif
+xe-$(CONFIG_PERF_EVENTS) += xe_pmu.o
+
obj-$(CONFIG_DRM_XE) += xe.o
obj-$(CONFIG_DRM_XE_KUNIT_TEST) += tests/
diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
index 51bb9d875268..bb48d111310a 100644
--- a/drivers/gpu/drm/xe/xe_device.c
+++ b/drivers/gpu/drm/xe/xe_device.c
@@ -769,6 +769,9 @@ int xe_device_probe(struct xe_device *xe)
for_each_gt(gt, xe, id)
xe_gt_sanitize_freq(gt);
+ if (!IS_SRIOV_VF(xe))
+ xe_pmu_register(&xe->pmu);
+
return devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
err_fini_display:
@@ -813,6 +816,9 @@ void xe_device_remove(struct xe_device *xe)
xe_heci_gsc_fini(xe);
+ if (!IS_SRIOV_VF(xe))
+ xe_pmu_unregister(&xe->pmu);
+
for_each_gt(gt, xe, id)
xe_gt_remove(gt);
}
diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
index 85bede4dd646..07122e8f6457 100644
--- a/drivers/gpu/drm/xe/xe_device_types.h
+++ b/drivers/gpu/drm/xe/xe_device_types.h
@@ -18,6 +18,7 @@
#include "xe_memirq_types.h"
#include "xe_oa.h"
#include "xe_platform_types.h"
+#include "xe_pmu.h"
#include "xe_pt_types.h"
#include "xe_sriov_types.h"
#include "xe_step_types.h"
@@ -516,6 +517,9 @@ struct xe_device {
int mode;
} wedged;
+ /** @pmu: performance monitoring unit */
+ struct xe_pmu pmu;
+
#ifdef TEST_VM_OPS_ERROR
/**
* @vm_inject_error_position: inject errors at different places in VM
diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
index 77ce9f9ca7a5..1bf2bf8447c0 100644
--- a/drivers/gpu/drm/xe/xe_module.c
+++ b/drivers/gpu/drm/xe/xe_module.c
@@ -14,6 +14,7 @@
#include "xe_hw_fence.h"
#include "xe_pci.h"
#include "xe_pm.h"
+#include "xe_pmu.h"
#include "xe_observation.h"
#include "xe_sched_job.h"
@@ -96,6 +97,10 @@ static const struct init_funcs init_funcs[] = {
.init = xe_sched_job_module_init,
.exit = xe_sched_job_module_exit,
},
+ {
+ .init = xe_pmu_init,
+ .exit = xe_pmu_exit,
+ },
{
.init = xe_register_pci_driver,
.exit = xe_unregister_pci_driver,
diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
new file mode 100644
index 000000000000..bba0ddc21df5
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_pmu.c
@@ -0,0 +1,577 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2024 Intel Corporation
+ */
+
+#include <drm/drm_drv.h>
+#include <drm/drm_managed.h>
+#include <drm/xe_drm.h>
+
+#include "regs/xe_gt_regs.h"
+#include "xe_device.h"
+#include "xe_force_wake.h"
+#include "xe_gt_clock.h"
+#include "xe_mmio.h"
+#include "xe_macros.h"
+#include "xe_pm.h"
+
+/**
+ * CPU mask is defined/initialized at a module level. All devices
+ * inside this module share this mask.
+ */
+static cpumask_t xe_pmu_cpumask;
+static unsigned int xe_pmu_target_cpu = -1;
+
+/**
+ * DOC: Xe PMU (Performance Monitoring Unit)
+ *
+ * Expose events/counters like C6 residency and GT frequency to user land.
+ * Perf tool can be used to list these counters from the command line.
+ *
+ * Example commands to list/record supported perf events-
+ *
+ * $ ls -ld /sys/bus/event_source/devices/xe_*
+ * $ ls /sys/bus/event_source/devices/xe_0000_00_02.0/events/
+ *
+ * You can also use the perf tool to grep for a certain event-
+ * $ perf list | grep rc6
+ *
+ * To list a specific event at regular intervals-
+ * $ perf stat -e <event_name> -I <interval>
+ *
+ */
+
+static unsigned int config_gt_id(const u64 config)
+{
+ return config >> __XE_PMU_GT_SHIFT;
+}
+
+static u64 config_counter(const u64 config)
+{
+ return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
+}
+
+static void xe_pmu_event_destroy(struct perf_event *event)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+
+ drm_WARN_ON(&xe->drm, event->parent);
+
+ drm_dev_put(&xe->drm);
+}
+
+static int
+config_status(struct xe_device *xe, u64 config)
+{
+ unsigned int gt_id = config_gt_id(config);
+
+ if (gt_id >= XE_MAX_GT_PER_TILE)
+ return -ENOENT;
+
+ switch (config_counter(config)) {
+ default:
+ return -ENOENT;
+ }
+
+ return 0;
+}
+
+static int xe_pmu_event_init(struct perf_event *event)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct xe_pmu *pmu = &xe->pmu;
+ int ret;
+
+ if (pmu->closed)
+ return -ENODEV;
+
+ if (event->attr.type != event->pmu->type)
+ return -ENOENT;
+
+ /* unsupported modes and filters */
+ if (event->attr.sample_period) /* no sampling */
+ return -EINVAL;
+
+ if (has_branch_stack(event))
+ return -EOPNOTSUPP;
+
+ if (event->cpu < 0)
+ return -EINVAL;
+
+ /* only allow running on one cpu at a time */
+ if (!cpumask_test_cpu(event->cpu, &xe_pmu_cpumask))
+ return -EINVAL;
+
+ ret = config_status(xe, event->attr.config);
+ if (ret)
+ return ret;
+
+ if (!event->parent) {
+ drm_dev_get(&xe->drm);
+ event->destroy = xe_pmu_event_destroy;
+ }
+
+ return 0;
+}
+
+static u64 __xe_pmu_event_read(struct perf_event *event)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ const unsigned int gt_id = config_gt_id(event->attr.config);
+ const u64 config = event->attr.config;
+ struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
+ u64 val = 0;
+
+ switch (config_counter(config)) {
+ default:
+ drm_warn(>->tile->xe->drm, "unknown pmu event\n");
+ }
+
+ return val;
+}
+
+static void xe_pmu_event_read(struct perf_event *event)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct hw_perf_event *hwc = &event->hw;
+ struct xe_pmu *pmu = &xe->pmu;
+ u64 prev, new;
+
+ if (pmu->closed) {
+ event->hw.state = PERF_HES_STOPPED;
+ return;
+ }
+again:
+ prev = local64_read(&hwc->prev_count);
+ new = __xe_pmu_event_read(event);
+
+ if (local64_cmpxchg(&hwc->prev_count, prev, new) != prev)
+ goto again;
+
+ local64_add(new - prev, &event->count);
+}
+
+static void xe_pmu_enable(struct perf_event *event)
+{
+ /*
+ * Store the current counter value so we can report the correct delta
+ * for all listeners. Even when the event was already enabled and has
+ * an existing non-zero value.
+ */
+ local64_set(&event->hw.prev_count, __xe_pmu_event_read(event));
+}
+
+static void xe_pmu_event_start(struct perf_event *event, int flags)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct xe_pmu *pmu = &xe->pmu;
+
+ if (pmu->closed)
+ return;
+
+ xe_pmu_enable(event);
+ event->hw.state = 0;
+}
+
+static void xe_pmu_event_stop(struct perf_event *event, int flags)
+{
+ if (flags & PERF_EF_UPDATE)
+ xe_pmu_event_read(event);
+
+ event->hw.state = PERF_HES_STOPPED;
+}
+
+static int xe_pmu_event_add(struct perf_event *event, int flags)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct xe_pmu *pmu = &xe->pmu;
+
+ if (pmu->closed)
+ return -ENODEV;
+
+ if (flags & PERF_EF_START)
+ xe_pmu_event_start(event, flags);
+
+ return 0;
+}
+
+static void xe_pmu_event_del(struct perf_event *event, int flags)
+{
+ xe_pmu_event_stop(event, PERF_EF_UPDATE);
+}
+
+static int xe_pmu_event_event_idx(struct perf_event *event)
+{
+ return 0;
+}
+
+struct xe_ext_attribute {
+ struct device_attribute attr;
+ unsigned long val;
+};
+
+static ssize_t xe_pmu_event_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct xe_ext_attribute *eattr;
+
+ eattr = container_of(attr, struct xe_ext_attribute, attr);
+ return sprintf(buf, "config=0x%lx\n", eattr->val);
+}
+
+static ssize_t cpumask_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ return cpumap_print_to_pagebuf(true, buf, &xe_pmu_cpumask);
+}
+
+static DEVICE_ATTR_RO(cpumask);
+
+static struct attribute *xe_cpumask_attrs[] = {
+ &dev_attr_cpumask.attr,
+ NULL,
+};
+
+static const struct attribute_group xe_pmu_cpumask_attr_group = {
+ .attrs = xe_cpumask_attrs,
+};
+
+#define __event(__counter, __name, __unit) \
+{ \
+ .counter = (__counter), \
+ .name = (__name), \
+ .unit = (__unit), \
+}
+
+static struct xe_ext_attribute *
+add_xe_attr(struct xe_ext_attribute *attr, const char *name, u64 config)
+{
+ sysfs_attr_init(&attr->attr.attr);
+ attr->attr.attr.name = name;
+ attr->attr.attr.mode = 0444;
+ attr->attr.show = xe_pmu_event_show;
+ attr->val = config;
+
+ return ++attr;
+}
+
+static struct perf_pmu_events_attr *
+add_pmu_attr(struct perf_pmu_events_attr *attr, const char *name,
+ const char *str)
+{
+ sysfs_attr_init(&attr->attr.attr);
+ attr->attr.attr.name = name;
+ attr->attr.attr.mode = 0444;
+ attr->attr.show = perf_event_sysfs_show;
+ attr->event_str = str;
+
+ return ++attr;
+}
+
+static struct attribute **
+create_event_attributes(struct xe_pmu *pmu)
+{
+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
+ static const struct {
+ unsigned int counter;
+ const char *name;
+ const char *unit;
+ } events[] = {
+ };
+
+ struct perf_pmu_events_attr *pmu_attr = NULL, *pmu_iter;
+ struct xe_ext_attribute *xe_attr = NULL, *xe_iter;
+ struct attribute **attr = NULL, **attr_iter;
+ unsigned int count = 0;
+ unsigned int i, j;
+ struct xe_gt *gt;
+
+ /* Count how many counters we will be exposing. */
+ for_each_gt(gt, xe, j) {
+ for (i = 0; i < ARRAY_SIZE(events); i++) {
+ u64 config = ___XE_PMU_OTHER(j, events[i].counter);
+
+ if (!config_status(xe, config))
+ count++;
+ }
+ }
+
+ /* Allocate attribute objects and table. */
+ xe_attr = kcalloc(count, sizeof(*xe_attr), GFP_KERNEL);
+ if (!xe_attr)
+ goto err_alloc;
+
+ pmu_attr = kcalloc(count, sizeof(*pmu_attr), GFP_KERNEL);
+ if (!pmu_attr)
+ goto err_alloc;
+
+ /* Max one pointer of each attribute type plus a termination entry. */
+ attr = kcalloc(count * 2 + 1, sizeof(*attr), GFP_KERNEL);
+ if (!attr)
+ goto err_alloc;
+
+ xe_iter = xe_attr;
+ pmu_iter = pmu_attr;
+ attr_iter = attr;
+
+ for_each_gt(gt, xe, j) {
+ for (i = 0; i < ARRAY_SIZE(events); i++) {
+ u64 config = ___XE_PMU_OTHER(j, events[i].counter);
+ char *str;
+
+ if (config_status(xe, config))
+ continue;
+
+ str = kasprintf(GFP_KERNEL, "%s-gt%u",
+ events[i].name, j);
+ if (!str)
+ goto err;
+
+ *attr_iter++ = &xe_iter->attr.attr;
+ xe_iter = add_xe_attr(xe_iter, str, config);
+
+ if (events[i].unit) {
+ str = kasprintf(GFP_KERNEL, "%s-gt%u.unit",
+ events[i].name, j);
+ if (!str)
+ goto err;
+
+ *attr_iter++ = &pmu_iter->attr.attr;
+ pmu_iter = add_pmu_attr(pmu_iter, str,
+ events[i].unit);
+ }
+ }
+ }
+
+ pmu->xe_attr = xe_attr;
+ pmu->pmu_attr = pmu_attr;
+
+ return attr;
+
+err:
+ for (attr_iter = attr; *attr_iter; attr_iter++)
+ kfree((*attr_iter)->name);
+
+err_alloc:
+ kfree(attr);
+ kfree(xe_attr);
+ kfree(pmu_attr);
+
+ return NULL;
+}
+
+static void free_event_attributes(struct xe_pmu *pmu)
+{
+ struct attribute **attr_iter = pmu->events_attr_group.attrs;
+
+ for (; *attr_iter; attr_iter++)
+ kfree((*attr_iter)->name);
+
+ kfree(pmu->events_attr_group.attrs);
+ kfree(pmu->xe_attr);
+ kfree(pmu->pmu_attr);
+
+ pmu->events_attr_group.attrs = NULL;
+ pmu->xe_attr = NULL;
+ pmu->pmu_attr = NULL;
+}
+
+static int xe_pmu_cpu_online(unsigned int cpu, struct hlist_node *node)
+{
+ struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
+
+ XE_WARN_ON(!pmu->base.event_init);
+
+ /* Select the first online CPU as a designated reader. */
+ if (cpumask_empty(&xe_pmu_cpumask))
+ cpumask_set_cpu(cpu, &xe_pmu_cpumask);
+
+ return 0;
+}
+
+static int xe_pmu_cpu_offline(unsigned int cpu, struct hlist_node *node)
+{
+ struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
+ unsigned int target = xe_pmu_target_cpu;
+
+ /*
+ * Unregistering an instance generates a CPU offline event which we must
+ * ignore to avoid incorrectly modifying the shared xe_pmu_cpumask.
+ */
+ if (pmu->closed)
+ return 0;
+
+ if (cpumask_test_and_clear_cpu(cpu, &xe_pmu_cpumask)) {
+ target = cpumask_any_but(topology_sibling_cpumask(cpu), cpu);
+
+ /* Migrate events if there is a valid target */
+ if (target < nr_cpu_ids) {
+ cpumask_set_cpu(target, &xe_pmu_cpumask);
+ xe_pmu_target_cpu = target;
+ }
+ }
+
+ if (target < nr_cpu_ids && target != pmu->cpuhp.cpu) {
+ perf_pmu_migrate_context(&pmu->base, cpu, target);
+ pmu->cpuhp.cpu = target;
+ }
+
+ return 0;
+}
+
+static enum cpuhp_state cpuhp_slot = CPUHP_INVALID;
+
+/**
+ * xe_pmu_init() - Setup CPU hotplug state/callbacks for Xe PMU
+ *
+ * Returns: 0 if successful, else error code
+ */
+int xe_pmu_init(void)
+{
+ int ret;
+
+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
+ "perf/x86/intel/xe:online",
+ xe_pmu_cpu_online,
+ xe_pmu_cpu_offline);
+ if (ret < 0)
+ pr_notice("Failed to setup cpuhp state for xe PMU! (%d)\n",
+ ret);
+ else
+ cpuhp_slot = ret;
+
+ return 0;
+}
+
+/**
+ * xe_pmu_exit() - Remove CPU hotplug state/callbacks for Xe PMU
+ */
+void xe_pmu_exit(void)
+{
+ if (cpuhp_slot != CPUHP_INVALID)
+ cpuhp_remove_multi_state(cpuhp_slot);
+}
+
+static int xe_pmu_register_cpuhp_state(struct xe_pmu *pmu)
+{
+ if (cpuhp_slot == CPUHP_INVALID)
+ return -EINVAL;
+
+ return cpuhp_state_add_instance(cpuhp_slot, &pmu->cpuhp.node);
+}
+
+static void xe_pmu_unregister_cpuhp_state(struct xe_pmu *pmu)
+{
+ cpuhp_state_remove_instance(cpuhp_slot, &pmu->cpuhp.node);
+}
+
+/**
+ * xe_pmu_unregister() - Remove/cleanup PMU registration
+ */
+void xe_pmu_unregister(void *arg)
+{
+ struct xe_pmu *pmu = arg;
+
+ if (!pmu->base.event_init)
+ return;
+
+ /*
+ * "Disconnect" the PMU callbacks - since all are atomic synchronize_rcu
+ * ensures all currently executing ones will have exited before we
+ * proceed with unregistration.
+ */
+ pmu->closed = true;
+ synchronize_rcu();
+
+ xe_pmu_unregister_cpuhp_state(pmu);
+
+ perf_pmu_unregister(&pmu->base);
+ pmu->base.event_init = NULL;
+ kfree(pmu->base.attr_groups);
+ kfree(pmu->name);
+ free_event_attributes(pmu);
+}
+
+/**
+ * xe_pmu_register() - Define basic PMU properties for Xe and add event callbacks.
+ *
+ */
+void xe_pmu_register(struct xe_pmu *pmu)
+{
+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
+ const struct attribute_group *attr_groups[] = {
+ &pmu->events_attr_group,
+ &xe_pmu_cpumask_attr_group,
+ NULL
+ };
+
+ int ret = -ENOMEM;
+
+ spin_lock_init(&pmu->lock);
+ pmu->cpuhp.cpu = -1;
+
+ pmu->name = kasprintf(GFP_KERNEL,
+ "xe_%s",
+ dev_name(xe->drm.dev));
+ if (pmu->name) {
+ /* tools/perf reserves colons as special. */
+ strreplace((char *)pmu->name, ':', '_');
+ }
+
+ if (!pmu->name)
+ goto err;
+
+ pmu->events_attr_group.name = "events";
+ pmu->events_attr_group.attrs = create_event_attributes(pmu);
+ if (!pmu->events_attr_group.attrs)
+ goto err_name;
+
+ pmu->base.attr_groups = kmemdup(attr_groups, sizeof(attr_groups),
+ GFP_KERNEL);
+ if (!pmu->base.attr_groups)
+ goto err_attr;
+
+ pmu->base.module = THIS_MODULE;
+ pmu->base.task_ctx_nr = perf_invalid_context;
+ pmu->base.event_init = xe_pmu_event_init;
+ pmu->base.add = xe_pmu_event_add;
+ pmu->base.del = xe_pmu_event_del;
+ pmu->base.start = xe_pmu_event_start;
+ pmu->base.stop = xe_pmu_event_stop;
+ pmu->base.read = xe_pmu_event_read;
+ pmu->base.event_idx = xe_pmu_event_event_idx;
+
+ ret = perf_pmu_register(&pmu->base, pmu->name, -1);
+ if (ret)
+ goto err_groups;
+
+ ret = xe_pmu_register_cpuhp_state(pmu);
+ if (ret)
+ goto err_unreg;
+
+ ret = devm_add_action_or_reset(xe->drm.dev, xe_pmu_unregister, pmu);
+ if (ret)
+ goto err_cpuhp;
+
+ return;
+
+err_cpuhp:
+ xe_pmu_unregister_cpuhp_state(pmu);
+err_unreg:
+ perf_pmu_unregister(&pmu->base);
+err_groups:
+ kfree(pmu->base.attr_groups);
+err_attr:
+ pmu->base.event_init = NULL;
+ free_event_attributes(pmu);
+err_name:
+ kfree(pmu->name);
+err:
+ drm_notice(&xe->drm, "Failed to register PMU!\n");
+}
diff --git a/drivers/gpu/drm/xe/xe_pmu.h b/drivers/gpu/drm/xe/xe_pmu.h
new file mode 100644
index 000000000000..d07e5dfdfec0
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_pmu.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2024 Intel Corporation
+ */
+
+#ifndef _XE_PMU_H_
+#define _XE_PMU_H_
+
+#include "xe_pmu_types.h"
+
+struct xe_gt;
+
+#if IS_ENABLED(CONFIG_PERF_EVENTS)
+int xe_pmu_init(void);
+void xe_pmu_exit(void);
+void xe_pmu_register(struct xe_pmu *pmu);
+void xe_pmu_unregister(void *arg);
+#else
+static inline int xe_pmu_init(void) { return 0; }
+static inline void xe_pmu_exit(void) {}
+static inline void xe_pmu_register(struct xe_pmu *pmu) {}
+static inline void xe_pmu_unregister(void *arg) {}
+#endif
+
+#endif
+
diff --git a/drivers/gpu/drm/xe/xe_pmu_types.h b/drivers/gpu/drm/xe/xe_pmu_types.h
new file mode 100644
index 000000000000..c79f000a4880
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_pmu_types.h
@@ -0,0 +1,70 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2024 Intel Corporation
+ */
+
+#ifndef _XE_PMU_TYPES_H_
+#define _XE_PMU_TYPES_H_
+
+#include <linux/perf_event.h>
+#include <linux/spinlock_types.h>
+
+enum {
+ __XE_NUM_PMU_SAMPLERS
+};
+
+#define XE_PMU_MAX_GT 2
+
+/*
+ * Top bits of every counter are GT id.
+ */
+#define __XE_PMU_GT_SHIFT (56)
+
+#define ___XE_PMU_OTHER(gt, x) \
+ (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
+
+struct xe_pmu {
+ /**
+ * @cpuhp: Struct used for CPU hotplug handling.
+ */
+ struct {
+ struct hlist_node node;
+ unsigned int cpu;
+ } cpuhp;
+ /**
+ * @base: PMU base.
+ */
+ struct pmu base;
+ /**
+ * @closed: xe is unregistering.
+ */
+ bool closed;
+ /**
+ * @name: Name as registered with perf core.
+ */
+ const char *name;
+ /**
+ * @lock: Lock protecting enable mask and ref count handling.
+ */
+ spinlock_t lock;
+ /**
+ * @sample: Current and previous (raw) counters.
+ *
+ * These counters are updated when the device is awake.
+ */
+ u64 sample[XE_PMU_MAX_GT][__XE_NUM_PMU_SAMPLERS];
+ /**
+ * @events_attr_group: Device events attribute group.
+ */
+ struct attribute_group events_attr_group;
+ /**
+ * @xe_attr: Memory block holding device attributes.
+ */
+ void *xe_attr;
+ /**
+ * @pmu_attr: Memory block holding device attributes.
+ */
+ void *pmu_attr;
+};
+
+#endif
--
2.38.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 1/3] drm/xe/pmu: Enable PMU interface
2024-11-08 18:15 [PATCH 0/3] drm/xe/pmu: PMU interface for Xe Vinay Belgaumkar
@ 2024-11-08 18:15 ` Vinay Belgaumkar
2024-11-08 22:30 ` Rodrigo Vivi
0 siblings, 1 reply; 19+ messages in thread
From: Vinay Belgaumkar @ 2024-11-08 18:15 UTC (permalink / raw)
To: intel-xe
Cc: Aravind Iddamsetty, Bommu Krishnaiah, Riana Tauro, Rodrigo Vivi,
Vinay Belgaumkar
From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
Basic PMU enabling patch. Setup the basic framework
for adding events/timers. This patch was previously
reviewed here -
https://patchwork.freedesktop.org/series/119504/
The pmu base implementation is still from the
i915 driver.
v2: Review comments(Rodrigo) and do not init pmu for VFs
as they don't have access to freq and c6 residency anyways.
v3: Fix kunit issue, move xe_pmu entry in Makefile (Jani) and
move drm uapi definitions (Lucas)
v4: Adapt Lucas's recent PMU fixes for i915
Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
Signed-off-by: Riana Tauro <riana.tauro@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
---
drivers/gpu/drm/xe/Makefile | 2 +
drivers/gpu/drm/xe/xe_device.c | 6 +
drivers/gpu/drm/xe/xe_device_types.h | 4 +
drivers/gpu/drm/xe/xe_module.c | 5 +
drivers/gpu/drm/xe/xe_pmu.c | 577 +++++++++++++++++++++++++++
drivers/gpu/drm/xe/xe_pmu.h | 26 ++
drivers/gpu/drm/xe/xe_pmu_types.h | 70 ++++
7 files changed, 690 insertions(+)
create mode 100644 drivers/gpu/drm/xe/xe_pmu.c
create mode 100644 drivers/gpu/drm/xe/xe_pmu.h
create mode 100644 drivers/gpu/drm/xe/xe_pmu_types.h
diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
index a93e6fcc0ad9..c231ecaf86b8 100644
--- a/drivers/gpu/drm/xe/Makefile
+++ b/drivers/gpu/drm/xe/Makefile
@@ -299,6 +299,8 @@ ifeq ($(CONFIG_DEBUG_FS),y)
i915-display/intel_pipe_crc.o
endif
+xe-$(CONFIG_PERF_EVENTS) += xe_pmu.o
+
obj-$(CONFIG_DRM_XE) += xe.o
obj-$(CONFIG_DRM_XE_KUNIT_TEST) += tests/
diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
index 0e2dd691bdae..89463cf7cc2c 100644
--- a/drivers/gpu/drm/xe/xe_device.c
+++ b/drivers/gpu/drm/xe/xe_device.c
@@ -759,6 +759,9 @@ int xe_device_probe(struct xe_device *xe)
for_each_gt(gt, xe, id)
xe_gt_sanitize_freq(gt);
+ if (!IS_SRIOV_VF(xe))
+ xe_pmu_register(&xe->pmu);
+
return devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
err_fini_display:
@@ -803,6 +806,9 @@ void xe_device_remove(struct xe_device *xe)
xe_heci_gsc_fini(xe);
+ if (!IS_SRIOV_VF(xe))
+ xe_pmu_unregister(&xe->pmu);
+
for_each_gt(gt, xe, id)
xe_gt_remove(gt);
}
diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
index bccca63c8a48..0cb8d650135a 100644
--- a/drivers/gpu/drm/xe/xe_device_types.h
+++ b/drivers/gpu/drm/xe/xe_device_types.h
@@ -18,6 +18,7 @@
#include "xe_memirq_types.h"
#include "xe_oa.h"
#include "xe_platform_types.h"
+#include "xe_pmu.h"
#include "xe_pt_types.h"
#include "xe_sriov_types.h"
#include "xe_step_types.h"
@@ -509,6 +510,9 @@ struct xe_device {
int mode;
} wedged;
+ /** @pmu: performance monitoring unit */
+ struct xe_pmu pmu;
+
#ifdef TEST_VM_OPS_ERROR
/**
* @vm_inject_error_position: inject errors at different places in VM
diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
index 77ce9f9ca7a5..1bf2bf8447c0 100644
--- a/drivers/gpu/drm/xe/xe_module.c
+++ b/drivers/gpu/drm/xe/xe_module.c
@@ -14,6 +14,7 @@
#include "xe_hw_fence.h"
#include "xe_pci.h"
#include "xe_pm.h"
+#include "xe_pmu.h"
#include "xe_observation.h"
#include "xe_sched_job.h"
@@ -96,6 +97,10 @@ static const struct init_funcs init_funcs[] = {
.init = xe_sched_job_module_init,
.exit = xe_sched_job_module_exit,
},
+ {
+ .init = xe_pmu_init,
+ .exit = xe_pmu_exit,
+ },
{
.init = xe_register_pci_driver,
.exit = xe_unregister_pci_driver,
diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
new file mode 100644
index 000000000000..7ce66c022e27
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_pmu.c
@@ -0,0 +1,577 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2024 Intel Corporation
+ */
+
+#include <drm/drm_drv.h>
+#include <drm/drm_managed.h>
+#include <drm/xe_drm.h>
+
+#include "regs/xe_gt_regs.h"
+#include "xe_device.h"
+#include "xe_force_wake.h"
+#include "xe_gt_clock.h"
+#include "xe_mmio.h"
+#include "xe_macros.h"
+#include "xe_pm.h"
+
+/**
+ * CPU mask is defined/initialized at a module level. All devices
+ * inside this module share this mask.
+ */
+static cpumask_t xe_pmu_cpumask;
+static unsigned int xe_pmu_target_cpu = -1;
+
+/**
+ * DOC: Xe PMU (Performance Monitoring Unit)
+ *
+ * Expose events/counters like C6 residency and GT frequency to user land.
+ * Perf tool can be used to list these counters from the command line.
+ *
+ * Example commands to list/record supported perf events-
+ *
+ * $ ls -ld /sys/bus/event_source/devices/xe_*
+ * $ ls /sys/bus/event_source/devices/xe_0000_00_02.0/events/
+ *
+ * You can also use the perf tool to grep for a certain event-
+ * $ perf list | grep rc6
+ *
+ * To list a specific event at regular intervals-
+ * $ perf stat -e <event_name> -I <interval>
+ *
+ */
+
+static unsigned int config_gt_id(const u64 config)
+{
+ return config >> __XE_PMU_GT_SHIFT;
+}
+
+static u64 config_counter(const u64 config)
+{
+ return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
+}
+
+static void xe_pmu_event_destroy(struct perf_event *event)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+
+ drm_WARN_ON(&xe->drm, event->parent);
+
+ drm_dev_put(&xe->drm);
+}
+
+static int
+config_status(struct xe_device *xe, u64 config)
+{
+ unsigned int gt_id = config_gt_id(config);
+
+ if (gt_id >= XE_MAX_GT_PER_TILE)
+ return -ENOENT;
+
+ switch (config_counter(config)) {
+ default:
+ return -ENOENT;
+ }
+
+ return 0;
+}
+
+static int xe_pmu_event_init(struct perf_event *event)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct xe_pmu *pmu = &xe->pmu;
+ int ret;
+
+ if (!pmu->registered)
+ return -ENODEV;
+
+ if (event->attr.type != event->pmu->type)
+ return -ENOENT;
+
+ /* unsupported modes and filters */
+ if (event->attr.sample_period) /* no sampling */
+ return -EINVAL;
+
+ if (has_branch_stack(event))
+ return -EOPNOTSUPP;
+
+ if (event->cpu < 0)
+ return -EINVAL;
+
+ /* only allow running on one cpu at a time */
+ if (!cpumask_test_cpu(event->cpu, &xe_pmu_cpumask))
+ return -EINVAL;
+
+ ret = config_status(xe, event->attr.config);
+ if (ret)
+ return ret;
+
+ if (!event->parent) {
+ drm_dev_get(&xe->drm);
+ event->destroy = xe_pmu_event_destroy;
+ }
+
+ return 0;
+}
+
+static u64 __xe_pmu_event_read(struct perf_event *event)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ const unsigned int gt_id = config_gt_id(event->attr.config);
+ const u64 config = event->attr.config;
+ struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
+ u64 val = 0;
+
+ switch (config_counter(config)) {
+ default:
+ drm_warn(>->tile->xe->drm, "unknown pmu event\n");
+ }
+
+ return val;
+}
+
+static void xe_pmu_event_read(struct perf_event *event)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct hw_perf_event *hwc = &event->hw;
+ struct xe_pmu *pmu = &xe->pmu;
+ u64 prev, new;
+
+ if (!pmu->registered) {
+ event->hw.state = PERF_HES_STOPPED;
+ return;
+ }
+again:
+ prev = local64_read(&hwc->prev_count);
+ new = __xe_pmu_event_read(event);
+
+ if (local64_cmpxchg(&hwc->prev_count, prev, new) != prev)
+ goto again;
+
+ local64_add(new - prev, &event->count);
+}
+
+static void xe_pmu_enable(struct perf_event *event)
+{
+ /*
+ * Store the current counter value so we can report the correct delta
+ * for all listeners. Even when the event was already enabled and has
+ * an existing non-zero value.
+ */
+ local64_set(&event->hw.prev_count, __xe_pmu_event_read(event));
+}
+
+static void xe_pmu_event_start(struct perf_event *event, int flags)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct xe_pmu *pmu = &xe->pmu;
+
+ if (!pmu->registered)
+ return;
+
+ xe_pmu_enable(event);
+ event->hw.state = 0;
+}
+
+static void xe_pmu_event_stop(struct perf_event *event, int flags)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct xe_pmu *pmu = &xe->pmu;
+
+ if (!pmu->registered)
+ goto out;
+
+ if (flags & PERF_EF_UPDATE)
+ xe_pmu_event_read(event);
+
+out:
+ event->hw.state = PERF_HES_STOPPED;
+}
+
+static int xe_pmu_event_add(struct perf_event *event, int flags)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct xe_pmu *pmu = &xe->pmu;
+
+ if (!pmu->registered)
+ return -ENODEV;
+
+ if (flags & PERF_EF_START)
+ xe_pmu_event_start(event, flags);
+
+ return 0;
+}
+
+static void xe_pmu_event_del(struct perf_event *event, int flags)
+{
+ xe_pmu_event_stop(event, PERF_EF_UPDATE);
+}
+
+static int xe_pmu_event_event_idx(struct perf_event *event)
+{
+ return 0;
+}
+
+struct xe_ext_attribute {
+ struct device_attribute attr;
+ unsigned long val;
+};
+
+static ssize_t xe_pmu_event_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct xe_ext_attribute *eattr;
+
+ eattr = container_of(attr, struct xe_ext_attribute, attr);
+ return sprintf(buf, "config=0x%lx\n", eattr->val);
+}
+
+static ssize_t cpumask_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ return cpumap_print_to_pagebuf(true, buf, &xe_pmu_cpumask);
+}
+
+static DEVICE_ATTR_RO(cpumask);
+
+static struct attribute *xe_cpumask_attrs[] = {
+ &dev_attr_cpumask.attr,
+ NULL,
+};
+
+static const struct attribute_group xe_pmu_cpumask_attr_group = {
+ .attrs = xe_cpumask_attrs,
+};
+
+#define __event(__counter, __name, __unit) \
+{ \
+ .counter = (__counter), \
+ .name = (__name), \
+ .unit = (__unit), \
+}
+
+static struct xe_ext_attribute *
+add_xe_attr(struct xe_ext_attribute *attr, const char *name, u64 config)
+{
+ sysfs_attr_init(&attr->attr.attr);
+ attr->attr.attr.name = name;
+ attr->attr.attr.mode = 0444;
+ attr->attr.show = xe_pmu_event_show;
+ attr->val = config;
+
+ return ++attr;
+}
+
+static struct perf_pmu_events_attr *
+add_pmu_attr(struct perf_pmu_events_attr *attr, const char *name,
+ const char *str)
+{
+ sysfs_attr_init(&attr->attr.attr);
+ attr->attr.attr.name = name;
+ attr->attr.attr.mode = 0444;
+ attr->attr.show = perf_event_sysfs_show;
+ attr->event_str = str;
+
+ return ++attr;
+}
+
+static struct attribute **
+create_event_attributes(struct xe_pmu *pmu)
+{
+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
+ static const struct {
+ unsigned int counter;
+ const char *name;
+ const char *unit;
+ } events[] = {
+ };
+
+ struct perf_pmu_events_attr *pmu_attr = NULL, *pmu_iter;
+ struct xe_ext_attribute *xe_attr = NULL, *xe_iter;
+ struct attribute **attr = NULL, **attr_iter;
+ unsigned int count = 0;
+ unsigned int i, j;
+ struct xe_gt *gt;
+
+ /* Count how many counters we will be exposing. */
+ for_each_gt(gt, xe, j) {
+ for (i = 0; i < ARRAY_SIZE(events); i++) {
+ u64 config = ___XE_PMU_OTHER(j, events[i].counter);
+
+ if (!config_status(xe, config))
+ count++;
+ }
+ }
+
+ /* Allocate attribute objects and table. */
+ xe_attr = kcalloc(count, sizeof(*xe_attr), GFP_KERNEL);
+ if (!xe_attr)
+ goto err_alloc;
+
+ pmu_attr = kcalloc(count, sizeof(*pmu_attr), GFP_KERNEL);
+ if (!pmu_attr)
+ goto err_alloc;
+
+ /* Max one pointer of each attribute type plus a termination entry. */
+ attr = kcalloc(count * 2 + 1, sizeof(*attr), GFP_KERNEL);
+ if (!attr)
+ goto err_alloc;
+
+ xe_iter = xe_attr;
+ pmu_iter = pmu_attr;
+ attr_iter = attr;
+
+ for_each_gt(gt, xe, j) {
+ for (i = 0; i < ARRAY_SIZE(events); i++) {
+ u64 config = ___XE_PMU_OTHER(j, events[i].counter);
+ char *str;
+
+ if (config_status(xe, config))
+ continue;
+
+ str = kasprintf(GFP_KERNEL, "%s-gt%u",
+ events[i].name, j);
+ if (!str)
+ goto err;
+
+ *attr_iter++ = &xe_iter->attr.attr;
+ xe_iter = add_xe_attr(xe_iter, str, config);
+
+ if (events[i].unit) {
+ str = kasprintf(GFP_KERNEL, "%s-gt%u.unit",
+ events[i].name, j);
+ if (!str)
+ goto err;
+
+ *attr_iter++ = &pmu_iter->attr.attr;
+ pmu_iter = add_pmu_attr(pmu_iter, str,
+ events[i].unit);
+ }
+ }
+ }
+
+ pmu->xe_attr = xe_attr;
+ pmu->pmu_attr = pmu_attr;
+
+ return attr;
+
+err:
+ for (attr_iter = attr; *attr_iter; attr_iter++)
+ kfree((*attr_iter)->name);
+
+err_alloc:
+ kfree(attr);
+ kfree(xe_attr);
+ kfree(pmu_attr);
+
+ return NULL;
+}
+
+static void free_event_attributes(struct xe_pmu *pmu)
+{
+ struct attribute **attr_iter = pmu->events_attr_group.attrs;
+
+ for (; *attr_iter; attr_iter++)
+ kfree((*attr_iter)->name);
+
+ kfree(pmu->events_attr_group.attrs);
+ kfree(pmu->xe_attr);
+ kfree(pmu->pmu_attr);
+
+ pmu->events_attr_group.attrs = NULL;
+ pmu->xe_attr = NULL;
+ pmu->pmu_attr = NULL;
+}
+
+static int xe_pmu_cpu_online(unsigned int cpu, struct hlist_node *node)
+{
+ struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
+
+ /* Select the first online CPU as a designated reader. */
+ if (cpumask_empty(&xe_pmu_cpumask))
+ cpumask_set_cpu(cpu, &xe_pmu_cpumask);
+
+ return 0;
+}
+
+static int xe_pmu_cpu_offline(unsigned int cpu, struct hlist_node *node)
+{
+ struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
+ unsigned int target = xe_pmu_target_cpu;
+
+ /*
+ * Unregistering an instance generates a CPU offline event which we must
+ * ignore to avoid incorrectly modifying the shared xe_pmu_cpumask.
+ */
+ if (!pmu->registered)
+ return 0;
+
+ if (cpumask_test_and_clear_cpu(cpu, &xe_pmu_cpumask)) {
+ target = cpumask_any_but(topology_sibling_cpumask(cpu), cpu);
+
+ /* Migrate events if there is a valid target */
+ if (target < nr_cpu_ids) {
+ cpumask_set_cpu(target, &xe_pmu_cpumask);
+ xe_pmu_target_cpu = target;
+ }
+ }
+
+ if (target < nr_cpu_ids && target != pmu->cpuhp.cpu) {
+ perf_pmu_migrate_context(&pmu->base, cpu, target);
+ pmu->cpuhp.cpu = target;
+ }
+
+ return 0;
+}
+
+static enum cpuhp_state cpuhp_state = CPUHP_INVALID;
+
+/**
+ * xe_pmu_init() - Setup CPU hotplug state/callbacks for Xe PMU
+ *
+ * Returns: 0 if successful, else error code
+ */
+int xe_pmu_init(void)
+{
+ int ret;
+
+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
+ "perf/x86/intel/xe:online",
+ xe_pmu_cpu_online,
+ xe_pmu_cpu_offline);
+ if (ret < 0)
+ pr_notice("Failed to setup cpuhp state for xe PMU! (%d)\n",
+ ret);
+ else
+ cpuhp_state = ret;
+
+ return 0;
+}
+
+/**
+ * xe_pmu_exit() - Remove CPU hotplug state/callbacks for Xe PMU
+ */
+void xe_pmu_exit(void)
+{
+ if (cpuhp_state != CPUHP_INVALID)
+ cpuhp_remove_multi_state(cpuhp_state);
+}
+
+static int xe_pmu_register_cpuhp_state(struct xe_pmu *pmu)
+{
+ if (cpuhp_state == CPUHP_INVALID)
+ return -EINVAL;
+
+ return cpuhp_state_add_instance(cpuhp_state, &pmu->cpuhp.node);
+}
+
+static void xe_pmu_unregister_cpuhp_state(struct xe_pmu *pmu)
+{
+ cpuhp_state_remove_instance(cpuhp_state, &pmu->cpuhp.node);
+}
+
+/**
+ * xe_pmu_unregister() - Remove/cleanup PMU registration
+ */
+void xe_pmu_unregister(void *arg)
+{
+ struct xe_pmu *pmu = arg;
+
+ if (!pmu->registered)
+ return;
+
+ pmu->registered = false;
+
+ xe_pmu_unregister_cpuhp_state(pmu);
+
+ perf_pmu_unregister(&pmu->base);
+ kfree(pmu->base.attr_groups);
+ kfree(pmu->name);
+ free_event_attributes(pmu);
+}
+
+/**
+ * xe_pmu_register() - Define basic PMU properties for Xe and add event callbacks.
+ *
+ */
+void xe_pmu_register(struct xe_pmu *pmu)
+{
+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
+ const struct attribute_group *attr_groups[] = {
+ &pmu->events_attr_group,
+ &xe_pmu_cpumask_attr_group,
+ NULL
+ };
+
+ int ret = -ENOMEM;
+
+ spin_lock_init(&pmu->lock);
+ pmu->cpuhp.cpu = -1;
+
+ pmu->name = kasprintf(GFP_KERNEL,
+ "xe_%s",
+ dev_name(xe->drm.dev));
+ if (pmu->name) {
+ /* tools/perf reserves colons as special. */
+ strreplace((char *)pmu->name, ':', '_');
+ }
+
+ if (!pmu->name)
+ goto err;
+
+ pmu->events_attr_group.name = "events";
+ pmu->events_attr_group.attrs = create_event_attributes(pmu);
+ if (!pmu->events_attr_group.attrs)
+ goto err_name;
+
+ pmu->base.attr_groups = kmemdup(attr_groups, sizeof(attr_groups),
+ GFP_KERNEL);
+ if (!pmu->base.attr_groups)
+ goto err_attr;
+
+ pmu->base.module = THIS_MODULE;
+ pmu->base.task_ctx_nr = perf_invalid_context;
+ pmu->base.event_init = xe_pmu_event_init;
+ pmu->base.add = xe_pmu_event_add;
+ pmu->base.del = xe_pmu_event_del;
+ pmu->base.start = xe_pmu_event_start;
+ pmu->base.stop = xe_pmu_event_stop;
+ pmu->base.read = xe_pmu_event_read;
+ pmu->base.event_idx = xe_pmu_event_event_idx;
+
+ ret = perf_pmu_register(&pmu->base, pmu->name, -1);
+ if (ret)
+ goto err_groups;
+
+ ret = xe_pmu_register_cpuhp_state(pmu);
+ if (ret)
+ goto err_unreg;
+
+ ret = devm_add_action_or_reset(xe->drm.dev, xe_pmu_unregister, pmu);
+ if (ret)
+ goto err_cpuhp;
+
+ pmu->registered = true;
+
+ return;
+
+err_cpuhp:
+ xe_pmu_unregister_cpuhp_state(pmu);
+err_unreg:
+ perf_pmu_unregister(&pmu->base);
+err_groups:
+ kfree(pmu->base.attr_groups);
+err_attr:
+ free_event_attributes(pmu);
+err_name:
+ kfree(pmu->name);
+err:
+ drm_notice(&xe->drm, "Failed to register PMU!\n");
+}
diff --git a/drivers/gpu/drm/xe/xe_pmu.h b/drivers/gpu/drm/xe/xe_pmu.h
new file mode 100644
index 000000000000..d07e5dfdfec0
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_pmu.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2024 Intel Corporation
+ */
+
+#ifndef _XE_PMU_H_
+#define _XE_PMU_H_
+
+#include "xe_pmu_types.h"
+
+struct xe_gt;
+
+#if IS_ENABLED(CONFIG_PERF_EVENTS)
+int xe_pmu_init(void);
+void xe_pmu_exit(void);
+void xe_pmu_register(struct xe_pmu *pmu);
+void xe_pmu_unregister(void *arg);
+#else
+static inline int xe_pmu_init(void) { return 0; }
+static inline void xe_pmu_exit(void) {}
+static inline void xe_pmu_register(struct xe_pmu *pmu) {}
+static inline void xe_pmu_unregister(void *arg) {}
+#endif
+
+#endif
+
diff --git a/drivers/gpu/drm/xe/xe_pmu_types.h b/drivers/gpu/drm/xe/xe_pmu_types.h
new file mode 100644
index 000000000000..4da96b8fadd1
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_pmu_types.h
@@ -0,0 +1,70 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2024 Intel Corporation
+ */
+
+#ifndef _XE_PMU_TYPES_H_
+#define _XE_PMU_TYPES_H_
+
+#include <linux/perf_event.h>
+#include <linux/spinlock_types.h>
+
+enum {
+ __XE_NUM_PMU_SAMPLERS
+};
+
+#define XE_PMU_MAX_GT 2
+
+/*
+ * Top bits of every counter are GT id.
+ */
+#define __XE_PMU_GT_SHIFT (56)
+
+#define ___XE_PMU_OTHER(gt, x) \
+ (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
+
+struct xe_pmu {
+ /**
+ * @cpuhp: Struct used for CPU hotplug handling.
+ */
+ struct {
+ struct hlist_node node;
+ unsigned int cpu;
+ } cpuhp;
+ /**
+ * @base: PMU base.
+ */
+ struct pmu base;
+ /**
+ * @registered: PMU is registered and not in the unregistering process.
+ */
+ bool registered;
+ /**
+ * @name: Name as registered with perf core.
+ */
+ const char *name;
+ /**
+ * @lock: Lock protecting enable mask and ref count handling.
+ */
+ spinlock_t lock;
+ /**
+ * @sample: Current and previous (raw) counters.
+ *
+ * These counters are updated when the device is awake.
+ */
+ u64 sample[XE_PMU_MAX_GT][__XE_NUM_PMU_SAMPLERS];
+ /**
+ * @events_attr_group: Device events attribute group.
+ */
+ struct attribute_group events_attr_group;
+ /**
+ * @xe_attr: Memory block holding device attributes.
+ */
+ void *xe_attr;
+ /**
+ * @pmu_attr: Memory block holding device attributes.
+ */
+ void *pmu_attr;
+};
+
+#endif
--
2.38.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PATCH 1/3] drm/xe/pmu: Enable PMU interface
2024-11-08 18:15 ` [PATCH 1/3] drm/xe/pmu: Enable PMU interface Vinay Belgaumkar
@ 2024-11-08 22:30 ` Rodrigo Vivi
0 siblings, 0 replies; 19+ messages in thread
From: Rodrigo Vivi @ 2024-11-08 22:30 UTC (permalink / raw)
To: Vinay Belgaumkar
Cc: intel-xe, Aravind Iddamsetty, Bommu Krishnaiah, Riana Tauro
On Fri, Nov 08, 2024 at 10:15:10AM -0800, Vinay Belgaumkar wrote:
> From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
>
> Basic PMU enabling patch. Setup the basic framework
> for adding events/timers. This patch was previously
> reviewed here -
> https://patchwork.freedesktop.org/series/119504/
>
> The pmu base implementation is still from the
> i915 driver.
>
> v2: Review comments(Rodrigo) and do not init pmu for VFs
> as they don't have access to freq and c6 residency anyways.
>
> v3: Fix kunit issue, move xe_pmu entry in Makefile (Jani) and
> move drm uapi definitions (Lucas)
>
> v4: Adapt Lucas's recent PMU fixes for i915
I believe this already deserves
Co-developed-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
>
> Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
> Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
> Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
> Signed-off-by: Riana Tauro <riana.tauro@intel.com>
> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> Signed-off-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
> ---
> drivers/gpu/drm/xe/Makefile | 2 +
> drivers/gpu/drm/xe/xe_device.c | 6 +
> drivers/gpu/drm/xe/xe_device_types.h | 4 +
> drivers/gpu/drm/xe/xe_module.c | 5 +
> drivers/gpu/drm/xe/xe_pmu.c | 577 +++++++++++++++++++++++++++
> drivers/gpu/drm/xe/xe_pmu.h | 26 ++
> drivers/gpu/drm/xe/xe_pmu_types.h | 70 ++++
> 7 files changed, 690 insertions(+)
> create mode 100644 drivers/gpu/drm/xe/xe_pmu.c
> create mode 100644 drivers/gpu/drm/xe/xe_pmu.h
> create mode 100644 drivers/gpu/drm/xe/xe_pmu_types.h
>
> diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
> index a93e6fcc0ad9..c231ecaf86b8 100644
> --- a/drivers/gpu/drm/xe/Makefile
> +++ b/drivers/gpu/drm/xe/Makefile
> @@ -299,6 +299,8 @@ ifeq ($(CONFIG_DEBUG_FS),y)
> i915-display/intel_pipe_crc.o
> endif
>
> +xe-$(CONFIG_PERF_EVENTS) += xe_pmu.o
> +
> obj-$(CONFIG_DRM_XE) += xe.o
> obj-$(CONFIG_DRM_XE_KUNIT_TEST) += tests/
>
> diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
> index 0e2dd691bdae..89463cf7cc2c 100644
> --- a/drivers/gpu/drm/xe/xe_device.c
> +++ b/drivers/gpu/drm/xe/xe_device.c
> @@ -759,6 +759,9 @@ int xe_device_probe(struct xe_device *xe)
> for_each_gt(gt, xe, id)
> xe_gt_sanitize_freq(gt);
>
> + if (!IS_SRIOV_VF(xe))
> + xe_pmu_register(&xe->pmu);
> +
> return devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
>
> err_fini_display:
> @@ -803,6 +806,9 @@ void xe_device_remove(struct xe_device *xe)
>
> xe_heci_gsc_fini(xe);
>
> + if (!IS_SRIOV_VF(xe))
> + xe_pmu_unregister(&xe->pmu);
> +
> for_each_gt(gt, xe, id)
> xe_gt_remove(gt);
> }
> diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
> index bccca63c8a48..0cb8d650135a 100644
> --- a/drivers/gpu/drm/xe/xe_device_types.h
> +++ b/drivers/gpu/drm/xe/xe_device_types.h
> @@ -18,6 +18,7 @@
> #include "xe_memirq_types.h"
> #include "xe_oa.h"
> #include "xe_platform_types.h"
> +#include "xe_pmu.h"
> #include "xe_pt_types.h"
> #include "xe_sriov_types.h"
> #include "xe_step_types.h"
> @@ -509,6 +510,9 @@ struct xe_device {
> int mode;
> } wedged;
>
> + /** @pmu: performance monitoring unit */
> + struct xe_pmu pmu;
> +
> #ifdef TEST_VM_OPS_ERROR
> /**
> * @vm_inject_error_position: inject errors at different places in VM
> diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
> index 77ce9f9ca7a5..1bf2bf8447c0 100644
> --- a/drivers/gpu/drm/xe/xe_module.c
> +++ b/drivers/gpu/drm/xe/xe_module.c
> @@ -14,6 +14,7 @@
> #include "xe_hw_fence.h"
> #include "xe_pci.h"
> #include "xe_pm.h"
> +#include "xe_pmu.h"
> #include "xe_observation.h"
> #include "xe_sched_job.h"
>
> @@ -96,6 +97,10 @@ static const struct init_funcs init_funcs[] = {
> .init = xe_sched_job_module_init,
> .exit = xe_sched_job_module_exit,
> },
> + {
> + .init = xe_pmu_init,
> + .exit = xe_pmu_exit,
> + },
> {
> .init = xe_register_pci_driver,
> .exit = xe_unregister_pci_driver,
> diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
> new file mode 100644
> index 000000000000..7ce66c022e27
> --- /dev/null
> +++ b/drivers/gpu/drm/xe/xe_pmu.c
> @@ -0,0 +1,577 @@
> +// SPDX-License-Identifier: MIT
> +/*
> + * Copyright © 2024 Intel Corporation
> + */
> +
> +#include <drm/drm_drv.h>
> +#include <drm/drm_managed.h>
> +#include <drm/xe_drm.h>
> +
> +#include "regs/xe_gt_regs.h"
> +#include "xe_device.h"
> +#include "xe_force_wake.h"
> +#include "xe_gt_clock.h"
> +#include "xe_mmio.h"
> +#include "xe_macros.h"
> +#include "xe_pm.h"
> +
> +/**
> + * CPU mask is defined/initialized at a module level. All devices
> + * inside this module share this mask.
> + */
> +static cpumask_t xe_pmu_cpumask;
> +static unsigned int xe_pmu_target_cpu = -1;
> +
> +/**
> + * DOC: Xe PMU (Performance Monitoring Unit)
> + *
> + * Expose events/counters like C6 residency and GT frequency to user land.
> + * Perf tool can be used to list these counters from the command line.
> + *
> + * Example commands to list/record supported perf events-
> + *
> + * $ ls -ld /sys/bus/event_source/devices/xe_*
> + * $ ls /sys/bus/event_source/devices/xe_0000_00_02.0/events/
> + *
> + * You can also use the perf tool to grep for a certain event-
> + * $ perf list | grep rc6
> + *
> + * To list a specific event at regular intervals-
> + * $ perf stat -e <event_name> -I <interval>
> + *
> + */
> +
> +static unsigned int config_gt_id(const u64 config)
> +{
> + return config >> __XE_PMU_GT_SHIFT;
> +}
> +
> +static u64 config_counter(const u64 config)
> +{
> + return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
> +}
> +
> +static void xe_pmu_event_destroy(struct perf_event *event)
> +{
> + struct xe_device *xe =
> + container_of(event->pmu, typeof(*xe), pmu.base);
> +
> + drm_WARN_ON(&xe->drm, event->parent);
> +
> + drm_dev_put(&xe->drm);
> +}
> +
> +static int
> +config_status(struct xe_device *xe, u64 config)
> +{
> + unsigned int gt_id = config_gt_id(config);
> +
> + if (gt_id >= XE_MAX_GT_PER_TILE)
> + return -ENOENT;
> +
> + switch (config_counter(config)) {
> + default:
> + return -ENOENT;
> + }
> +
> + return 0;
> +}
> +
> +static int xe_pmu_event_init(struct perf_event *event)
> +{
> + struct xe_device *xe =
> + container_of(event->pmu, typeof(*xe), pmu.base);
> + struct xe_pmu *pmu = &xe->pmu;
> + int ret;
> +
> + if (!pmu->registered)
> + return -ENODEV;
> +
> + if (event->attr.type != event->pmu->type)
> + return -ENOENT;
> +
> + /* unsupported modes and filters */
> + if (event->attr.sample_period) /* no sampling */
> + return -EINVAL;
> +
> + if (has_branch_stack(event))
> + return -EOPNOTSUPP;
> +
> + if (event->cpu < 0)
> + return -EINVAL;
> +
> + /* only allow running on one cpu at a time */
> + if (!cpumask_test_cpu(event->cpu, &xe_pmu_cpumask))
> + return -EINVAL;
> +
> + ret = config_status(xe, event->attr.config);
> + if (ret)
> + return ret;
> +
> + if (!event->parent) {
> + drm_dev_get(&xe->drm);
> + event->destroy = xe_pmu_event_destroy;
> + }
> +
> + return 0;
> +}
> +
> +static u64 __xe_pmu_event_read(struct perf_event *event)
> +{
> + struct xe_device *xe =
> + container_of(event->pmu, typeof(*xe), pmu.base);
> + const unsigned int gt_id = config_gt_id(event->attr.config);
> + const u64 config = event->attr.config;
> + struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
> + u64 val = 0;
> +
> + switch (config_counter(config)) {
> + default:
> + drm_warn(>->tile->xe->drm, "unknown pmu event\n");
> + }
> +
> + return val;
> +}
> +
> +static void xe_pmu_event_read(struct perf_event *event)
> +{
> + struct xe_device *xe =
> + container_of(event->pmu, typeof(*xe), pmu.base);
> + struct hw_perf_event *hwc = &event->hw;
> + struct xe_pmu *pmu = &xe->pmu;
> + u64 prev, new;
> +
> + if (!pmu->registered) {
> + event->hw.state = PERF_HES_STOPPED;
> + return;
> + }
> +again:
> + prev = local64_read(&hwc->prev_count);
> + new = __xe_pmu_event_read(event);
> +
> + if (local64_cmpxchg(&hwc->prev_count, prev, new) != prev)
> + goto again;
> +
> + local64_add(new - prev, &event->count);
> +}
> +
> +static void xe_pmu_enable(struct perf_event *event)
> +{
> + /*
> + * Store the current counter value so we can report the correct delta
> + * for all listeners. Even when the event was already enabled and has
> + * an existing non-zero value.
> + */
> + local64_set(&event->hw.prev_count, __xe_pmu_event_read(event));
> +}
> +
> +static void xe_pmu_event_start(struct perf_event *event, int flags)
> +{
> + struct xe_device *xe =
> + container_of(event->pmu, typeof(*xe), pmu.base);
> + struct xe_pmu *pmu = &xe->pmu;
> +
> + if (!pmu->registered)
> + return;
> +
> + xe_pmu_enable(event);
> + event->hw.state = 0;
> +}
> +
> +static void xe_pmu_event_stop(struct perf_event *event, int flags)
> +{
> + struct xe_device *xe =
> + container_of(event->pmu, typeof(*xe), pmu.base);
> + struct xe_pmu *pmu = &xe->pmu;
> +
> + if (!pmu->registered)
> + goto out;
> +
> + if (flags & PERF_EF_UPDATE)
> + xe_pmu_event_read(event);
> +
> +out:
> + event->hw.state = PERF_HES_STOPPED;
> +}
> +
> +static int xe_pmu_event_add(struct perf_event *event, int flags)
> +{
> + struct xe_device *xe =
> + container_of(event->pmu, typeof(*xe), pmu.base);
> + struct xe_pmu *pmu = &xe->pmu;
> +
> + if (!pmu->registered)
> + return -ENODEV;
> +
> + if (flags & PERF_EF_START)
> + xe_pmu_event_start(event, flags);
> +
> + return 0;
> +}
> +
> +static void xe_pmu_event_del(struct perf_event *event, int flags)
> +{
> + xe_pmu_event_stop(event, PERF_EF_UPDATE);
> +}
> +
> +static int xe_pmu_event_event_idx(struct perf_event *event)
> +{
> + return 0;
> +}
> +
> +struct xe_ext_attribute {
> + struct device_attribute attr;
> + unsigned long val;
> +};
> +
> +static ssize_t xe_pmu_event_show(struct device *dev,
> + struct device_attribute *attr, char *buf)
> +{
> + struct xe_ext_attribute *eattr;
> +
> + eattr = container_of(attr, struct xe_ext_attribute, attr);
> + return sprintf(buf, "config=0x%lx\n", eattr->val);
> +}
> +
> +static ssize_t cpumask_show(struct device *dev,
> + struct device_attribute *attr, char *buf)
> +{
> + return cpumap_print_to_pagebuf(true, buf, &xe_pmu_cpumask);
> +}
> +
> +static DEVICE_ATTR_RO(cpumask);
> +
> +static struct attribute *xe_cpumask_attrs[] = {
> + &dev_attr_cpumask.attr,
> + NULL,
> +};
> +
> +static const struct attribute_group xe_pmu_cpumask_attr_group = {
> + .attrs = xe_cpumask_attrs,
> +};
> +
> +#define __event(__counter, __name, __unit) \
> +{ \
> + .counter = (__counter), \
> + .name = (__name), \
> + .unit = (__unit), \
> +}
> +
> +static struct xe_ext_attribute *
> +add_xe_attr(struct xe_ext_attribute *attr, const char *name, u64 config)
> +{
> + sysfs_attr_init(&attr->attr.attr);
> + attr->attr.attr.name = name;
> + attr->attr.attr.mode = 0444;
> + attr->attr.show = xe_pmu_event_show;
> + attr->val = config;
> +
> + return ++attr;
> +}
> +
> +static struct perf_pmu_events_attr *
> +add_pmu_attr(struct perf_pmu_events_attr *attr, const char *name,
> + const char *str)
> +{
> + sysfs_attr_init(&attr->attr.attr);
> + attr->attr.attr.name = name;
> + attr->attr.attr.mode = 0444;
> + attr->attr.show = perf_event_sysfs_show;
> + attr->event_str = str;
> +
> + return ++attr;
> +}
> +
> +static struct attribute **
> +create_event_attributes(struct xe_pmu *pmu)
> +{
> + struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
> + static const struct {
> + unsigned int counter;
> + const char *name;
> + const char *unit;
> + } events[] = {
> + };
> +
> + struct perf_pmu_events_attr *pmu_attr = NULL, *pmu_iter;
> + struct xe_ext_attribute *xe_attr = NULL, *xe_iter;
> + struct attribute **attr = NULL, **attr_iter;
> + unsigned int count = 0;
> + unsigned int i, j;
> + struct xe_gt *gt;
> +
> + /* Count how many counters we will be exposing. */
> + for_each_gt(gt, xe, j) {
> + for (i = 0; i < ARRAY_SIZE(events); i++) {
> + u64 config = ___XE_PMU_OTHER(j, events[i].counter);
> +
> + if (!config_status(xe, config))
> + count++;
> + }
> + }
> +
> + /* Allocate attribute objects and table. */
> + xe_attr = kcalloc(count, sizeof(*xe_attr), GFP_KERNEL);
> + if (!xe_attr)
> + goto err_alloc;
> +
> + pmu_attr = kcalloc(count, sizeof(*pmu_attr), GFP_KERNEL);
> + if (!pmu_attr)
> + goto err_alloc;
> +
> + /* Max one pointer of each attribute type plus a termination entry. */
> + attr = kcalloc(count * 2 + 1, sizeof(*attr), GFP_KERNEL);
> + if (!attr)
> + goto err_alloc;
> +
> + xe_iter = xe_attr;
> + pmu_iter = pmu_attr;
> + attr_iter = attr;
> +
> + for_each_gt(gt, xe, j) {
> + for (i = 0; i < ARRAY_SIZE(events); i++) {
> + u64 config = ___XE_PMU_OTHER(j, events[i].counter);
> + char *str;
> +
> + if (config_status(xe, config))
> + continue;
> +
> + str = kasprintf(GFP_KERNEL, "%s-gt%u",
> + events[i].name, j);
> + if (!str)
> + goto err;
> +
> + *attr_iter++ = &xe_iter->attr.attr;
> + xe_iter = add_xe_attr(xe_iter, str, config);
> +
> + if (events[i].unit) {
> + str = kasprintf(GFP_KERNEL, "%s-gt%u.unit",
> + events[i].name, j);
> + if (!str)
> + goto err;
> +
> + *attr_iter++ = &pmu_iter->attr.attr;
> + pmu_iter = add_pmu_attr(pmu_iter, str,
> + events[i].unit);
> + }
> + }
> + }
> +
> + pmu->xe_attr = xe_attr;
> + pmu->pmu_attr = pmu_attr;
> +
> + return attr;
> +
> +err:
> + for (attr_iter = attr; *attr_iter; attr_iter++)
> + kfree((*attr_iter)->name);
> +
> +err_alloc:
> + kfree(attr);
> + kfree(xe_attr);
> + kfree(pmu_attr);
> +
> + return NULL;
> +}
> +
> +static void free_event_attributes(struct xe_pmu *pmu)
> +{
> + struct attribute **attr_iter = pmu->events_attr_group.attrs;
> +
> + for (; *attr_iter; attr_iter++)
> + kfree((*attr_iter)->name);
> +
> + kfree(pmu->events_attr_group.attrs);
> + kfree(pmu->xe_attr);
> + kfree(pmu->pmu_attr);
> +
> + pmu->events_attr_group.attrs = NULL;
> + pmu->xe_attr = NULL;
> + pmu->pmu_attr = NULL;
> +}
> +
> +static int xe_pmu_cpu_online(unsigned int cpu, struct hlist_node *node)
> +{
> + struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
> +
> + /* Select the first online CPU as a designated reader. */
> + if (cpumask_empty(&xe_pmu_cpumask))
> + cpumask_set_cpu(cpu, &xe_pmu_cpumask);
> +
> + return 0;
> +}
> +
> +static int xe_pmu_cpu_offline(unsigned int cpu, struct hlist_node *node)
> +{
> + struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
> + unsigned int target = xe_pmu_target_cpu;
> +
> + /*
> + * Unregistering an instance generates a CPU offline event which we must
> + * ignore to avoid incorrectly modifying the shared xe_pmu_cpumask.
> + */
> + if (!pmu->registered)
> + return 0;
> +
> + if (cpumask_test_and_clear_cpu(cpu, &xe_pmu_cpumask)) {
> + target = cpumask_any_but(topology_sibling_cpumask(cpu), cpu);
> +
> + /* Migrate events if there is a valid target */
> + if (target < nr_cpu_ids) {
> + cpumask_set_cpu(target, &xe_pmu_cpumask);
> + xe_pmu_target_cpu = target;
> + }
> + }
> +
> + if (target < nr_cpu_ids && target != pmu->cpuhp.cpu) {
> + perf_pmu_migrate_context(&pmu->base, cpu, target);
> + pmu->cpuhp.cpu = target;
> + }
> +
> + return 0;
> +}
> +
> +static enum cpuhp_state cpuhp_state = CPUHP_INVALID;
> +
> +/**
> + * xe_pmu_init() - Setup CPU hotplug state/callbacks for Xe PMU
> + *
> + * Returns: 0 if successful, else error code
> + */
> +int xe_pmu_init(void)
> +{
> + int ret;
> +
> + ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
> + "perf/x86/intel/xe:online",
> + xe_pmu_cpu_online,
> + xe_pmu_cpu_offline);
> + if (ret < 0)
> + pr_notice("Failed to setup cpuhp state for xe PMU! (%d)\n",
> + ret);
> + else
> + cpuhp_state = ret;
> +
> + return 0;
> +}
> +
> +/**
> + * xe_pmu_exit() - Remove CPU hotplug state/callbacks for Xe PMU
> + */
> +void xe_pmu_exit(void)
> +{
> + if (cpuhp_state != CPUHP_INVALID)
> + cpuhp_remove_multi_state(cpuhp_state);
> +}
> +
> +static int xe_pmu_register_cpuhp_state(struct xe_pmu *pmu)
> +{
> + if (cpuhp_state == CPUHP_INVALID)
> + return -EINVAL;
> +
> + return cpuhp_state_add_instance(cpuhp_state, &pmu->cpuhp.node);
> +}
> +
> +static void xe_pmu_unregister_cpuhp_state(struct xe_pmu *pmu)
> +{
> + cpuhp_state_remove_instance(cpuhp_state, &pmu->cpuhp.node);
> +}
> +
> +/**
> + * xe_pmu_unregister() - Remove/cleanup PMU registration
> + */
> +void xe_pmu_unregister(void *arg)
> +{
> + struct xe_pmu *pmu = arg;
> +
> + if (!pmu->registered)
> + return;
> +
> + pmu->registered = false;
> +
> + xe_pmu_unregister_cpuhp_state(pmu);
> +
> + perf_pmu_unregister(&pmu->base);
> + kfree(pmu->base.attr_groups);
> + kfree(pmu->name);
> + free_event_attributes(pmu);
> +}
> +
> +/**
> + * xe_pmu_register() - Define basic PMU properties for Xe and add event callbacks.
> + *
> + */
> +void xe_pmu_register(struct xe_pmu *pmu)
> +{
> + struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
> + const struct attribute_group *attr_groups[] = {
> + &pmu->events_attr_group,
> + &xe_pmu_cpumask_attr_group,
> + NULL
> + };
> +
> + int ret = -ENOMEM;
> +
> + spin_lock_init(&pmu->lock);
> + pmu->cpuhp.cpu = -1;
> +
> + pmu->name = kasprintf(GFP_KERNEL,
> + "xe_%s",
> + dev_name(xe->drm.dev));
> + if (pmu->name) {
> + /* tools/perf reserves colons as special. */
> + strreplace((char *)pmu->name, ':', '_');
> + }
> +
> + if (!pmu->name)
> + goto err;
> +
> + pmu->events_attr_group.name = "events";
> + pmu->events_attr_group.attrs = create_event_attributes(pmu);
> + if (!pmu->events_attr_group.attrs)
> + goto err_name;
> +
> + pmu->base.attr_groups = kmemdup(attr_groups, sizeof(attr_groups),
> + GFP_KERNEL);
> + if (!pmu->base.attr_groups)
> + goto err_attr;
> +
> + pmu->base.module = THIS_MODULE;
> + pmu->base.task_ctx_nr = perf_invalid_context;
> + pmu->base.event_init = xe_pmu_event_init;
> + pmu->base.add = xe_pmu_event_add;
> + pmu->base.del = xe_pmu_event_del;
> + pmu->base.start = xe_pmu_event_start;
> + pmu->base.stop = xe_pmu_event_stop;
> + pmu->base.read = xe_pmu_event_read;
> + pmu->base.event_idx = xe_pmu_event_event_idx;
> +
> + ret = perf_pmu_register(&pmu->base, pmu->name, -1);
> + if (ret)
> + goto err_groups;
> +
> + ret = xe_pmu_register_cpuhp_state(pmu);
> + if (ret)
> + goto err_unreg;
> +
> + ret = devm_add_action_or_reset(xe->drm.dev, xe_pmu_unregister, pmu);
> + if (ret)
> + goto err_cpuhp;
> +
> + pmu->registered = true;
> +
> + return;
> +
> +err_cpuhp:
> + xe_pmu_unregister_cpuhp_state(pmu);
> +err_unreg:
> + perf_pmu_unregister(&pmu->base);
> +err_groups:
> + kfree(pmu->base.attr_groups);
> +err_attr:
> + free_event_attributes(pmu);
> +err_name:
> + kfree(pmu->name);
> +err:
> + drm_notice(&xe->drm, "Failed to register PMU!\n");
> +}
> diff --git a/drivers/gpu/drm/xe/xe_pmu.h b/drivers/gpu/drm/xe/xe_pmu.h
> new file mode 100644
> index 000000000000..d07e5dfdfec0
> --- /dev/null
> +++ b/drivers/gpu/drm/xe/xe_pmu.h
> @@ -0,0 +1,26 @@
> +/* SPDX-License-Identifier: MIT */
> +/*
> + * Copyright © 2024 Intel Corporation
> + */
> +
> +#ifndef _XE_PMU_H_
> +#define _XE_PMU_H_
> +
> +#include "xe_pmu_types.h"
> +
> +struct xe_gt;
> +
> +#if IS_ENABLED(CONFIG_PERF_EVENTS)
> +int xe_pmu_init(void);
> +void xe_pmu_exit(void);
> +void xe_pmu_register(struct xe_pmu *pmu);
> +void xe_pmu_unregister(void *arg);
> +#else
> +static inline int xe_pmu_init(void) { return 0; }
> +static inline void xe_pmu_exit(void) {}
> +static inline void xe_pmu_register(struct xe_pmu *pmu) {}
> +static inline void xe_pmu_unregister(void *arg) {}
> +#endif
> +
> +#endif
> +
> diff --git a/drivers/gpu/drm/xe/xe_pmu_types.h b/drivers/gpu/drm/xe/xe_pmu_types.h
> new file mode 100644
> index 000000000000..4da96b8fadd1
> --- /dev/null
> +++ b/drivers/gpu/drm/xe/xe_pmu_types.h
> @@ -0,0 +1,70 @@
> +/* SPDX-License-Identifier: MIT */
> +/*
> + * Copyright © 2024 Intel Corporation
> + */
> +
> +#ifndef _XE_PMU_TYPES_H_
> +#define _XE_PMU_TYPES_H_
> +
> +#include <linux/perf_event.h>
> +#include <linux/spinlock_types.h>
> +
> +enum {
> + __XE_NUM_PMU_SAMPLERS
> +};
> +
> +#define XE_PMU_MAX_GT 2
> +
> +/*
> + * Top bits of every counter are GT id.
> + */
> +#define __XE_PMU_GT_SHIFT (56)
> +
> +#define ___XE_PMU_OTHER(gt, x) \
> + (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
> +
> +struct xe_pmu {
> + /**
> + * @cpuhp: Struct used for CPU hotplug handling.
> + */
> + struct {
> + struct hlist_node node;
> + unsigned int cpu;
> + } cpuhp;
> + /**
> + * @base: PMU base.
> + */
> + struct pmu base;
> + /**
> + * @registered: PMU is registered and not in the unregistering process.
> + */
> + bool registered;
> + /**
> + * @name: Name as registered with perf core.
> + */
> + const char *name;
> + /**
> + * @lock: Lock protecting enable mask and ref count handling.
> + */
> + spinlock_t lock;
> + /**
> + * @sample: Current and previous (raw) counters.
> + *
> + * These counters are updated when the device is awake.
> + */
> + u64 sample[XE_PMU_MAX_GT][__XE_NUM_PMU_SAMPLERS];
> + /**
> + * @events_attr_group: Device events attribute group.
> + */
> + struct attribute_group events_attr_group;
> + /**
> + * @xe_attr: Memory block holding device attributes.
> + */
> + void *xe_attr;
> + /**
> + * @pmu_attr: Memory block holding device attributes.
> + */
> + void *pmu_attr;
> +};
> +
> +#endif
> --
> 2.38.1
>
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH 1/3] drm/xe/pmu: Enable PMU interface
2024-11-12 20:51 [PATCH 0/3] drm/xe/pmu: PMU interface for Xe Vinay Belgaumkar
@ 2024-11-12 20:51 ` Vinay Belgaumkar
0 siblings, 0 replies; 19+ messages in thread
From: Vinay Belgaumkar @ 2024-11-12 20:51 UTC (permalink / raw)
To: intel-xe
Cc: Aravind Iddamsetty, Bommu Krishnaiah, Vinay Belgaumkar,
Riana Tauro, Rodrigo Vivi
From: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
Basic PMU enabling patch. Setup the basic framework
for adding events/timers. This patch was previously
reviewed here -
https://patchwork.freedesktop.org/series/119504/
The pmu base implementation is still from the
i915 driver.
v2: Review comments(Rodrigo) and do not init pmu for VFs
as they don't have access to freq and c6 residency anyways.
v3: Fix kunit issue, move xe_pmu entry in Makefile (Jani) and
move drm uapi definitions (Lucas)
v4: Adapt Lucas's recent PMU fixes for i915
v5: Fix some kernel doc issues
Co-developed-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
Co-developed-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
Signed-off-by: Riana Tauro <riana.tauro@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
---
drivers/gpu/drm/xe/Makefile | 2 +
drivers/gpu/drm/xe/xe_device.c | 6 +
drivers/gpu/drm/xe/xe_device_types.h | 4 +
drivers/gpu/drm/xe/xe_module.c | 5 +
drivers/gpu/drm/xe/xe_pmu.c | 579 +++++++++++++++++++++++++++
drivers/gpu/drm/xe/xe_pmu.h | 26 ++
drivers/gpu/drm/xe/xe_pmu_types.h | 70 ++++
7 files changed, 692 insertions(+)
create mode 100644 drivers/gpu/drm/xe/xe_pmu.c
create mode 100644 drivers/gpu/drm/xe/xe_pmu.h
create mode 100644 drivers/gpu/drm/xe/xe_pmu_types.h
diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
index a93e6fcc0ad9..c231ecaf86b8 100644
--- a/drivers/gpu/drm/xe/Makefile
+++ b/drivers/gpu/drm/xe/Makefile
@@ -299,6 +299,8 @@ ifeq ($(CONFIG_DEBUG_FS),y)
i915-display/intel_pipe_crc.o
endif
+xe-$(CONFIG_PERF_EVENTS) += xe_pmu.o
+
obj-$(CONFIG_DRM_XE) += xe.o
obj-$(CONFIG_DRM_XE_KUNIT_TEST) += tests/
diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
index 0e2dd691bdae..89463cf7cc2c 100644
--- a/drivers/gpu/drm/xe/xe_device.c
+++ b/drivers/gpu/drm/xe/xe_device.c
@@ -759,6 +759,9 @@ int xe_device_probe(struct xe_device *xe)
for_each_gt(gt, xe, id)
xe_gt_sanitize_freq(gt);
+ if (!IS_SRIOV_VF(xe))
+ xe_pmu_register(&xe->pmu);
+
return devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
err_fini_display:
@@ -803,6 +806,9 @@ void xe_device_remove(struct xe_device *xe)
xe_heci_gsc_fini(xe);
+ if (!IS_SRIOV_VF(xe))
+ xe_pmu_unregister(&xe->pmu);
+
for_each_gt(gt, xe, id)
xe_gt_remove(gt);
}
diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
index bccca63c8a48..0cb8d650135a 100644
--- a/drivers/gpu/drm/xe/xe_device_types.h
+++ b/drivers/gpu/drm/xe/xe_device_types.h
@@ -18,6 +18,7 @@
#include "xe_memirq_types.h"
#include "xe_oa.h"
#include "xe_platform_types.h"
+#include "xe_pmu.h"
#include "xe_pt_types.h"
#include "xe_sriov_types.h"
#include "xe_step_types.h"
@@ -509,6 +510,9 @@ struct xe_device {
int mode;
} wedged;
+ /** @pmu: performance monitoring unit */
+ struct xe_pmu pmu;
+
#ifdef TEST_VM_OPS_ERROR
/**
* @vm_inject_error_position: inject errors at different places in VM
diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
index 77ce9f9ca7a5..1bf2bf8447c0 100644
--- a/drivers/gpu/drm/xe/xe_module.c
+++ b/drivers/gpu/drm/xe/xe_module.c
@@ -14,6 +14,7 @@
#include "xe_hw_fence.h"
#include "xe_pci.h"
#include "xe_pm.h"
+#include "xe_pmu.h"
#include "xe_observation.h"
#include "xe_sched_job.h"
@@ -96,6 +97,10 @@ static const struct init_funcs init_funcs[] = {
.init = xe_sched_job_module_init,
.exit = xe_sched_job_module_exit,
},
+ {
+ .init = xe_pmu_init,
+ .exit = xe_pmu_exit,
+ },
{
.init = xe_register_pci_driver,
.exit = xe_unregister_pci_driver,
diff --git a/drivers/gpu/drm/xe/xe_pmu.c b/drivers/gpu/drm/xe/xe_pmu.c
new file mode 100644
index 000000000000..2bd98c91ab0f
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_pmu.c
@@ -0,0 +1,579 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2024 Intel Corporation
+ */
+
+#include <drm/drm_drv.h>
+#include <drm/drm_managed.h>
+#include <drm/xe_drm.h>
+
+#include "regs/xe_gt_regs.h"
+#include "xe_device.h"
+#include "xe_force_wake.h"
+#include "xe_gt_clock.h"
+#include "xe_mmio.h"
+#include "xe_macros.h"
+#include "xe_pm.h"
+
+/*
+ * CPU mask is defined/initialized at a module level. All devices
+ * inside this module share this mask.
+ */
+static cpumask_t xe_pmu_cpumask;
+static unsigned int xe_pmu_target_cpu = -1;
+
+/**
+ * DOC: Xe PMU (Performance Monitoring Unit)
+ *
+ * Expose events/counters like C6 residency and GT frequency to user land.
+ * Perf tool can be used to list these counters from the command line.
+ *
+ * Example commands to list/record supported perf events-
+ *
+ * $ ls -ld /sys/bus/event_source/devices/xe_*
+ * $ ls /sys/bus/event_source/devices/xe_0000_00_02.0/events/
+ *
+ * You can also use the perf tool to grep for a certain event-
+ * $ perf list | grep rc6
+ *
+ * To list a specific event at regular intervals-
+ * $ perf stat -e <event_name> -I <interval>
+ *
+ */
+
+static unsigned int config_gt_id(const u64 config)
+{
+ return config >> __XE_PMU_GT_SHIFT;
+}
+
+static u64 config_counter(const u64 config)
+{
+ return config & ~(~0ULL << __XE_PMU_GT_SHIFT);
+}
+
+static void xe_pmu_event_destroy(struct perf_event *event)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+
+ drm_WARN_ON(&xe->drm, event->parent);
+
+ drm_dev_put(&xe->drm);
+}
+
+static int
+config_status(struct xe_device *xe, u64 config)
+{
+ unsigned int gt_id = config_gt_id(config);
+
+ if (gt_id >= XE_MAX_GT_PER_TILE)
+ return -ENOENT;
+
+ switch (config_counter(config)) {
+ default:
+ return -ENOENT;
+ }
+
+ return 0;
+}
+
+static int xe_pmu_event_init(struct perf_event *event)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct xe_pmu *pmu = &xe->pmu;
+ int ret;
+
+ if (!pmu->registered)
+ return -ENODEV;
+
+ if (event->attr.type != event->pmu->type)
+ return -ENOENT;
+
+ /* unsupported modes and filters */
+ if (event->attr.sample_period) /* no sampling */
+ return -EINVAL;
+
+ if (has_branch_stack(event))
+ return -EOPNOTSUPP;
+
+ if (event->cpu < 0)
+ return -EINVAL;
+
+ /* only allow running on one cpu at a time */
+ if (!cpumask_test_cpu(event->cpu, &xe_pmu_cpumask))
+ return -EINVAL;
+
+ ret = config_status(xe, event->attr.config);
+ if (ret)
+ return ret;
+
+ if (!event->parent) {
+ drm_dev_get(&xe->drm);
+ event->destroy = xe_pmu_event_destroy;
+ }
+
+ return 0;
+}
+
+static u64 __xe_pmu_event_read(struct perf_event *event)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ const unsigned int gt_id = config_gt_id(event->attr.config);
+ const u64 config = event->attr.config;
+ struct xe_gt *gt = xe_device_get_gt(xe, gt_id);
+ u64 val = 0;
+
+ switch (config_counter(config)) {
+ default:
+ drm_warn(>->tile->xe->drm, "unknown pmu event\n");
+ }
+
+ return val;
+}
+
+static void xe_pmu_event_read(struct perf_event *event)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct hw_perf_event *hwc = &event->hw;
+ struct xe_pmu *pmu = &xe->pmu;
+ u64 prev, new;
+
+ if (!pmu->registered) {
+ event->hw.state = PERF_HES_STOPPED;
+ return;
+ }
+again:
+ prev = local64_read(&hwc->prev_count);
+ new = __xe_pmu_event_read(event);
+
+ if (local64_cmpxchg(&hwc->prev_count, prev, new) != prev)
+ goto again;
+
+ local64_add(new - prev, &event->count);
+}
+
+static void xe_pmu_enable(struct perf_event *event)
+{
+ /*
+ * Store the current counter value so we can report the correct delta
+ * for all listeners. Even when the event was already enabled and has
+ * an existing non-zero value.
+ */
+ local64_set(&event->hw.prev_count, __xe_pmu_event_read(event));
+}
+
+static void xe_pmu_event_start(struct perf_event *event, int flags)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct xe_pmu *pmu = &xe->pmu;
+
+ if (!pmu->registered)
+ return;
+
+ xe_pmu_enable(event);
+ event->hw.state = 0;
+}
+
+static void xe_pmu_event_stop(struct perf_event *event, int flags)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct xe_pmu *pmu = &xe->pmu;
+
+ if (!pmu->registered)
+ goto out;
+
+ if (flags & PERF_EF_UPDATE)
+ xe_pmu_event_read(event);
+
+out:
+ event->hw.state = PERF_HES_STOPPED;
+}
+
+static int xe_pmu_event_add(struct perf_event *event, int flags)
+{
+ struct xe_device *xe =
+ container_of(event->pmu, typeof(*xe), pmu.base);
+ struct xe_pmu *pmu = &xe->pmu;
+
+ if (!pmu->registered)
+ return -ENODEV;
+
+ if (flags & PERF_EF_START)
+ xe_pmu_event_start(event, flags);
+
+ return 0;
+}
+
+static void xe_pmu_event_del(struct perf_event *event, int flags)
+{
+ xe_pmu_event_stop(event, PERF_EF_UPDATE);
+}
+
+static int xe_pmu_event_event_idx(struct perf_event *event)
+{
+ return 0;
+}
+
+struct xe_ext_attribute {
+ struct device_attribute attr;
+ unsigned long val;
+};
+
+static ssize_t xe_pmu_event_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct xe_ext_attribute *eattr;
+
+ eattr = container_of(attr, struct xe_ext_attribute, attr);
+ return sprintf(buf, "config=0x%lx\n", eattr->val);
+}
+
+static ssize_t cpumask_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ return cpumap_print_to_pagebuf(true, buf, &xe_pmu_cpumask);
+}
+
+static DEVICE_ATTR_RO(cpumask);
+
+static struct attribute *xe_cpumask_attrs[] = {
+ &dev_attr_cpumask.attr,
+ NULL,
+};
+
+static const struct attribute_group xe_pmu_cpumask_attr_group = {
+ .attrs = xe_cpumask_attrs,
+};
+
+#define __event(__counter, __name, __unit) \
+{ \
+ .counter = (__counter), \
+ .name = (__name), \
+ .unit = (__unit), \
+}
+
+static struct xe_ext_attribute *
+add_xe_attr(struct xe_ext_attribute *attr, const char *name, u64 config)
+{
+ sysfs_attr_init(&attr->attr.attr);
+ attr->attr.attr.name = name;
+ attr->attr.attr.mode = 0444;
+ attr->attr.show = xe_pmu_event_show;
+ attr->val = config;
+
+ return ++attr;
+}
+
+static struct perf_pmu_events_attr *
+add_pmu_attr(struct perf_pmu_events_attr *attr, const char *name,
+ const char *str)
+{
+ sysfs_attr_init(&attr->attr.attr);
+ attr->attr.attr.name = name;
+ attr->attr.attr.mode = 0444;
+ attr->attr.show = perf_event_sysfs_show;
+ attr->event_str = str;
+
+ return ++attr;
+}
+
+static struct attribute **
+create_event_attributes(struct xe_pmu *pmu)
+{
+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
+ static const struct {
+ unsigned int counter;
+ const char *name;
+ const char *unit;
+ } events[] = {
+ };
+
+ struct perf_pmu_events_attr *pmu_attr = NULL, *pmu_iter;
+ struct xe_ext_attribute *xe_attr = NULL, *xe_iter;
+ struct attribute **attr = NULL, **attr_iter;
+ unsigned int count = 0;
+ unsigned int i, j;
+ struct xe_gt *gt;
+
+ /* Count how many counters we will be exposing. */
+ for_each_gt(gt, xe, j) {
+ for (i = 0; i < ARRAY_SIZE(events); i++) {
+ u64 config = ___XE_PMU_OTHER(j, events[i].counter);
+
+ if (!config_status(xe, config))
+ count++;
+ }
+ }
+
+ /* Allocate attribute objects and table. */
+ xe_attr = kcalloc(count, sizeof(*xe_attr), GFP_KERNEL);
+ if (!xe_attr)
+ goto err_alloc;
+
+ pmu_attr = kcalloc(count, sizeof(*pmu_attr), GFP_KERNEL);
+ if (!pmu_attr)
+ goto err_alloc;
+
+ /* Max one pointer of each attribute type plus a termination entry. */
+ attr = kcalloc(count * 2 + 1, sizeof(*attr), GFP_KERNEL);
+ if (!attr)
+ goto err_alloc;
+
+ xe_iter = xe_attr;
+ pmu_iter = pmu_attr;
+ attr_iter = attr;
+
+ for_each_gt(gt, xe, j) {
+ for (i = 0; i < ARRAY_SIZE(events); i++) {
+ u64 config = ___XE_PMU_OTHER(j, events[i].counter);
+ char *str;
+
+ if (config_status(xe, config))
+ continue;
+
+ str = kasprintf(GFP_KERNEL, "%s-gt%u",
+ events[i].name, j);
+ if (!str)
+ goto err;
+
+ *attr_iter++ = &xe_iter->attr.attr;
+ xe_iter = add_xe_attr(xe_iter, str, config);
+
+ if (events[i].unit) {
+ str = kasprintf(GFP_KERNEL, "%s-gt%u.unit",
+ events[i].name, j);
+ if (!str)
+ goto err;
+
+ *attr_iter++ = &pmu_iter->attr.attr;
+ pmu_iter = add_pmu_attr(pmu_iter, str,
+ events[i].unit);
+ }
+ }
+ }
+
+ pmu->xe_attr = xe_attr;
+ pmu->pmu_attr = pmu_attr;
+
+ return attr;
+
+err:
+ for (attr_iter = attr; *attr_iter; attr_iter++)
+ kfree((*attr_iter)->name);
+
+err_alloc:
+ kfree(attr);
+ kfree(xe_attr);
+ kfree(pmu_attr);
+
+ return NULL;
+}
+
+static void free_event_attributes(struct xe_pmu *pmu)
+{
+ struct attribute **attr_iter = pmu->events_attr_group.attrs;
+
+ for (; *attr_iter; attr_iter++)
+ kfree((*attr_iter)->name);
+
+ kfree(pmu->events_attr_group.attrs);
+ kfree(pmu->xe_attr);
+ kfree(pmu->pmu_attr);
+
+ pmu->events_attr_group.attrs = NULL;
+ pmu->xe_attr = NULL;
+ pmu->pmu_attr = NULL;
+}
+
+static int xe_pmu_cpu_online(unsigned int cpu, struct hlist_node *node)
+{
+ struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
+
+ /* Select the first online CPU as a designated reader. */
+ if (cpumask_empty(&xe_pmu_cpumask))
+ cpumask_set_cpu(cpu, &xe_pmu_cpumask);
+
+ return 0;
+}
+
+static int xe_pmu_cpu_offline(unsigned int cpu, struct hlist_node *node)
+{
+ struct xe_pmu *pmu = hlist_entry_safe(node, typeof(*pmu), cpuhp.node);
+ unsigned int target = xe_pmu_target_cpu;
+
+ /*
+ * Unregistering an instance generates a CPU offline event which we must
+ * ignore to avoid incorrectly modifying the shared xe_pmu_cpumask.
+ */
+ if (!pmu->registered)
+ return 0;
+
+ if (cpumask_test_and_clear_cpu(cpu, &xe_pmu_cpumask)) {
+ target = cpumask_any_but(topology_sibling_cpumask(cpu), cpu);
+
+ /* Migrate events if there is a valid target */
+ if (target < nr_cpu_ids) {
+ cpumask_set_cpu(target, &xe_pmu_cpumask);
+ xe_pmu_target_cpu = target;
+ }
+ }
+
+ if (target < nr_cpu_ids && target != pmu->cpuhp.cpu) {
+ perf_pmu_migrate_context(&pmu->base, cpu, target);
+ pmu->cpuhp.cpu = target;
+ }
+
+ return 0;
+}
+
+static enum cpuhp_state cpuhp_state = CPUHP_INVALID;
+
+/**
+ * xe_pmu_init() - Setup CPU hotplug state/callbacks for Xe PMU
+ *
+ * Returns: 0 if successful, else error code
+ */
+int xe_pmu_init(void)
+{
+ int ret;
+
+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
+ "perf/x86/intel/xe:online",
+ xe_pmu_cpu_online,
+ xe_pmu_cpu_offline);
+ if (ret < 0)
+ pr_notice("Failed to setup cpuhp state for xe PMU! (%d)\n",
+ ret);
+ else
+ cpuhp_state = ret;
+
+ return 0;
+}
+
+/**
+ * xe_pmu_exit() - Remove CPU hotplug state/callbacks for Xe PMU
+ */
+void xe_pmu_exit(void)
+{
+ if (cpuhp_state != CPUHP_INVALID)
+ cpuhp_remove_multi_state(cpuhp_state);
+}
+
+static int xe_pmu_register_cpuhp_state(struct xe_pmu *pmu)
+{
+ if (cpuhp_state == CPUHP_INVALID)
+ return -EINVAL;
+
+ return cpuhp_state_add_instance(cpuhp_state, &pmu->cpuhp.node);
+}
+
+static void xe_pmu_unregister_cpuhp_state(struct xe_pmu *pmu)
+{
+ cpuhp_state_remove_instance(cpuhp_state, &pmu->cpuhp.node);
+}
+
+/**
+ * xe_pmu_unregister() - Remove/cleanup PMU registration
+ * @arg: Ptr to pmu
+ */
+void xe_pmu_unregister(void *arg)
+{
+ struct xe_pmu *pmu = arg;
+
+ if (!pmu->registered)
+ return;
+
+ pmu->registered = false;
+
+ xe_pmu_unregister_cpuhp_state(pmu);
+
+ perf_pmu_unregister(&pmu->base);
+ kfree(pmu->base.attr_groups);
+ kfree(pmu->name);
+ free_event_attributes(pmu);
+}
+
+/**
+ * xe_pmu_register() - Define basic PMU properties for Xe and add event callbacks.
+ * @pmu: the PMU object
+ *
+ */
+void xe_pmu_register(struct xe_pmu *pmu)
+{
+ struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
+ const struct attribute_group *attr_groups[] = {
+ &pmu->events_attr_group,
+ &xe_pmu_cpumask_attr_group,
+ NULL
+ };
+
+ int ret = -ENOMEM;
+
+ spin_lock_init(&pmu->lock);
+ pmu->cpuhp.cpu = -1;
+
+ pmu->name = kasprintf(GFP_KERNEL,
+ "xe_%s",
+ dev_name(xe->drm.dev));
+ if (pmu->name) {
+ /* tools/perf reserves colons as special. */
+ strreplace((char *)pmu->name, ':', '_');
+ }
+
+ if (!pmu->name)
+ goto err;
+
+ pmu->events_attr_group.name = "events";
+ pmu->events_attr_group.attrs = create_event_attributes(pmu);
+ if (!pmu->events_attr_group.attrs)
+ goto err_name;
+
+ pmu->base.attr_groups = kmemdup(attr_groups, sizeof(attr_groups),
+ GFP_KERNEL);
+ if (!pmu->base.attr_groups)
+ goto err_attr;
+
+ pmu->base.module = THIS_MODULE;
+ pmu->base.task_ctx_nr = perf_invalid_context;
+ pmu->base.event_init = xe_pmu_event_init;
+ pmu->base.add = xe_pmu_event_add;
+ pmu->base.del = xe_pmu_event_del;
+ pmu->base.start = xe_pmu_event_start;
+ pmu->base.stop = xe_pmu_event_stop;
+ pmu->base.read = xe_pmu_event_read;
+ pmu->base.event_idx = xe_pmu_event_event_idx;
+
+ ret = perf_pmu_register(&pmu->base, pmu->name, -1);
+ if (ret)
+ goto err_groups;
+
+ ret = xe_pmu_register_cpuhp_state(pmu);
+ if (ret)
+ goto err_unreg;
+
+ ret = devm_add_action_or_reset(xe->drm.dev, xe_pmu_unregister, pmu);
+ if (ret)
+ goto err_cpuhp;
+
+ pmu->registered = true;
+
+ return;
+
+err_cpuhp:
+ xe_pmu_unregister_cpuhp_state(pmu);
+err_unreg:
+ perf_pmu_unregister(&pmu->base);
+err_groups:
+ kfree(pmu->base.attr_groups);
+err_attr:
+ free_event_attributes(pmu);
+err_name:
+ kfree(pmu->name);
+err:
+ drm_notice(&xe->drm, "Failed to register PMU!\n");
+}
diff --git a/drivers/gpu/drm/xe/xe_pmu.h b/drivers/gpu/drm/xe/xe_pmu.h
new file mode 100644
index 000000000000..d07e5dfdfec0
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_pmu.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2024 Intel Corporation
+ */
+
+#ifndef _XE_PMU_H_
+#define _XE_PMU_H_
+
+#include "xe_pmu_types.h"
+
+struct xe_gt;
+
+#if IS_ENABLED(CONFIG_PERF_EVENTS)
+int xe_pmu_init(void);
+void xe_pmu_exit(void);
+void xe_pmu_register(struct xe_pmu *pmu);
+void xe_pmu_unregister(void *arg);
+#else
+static inline int xe_pmu_init(void) { return 0; }
+static inline void xe_pmu_exit(void) {}
+static inline void xe_pmu_register(struct xe_pmu *pmu) {}
+static inline void xe_pmu_unregister(void *arg) {}
+#endif
+
+#endif
+
diff --git a/drivers/gpu/drm/xe/xe_pmu_types.h b/drivers/gpu/drm/xe/xe_pmu_types.h
new file mode 100644
index 000000000000..4da96b8fadd1
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_pmu_types.h
@@ -0,0 +1,70 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2024 Intel Corporation
+ */
+
+#ifndef _XE_PMU_TYPES_H_
+#define _XE_PMU_TYPES_H_
+
+#include <linux/perf_event.h>
+#include <linux/spinlock_types.h>
+
+enum {
+ __XE_NUM_PMU_SAMPLERS
+};
+
+#define XE_PMU_MAX_GT 2
+
+/*
+ * Top bits of every counter are GT id.
+ */
+#define __XE_PMU_GT_SHIFT (56)
+
+#define ___XE_PMU_OTHER(gt, x) \
+ (((__u64)(x)) | ((__u64)(gt) << __XE_PMU_GT_SHIFT))
+
+struct xe_pmu {
+ /**
+ * @cpuhp: Struct used for CPU hotplug handling.
+ */
+ struct {
+ struct hlist_node node;
+ unsigned int cpu;
+ } cpuhp;
+ /**
+ * @base: PMU base.
+ */
+ struct pmu base;
+ /**
+ * @registered: PMU is registered and not in the unregistering process.
+ */
+ bool registered;
+ /**
+ * @name: Name as registered with perf core.
+ */
+ const char *name;
+ /**
+ * @lock: Lock protecting enable mask and ref count handling.
+ */
+ spinlock_t lock;
+ /**
+ * @sample: Current and previous (raw) counters.
+ *
+ * These counters are updated when the device is awake.
+ */
+ u64 sample[XE_PMU_MAX_GT][__XE_NUM_PMU_SAMPLERS];
+ /**
+ * @events_attr_group: Device events attribute group.
+ */
+ struct attribute_group events_attr_group;
+ /**
+ * @xe_attr: Memory block holding device attributes.
+ */
+ void *xe_attr;
+ /**
+ * @pmu_attr: Memory block holding device attributes.
+ */
+ void *pmu_attr;
+};
+
+#endif
--
2.38.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
end of thread, other threads:[~2024-11-12 20:55 UTC | newest]
Thread overview: 19+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-09-25 23:21 [PATCH 0/3] drm/xe/pmu: PMU interface for Xe Vinay Belgaumkar
2024-09-25 23:21 ` [PATCH 1/3] drm/xe/pmu: Enable PMU interface Vinay Belgaumkar
2024-09-26 9:25 ` Jani Nikula
2024-09-25 23:21 ` [PATCH 2/3] drm/xe/pmu: Add GT C6 events Vinay Belgaumkar
2024-09-25 23:21 ` [PATCH 3/3] drm/xe/pmu: Add GT frequency events Vinay Belgaumkar
2024-09-26 16:57 ` ✓ CI.Patch_applied: success for drm/xe/pmu: PMU interface for Xe Patchwork
2024-09-26 16:57 ` ✗ CI.checkpatch: warning " Patchwork
2024-09-26 16:58 ` ✗ CI.KUnit: failure " Patchwork
-- strict thread matches above, loose matches on Subject: below --
2024-09-27 0:23 [PATCH 0/3] " Vinay Belgaumkar
2024-09-27 0:23 ` [PATCH 1/3] drm/xe/pmu: Enable PMU interface Vinay Belgaumkar
2024-09-27 18:26 ` Rodrigo Vivi
2024-09-27 19:45 ` Belgaumkar, Vinay
2024-10-24 22:58 ` Belgaumkar, Vinay
2024-10-01 21:32 ` Lucas De Marchi
2024-10-01 22:08 ` Belgaumkar, Vinay
2024-10-02 14:51 ` Lucas De Marchi
2024-10-28 19:23 [PATCH 0/3] drm/xe/pmu: PMU interface for Xe Vinay Belgaumkar
2024-10-28 19:24 ` [PATCH 1/3] drm/xe/pmu: Enable PMU interface Vinay Belgaumkar
2024-11-08 18:15 [PATCH 0/3] drm/xe/pmu: PMU interface for Xe Vinay Belgaumkar
2024-11-08 18:15 ` [PATCH 1/3] drm/xe/pmu: Enable PMU interface Vinay Belgaumkar
2024-11-08 22:30 ` Rodrigo Vivi
2024-11-12 20:51 [PATCH 0/3] drm/xe/pmu: PMU interface for Xe Vinay Belgaumkar
2024-11-12 20:51 ` [PATCH 1/3] drm/xe/pmu: Enable PMU interface Vinay Belgaumkar
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).