Re: [PATCH v2] drm/xe: GPU frequency tracing support

[Matthew Brost <matthew.brost@intel.com>]

On Mon, Aug 11, 2025 at 06:13:59PM -0300, Lucas De Marchi wrote:
> On Mon, Aug 11, 2025 at 02:04:14PM +0530, S Sebinraj wrote:
> > A periodic GPU frequency monitoring and tracing program for the
> > Xe driver. The implementation provides periodic sampling of
> > GPU frequency through the Linux ftrace infrastructure.
> > 
> > Key features:
> > - Periodic GPU frequency sampling with configurable intervals
> > - Immediate frequency change reporting via tracepoints
> > - Integration with Linux ftrace subsystem under 'power' events
> > - Per-GT (Graphics Technology) monitoring support
> > - Dedicated workqueue for non-blocking frequency sampling
> > - Configurable via CONFIG_DRM_XE_GPUFREQTRACER kernel option
> > - The monitoring interval can be configured at runtime via the sysfs
> >  (default 5sec).
> > 
> > The sysfs entry is at:
> > /sys/module/xe/parameters/gpufreq_monitoring_interval_ms
> 
> This is actually a module parameter, which is global to all devices xe
> binds to.
> 
> > 
> > The tracepoint is exposed at:
> > /sys/kernel/debug/tracing/events/power/gpu_frequency
> > 
> > Format: {unsigned int state, unsigned int gpu_id}
> > - state: GPU frequency in KHz
> > - gpu_id: GPU clock domain identifier
> > 
> > This enables userspace tools and system monitoring applications to track
> > GPU frequency changes for power management analysis, performance tuning,
> > and debugging purposes.
> > 
> > Signed-off-by: S Sebinraj <s.sebinraj@intel.com>
> > ---
> > drivers/gpu/drm/xe/Kconfig                  |  22 ++
> > drivers/gpu/drm/xe/Makefile                 |   3 +
> > drivers/gpu/drm/xe/xe_device.c              |   7 +
> > drivers/gpu/drm/xe/xe_device_types.h        |   4 +
> > drivers/gpu/drm/xe/xe_gpufreqtracer.c       | 292 ++++++++++++++++++++
> > drivers/gpu/drm/xe/xe_gpufreqtracer.h       |  30 ++
> > drivers/gpu/drm/xe/xe_gpufreqtracer_trace.h |  48 ++++
> > drivers/gpu/drm/xe/xe_module.c              |  12 +
> > drivers/gpu/drm/xe/xe_module.h              |   3 +
> > 9 files changed, 421 insertions(+)
> > create mode 100644 drivers/gpu/drm/xe/xe_gpufreqtracer.c
> > create mode 100644 drivers/gpu/drm/xe/xe_gpufreqtracer.h
> > create mode 100644 drivers/gpu/drm/xe/xe_gpufreqtracer_trace.h
> > 
> > diff --git a/drivers/gpu/drm/xe/Kconfig b/drivers/gpu/drm/xe/Kconfig
> > index 714d5702dfd7..d9d07ec69875 100644
> > --- a/drivers/gpu/drm/xe/Kconfig
> > +++ b/drivers/gpu/drm/xe/Kconfig
> > @@ -129,6 +129,28 @@ config DRM_XE_FORCE_PROBE
> > 
> > 	  Use "!*" to block the probe of the driver for all known devices.
> > 
> > +config DRM_XE_GPUFREQTRACER
> > +	bool "Enable XE GPU frequency tracing"
> > +	depends on DRM_XE && FTRACE
> > +	default n
> > +	help
> > +	  Enable GPU frequency tracing support for Intel XE driver.
> > +	  This adds an ftrace tracepoint that reports GPU frequency changes
> > +	  at periodic boundaries (default 5 secs, configurable via the
> > +	  gpufreq_monitoring_interval_ms module parameter) and
> > +	  on direct frequency change events.
> > +
> > +	  The monitoring interval can be configured at runtime via the sysfs module parameter:
> > +	  /sys/module/xe/parameters/gpufreq_monitoring_interval_ms
> > +
> > +	  The tracepoint will be available at:
> > +	  /sys/kernel/debug/tracing/events/power/gpu_frequency
> > +
> > +	  Format: {unsigned int state, unsigned int gpu_id}
> > +	  Where state is the frequency in KHz and gpu_id is the GPU clock domain.
> > +
> > +	  If unsure, say N.
> > +
> > menu "drm/Xe Debugging"
> > depends on DRM_XE
> > depends on EXPERT
> > diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
> > index 8e0c3412a757..61d46cc4dc25 100644
> > --- a/drivers/gpu/drm/xe/Makefile
> > +++ b/drivers/gpu/drm/xe/Makefile
> > @@ -170,6 +170,9 @@ xe-$(CONFIG_PCI_IOV) += \
> > 	xe_sriov_pf.o \
> > 	xe_sriov_pf_service.o
> > 
> > +# GPU frequency tracer
> > +xe-$(CONFIG_DRM_XE_GPUFREQTRACER) += xe_gpufreqtracer.o
> > +
> > # include helpers for tests even when XE is built-in
> > ifdef CONFIG_DRM_XE_KUNIT_TEST
> > xe-y += tests/xe_kunit_helpers.o
> > diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
> > index 57edbc63da6f..88198ceb519b 100644
> > --- a/drivers/gpu/drm/xe/xe_device.c
> > +++ b/drivers/gpu/drm/xe/xe_device.c
> > @@ -34,6 +34,7 @@
> > #include "xe_exec_queue.h"
> > #include "xe_force_wake.h"
> > #include "xe_ggtt.h"
> > +#include "xe_gpufreqtracer.h"
> > #include "xe_gsc_proxy.h"
> > #include "xe_gt.h"
> > #include "xe_gt_mcr.h"
> > @@ -896,6 +897,12 @@ int xe_device_probe(struct xe_device *xe)
> > 	if (err)
> > 		return err;
> > 
> > +#if IS_ENABLED(CONFIG_DRM_XE_GPUFREQTRACER)
> > +	err = xe_gpufreqtracer_init(xe);
> > +	if (err)
> > +		return err;
> > +#endif
> > +
> > 	err = xe_oa_init(xe);
> > 	if (err)
> > 		return err;
> > diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
> > index 01e8fa0d2f9f..0634dbbbdc3f 100644
> > --- a/drivers/gpu/drm/xe/xe_device_types.h
> > +++ b/drivers/gpu/drm/xe/xe_device_types.h
> > @@ -35,6 +35,7 @@ struct dram_info;
> > struct intel_display;
> > struct intel_dg_nvm_dev;
> > struct xe_ggtt;
> > +struct xe_gpufreqtracer_data;
> > struct xe_i2c;
> > struct xe_pat_ops;
> > struct xe_pxp;
> > @@ -529,6 +530,9 @@ struct xe_device {
> > 	/** @oa: oa observation subsystem */
> > 	struct xe_oa oa;
> > 
> > +	/** @gpufreqtracer_data: GPU frequency tracer data */
> > +	struct xe_gpufreqtracer_data *gpufreqtracer_data;
> > +
> > 	/** @pxp: Encapsulate Protected Xe Path support */
> > 	struct xe_pxp *pxp;
> > 
> > diff --git a/drivers/gpu/drm/xe/xe_gpufreqtracer.c b/drivers/gpu/drm/xe/xe_gpufreqtracer.c
> > new file mode 100644
> > index 000000000000..07eedfb79e93
> > --- /dev/null
> > +++ b/drivers/gpu/drm/xe/xe_gpufreqtracer.c
> > @@ -0,0 +1,292 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright © 2025 Intel Corporation
> > + */
> > +
> > +#include "xe_gpufreqtracer.h"
> > +
> > +#include <linux/workqueue.h>
> > +#include <linux/slab.h>
> > +#include <linux/jiffies.h>
> > +#include <linux/kernel.h>
> > +#include <linux/moduleparam.h>
> > +#include <linux/types.h>
> > +#include <linux/container_of.h>
> > +#include <linux/gfp.h>
> > +#include <linux/atomic.h>
> > +#include <drm/drm_managed.h>
> > +
> > +#include "xe_device.h"
> > +#include "xe_gt.h"
> > +#include "xe_gt_types.h"
> > +#include "xe_guc_pc.h"
> > +#include "xe_module.h"
> > +
> > +/* GPU frequency monitoring interval constants (in milliseconds) */
> > +#define XE_GPUFREQ_MONITORING_MIN_INTERVAL_MS	100
> > +#define XE_GPUFREQ_MONITORING_MAX_INTERVAL_MS	10000
> > +#define XE_GPUFREQ_MONITORING_DEFAULT_INTERVAL_MS	5000
> > +
> > +#define CREATE_TRACE_POINTS
> > +#include "xe_gpufreqtracer_trace.h"
> > +
> > +/**
> > + * struct xe_gpufreqtracer_gt_data - Per-GT frequency monitoring data
> > + * @gt: Reference to the GT
> > + * @delayed_work: Delayed work for periodic monitoring
> > + * @last_frequency: Last reported frequency to avoid duplicate reports
> > + * @monitoring_active: Whether monitoring is currently active
> > + */
> > +struct xe_gpufreqtracer_gt_data {
> > +	struct xe_gt *gt;
> > +	struct delayed_work delayed_work;
> > +	atomic_t last_frequency;
> > +	atomic_t monitoring_active;
> > +};
> > +
> > +/**
> > + * struct xe_gpufreqtracer_data - Per-device frequency tracer data
> > + * @xe: Reference to the XE device
> > + * @gt_data: Array of per-GT monitoring data
> > + */
> > +struct xe_gpufreqtracer_data {
> > +	struct xe_device *xe;
> > +	struct xe_gpufreqtracer_gt_data *gt_data;
> > +};
> > +
> > +
> > +/**
> > + * xe_gpufreqtracer_sample_work - Worker function to sample GPU frequency.
> > + * @work: Pointer to the delayed_work_struct representing the scheduled work.
> > + *
> > + * This function is executed in a workqueue context to periodically sample
> > + * the GPU frequency and perform any necessary tracing or logging operations.
> > + * It reschedules itself for the next sampling interval.
> > + */
> > +static void xe_gpufreqtracer_sample_work(struct work_struct *work)
> 
> this looks like a weird abstraction that creates issues (see Matt
> Brost's reply) that are already a solved problem. Why can't the
> userspace interested in it use perf for that? We already sample
> frequency changes there. See drivers/gpu/drm/xe/xe_pmu.c
> 

I was actually going to follow up with this question after looking into
the issues with hotplug, runtime PM, and FW, as xe_pmu.c appears to do
basically the same thing (monitoring the result of
xe_guc_pc_get_act_freq) and has all the correct hotplug, runtime PM, and
FW refs in place.

Matt

> Lucas De Marchi