* [PATCH 1/4] RFC: ANDROID: drm/xe: add adaptive GPU memory growth monitor xe_mem_monitor
2026-09-07 6:07 [PATCH 0/4] RFC: drm/xe: Dynamic Xe Memory Monitor S Sebinraj
@ 2026-09-07 6:07 ` S Sebinraj
2026-09-07 6:07 ` [PATCH 2/4] RFC: ANDROID: drm/xe: register ttm page alloc vendor hooks internally S Sebinraj
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: S Sebinraj @ 2026-09-07 6:07 UTC (permalink / raw)
To: intel-xe; +Cc: S Sebinraj, Carlos Santa, Ryan Neph, Renato Pereyra, S Sebinraj
From: S Sebinraj <s.sebinraj@intel.corp-partner.google.com>
Introduce xe_mem_monitor, a periodic worker that samples
xe->global_total_pages (the same counter that feeds the
gpu_mem/gpu_mem_total tracepoint) and tracks GPU memory growth using an
exponential moving average (EMA) over per-interval deltas:
delta(t) = max(0, mem(t) - mem(t-1))
EMA(t) = alpha * delta(t) + (1 - alpha) * EMA(t-1)
where alpha = 0.8, applied to every real sample
regardless of mode.
A single alpha and a single polling cadence (mem_monitor_poll_ms)
are used in both LATENCY and HIGH_THROUGHPUT mode, once
HIGH_THROUGHPUT mode is entered there is little benefit to re-checking
any faster, since mem_monitor_debounce_count consecutive
below-threshold samples are already required before switching back,
which itself provides the desired minimum dwell time
(debounce_count * poll_ms) at this single cadence.
When the EMA of growth exceeds a threshold, xe->gpu_mem_mode is switched
from XE_GPU_MEM_MODE_LATENCY to XE_GPU_MEM_MODE_HIGH_THROUGHPUT, and
back once mem_monitor_debounce_count consecutive below-threshold
samples are seen.
xe->gpu_mem_mode is intended to be read by the TTM page allocation path
to trade allocation latency for throughput during GPU memory growth
bursts; the GFP flag adjustment itself is added in a follow-up change.
For power, the worker sleeps after mem_monitor_idle_timeout_ms of no
significant activity in LATENCY mode. The worker is re-armed on
activity.
The worker is suspended/resumed across system and runtime PM
transitions with notify_activity() also disabled while suspended.
All tuning is exposed as module parameters (mem_monitor_*) under
/sys/module/xe/parameters/. Defaults:
mem_monitor_enabled false
mem_monitor_growth_threshold_mb 250
mem_monitor_poll_ms 2000
mem_monitor_debounce_count 5
mem_monitor_idle_timeout_ms 10000
Guarded by new CONFIG_DRM_XE_MEM_MONITOR (depends on DRM_XE, selects
TRACE_GPU_MEM), enabled by default in defconfig_xe.
mem_monitor_enabled for fatcat is set via kernel arg xe.mem_monitor_enabled=1
Test: Verified via kernel logs while running WebGL-Aquarium / WLEU benchmark,
mode switches to HIGH_THROUGHPUT on rapid allocation bursts and
back once memory stabilizes. Confirmed idle-sleep/wake, EMA decay
across sleep, clean suspend/resume behavior.
Cc: Carlos Santa <carlos.santa@intel.com>
Cc: Ryan Neph <ryanneph@google.com>
Cc: Renato Pereyra <renatopereyra@google.com>
Signed-off-by: S Sebinraj <s.sebinraj@intel.com>
---
drivers/gpu/drm/BUILD.bazel | 3 +
drivers/gpu/drm/defconfig_xe | 1 +
drivers/gpu/drm/xe/Kconfig | 14 +
drivers/gpu/drm/xe/xe_bo.c | 3 +
drivers/gpu/drm/xe/xe_device.c | 5 +
drivers/gpu/drm/xe/xe_device_types.h | 12 +
drivers/gpu/drm/xe/xe_mem_monitor.c | 462 +++++++++++++++++++++++++++
drivers/gpu/drm/xe/xe_mem_monitor.h | 82 +++++
drivers/gpu/drm/xe/xe_module.c | 55 ++++
drivers/gpu/drm/xe/xe_module.h | 13 +
drivers/gpu/drm/xe/xe_pm.c | 5 +
11 files changed, 655 insertions(+)
create mode 100644 drivers/gpu/drm/xe/xe_mem_monitor.c
create mode 100644 drivers/gpu/drm/xe/xe_mem_monitor.h
diff --git a/drivers/gpu/drm/BUILD.bazel b/drivers/gpu/drm/BUILD.bazel
index ac32f5cdecb9..274a1aa4ca93 100644
--- a/drivers/gpu/drm/BUILD.bazel
+++ b/drivers/gpu/drm/BUILD.bazel
@@ -154,6 +154,9 @@ ddk_module(
"CONFIG_DRM_XE_GPUFREQTRACER": {
True: ["xe/xe_gpufreqtracer.c"],
},
+ "CONFIG_DRM_XE_MEM_MONITOR": {
+ True: ["xe/xe_mem_monitor.c"],
+ },
"CONFIG_DRM_XE_GPUSVM": {
True: ["xe/xe_svm.c"],
},
diff --git a/drivers/gpu/drm/defconfig_xe b/drivers/gpu/drm/defconfig_xe
index c6e1c69fa663..4dc505580a37 100644
--- a/drivers/gpu/drm/defconfig_xe
+++ b/drivers/gpu/drm/defconfig_xe
@@ -3,3 +3,4 @@ CONFIG_DRM_XE_DISPLAY=y
CONFIG_DRM_XE_DP_TUNNEL=y
CONFIG_DRM_XE_FORCE_PROBE="*"
CONFIG_DRM_XE_GPUFREQTRACER=y
+CONFIG_DRM_XE_MEM_MONITOR=y
diff --git a/drivers/gpu/drm/xe/Kconfig b/drivers/gpu/drm/xe/Kconfig
index 1521975c35fc..d0a0fcb33d64 100644
--- a/drivers/gpu/drm/xe/Kconfig
+++ b/drivers/gpu/drm/xe/Kconfig
@@ -150,6 +150,20 @@ config DRM_XE_GPUFREQTRACER
If unsure, say N.
+config DRM_XE_MEM_MONITOR
+ bool "Enable XE GPU memory growth monitor"
+ depends on DRM_XE
+ select TRACE_GPU_MEM
+ default n
+ help
+ Enable adaptive GPU memory growth monitoring for the Intel XE driver.
+ Polls xe->global_total_pages at 500ms (active) or 2s (idle) intervals
+ and applies an exponential moving average to detect rapid allocation
+ bursts (>200 MB/interval). When a burst is detected the TTM vendor
+ hook is disabled so that the kernel's normal direct-reclaim path can
+ apply backpressure on high-order page allocations, preventing stalls
+ from an unresponsive page cache.
+
menu "drm/Xe Debugging"
depends on DRM_XE
depends on EXPERT
diff --git a/drivers/gpu/drm/xe/xe_bo.c b/drivers/gpu/drm/xe/xe_bo.c
index 493619229376..f2634d048cd4 100644
--- a/drivers/gpu/drm/xe/xe_bo.c
+++ b/drivers/gpu/drm/xe/xe_bo.c
@@ -27,6 +27,7 @@
#include "xe_ggtt.h"
#include "xe_gt.h"
#include "xe_map.h"
+#include "xe_mem_monitor.h"
#include "xe_migrate.h"
#include "xe_pm.h"
#include "xe_preempt_fence.h"
@@ -436,6 +437,8 @@ static void update_global_total_pages(struct ttm_device *ttm_dev,
trace_gpu_mem_total(xe->drm.primary->index, 0,
global_total_pages << PAGE_SHIFT);
+
+ xe_mem_monitor_notify_activity(xe);
#endif
}
diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
index e405dbedd6b8..676ffc258b94 100644
--- a/drivers/gpu/drm/xe/xe_device.c
+++ b/drivers/gpu/drm/xe/xe_device.c
@@ -36,6 +36,7 @@
#include "xe_force_wake.h"
#include "xe_ggtt.h"
#include "xe_gpufreqtracer.h"
+#include "xe_mem_monitor.h"
#include "xe_gsc_proxy.h"
#include "xe_gt.h"
#include "xe_gt_mcr.h"
@@ -931,6 +932,10 @@ int xe_device_probe(struct xe_device *xe)
if (err)
return err;
+ err = xe_mem_monitor_init(xe);
+ if (err)
+ return err;
+
err = xe_late_bind_init(&xe->late_bind);
if (err)
return err;
diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
index 7b6c4e9a12b1..48aa9d8f6f56 100644
--- a/drivers/gpu/drm/xe/xe_device_types.h
+++ b/drivers/gpu/drm/xe/xe_device_types.h
@@ -39,6 +39,7 @@ struct intel_display;
struct intel_dg_nvm_dev;
struct xe_ggtt;
struct xe_gpufreqtracer_data;
+struct xe_mem_monitor_data;
struct xe_i2c;
struct xe_pat_ops;
struct xe_pxp;
@@ -583,6 +584,9 @@ struct xe_device {
/** @gpufreqtracer_data: GPU frequency tracer data */
struct xe_gpufreqtracer_data *gpufreqtracer_data;
+ /** @mem_monitor: GPU memory growth monitor data */
+ struct xe_mem_monitor_data *mem_monitor;
+
/** @needs_flr_on_fini: requests function-reset on fini */
bool needs_flr_on_fini;
@@ -630,6 +634,14 @@ struct xe_device {
*/
atomic64_t global_total_pages;
#endif
+
+ /**
+ * @gpu_mem_mode: current GPU memory allocation mode (enum xe_gpu_mem_mode),
+ * switched between latency and high-throughput by xe_mem_monitor based on
+ * GPU memory growth rate.
+ */
+ atomic_t gpu_mem_mode;
+
/** @val: The domain for exhaustive eviction, which is currently per device. */
struct xe_validation_device val;
diff --git a/drivers/gpu/drm/xe/xe_mem_monitor.c b/drivers/gpu/drm/xe/xe_mem_monitor.c
new file mode 100644
index 000000000000..06fbd0624da7
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_mem_monitor.c
@@ -0,0 +1,462 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright © 2026 Intel Corporation
+ *
+ * xe_mem_monitor - Adaptive GPU memory growth monitor
+ *
+ * Periodically samples xe->global_total_pages (the same counter that feeds
+ * the gpu_mem/gpu_mem_total tracepoint) and applies an exponential moving
+ * average (EMA) over per-interval growth. When growth exceeds a threshold
+ * xe->gpu_mem_mode is switched to XE_GPU_MEM_MODE_HIGH_THROUGHPUT, allowing
+ * the kernel's normal reclaim path to apply backpressure on TTM page
+ * allocations. Once memory stabilises, the mode is switched back to
+ * XE_GPU_MEM_MODE_LATENCY.
+ *
+ * Algorithm:
+ *
+ * delta(t) = max(0, mem(t) - mem(t-1)) // clamp frees to zero
+ * EMA(t) = (ALPHA_NUM * delta(t) +
+ * (2^ALPHA_SHIFT - ALPHA_NUM) * EMA(t-1) ) >> ALPHA_SHIFT
+ *
+ * where ALPHA_NUM/2^ALPHA_SHIFT = alpha = 13/16 = 0.8125,
+ * applied to every real sample regardless of mode, where ALPHA_SHIFT
+ * = 4, (2^4 = 16). Both modes poll at the same mem_monitor_poll_ms
+ * cadence and use the same smoothing weight.
+ *
+ * State machine (xe->gpu_mem_mode):
+ * LATENCY mode, poll
+ * -> EMA > mem_monitor_growth_threshold_mb: switch to HIGH_THROUGHPUT
+ * HIGH_THROUGHPUT mode, poll (single polling rate - see
+ * xe_mem_monitor_work()'s reschedule logic)
+ * -> EMA below threshold for mem_monitor_debounce_count consecutive
+ * samples -> switch back to LATENCY mode. Since samples are taken
+ * every mem_monitor_poll_ms regardless of mode,
+ * mem_monitor_debounce_count also acts as the implicit minimum dwell
+ * time in HIGH_THROUGHPUT mode (debounce_count * poll_ms).
+ *
+ * Power saving: while in LATENCY mode, if no GPU allocation activity has been
+ * observed (see xe_mem_monitor_notify_activity()) for mem_monitor_idle_timeout_ms,
+ * the polling worker stops rescheduling itself entirely (goes to sleep, zero
+ * wakeups) rather than continuing to poll indefinitely. It is re-armed the
+ * next time real GPU memory growth is observed via
+ * xe_mem_monitor_notify_activity(). Since the state machine only ever reacts
+ * to growth (frees are clamped to zero), notify_activity() itself only
+ * resets the idle timer / wakes a sleeping worker when the live memory total
+ * has actually grown since the last real sample, a pure free or a
+ * populate/unpopulate pair that nets to no change, is ignored rather than
+ * forcing a full poll+decay cycle that could never affect the LATENCY/
+ * HIGH_THROUGHPUT decision.
+ *
+ * Since the EMA depends on samples being taken at a roughly regular cadence,
+ * and no samples are taken while asleep, the worker treats any skipped
+ * mem_monitor_poll_ms intervals since its last real sample as implicit
+ * zero-growth samples and fast-forwards the EMA decay for them before
+ * folding in the real delta observed on the sample that woke it up. This
+ * prevents stale, pre-sleep growth history from persisting indefinitely
+ * across arbitrarily long idle gaps.
+ */
+
+#include "xe_mem_monitor.h"
+
+#include <linux/atomic.h>
+#include <linux/jiffies.h>
+#include <linux/minmax.h>
+#include <linux/slab.h>
+#include <linux/workqueue.h>
+#include <drm/drm_managed.h>
+#include <drm/drm_print.h>
+
+#include "xe_device.h"
+#include "xe_device_types.h"
+#include "xe_module.h"
+
+/*
+ * EMA smoothing factor. alpha (13/16 = 0.8125) applies to every real
+ * sample, regardless of mode,
+ * since both LATENCY and HIGH_THROUGHPUT modes poll at the same
+ * mem_monitor_poll_ms cadence (see xe_mem_monitor_work()'s reschedule
+ * logic) and share the same smoothing weight.
+ */
+#define ALPHA_NUM 13U
+#define ALPHA_RETAINED 3U /* 16 - 13 */
+#define ALPHA_SHIFT 4U /* /16 */
+
+/*
+ * This is the max number of implicit zero-growth decay steps applied when waking
+ * from sleep (see xe_mem_monitor_work()). Each step degrades EMA by
+ * ALPHA_RETAINED/2^ALPHA_SHIFT = 3/16 per step,
+ * so after the number of steps below the EMA has already decayed to 0
+ * via integer truncation regardless of its starting value. No need to loop
+ * further for arbitrarily long sleeps.
+ */
+#define EMA_DECAY_SATURATE_PERIODS 32
+
+/*
+ * Minimum absolute change (bytes) since the worker's last real sample
+ * required for xe_mem_monitor_notify_activity() to treat it as activity
+ * worth acting on. Filters out negligible/noise-level trickle (a few KB of
+ * housekeeping churn) that would otherwise repeatedly wake the worker for no
+ * meaningful reason. Checked in both directions (growth or shrink): a
+ * significant free must also resync prev_mem_bytes, otherwise it can be left
+ * stuck at a stale high-water mark that later, real growth from a much lower
+ * baseline would never be able to cross.
+ * Current allocated bytes is compared to the prev bytes so we won't accumulate
+ * small errors less than 1 MB over multiple calls
+ */
+#define ACTIVITY_MIN_DELTA_BYTES (1ULL << 20) /* 1 MB */
+
+/**
+ * struct xe_mem_monitor_data - per-device GPU memory monitor state
+ * @xe: back-pointer to the xe device
+ * @work: self-rescheduling delayed work
+ * @prev_mem_bytes: GPU memory total from the previous poll. atomic64_t
+ * because it is also read from
+ * xe_mem_monitor_notify_activity(), which can run
+ * concurrently with the worker on another thread.
+ * @has_prev_sample: false until the first sample is taken (baseline)
+ * @ema_growth_bytes: EMA of per-interval memory growth (bytes)
+ * @is_high_throughput: true while in HIGH_THROUGHPUT mode; only gates
+ * the idle-sleep check below (idle-sleep only
+ * applies in LATENCY mode).
+ * @below_threshold_count: consecutive below-threshold samples since entering
+ * HIGH_THROUGHPUT mode
+ * @last_activity_jiffies: jiffies value of the most recent GPU memory growth
+ * notification (see xe_mem_monitor_notify_activity())
+ * @last_poll_jiffies: jiffies value of the most recent real sample taken
+ * by xe_mem_monitor_work(), used to detect and decay
+ * the EMA across skipped (slept-through) intervals
+ * @suspended: set while the device is suspended (system or
+ * runtime PM); xe_mem_monitor_notify_activity() is a
+ * no-op while set, so that BO eviction/restore
+ * traffic generated by suspend/resume itself cannot
+ * re-arm the worker mid-transition. atomic_t because
+ * it is read from xe_mem_monitor_notify_activity() on
+ * arbitrary caller threads.
+ * @was_pending_before_suspend: whether the worker had a poll actually
+ * scheduled at the moment xe_mem_monitor_suspend()
+ * cancelled it; used by xe_mem_monitor_resume() to
+ * decide whether to reschedule, so that a monitor
+ * that was legitimately asleep before suspend stays
+ * asleep after resume instead of being woken
+ * unconditionally.
+ */
+struct xe_mem_monitor_data {
+ struct xe_device *xe;
+ struct delayed_work work;
+
+ atomic64_t prev_mem_bytes;
+ bool has_prev_sample;
+ u64 ema_growth_bytes;
+ int below_threshold_count;
+ unsigned long last_activity_jiffies;
+ unsigned long last_poll_jiffies;
+ bool is_high_throughput;
+ atomic_t suspended;
+ bool was_pending_before_suspend;
+};
+
+static void xe_mem_monitor_set_high_throughput_mode(struct xe_mem_monitor_data *mon)
+{
+ /* TODO: wire up to the actual vendor hook (adds __GFP_RETRY_MAYFAIL, TBD) */
+ drm_dbg(&mon->xe->drm,
+ "xe_mem_monitor: switching to HIGH_THROUGHPUT mode (TBD)\n");
+ atomic_set(&mon->xe->gpu_mem_mode, XE_GPU_MEM_MODE_HIGH_THROUGHPUT);
+}
+
+static void xe_mem_monitor_set_latency_mode(struct xe_mem_monitor_data *mon)
+{
+ /* TODO: wire up to the actual vendor hook (TBD) */
+ drm_dbg(&mon->xe->drm,
+ "xe_mem_monitor: switching to LATENCY mode (TBD)\n");
+ atomic_set(&mon->xe->gpu_mem_mode, XE_GPU_MEM_MODE_LATENCY);
+}
+
+static void xe_mem_monitor_work(struct work_struct *work)
+{
+ struct xe_mem_monitor_data *mon =
+ container_of(work, struct xe_mem_monitor_data, work.work);
+ struct xe_device *xe = mon->xe;
+ u64 current_bytes, clamped_delta;
+ u64 growth_threshold_bytes;
+ s64 delta_signed;
+ unsigned long now = jiffies;
+ unsigned long next_delay;
+
+ /*
+ * Read the GPU memory total directly from Xe's own accounting counter.
+ * This is the same value that drives the gpu_mem/gpu_mem_total
+ * tracepoint, but read without any BPF or sysfs indirection.
+ */
+ current_bytes = (u64)atomic64_read(&xe->global_total_pages) << PAGE_SHIFT;
+
+ if (!mon->has_prev_sample) {
+ drm_dbg(&xe->drm, "xe_mem_monitor: baseline = %llu MB\n",
+ current_bytes >> 20);
+ atomic64_set(&mon->prev_mem_bytes, current_bytes);
+ mon->has_prev_sample = true;
+ mon->last_poll_jiffies = now;
+ goto reschedule;
+ }
+
+ /*
+ * If more than one nominal poll interval has elapsed since the
+ * last real sample, the worker must have been asleep (or otherwise
+ * delayed) for the extra time. Since no growth can occur unnoticed
+ * while asleep (any GPU allocation activity would have woken it via
+ * xe_mem_monitor_notify_activity()), treat each skipped interval as
+ * an implicit zero-growth sample and fast-forward the EMA decay for
+ * them before folding in the real delta observed below. Otherwise
+ * stale, pre-sleep growth history would persist indefinitely across
+ * arbitrarily long idle gaps.
+ */
+ {
+ unsigned long poll_jiffies =
+ msecs_to_jiffies(xe_modparam.mem_monitor_poll_ms);
+ unsigned long skipped_periods = poll_jiffies ?
+ (now - mon->last_poll_jiffies) / poll_jiffies : 0;
+
+ if (skipped_periods > 1) {
+ unsigned long decay_periods = min_t(unsigned long,
+ skipped_periods - 1,
+ EMA_DECAY_SATURATE_PERIODS);
+
+ drm_dbg(&xe->drm,
+ "xe_mem_monitor: woke after %lu skipped poll interval(s), "
+ "fast-forwarding EMA decay by %lu\n",
+ skipped_periods - 1, decay_periods);
+
+ while (decay_periods-- > 0)
+ mon->ema_growth_bytes = (ALPHA_RETAINED *
+ mon->ema_growth_bytes) >> ALPHA_SHIFT;
+ }
+ }
+
+ /* Clamp negative deltas (memory freed) to zero; only track growth. */
+ delta_signed = (s64)current_bytes - (s64)atomic64_read(&mon->prev_mem_bytes);
+ clamped_delta = delta_signed > 0 ? (u64)delta_signed : 0;
+
+ /* A single alpha smoothing weight is used regardless of mode. */
+ mon->ema_growth_bytes = (ALPHA_NUM * clamped_delta +
+ ALPHA_RETAINED * mon->ema_growth_bytes)
+ >> ALPHA_SHIFT;
+
+ atomic64_set(&mon->prev_mem_bytes, current_bytes);
+ mon->last_poll_jiffies = now;
+
+ growth_threshold_bytes = (u64)xe_modparam.mem_monitor_growth_threshold_mb << 20;
+
+ if (atomic_read(&xe->gpu_mem_mode) == XE_GPU_MEM_MODE_LATENCY) {
+ if (mon->ema_growth_bytes > growth_threshold_bytes) {
+ drm_dbg(&xe->drm,
+ "xe_mem_monitor: rapid GPU memory growth detected, delta=%llu MB, "
+ "(EMA=%llu MB/interval > threshold=%llu MB)\n",
+ clamped_delta >> 20,
+ mon->ema_growth_bytes >> 20,
+ growth_threshold_bytes >> 20);
+ xe_mem_monitor_set_high_throughput_mode(mon);
+ mon->is_high_throughput = true;
+ mon->below_threshold_count = 0;
+ }
+ } else {
+ /*
+ * mem_monitor_debounce_count consecutive below-threshold
+ * samples (taken at mem_monitor_poll_ms, same single
+ * polling rate used in both modes) are required before
+ * switching back to LATENCY mode. This also acts as the
+ * implicit minimum dwell time in HIGH_THROUGHPUT mode.
+ */
+ if (mon->ema_growth_bytes < growth_threshold_bytes) {
+ mon->below_threshold_count++;
+ drm_dbg(&xe->drm,
+ "xe_mem_monitor: GPU memory growth slowing "
+ "(EMA=%llu MB/interval), below-threshold sample %d/%u\n",
+ mon->ema_growth_bytes >> 20,
+ mon->below_threshold_count, xe_modparam.mem_monitor_debounce_count);
+ if (mon->below_threshold_count >= xe_modparam.mem_monitor_debounce_count) {
+ drm_dbg(&xe->drm,
+ "xe_mem_monitor: GPU memory stable for %u "
+ "consecutive samples\n",
+ xe_modparam.mem_monitor_debounce_count);
+ xe_mem_monitor_set_latency_mode(mon);
+ mon->is_high_throughput = false;
+ mon->below_threshold_count = 0;
+ }
+ } else {
+ drm_dbg(&xe->drm,
+ "xe_mem_monitor: GPU memory still growing "
+ "(EMA=%llu MB/interval), resetting debounce counter\n",
+ mon->ema_growth_bytes >> 20);
+ mon->below_threshold_count = 0;
+ }
+ }
+
+reschedule:
+ if (!mon->is_high_throughput &&
+ jiffies_to_msecs(jiffies - mon->last_activity_jiffies) >
+ xe_modparam.mem_monitor_idle_timeout_ms) {
+ drm_dbg(&xe->drm,
+ "xe_mem_monitor: no activity for %u ms, going to sleep\n",
+ xe_modparam.mem_monitor_idle_timeout_ms);
+ return;
+ }
+
+ /*
+ * Single polling rate (mem_monitor_poll_ms) regardless of mode -
+ * per review discussion, once HIGH_THROUGHPUT mode is entered we
+ * don't need to re-check any faster, since mem_monitor_debounce_count
+ * already provides the equivalent minimum dwell time at this cadence.
+ */
+ next_delay = msecs_to_jiffies(xe_modparam.mem_monitor_poll_ms);
+ schedule_delayed_work(&mon->work, next_delay);
+}
+
+static void xe_mem_monitor_cleanup(struct drm_device *drm, void *arg)
+{
+ struct xe_mem_monitor_data *mon = arg;
+
+ cancel_delayed_work_sync(&mon->work);
+}
+
+/**
+ * xe_mem_monitor_notify_activity - notify the monitor of GPU memory changes
+ * @xe: the Xe device
+ *
+ * Called from the TTM populate/unpopulate path whenever GPU memory is
+ * allocated or freed. No-op while the device is suspended (see
+ * xe_mem_monitor_suspend()), since suspend/resume's own BO eviction/restore
+ * traffic would otherwise re-arm the worker mid-transition. Otherwise only
+ * acts when the live memory total has changed by at least
+ * ACTIVITY_MIN_DELTA_BYTES (in either direction) since the worker's last real
+ * sample, negligible trickle growth/shrink is ignored to avoid waking the
+ * worker for no meaningful reason. A significant free must still be acted on
+ * (not just growth): it resyncs prev_mem_bytes down to reality by letting the
+ * worker run once, which is required so that later real growth measured from
+ * that new, lower baseline can still be detected, otherwise prev_mem_bytes
+ * would stay stuck at a stale high-water mark that new growth might never
+ * cross. If the worker is already scheduled (delayed_work_pending()), it
+ * will read the live counter fresh when it runs and pick up whatever
+ * accumulated, so there's no need to re-schedule again here.
+ */
+void xe_mem_monitor_notify_activity(struct xe_device *xe)
+{
+ struct xe_mem_monitor_data *mon = xe->mem_monitor;
+ u64 current_bytes;
+ s64 delta_signed, abs_delta;
+
+ if (!mon)
+ return;
+
+ if (atomic_read(&mon->suspended))
+ return;
+
+ current_bytes = (u64)atomic64_read(&xe->global_total_pages) << PAGE_SHIFT;
+ delta_signed = (s64)current_bytes - (s64)atomic64_read(&mon->prev_mem_bytes);
+ abs_delta = delta_signed < 0 ? -delta_signed : delta_signed;
+
+ if (abs_delta < ACTIVITY_MIN_DELTA_BYTES)
+ return;
+
+ mon->last_activity_jiffies = jiffies;
+
+ if (delayed_work_pending(&mon->work))
+ return;
+
+ schedule_delayed_work(&mon->work, 0);
+}
+
+/**
+ * xe_mem_monitor_suspend - stop the polling worker for a PM transition
+ * @xe: the Xe device
+ *
+ * Sets the suspended flag first so any xe_mem_monitor_notify_activity()
+ * call racing with (or generated by) the suspend sequence itself (e.g.
+ * xe_bo_evict_all()) is a no-op, then synchronously cancels the worker.
+ * Whether a poll was actually pending at that point is recorded so
+ * xe_mem_monitor_resume() can decide whether to reschedule.
+ */
+void xe_mem_monitor_suspend(struct xe_device *xe)
+{
+ struct xe_mem_monitor_data *mon = xe->mem_monitor;
+
+ if (!mon)
+ return;
+
+ atomic_set(&mon->suspended, 1);
+ mon->was_pending_before_suspend = cancel_delayed_work_sync(&mon->work);
+}
+
+/**
+ * xe_mem_monitor_resume - resume the polling worker after a PM transition
+ * @xe: the Xe device
+ *
+ * Only reschedules the worker if it was actually pending at the time
+ * xe_mem_monitor_suspend() cancelled it. If the monitor had already gone to
+ * sleep (idle timeout) before suspend, it stays asleep across the
+ * transition rather than being unconditionally woken.
+ */
+void xe_mem_monitor_resume(struct xe_device *xe)
+{
+ struct xe_mem_monitor_data *mon = xe->mem_monitor;
+
+ if (!mon)
+ return;
+
+ atomic_set(&mon->suspended, 0);
+
+ if (mon->was_pending_before_suspend) {
+ unsigned long delay = msecs_to_jiffies(xe_modparam.mem_monitor_poll_ms);
+
+ schedule_delayed_work(&mon->work, delay);
+ }
+}
+
+/**
+ * xe_mem_monitor_init - initialise and start the GPU memory monitor
+ * @xe: the Xe device
+ *
+ * Allocates monitor state, registers a cleanup action via drmm, and
+ * schedules the first poll after one poll interval.
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+int xe_mem_monitor_init(struct xe_device *xe)
+{
+ struct xe_mem_monitor_data *mon;
+ int ret;
+
+ if (!xe_modparam.mem_monitor_enabled) {
+ drm_info(&xe->drm,
+ "xe_mem_monitor: disabled via mem_monitor_enabled module parameter\n");
+ return 0;
+ }
+
+ mon = drmm_kzalloc(&xe->drm, sizeof(*mon), GFP_KERNEL);
+ if (!mon)
+ return -ENOMEM;
+
+ mon->xe = xe;
+ atomic_set(&xe->gpu_mem_mode, XE_GPU_MEM_MODE_LATENCY);
+ mon->last_activity_jiffies = jiffies;
+ INIT_DELAYED_WORK(&mon->work, xe_mem_monitor_work);
+
+ xe->mem_monitor = mon;
+
+ ret = drmm_add_action_or_reset(&xe->drm, xe_mem_monitor_cleanup, mon);
+ if (ret)
+ return ret;
+
+ schedule_delayed_work(&mon->work,
+ msecs_to_jiffies(xe_modparam.mem_monitor_poll_ms));
+
+ drm_info(&xe->drm,
+ "xe_mem_monitor: initialized: poll=%u ms, "
+ "threshold=%u MB, alpha=0.%03u, "
+ "debounce=%u, idle_timeout=%u ms\n",
+ xe_modparam.mem_monitor_poll_ms,
+ xe_modparam.mem_monitor_growth_threshold_mb,
+ (ALPHA_NUM * 1000U + (1U << (ALPHA_SHIFT - 1))) >> ALPHA_SHIFT,
+ xe_modparam.mem_monitor_debounce_count,
+ xe_modparam.mem_monitor_idle_timeout_ms);
+
+ return 0;
+}
diff --git a/drivers/gpu/drm/xe/xe_mem_monitor.h b/drivers/gpu/drm/xe/xe_mem_monitor.h
new file mode 100644
index 000000000000..846c0628ed93
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_mem_monitor.h
@@ -0,0 +1,82 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright © 2026 Intel Corporation
+ */
+
+#ifndef _XE_MEM_MONITOR_H_
+#define _XE_MEM_MONITOR_H_
+
+struct xe_device;
+
+/**
+ * enum xe_gpu_mem_mode - GPU memory allocation mode controlled by xe_mem_monitor
+ * @XE_GPU_MEM_MODE_LATENCY: default mode; vendor hook strips reclaim flags
+ * from high-order TTM allocations for low latency.
+ * @XE_GPU_MEM_MODE_HIGH_THROUGHPUT: entered during rapid GPU memory growth;
+ * reclaim is allowed (and __GFP_RETRY_MAYFAIL added)
+ * so the kernel can shrink the page cache to satisfy
+ * high-order allocations.
+ */
+enum xe_gpu_mem_mode {
+ XE_GPU_MEM_MODE_LATENCY = 0,
+ XE_GPU_MEM_MODE_HIGH_THROUGHPUT,
+};
+
+/* Default values for module parameters, see xe_module.h/xe_module.c */
+#define XE_MEM_MONITOR_DEFAULT_ENABLED false
+#define XE_MEM_MONITOR_DEFAULT_GROWTH_THRESHOLD_MB 250
+#define XE_MEM_MONITOR_DEFAULT_POLL_MS 2000
+#define XE_MEM_MONITOR_DEFAULT_DEBOUNCE_COUNT 5
+#define XE_MEM_MONITOR_DEFAULT_IDLE_TIMEOUT_MS 10000
+
+/*
+ * Minimum enforced value for mem_monitor_poll_ms (no maximum is
+ * enforced). Guards against a misconfigured/absurdly low value (e.g. 0)
+ * causing the delayed work to effectively busy-loop.
+ */
+#define XE_MEM_MONITOR_MIN_POLL_MS 500
+
+#ifdef CONFIG_DRM_XE_MEM_MONITOR
+
+int xe_mem_monitor_init(struct xe_device *xe);
+
+/*
+ * Notify the monitor that GPU allocation activity occurred (called from the
+ * TTM populate/unpopulate path). Updates the last-activity timestamp and
+ * re-arms the polling worker if it had gone to sleep due to inactivity.
+ */
+void xe_mem_monitor_notify_activity(struct xe_device *xe);
+
+/*
+ * Suspend/resume the polling worker across system and runtime PM
+ * transitions. Must be called from the same suspend/resume paths for
+ * the same reason: schedule_delayed_work() on system_wq is not freezable, so
+ * without this the worker could keep firing (or be re-armed by the BO
+ * eviction/restore traffic that suspend/resume itself generates) across the
+ * transition.
+ */
+void xe_mem_monitor_suspend(struct xe_device *xe);
+void xe_mem_monitor_resume(struct xe_device *xe);
+
+#else /* CONFIG_DRM_XE_MEM_MONITOR */
+
+static inline int xe_mem_monitor_init(struct xe_device *xe)
+{
+ return 0;
+}
+
+static inline void xe_mem_monitor_notify_activity(struct xe_device *xe)
+{
+}
+
+static inline void xe_mem_monitor_suspend(struct xe_device *xe)
+{
+}
+
+static inline void xe_mem_monitor_resume(struct xe_device *xe)
+{
+}
+
+#endif /* CONFIG_DRM_XE_MEM_MONITOR */
+
+#endif /* _XE_MEM_MONITOR_H_ */
diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
index 4878463734eb..a49b0c0f958b 100644
--- a/drivers/gpu/drm/xe/xe_module.c
+++ b/drivers/gpu/drm/xe/xe_module.c
@@ -15,6 +15,7 @@
#include "xe_configfs.h"
#include "xe_gpufreqtracer.h"
#include "xe_hw_fence.h"
+#include "xe_mem_monitor.h"
#include "xe_pci.h"
#include "xe_pm.h"
#include "xe_observation.h"
@@ -45,6 +46,13 @@ struct xe_modparam xe_modparam = {
.svm_notifier_size = DEFAULT_SVM_NOTIFIER_SIZE,
#ifdef CONFIG_DRM_XE_GPUFREQTRACER
.gpufreq_monitoring_interval_ms = XE_GPUFREQ_MONITORING_DEFAULT_INTERVAL_MS,
+#endif
+#ifdef CONFIG_DRM_XE_MEM_MONITOR
+ .mem_monitor_enabled = XE_MEM_MONITOR_DEFAULT_ENABLED,
+ .mem_monitor_growth_threshold_mb = XE_MEM_MONITOR_DEFAULT_GROWTH_THRESHOLD_MB,
+ .mem_monitor_poll_ms = XE_MEM_MONITOR_DEFAULT_POLL_MS,
+ .mem_monitor_debounce_count = XE_MEM_MONITOR_DEFAULT_DEBOUNCE_COUNT,
+ .mem_monitor_idle_timeout_ms = XE_MEM_MONITOR_DEFAULT_IDLE_TIMEOUT_MS,
#endif
/* the rest are 0 by default */
};
@@ -108,6 +116,53 @@ MODULE_PARM_DESC(gpufreq_monitoring_interval_ms,
__stringify(XE_GPUFREQ_MONITORING_DEFAULT_INTERVAL_MS) ")");
#endif
+#ifdef CONFIG_DRM_XE_MEM_MONITOR
+module_param_named_unsafe(mem_monitor_enabled, xe_modparam.mem_monitor_enabled, bool, 0444);
+MODULE_PARM_DESC(mem_monitor_enabled,
+ "Master enable switch for the GPU memory growth monitor and its "
+ "vendor hooks. Intended to be set as a kernel boot argument"
+ " [default=" __stringify(XE_MEM_MONITOR_DEFAULT_ENABLED) " (enabled)]");
+
+module_param_named(mem_monitor_growth_threshold_mb,
+ xe_modparam.mem_monitor_growth_threshold_mb, uint, 0644);
+MODULE_PARM_DESC(mem_monitor_growth_threshold_mb,
+ "GPU memory growth EMA threshold in MiB per interval that triggers "
+ "HIGH_THROUGHPUT mode [default="
+ __stringify(XE_MEM_MONITOR_DEFAULT_GROWTH_THRESHOLD_MB) "])");
+
+static int param_set_mem_monitor_poll_ms(const char *val, const struct kernel_param *kp)
+{
+ return param_set_uint_minmax(val, kp, XE_MEM_MONITOR_MIN_POLL_MS, UINT_MAX);
+}
+
+static const struct kernel_param_ops param_ops_mem_monitor_poll_ms = {
+ .set = param_set_mem_monitor_poll_ms,
+ .get = param_get_uint,
+};
+
+module_param_cb(mem_monitor_poll_ms, ¶m_ops_mem_monitor_poll_ms,
+ &xe_modparam.mem_monitor_poll_ms, 0644);
+MODULE_PARM_DESC(mem_monitor_poll_ms,
+ "GPU memory monitor polling interval in milliseconds, minimum "
+ __stringify(XE_MEM_MONITOR_MIN_POLL_MS) "ms "
+ "[default=" __stringify(XE_MEM_MONITOR_DEFAULT_POLL_MS) "])");
+
+module_param_named(mem_monitor_debounce_count,
+ xe_modparam.mem_monitor_debounce_count, uint, 0644);
+MODULE_PARM_DESC(mem_monitor_debounce_count,
+ "Number of consecutive below-threshold samples required before "
+ "switching back to LATENCY mode [default="
+ __stringify(XE_MEM_MONITOR_DEFAULT_DEBOUNCE_COUNT) "])");
+
+module_param_named(mem_monitor_idle_timeout_ms,
+ xe_modparam.mem_monitor_idle_timeout_ms, uint, 0644);
+MODULE_PARM_DESC(mem_monitor_idle_timeout_ms,
+ "Time in milliseconds with no GPU allocation activity in LATENCY "
+ "mode after which the monitor's polling worker goes to sleep "
+ "(woken again on the next allocation) [default="
+ __stringify(XE_MEM_MONITOR_DEFAULT_IDLE_TIMEOUT_MS) "])");
+#endif
+
static int xe_check_nomodeset(void)
{
if (drm_firmware_drivers_only())
diff --git a/drivers/gpu/drm/xe/xe_module.h b/drivers/gpu/drm/xe/xe_module.h
index 6a64ee221da6..24e0b52e48ca 100644
--- a/drivers/gpu/drm/xe/xe_module.h
+++ b/drivers/gpu/drm/xe/xe_module.h
@@ -26,6 +26,19 @@ struct xe_modparam {
#ifdef CONFIG_DRM_XE_GPUFREQTRACER
u32 gpufreq_monitoring_interval_ms;
#endif
+#ifdef CONFIG_DRM_XE_MEM_MONITOR
+ /*
+ * Master enable switch for the GPU memory growth monitor and its
+ * associated vendor hooks. Defaults is set in xe_mem_monitor.h
+ * (XE_MEM_MONITOR_DEFAULT_ENABLED). Intended to be
+ * settable as a kernel boot argument
+ */
+ bool mem_monitor_enabled;
+ u32 mem_monitor_growth_threshold_mb;
+ u32 mem_monitor_poll_ms;
+ u32 mem_monitor_debounce_count;
+ u32 mem_monitor_idle_timeout_ms;
+#endif
};
extern struct xe_modparam xe_modparam;
diff --git a/drivers/gpu/drm/xe/xe_pm.c b/drivers/gpu/drm/xe/xe_pm.c
index b291265d18fc..a330a42ea187 100644
--- a/drivers/gpu/drm/xe/xe_pm.c
+++ b/drivers/gpu/drm/xe/xe_pm.c
@@ -24,6 +24,7 @@
#include "xe_i2c.h"
#include "xe_irq.h"
#include "xe_late_bind_fw.h"
+#include "xe_mem_monitor.h"
#include "xe_pcode.h"
#include "xe_pxp.h"
#include "xe_sriov_vf_ccs.h"
@@ -129,6 +130,7 @@ int xe_pm_suspend(struct xe_device *xe)
trace_xe_pm_suspend(xe, __builtin_return_address(0));
xe_gpufreqtracer_suspend_workers(xe);
+ xe_mem_monitor_suspend(xe);
err = xe_pxp_pm_suspend(xe->pxp);
if (err)
@@ -219,6 +221,7 @@ int xe_pm_resume(struct xe_device *xe)
goto err;
xe_gpufreqtracer_resume_workers(xe);
+ xe_mem_monitor_resume(xe);
xe_pxp_pm_resume(xe->pxp);
@@ -515,6 +518,7 @@ int xe_pm_runtime_suspend(struct xe_device *xe)
xe_rpm_lockmap_acquire(xe);
xe_gpufreqtracer_suspend_workers(xe);
+ xe_mem_monitor_suspend(xe);
err = xe_pxp_pm_suspend(xe->pxp);
if (err)
@@ -616,6 +620,7 @@ int xe_pm_runtime_resume(struct xe_device *xe)
}
xe_gpufreqtracer_resume_workers(xe);
+ xe_mem_monitor_resume(xe);
xe_pxp_pm_resume(xe->pxp);
--
2.34.1
^ permalink raw reply related [flat|nested] 5+ messages in thread* [PATCH 2/4] RFC: ANDROID: drm/xe: register ttm page alloc vendor hooks internally
2026-09-07 6:07 [PATCH 0/4] RFC: drm/xe: Dynamic Xe Memory Monitor S Sebinraj
2026-09-07 6:07 ` [PATCH 1/4] RFC: ANDROID: drm/xe: add adaptive GPU memory growth monitor xe_mem_monitor S Sebinraj
@ 2026-09-07 6:07 ` S Sebinraj
2026-09-07 6:07 ` [PATCH 3/4] RFC: ANDROID: drm/xe: provide safe mem_monitor enable toggle via debugfs S Sebinraj
2026-09-07 6:07 ` [PATCH 4/4] RFC: ANDROID: drm/xe: Gate mem_monitor wake ups on order 9 external fragmentation S Sebinraj
3 siblings, 0 replies; 5+ messages in thread
From: S Sebinraj @ 2026-09-07 6:07 UTC (permalink / raw)
To: intel-xe; +Cc: S Sebinraj, Carlos Santa, Ryan Neph, S Sebinraj
From: S Sebinraj <s.sebinraj@intel.corp-partner.google.com>
Register the TTM page allocation vendor hooks
(android_vh_ttm_pool_alloc_max_page_order,
android_vh_ttm_pool_alloc_page_flags) directly from Xe, rather than via
the standalone intel_ttm_vendor_hooks.ko module, for direct, low
overhead access to xe_device state.
xe_register_android_vendor_hooks()/xe_unregister_android_vendor_hooks()
are called from xe_device_create()/xe_device_destroy() respectively.
Both are gated on the xe_modparam.mem_monitor_enabled master switch
(shared with xe_mem_monitor): if disabled, the hooks are never
registered at all, leaving TTM's default GFP flag behavior completely
untouched.
The hook caps the max TTM page order at 9 (2M pages) and adjusts GFP
flags for that order only based on xe->gpu_mem_mode, which is maintained
by xe_mem_monitor:
XE_GPU_MEM_MODE_LATENCY (default):
strip __GFP_DIRECT_RECLAIM and __GFP_KSWAPD_RECLAIM so 2M page
allocations fail fast rather than stalling on reclaim.
__GFP_NORETRY (already set by TTM for order>0) is left untouched.
XE_GPU_MEM_MODE_HIGH_THROUGHPUT:
explicitly set __GFP_DIRECT_RECLAIM and __GFP_KSWAPD_RECLAIM, clear
__GFP_NORETRY, and add __GFP_RETRY_MAYFAIL so the allocator retries
harder (including shrinking the page cache) to satisfy 2M page
allocations during bursts of GPU memory growth.
__GFP_NORETRY and __GFP_RETRY_MAYFAIL represent mutually exclusive
points on the same "costly allocation" retry spectrum (see
Documentation / gfp_types.h); __GFP_NORETRY is cleared explicitly
here since TTM sets it unconditionally for order>0 allocations and
leaving it set would blunt the effect of __GFP_RETRY_MAYFAIL.
- order-0: leave flags unmodified
- order-9: select based on gpu_mem_mode
- all others: ~gfp_flags &= ~(__GFP_DIRECT_RECLAIM | __GFP_KSWAPD_RECLAIM)
A message (rate-limited to mode transitions only, not per allocation) is
logged whenever the effective mode changes, to aid verification. A new
xe_gpu_mem_mode_switch tracepoint is also emitted on every mode
transition (from xe_mem_monitor, see xe_trace.h) so the switching
behavior can be recorded in Perfetto/ftrace traces for debugging and
parameter tuning.
Test: confirmed hook fires and GFP flags change correctly across mode
transitions via dmesg while running WebGL Aquarium / WLEU
benchmark. Confirmed xe_gpu_mem_mode_switch tracepoint fires on
each transition via ftrace. Confirmed hooks are not registered
when mem_monitor_enabled=0 is passed as a boot argument.
Cc: Carlos Santa <carlos.santa@intel.com>
Signed-off-by: Ryan Neph <ryanneph@google.com>
Signed-off-by: S Sebinraj <s.sebinraj@intel.com>
---
drivers/gpu/drm/BUILD.bazel | 1 +
drivers/gpu/drm/xe/xe_device.c | 6 +
drivers/gpu/drm/xe/xe_mem_monitor.c | 15 ++-
drivers/gpu/drm/xe/xe_trace.h | 16 +++
drivers/gpu/drm/xe/xe_ttm_vendor_hooks.c | 135 +++++++++++++++++++++++
drivers/gpu/drm/xe/xe_ttm_vendor_hooks.h | 9 ++
6 files changed, 178 insertions(+), 4 deletions(-)
create mode 100644 drivers/gpu/drm/xe/xe_ttm_vendor_hooks.c
create mode 100644 drivers/gpu/drm/xe/xe_ttm_vendor_hooks.h
diff --git a/drivers/gpu/drm/BUILD.bazel b/drivers/gpu/drm/BUILD.bazel
index 274a1aa4ca93..f4fc4b94f71f 100644
--- a/drivers/gpu/drm/BUILD.bazel
+++ b/drivers/gpu/drm/BUILD.bazel
@@ -127,6 +127,7 @@ ddk_module(
"xe/xe_trace_lrc.c",
"xe/xe_ttm_stolen_mgr.c",
"xe/xe_ttm_sys_mgr.c",
+ "xe/xe_ttm_vendor_hooks.c",
"xe/xe_ttm_vram_mgr.c",
"xe/xe_tuning.c",
"xe/xe_uc.c",
diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
index 676ffc258b94..d21068c1effc 100644
--- a/drivers/gpu/drm/xe/xe_device.c
+++ b/drivers/gpu/drm/xe/xe_device.c
@@ -69,6 +69,7 @@
#include "xe_tile.h"
#include "xe_ttm_stolen_mgr.h"
#include "xe_ttm_sys_mgr.h"
+#include "xe_ttm_vendor_hooks.h"
#include "xe_vm.h"
#include "xe_vm_madvise.h"
#include "xe_vram.h"
@@ -419,6 +420,7 @@ static void xe_device_destroy(struct drm_device *dev, void *dummy)
{
struct xe_device *xe = to_xe_device(dev);
+ xe_unregister_android_vendor_hooks(xe);
xe_bo_dev_fini(&xe->bo_device);
if (xe->preempt_fence_wq)
@@ -464,6 +466,10 @@ struct xe_device *xe_device_create(struct pci_dev *pdev,
if (err)
goto err;
+ err = xe_register_android_vendor_hooks(xe);
+ if (err)
+ goto err;
+
err = xe_shrinker_create(xe);
if (err)
goto err;
diff --git a/drivers/gpu/drm/xe/xe_mem_monitor.c b/drivers/gpu/drm/xe/xe_mem_monitor.c
index 06fbd0624da7..9205417bfb0b 100644
--- a/drivers/gpu/drm/xe/xe_mem_monitor.c
+++ b/drivers/gpu/drm/xe/xe_mem_monitor.c
@@ -69,6 +69,7 @@
#include "xe_device.h"
#include "xe_device_types.h"
#include "xe_module.h"
+#include "xe_trace.h"
/*
* EMA smoothing factor. alpha (13/16 = 0.8125) applies to every real
@@ -157,18 +158,24 @@ struct xe_mem_monitor_data {
static void xe_mem_monitor_set_high_throughput_mode(struct xe_mem_monitor_data *mon)
{
- /* TODO: wire up to the actual vendor hook (adds __GFP_RETRY_MAYFAIL, TBD) */
+ /*
+ * The TTM vendor hook (xe_ttm_vendor_hooks.c) reads xe->gpu_mem_mode
+ * directly on the next 2M page allocation, so updating the mode here
+ * is sufficient - no separate hook call is needed.
+ */
drm_dbg(&mon->xe->drm,
- "xe_mem_monitor: switching to HIGH_THROUGHPUT mode (TBD)\n");
+ "xe_mem_monitor: switching to HIGH_THROUGHPUT mode\n");
atomic_set(&mon->xe->gpu_mem_mode, XE_GPU_MEM_MODE_HIGH_THROUGHPUT);
+ trace_xe_gpu_mem_mode_switch(XE_GPU_MEM_MODE_HIGH_THROUGHPUT, mon->ema_growth_bytes);
}
static void xe_mem_monitor_set_latency_mode(struct xe_mem_monitor_data *mon)
{
- /* TODO: wire up to the actual vendor hook (TBD) */
+ /* See xe_mem_monitor_set_high_throughput_mode() above. */
drm_dbg(&mon->xe->drm,
- "xe_mem_monitor: switching to LATENCY mode (TBD)\n");
+ "xe_mem_monitor: switching to LATENCY mode\n");
atomic_set(&mon->xe->gpu_mem_mode, XE_GPU_MEM_MODE_LATENCY);
+ trace_xe_gpu_mem_mode_switch(XE_GPU_MEM_MODE_LATENCY, mon->ema_growth_bytes);
}
static void xe_mem_monitor_work(struct work_struct *work)
diff --git a/drivers/gpu/drm/xe/xe_trace.h b/drivers/gpu/drm/xe/xe_trace.h
index 314f42fcbcbd..77bfc08613b9 100644
--- a/drivers/gpu/drm/xe/xe_trace.h
+++ b/drivers/gpu/drm/xe/xe_trace.h
@@ -441,6 +441,22 @@ TRACE_EVENT(xe_eu_stall_data_read,
__entry->read_size, __entry->total_size)
);
+TRACE_EVENT(xe_gpu_mem_mode_switch,
+ TP_PROTO(int mode, u64 growth_bytes),
+ TP_ARGS(mode, growth_bytes),
+
+ TP_STRUCT__entry(__field(int, mode)
+ __field(u64, growth_bytes)
+ ),
+
+ TP_fast_assign(__entry->mode = mode;
+ __entry->growth_bytes = growth_bytes;
+ ),
+
+ TP_printk("mode=%d, growth_bytes=%llu", __entry->mode,
+ __entry->growth_bytes)
+);
+
#endif
/* This part must be outside protection */
diff --git a/drivers/gpu/drm/xe/xe_ttm_vendor_hooks.c b/drivers/gpu/drm/xe/xe_ttm_vendor_hooks.c
new file mode 100644
index 000000000000..b4136ac6cc88
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_ttm_vendor_hooks.c
@@ -0,0 +1,135 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2026 Google LLC.
+ *
+ * TTM page allocation vendor hooks, registered internally by Xe (rather than
+ * via the standalone intel_ttm_vendor_hooks.ko module) for direct, low
+ * overhead access to xe_device state.
+ *
+ * The GFP flags for 2M (order-9) page allocations are adjusted based on
+ * xe->gpu_mem_mode, which is maintained by xe_mem_monitor:
+ *
+ * XE_GPU_MEM_MODE_LATENCY: strip __GFP_DIRECT_RECLAIM and
+ * __GFP_KSWAPD_RECLAIM for fast, low
+ * latency allocation. __GFP_NORETRY (set
+ * by TTM for order>0) is left untouched.
+ * XE_GPU_MEM_MODE_HIGH_THROUGHPUT: ensure __GFP_DIRECT_RECLAIM and
+ * __GFP_KSWAPD_RECLAIM are set, clear
+ * __GFP_NORETRY (set by TTM for order>0,
+ * and mutually exclusive in intent with
+ * __GFP_RETRY_MAYFAIL below), and add
+ * __GFP_RETRY_MAYFAIL to maximize the
+ * chance of a successful 2M page
+ * allocation.
+ *
+ * All other orders (including order-0) are left completely unmodified.
+ */
+
+#include <linux/atomic.h>
+#include <linux/gfp.h>
+#include <linux/printk.h>
+#include <trace/hooks/ttm_pool.h>
+
+#include "xe_device_types.h"
+#include "xe_mem_monitor.h"
+#include "xe_module.h"
+#include "xe_ttm_vendor_hooks.h"
+
+/* 2M pages (order-9) are the only order whose GFP flags are adjusted. */
+#define XE_TTM_VENDOR_HOOKS_ORDER 9
+
+static void xe_android_vh_ttm_pool_alloc_max_page_order(void *data,
+ unsigned int *max_order)
+{
+ *max_order = XE_TTM_VENDOR_HOOKS_ORDER;
+}
+
+static void xe_android_vh_ttm_pool_alloc_page_flags(void *data,
+ unsigned int order,
+ gfp_t *gfp_flags)
+{
+ struct xe_device *xe = (struct xe_device *)data;
+ int mode;
+
+ /* order-0 must always be with __GFP_DIRECT_RECLAIM to avoid ENOMEM
+ * unless it is absolutely necessary (system memory is completely
+ * exhausted).
+ */
+ if (!order)
+ return;
+
+ mode = atomic_read(&xe->gpu_mem_mode);
+
+ if (order == XE_TTM_VENDOR_HOOKS_ORDER &&
+ mode == XE_GPU_MEM_MODE_HIGH_THROUGHPUT) {
+ /*
+ * Ensure reclaim is allowed, clear __GFP_NORETRY (set by TTM for
+ * order>0, but mutually exclusive in intent with
+ * __GFP_RETRY_MAYFAIL) and add __GFP_RETRY_MAYFAIL so the
+ * allocator retries harder for a 2M page instead of failing fast.
+ * Order-9 (2MB) is specifically targeted here as the underlying IOMMU
+ * hardware and Xe architecture operate most efficiently with 2MB alignments.
+ */
+ *gfp_flags |= (__GFP_DIRECT_RECLAIM | __GFP_KSWAPD_RECLAIM | __GFP_RETRY_MAYFAIL);
+ *gfp_flags &= ~__GFP_NORETRY;
+ } else {
+ *gfp_flags &= ~(__GFP_DIRECT_RECLAIM | __GFP_KSWAPD_RECLAIM);
+ }
+}
+
+int xe_register_android_vendor_hooks(struct xe_device *xe)
+{
+ int ret;
+
+ /*
+ * Gated by the same master switch as xe_mem_monitor: if the feature is
+ * disabled, do not register the hooks at all so TTM's default GFP flag
+ * behavior is left completely untouched.
+ */
+ if (!xe_modparam.mem_monitor_enabled)
+ return 0;
+
+ ret = register_trace_android_vh_ttm_pool_alloc_max_page_order(
+ xe_android_vh_ttm_pool_alloc_max_page_order, (void *)xe);
+ if (ret) {
+ pr_warn("failed to register ttm_pool_alloc_max_page_order hook: %d\n",
+ ret);
+ goto fail;
+ }
+
+ ret = register_trace_android_vh_ttm_pool_alloc_page_flags(
+ xe_android_vh_ttm_pool_alloc_page_flags, (void *)xe);
+ if (ret) {
+ pr_warn("failed to register ttm_pool_alloc_page_flags hook: %d\n",
+ ret);
+ goto fail_alloc_page_flags;
+ }
+
+ return 0;
+
+fail_alloc_page_flags:
+ unregister_trace_android_vh_ttm_pool_alloc_max_page_order(
+ xe_android_vh_ttm_pool_alloc_max_page_order, (void *)xe);
+fail:
+ return ret;
+}
+
+void xe_unregister_android_vendor_hooks(struct xe_device *xe)
+{
+ int ret;
+
+ if (!xe_modparam.mem_monitor_enabled)
+ return;
+
+ ret = unregister_trace_android_vh_ttm_pool_alloc_max_page_order(
+ xe_android_vh_ttm_pool_alloc_max_page_order, (void *)xe);
+ if (ret)
+ pr_warn("failed to unregister ttm_pool_alloc_max_page_order hook: %d\n",
+ ret);
+
+ ret = unregister_trace_android_vh_ttm_pool_alloc_page_flags(
+ xe_android_vh_ttm_pool_alloc_page_flags, (void *)xe);
+ if (ret)
+ pr_warn("failed to unregister ttm_pool_alloc_page_flags hook: %d\n",
+ ret);
+}
diff --git a/drivers/gpu/drm/xe/xe_ttm_vendor_hooks.h b/drivers/gpu/drm/xe/xe_ttm_vendor_hooks.h
new file mode 100644
index 000000000000..f2059a3822ae
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_ttm_vendor_hooks.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2026 Google LLC.
+ */
+
+struct xe_device;
+
+int xe_register_android_vendor_hooks(struct xe_device *xe);
+void xe_unregister_android_vendor_hooks(struct xe_device *xe);
--
2.34.1
^ permalink raw reply related [flat|nested] 5+ messages in thread* [PATCH 3/4] RFC: ANDROID: drm/xe: provide safe mem_monitor enable toggle via debugfs
2026-09-07 6:07 [PATCH 0/4] RFC: drm/xe: Dynamic Xe Memory Monitor S Sebinraj
2026-09-07 6:07 ` [PATCH 1/4] RFC: ANDROID: drm/xe: add adaptive GPU memory growth monitor xe_mem_monitor S Sebinraj
2026-09-07 6:07 ` [PATCH 2/4] RFC: ANDROID: drm/xe: register ttm page alloc vendor hooks internally S Sebinraj
@ 2026-09-07 6:07 ` S Sebinraj
2026-09-07 6:07 ` [PATCH 4/4] RFC: ANDROID: drm/xe: Gate mem_monitor wake ups on order 9 external fragmentation S Sebinraj
3 siblings, 0 replies; 5+ messages in thread
From: S Sebinraj @ 2026-09-07 6:07 UTC (permalink / raw)
To: intel-xe; +Cc: Ryan Neph, S Sebinraj
From: Ryan Neph <ryanneph@google.com>
Provide a way to safely enable/disable mem_monitor at runtime and modify
its parameters with the sequence:
```
su
echo 0 > /sys/kernel/debug/dri/0/mem_monitor/enabled
; # modify params: /sys/module/xe/parameters/mem_monitor_*
echo 1 > /sys/kernel/debug/dri/0/mem_monitor/enabled
```
Also adds to /sys/kernel/debug/dri/0/mem_monitor/:
- ./info: to show state of the mem_monitor
- ./mode: to show/set the current mode
Set the mode by writing "latency" or "throughput" to the file.
If mem_monitor is:
- disabled, this mode is persistent.
- enabled, this mode may be changed by the mem_monitor
Test: echo 0 > /sys/kernel/dri/0/mem_monitor/enabled # disables worker
Test: echo 1 > /sys/kernel/dri/0/mem_monitor/enabled # re-enables worker
Signed-off-by: Ryan Neph <ryanneph@google.com>
Signed-off-by: S Sebinraj <s.sebinraj@intel.com>
---
drivers/gpu/drm/xe/xe_device.c | 1 +
drivers/gpu/drm/xe/xe_device_types.h | 1 +
drivers/gpu/drm/xe/xe_mem_monitor.c | 153 ++++++++++++++++++++++++++-
drivers/gpu/drm/xe/xe_mem_monitor.h | 11 ++
drivers/gpu/drm/xe/xe_module.c | 1 +
5 files changed, 166 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
index d21068c1effc..c8cec778cc97 100644
--- a/drivers/gpu/drm/xe/xe_device.c
+++ b/drivers/gpu/drm/xe/xe_device.c
@@ -941,6 +941,7 @@ int xe_device_probe(struct xe_device *xe)
err = xe_mem_monitor_init(xe);
if (err)
return err;
+ xe_mem_monitor_debugfs_register(xe);
err = xe_late_bind_init(&xe->late_bind);
if (err)
diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
index 48aa9d8f6f56..b920a412026b 100644
--- a/drivers/gpu/drm/xe/xe_device_types.h
+++ b/drivers/gpu/drm/xe/xe_device_types.h
@@ -586,6 +586,7 @@ struct xe_device {
/** @mem_monitor: GPU memory growth monitor data */
struct xe_mem_monitor_data *mem_monitor;
+ struct dentry *mem_monitor_debugfs_root;
/** @needs_flr_on_fini: requests function-reset on fini */
bool needs_flr_on_fini;
diff --git a/drivers/gpu/drm/xe/xe_mem_monitor.c b/drivers/gpu/drm/xe/xe_mem_monitor.c
index 9205417bfb0b..a34cafe56976 100644
--- a/drivers/gpu/drm/xe/xe_mem_monitor.c
+++ b/drivers/gpu/drm/xe/xe_mem_monitor.c
@@ -59,9 +59,12 @@
#include "xe_mem_monitor.h"
#include <linux/atomic.h>
+#include <linux/debugfs.h>
#include <linux/jiffies.h>
#include <linux/minmax.h>
+#include <linux/seq_file.h>
#include <linux/slab.h>
+#include <linux/string.h>
#include <linux/workqueue.h>
#include <drm/drm_managed.h>
#include <drm/drm_print.h>
@@ -154,6 +157,11 @@ struct xe_mem_monitor_data {
bool is_high_throughput;
atomic_t suspended;
bool was_pending_before_suspend;
+
+ struct {
+ atomic64_t current_bytes;
+ atomic64_t delta_signed;
+ } info;
};
static void xe_mem_monitor_set_high_throughput_mode(struct xe_mem_monitor_data *mon)
@@ -247,6 +255,9 @@ static void xe_mem_monitor_work(struct work_struct *work)
ALPHA_RETAINED * mon->ema_growth_bytes)
>> ALPHA_SHIFT;
+ atomic64_set(&mon->info.current_bytes, current_bytes);
+ atomic64_set(&mon->info.delta_signed, delta_signed);
+
atomic64_set(&mon->prev_mem_bytes, current_bytes);
mon->last_poll_jiffies = now;
@@ -320,8 +331,21 @@ static void xe_mem_monitor_work(struct work_struct *work)
static void xe_mem_monitor_cleanup(struct drm_device *drm, void *arg)
{
struct xe_mem_monitor_data *mon = arg;
+ struct xe_device *xe = mon->xe;
+ WARN_ON_ONCE(!xe->mem_monitor);
cancel_delayed_work_sync(&mon->work);
+ kfree(mon);
+ xe->mem_monitor = NULL;
+}
+
+void xe_mem_monitor_fini(struct xe_device *xe)
+{
+ if (!xe->mem_monitor)
+ return;
+
+ drmm_release_action(&xe->drm, xe_mem_monitor_cleanup, (void *)xe->mem_monitor);
+ drm_info(&xe->drm, "xe_mem_monitor: stopped");
}
/**
@@ -437,7 +461,7 @@ int xe_mem_monitor_init(struct xe_device *xe)
return 0;
}
- mon = drmm_kzalloc(&xe->drm, sizeof(*mon), GFP_KERNEL);
+ mon = kzalloc(sizeof(*mon), GFP_KERNEL);
if (!mon)
return -ENOMEM;
@@ -467,3 +491,130 @@ int xe_mem_monitor_init(struct xe_device *xe)
return 0;
}
+
+static ssize_t enabled_write(struct file *file,
+ const char __user *userbuf,
+ size_t count, loff_t *ppos)
+{
+ struct seq_file *s = file->private_data;
+ struct xe_device *xe = s->private;
+ bool enable;
+ int ret;
+
+ ret = kstrtobool_from_user(userbuf, count, &enable);
+ if (ret)
+ return ret;
+
+ if (!enable && xe->mem_monitor)
+ xe_mem_monitor_fini(xe);
+ else if (enable && !xe->mem_monitor)
+ xe_mem_monitor_init(xe);
+
+ return count;
+}
+static int enabled_show(struct seq_file *s, void *data)
+{
+ struct xe_device *xe = s->private;
+
+ seq_printf(s, "%c\n", xe->mem_monitor ? 'Y' : 'N');
+ return 0;
+}
+DEFINE_SHOW_STORE_ATTRIBUTE(enabled);
+
+static int info_show(struct seq_file *s, void *data)
+{
+ struct xe_device *xe = s->private;
+ struct xe_mem_monitor_data *mon = xe->mem_monitor;
+
+ if (!mon)
+ return 1;
+
+ seq_printf(s, "current=%llu MB, delta=%lld MB, EMA=%llu MB mode=%d\n",
+ atomic64_read(&mon->info.current_bytes) >> 20,
+ atomic64_read(&mon->info.delta_signed) >> 20,
+ mon->ema_growth_bytes >> 20,
+ atomic_read(&xe->gpu_mem_mode));
+ return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(info);
+
+static ssize_t mode_write(struct file *file, const char __user *userbuf,
+ size_t count, loff_t *ppos)
+{
+ struct seq_file *s = file->private_data;
+ struct xe_device *xe = s->private;
+ char buf[32];
+ size_t len;
+
+ len = min(count, sizeof(buf) - 1);
+ if (copy_from_user(buf, userbuf, len))
+ return -EFAULT;
+ buf[len] = '\0';
+
+ if (sysfs_streq(buf, "latency"))
+ atomic_set(&xe->gpu_mem_mode, XE_GPU_MEM_MODE_LATENCY);
+ else if (sysfs_streq(buf, "throughput"))
+ atomic_set(&xe->gpu_mem_mode, XE_GPU_MEM_MODE_HIGH_THROUGHPUT);
+
+ return count;
+}
+
+static int mode_show(struct seq_file *s, void *data)
+{
+ struct xe_device *xe = s->private;
+ enum xe_gpu_mem_mode mode = atomic_read(&xe->gpu_mem_mode);
+
+ /*
+ * Print the options, placing brackets around the currently active mode.
+ */
+ switch (mode) {
+ case XE_GPU_MEM_MODE_LATENCY:
+ seq_puts(s, "[latency] throughput\n");
+ break;
+ case XE_GPU_MEM_MODE_HIGH_THROUGHPUT:
+ seq_puts(s, "latency [throughput]\n");
+ break;
+ default:
+ seq_puts(s, "latency throughput\n");
+ break;
+ }
+
+ return 0;
+}
+DEFINE_SHOW_STORE_ATTRIBUTE(mode);
+
+static void xe_mem_monitor_debugfs_unregister(struct drm_device *dev,
+ void *data)
+{
+ struct xe_device *xe = data;
+
+ debugfs_remove(xe->mem_monitor_debugfs_root);
+}
+
+/* Exposes controls through debugfs under the drm root for this device:
+ * /sys/kernel/dri/0/mem_monitor/
+ */
+int xe_mem_monitor_debugfs_register(struct xe_device *xe)
+{
+ struct dentry *root;
+ int ret;
+
+ root = debugfs_create_dir("mem_monitor", xe->drm.debugfs_root);
+ if (IS_ERR(root)) {
+ drm_warn(&xe->drm, "failed to create mem_monitor debug directory");
+ return 1;
+ }
+ root->d_inode->i_private = xe;
+ xe->mem_monitor_debugfs_root = root;
+
+ if (xe_modparam.mem_monitor_enabled) {
+ WARN_ON(!xe->mem_monitor);
+ debugfs_create_file("enabled", 0600, root, xe, &enabled_fops);
+ debugfs_create_file("info", 0400, root, xe, &info_fops);
+ }
+ debugfs_create_file("mode", 0600, root, xe, &mode_fops);
+
+ ret = drmm_add_action_or_reset(&xe->drm,
+ xe_mem_monitor_debugfs_unregister, xe);
+ return ret;
+}
diff --git a/drivers/gpu/drm/xe/xe_mem_monitor.h b/drivers/gpu/drm/xe/xe_mem_monitor.h
index 846c0628ed93..c617c10c1a12 100644
--- a/drivers/gpu/drm/xe/xe_mem_monitor.h
+++ b/drivers/gpu/drm/xe/xe_mem_monitor.h
@@ -39,6 +39,8 @@ enum xe_gpu_mem_mode {
#ifdef CONFIG_DRM_XE_MEM_MONITOR
int xe_mem_monitor_init(struct xe_device *xe);
+void xe_mem_monitor_fini(struct xe_device *xe);
+int xe_mem_monitor_debugfs_register(struct xe_device *xe);
/*
* Notify the monitor that GPU allocation activity occurred (called from the
@@ -77,6 +79,15 @@ static inline void xe_mem_monitor_resume(struct xe_device *xe)
{
}
+static inline void xe_mem_monitor_fini(struct xe_device *xe)
+{
+}
+
+static inline int xe_mem_monitor_debugfs_register(struct xe_device *xe)
+{
+ return 0;
+}
+
#endif /* CONFIG_DRM_XE_MEM_MONITOR */
#endif /* _XE_MEM_MONITOR_H_ */
diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
index a49b0c0f958b..a38776734544 100644
--- a/drivers/gpu/drm/xe/xe_module.c
+++ b/drivers/gpu/drm/xe/xe_module.c
@@ -5,6 +5,7 @@
#include "xe_module.h"
+#include "linux/log2.h"
#include <linux/init.h>
#include <linux/module.h>
#include <linux/stringify.h>
--
2.34.1
^ permalink raw reply related [flat|nested] 5+ messages in thread* [PATCH 4/4] RFC: ANDROID: drm/xe: Gate mem_monitor wake ups on order 9 external fragmentation
2026-09-07 6:07 [PATCH 0/4] RFC: drm/xe: Dynamic Xe Memory Monitor S Sebinraj
` (2 preceding siblings ...)
2026-09-07 6:07 ` [PATCH 3/4] RFC: ANDROID: drm/xe: provide safe mem_monitor enable toggle via debugfs S Sebinraj
@ 2026-09-07 6:07 ` S Sebinraj
3 siblings, 0 replies; 5+ messages in thread
From: S Sebinraj @ 2026-09-07 6:07 UTC (permalink / raw)
To: intel-xe
Cc: S Sebinraj, Carlos Santa, Erin Park, Ryan Neph, Renato Pereyra,
S Sebinraj
From: S Sebinraj <s.sebinraj@intel.corp-partner.google.com>
xe_mem_monitor_notify_activity() previously woke the polling worker
based only on a raw byte-level delta (ACTIVITY_MIN_DELTA_BYTES), which
doesn't capture whether order 9 (2M) TTM allocations are actually
under any pressure.
Add xe_mem_monitor_zone_order9_free_bytes(), which reads the amount of
order 9 or larger free memory directly from struct zone's free_area
counts.
Add xe_mem_monitor_order9_supply_low(), which walks the populated zones
and returns true if any zone's order 9 free supply has dropped below
an adaptive floor. The floor is mon->max_delta_bytes, the largest
growth sample xe_mem_monitor_work() has ever observed on this device
clamped between the mem_monitor_order9_supply_min_mb and
mem_monitor_order9_supply_max_mb module parameters (in MiB).
Adapting the floor to the largest growth sample actually observed
avoids false positives from a single large but otherwise harmless
allocation burst when overall order 9 supply is still healthy.
The upper clamp (mem_monitor_order9_supply_max_mb) stops a single
anomalously large allocation from permanently pinning the floor too high.
Use this as an additional guard in xe_mem_monitor_notify_activity
both the existing byte-delta check and this free-supply check must
now indicate pressure before the worker is woken.
Test: Tested by running WLEU on a fragmented system and confirmed the
worker is only woken when some populated zone's order-9 free
supply drops below ORDER9_SUPPLY_MIN_PCT of total RAM.
Cc: Carlos Santa <carlos.santa@intel.com>
Cc: Erin Park <erin.park@intel.com>
Cc: Ryan Neph <ryanneph@google.com>
Cc: Renato Pereyra <renatopereyra@google.com>
Signed-off-by: S Sebinraj <s.sebinraj@intel.com>
---
drivers/gpu/drm/xe/xe_mem_monitor.c | 115 +++++++++++++++++++++++++++-
drivers/gpu/drm/xe/xe_mem_monitor.h | 2 +
drivers/gpu/drm/xe/xe_module.c | 22 ++++++
drivers/gpu/drm/xe/xe_module.h | 2 +
4 files changed, 137 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/xe/xe_mem_monitor.c b/drivers/gpu/drm/xe/xe_mem_monitor.c
index a34cafe56976..24458af2700c 100644
--- a/drivers/gpu/drm/xe/xe_mem_monitor.c
+++ b/drivers/gpu/drm/xe/xe_mem_monitor.c
@@ -59,9 +59,11 @@
#include "xe_mem_monitor.h"
#include <linux/atomic.h>
+#include <linux/compiler.h>
#include <linux/debugfs.h>
#include <linux/jiffies.h>
#include <linux/minmax.h>
+#include <linux/mmzone.h>
#include <linux/seq_file.h>
#include <linux/slab.h>
#include <linux/string.h>
@@ -129,6 +131,13 @@
* @last_poll_jiffies: jiffies value of the most recent real sample taken
* by xe_mem_monitor_work(), used to detect and decay
* the EMA across skipped (slept-through) intervals
+ * @max_delta_bytes: largest clamped_delta (growth-only, per real
+ * sample) ever observed by xe_mem_monitor_work(),
+ * used by xe_mem_monitor_order9_supply_low() to
+ * adapt its floor upward to this workload's
+ * demonstrated behaviour (clamped between
+ * mem_monitor_order9_supply_min_mb and
+ * mem_monitor_order9_supply_max_mb there).
* @suspended: set while the device is suspended (system or
* runtime PM); xe_mem_monitor_notify_activity() is a
* no-op while set, so that BO eviction/restore
@@ -155,6 +164,7 @@ struct xe_mem_monitor_data {
unsigned long last_activity_jiffies;
unsigned long last_poll_jiffies;
bool is_high_throughput;
+ u64 max_delta_bytes;
atomic_t suspended;
bool was_pending_before_suspend;
@@ -186,6 +196,26 @@ static void xe_mem_monitor_set_latency_mode(struct xe_mem_monitor_data *mon)
trace_xe_gpu_mem_mode_switch(XE_GPU_MEM_MODE_LATENCY, mon->ema_growth_bytes);
}
+/*
+ * Returns the total amount of memory (bytes) available as order-9 (2M) or
+ * larger contiguous blocks in the given zone. Computed directly from
+ * struct zone's free_area[] counts (see <linux/mmzone.h>)
+ */
+static u64 xe_mem_monitor_zone_order9_free_bytes(struct zone *zone)
+{
+ unsigned long free_blocks_suitable = 0;
+ unsigned int order;
+
+ for (order = 9; order < NR_PAGE_ORDERS; order++) {
+ /* nr_free is lockless/diagnostic-only */
+ unsigned long blocks = data_race(zone->free_area[order].nr_free);
+
+ free_blocks_suitable += blocks << (order - 9);
+ }
+
+ return (u64)free_blocks_suitable << (9 + PAGE_SHIFT);
+}
+
static void xe_mem_monitor_work(struct work_struct *work)
{
struct xe_mem_monitor_data *mon =
@@ -255,6 +285,8 @@ static void xe_mem_monitor_work(struct work_struct *work)
ALPHA_RETAINED * mon->ema_growth_bytes)
>> ALPHA_SHIFT;
+ mon->max_delta_bytes = max_t(u64, mon->max_delta_bytes, clamped_delta);
+
atomic64_set(&mon->info.current_bytes, current_bytes);
atomic64_set(&mon->info.delta_signed, delta_signed);
@@ -348,6 +380,58 @@ void xe_mem_monitor_fini(struct xe_device *xe)
drm_info(&xe->drm, "xe_mem_monitor: stopped");
}
+/*
+ * Companion check to ACTIVITY_MIN_DELTA_BYTES for
+ * xe_mem_monitor_notify_activity(): a large byte-level delta doesn't
+ * necessarily mean order-9 (2M) allocations are under any real pressure, so
+ * this looks instead at whether any single populated zone's absolute
+ * order-9-and-larger free supply has dropped below the adaptive floor
+ * clamp_t(mon->max_delta_bytes, mem_monitor_order9_supply_min_mb,
+ * mem_monitor_order9_supply_max_mb) independent of the size of whatever
+ * delta triggered this call, since a single large delta is not itself
+ * evidence that supply is actually running low. Taking mon->max_delta_bytes
+ * (the largest growth sample ever observed on this device, see
+ * xe_mem_monitor_work()) as the basis, clamped between the two module
+ * parameters, means the floor adapts upward to this workload's own
+ * demonstrated behaviour without being able to grow unbounded from a
+ * single anomalous allocation.
+ */
+#if IS_ENABLED(CONFIG_COMPACTION)
+static bool xe_mem_monitor_order9_supply_low(struct xe_mem_monitor_data *mon)
+{
+ struct xe_device *xe = mon->xe;
+ struct pglist_data *pgdat = NODE_DATA(0);
+ u64 min_free_bytes = clamp_t(u64, mon->max_delta_bytes,
+ (u64)xe_modparam.mem_monitor_order9_supply_min_mb << 20,
+ (u64)xe_modparam.mem_monitor_order9_supply_max_mb << 20);
+ unsigned int z;
+
+ for (z = 0; z < MAX_NR_ZONES; z++) {
+ struct zone *zone = &pgdat->node_zones[z];
+
+ if (!populated_zone(zone))
+ continue;
+
+ if (xe_mem_monitor_zone_order9_free_bytes(zone) < min_free_bytes) {
+ drm_dbg(&xe->drm,
+ "xe_mem_monitor: zone=%s order9_free=%llu MB < min=%llu MB\n",
+ zone->name,
+ xe_mem_monitor_zone_order9_free_bytes(zone) >> 20,
+ min_free_bytes >> 20);
+ return true;
+ }
+ }
+
+ return false;
+}
+#else
+static inline bool xe_mem_monitor_order9_supply_low(struct xe_mem_monitor_data *mon)
+{
+ /* No zone data available - don't gate on it. */
+ return true;
+}
+#endif
+
/**
* xe_mem_monitor_notify_activity - notify the monitor of GPU memory changes
* @xe: the Xe device
@@ -367,6 +451,12 @@ void xe_mem_monitor_fini(struct xe_device *xe)
* cross. If the worker is already scheduled (delayed_work_pending()), it
* will read the live counter fresh when it runs and pick up whatever
* accumulated, so there's no need to re-schedule again here.
+ *
+ * As a further guard, even a significant byte-level change is ignored unless
+ * xe_mem_monitor_order9_supply_low() also reports that some populated
+ * zone's order-9-and-larger free supply has dropped below the adaptive
+ * floor described there. A byte-level delta alone doesn't imply 2M
+ * allocations are under any real pressure.
*/
void xe_mem_monitor_notify_activity(struct xe_device *xe)
{
@@ -387,11 +477,14 @@ void xe_mem_monitor_notify_activity(struct xe_device *xe)
if (abs_delta < ACTIVITY_MIN_DELTA_BYTES)
return;
- mon->last_activity_jiffies = jiffies;
-
if (delayed_work_pending(&mon->work))
return;
+ if (!xe_mem_monitor_order9_supply_low(mon))
+ return;
+
+ mon->last_activity_jiffies = jiffies;
+
schedule_delayed_work(&mon->work, 0);
}
@@ -525,15 +618,29 @@ static int info_show(struct seq_file *s, void *data)
{
struct xe_device *xe = s->private;
struct xe_mem_monitor_data *mon = xe->mem_monitor;
+ int mode = atomic_read(&xe->gpu_mem_mode);
+ struct pglist_data *pgdat = NODE_DATA(0);
+ unsigned long long order9_free_bytes = 0;
if (!mon)
return 1;
- seq_printf(s, "current=%llu MB, delta=%lld MB, EMA=%llu MB mode=%d\n",
+ for (int z = 0; z < MAX_NR_ZONES; z++) {
+ struct zone *zone = &pgdat->node_zones[z];
+
+ if (!populated_zone(zone))
+ continue;
+ order9_free_bytes += xe_mem_monitor_zone_order9_free_bytes(zone);
+ }
+
+ seq_printf(s, "current=%llu MB, delta=%lld MB, EMA=%llu MB mode=%s, alpha=0.%03u, "
+ "order9_free_total=%llu MB\n",
atomic64_read(&mon->info.current_bytes) >> 20,
atomic64_read(&mon->info.delta_signed) >> 20,
mon->ema_growth_bytes >> 20,
- atomic_read(&xe->gpu_mem_mode));
+ mode == XE_GPU_MEM_MODE_LATENCY ? "latency" : "throughput",
+ (ALPHA_NUM * 1000U + (1U << (ALPHA_SHIFT - 1))) >> ALPHA_SHIFT,
+ order9_free_bytes >> 20);
return 0;
}
DEFINE_SHOW_ATTRIBUTE(info);
diff --git a/drivers/gpu/drm/xe/xe_mem_monitor.h b/drivers/gpu/drm/xe/xe_mem_monitor.h
index c617c10c1a12..a30fabd5a8ea 100644
--- a/drivers/gpu/drm/xe/xe_mem_monitor.h
+++ b/drivers/gpu/drm/xe/xe_mem_monitor.h
@@ -28,6 +28,8 @@ enum xe_gpu_mem_mode {
#define XE_MEM_MONITOR_DEFAULT_POLL_MS 2000
#define XE_MEM_MONITOR_DEFAULT_DEBOUNCE_COUNT 5
#define XE_MEM_MONITOR_DEFAULT_IDLE_TIMEOUT_MS 10000
+#define XE_MEM_MONITOR_DEFAULT_ORDER9_SUPPLY_MIN_MB 512
+#define XE_MEM_MONITOR_DEFAULT_ORDER9_SUPPLY_MAX_MB 1024
/*
* Minimum enforced value for mem_monitor_poll_ms (no maximum is
diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
index a38776734544..69825baf100c 100644
--- a/drivers/gpu/drm/xe/xe_module.c
+++ b/drivers/gpu/drm/xe/xe_module.c
@@ -54,6 +54,8 @@ struct xe_modparam xe_modparam = {
.mem_monitor_poll_ms = XE_MEM_MONITOR_DEFAULT_POLL_MS,
.mem_monitor_debounce_count = XE_MEM_MONITOR_DEFAULT_DEBOUNCE_COUNT,
.mem_monitor_idle_timeout_ms = XE_MEM_MONITOR_DEFAULT_IDLE_TIMEOUT_MS,
+ .mem_monitor_order9_supply_min_mb = XE_MEM_MONITOR_DEFAULT_ORDER9_SUPPLY_MIN_MB,
+ .mem_monitor_order9_supply_max_mb = XE_MEM_MONITOR_DEFAULT_ORDER9_SUPPLY_MAX_MB,
#endif
/* the rest are 0 by default */
};
@@ -162,6 +164,26 @@ MODULE_PARM_DESC(mem_monitor_idle_timeout_ms,
"mode after which the monitor's polling worker goes to sleep "
"(woken again on the next allocation) [default="
__stringify(XE_MEM_MONITOR_DEFAULT_IDLE_TIMEOUT_MS) "])");
+
+module_param_named(mem_monitor_order9_supply_min_mb,
+ xe_modparam.mem_monitor_order9_supply_min_mb, uint, 0644);
+MODULE_PARM_DESC(mem_monitor_order9_supply_min_mb,
+ "Minimum order-9-and-larger free supply, in MiB, required per "
+ "populated zone before a significant GPU memory change wakes the "
+ "monitor's polling worker. The effective floor actually used is "
+ "the largest single growth sample ever observed by the worker, "
+ "clamped between this value and mem_monitor_order9_supply_max_mb, "
+ "so it adapts to this workload's own demonstrated behaviour without "
+ "growing unbounded from a single anomalous allocation "
+ "[default=" __stringify(XE_MEM_MONITOR_DEFAULT_ORDER9_SUPPLY_MIN_MB) "])");
+
+module_param_named(mem_monitor_order9_supply_max_mb,
+ xe_modparam.mem_monitor_order9_supply_max_mb, uint, 0644);
+MODULE_PARM_DESC(mem_monitor_order9_supply_max_mb,
+ "Upper bound, in MiB, on the adaptive order-9 free supply floor "
+ "described under mem_monitor_order9_supply_min_mb - caps how high "
+ "a single unusually large growth sample can permanently push that "
+ "floor [default=" __stringify(XE_MEM_MONITOR_DEFAULT_ORDER9_SUPPLY_MAX_MB) "])");
#endif
static int xe_check_nomodeset(void)
diff --git a/drivers/gpu/drm/xe/xe_module.h b/drivers/gpu/drm/xe/xe_module.h
index 24e0b52e48ca..19d0de96e078 100644
--- a/drivers/gpu/drm/xe/xe_module.h
+++ b/drivers/gpu/drm/xe/xe_module.h
@@ -38,6 +38,8 @@ struct xe_modparam {
u32 mem_monitor_poll_ms;
u32 mem_monitor_debounce_count;
u32 mem_monitor_idle_timeout_ms;
+ u32 mem_monitor_order9_supply_min_mb;
+ u32 mem_monitor_order9_supply_max_mb;
#endif
};
--
2.34.1
^ permalink raw reply related [flat|nested] 5+ messages in thread