[PATCH 0/4] RFC: drm/xe: Dynamic Xe Memory Monitor

[PATCH 0/4] RFC: drm/xe: Dynamic Xe Memory Monitor

[S Sebinraj]

Introduce xe_mem_monitor, a periodic worker that samples
xe->global_total_pages (the same counter that feeds the
gpu_mem/gpu_mem_total tracepoint) and tracks GPU memory growth using an
exponential moving average (EMA) over per-interval deltas:

    delta(t) = max(0, mem(t) - mem(t-1))
    EMA(t)   = alpha * delta(t) + (1 - alpha) * EMA(t-1)

    where alpha = 0.8, applied to every real sample
    regardless of mode.

A single alpha and a single polling cadence (mem_monitor_poll_ms)
are used in both LATENCY and HIGH_THROUGHPUT mode, once
HIGH_THROUGHPUT mode is entered there is little benefit to re-checking
any faster, since mem_monitor_debounce_count consecutive
below-threshold samples are already required before switching back,
which itself provides the desired minimum dwell time
(debounce_count * poll_ms) at this single cadence.

When the EMA of growth exceeds a threshold, xe->gpu_mem_mode is switched
from XE_GPU_MEM_MODE_LATENCY to XE_GPU_MEM_MODE_HIGH_THROUGHPUT, and
back once mem_monitor_debounce_count consecutive below-threshold
samples are seen.

xe->gpu_mem_mode is intended to be read by the TTM page allocation path
to trade allocation latency for throughput during GPU memory growth
bursts; the GFP flag adjustment itself is added in a follow-up change.

For power, the worker sleeps after mem_monitor_idle_timeout_ms of no
significant activity in LATENCY mode. The worker is re-armed on
activity.

The worker is suspended/resumed across system and runtime PM
transitions with notify_activity() also disabled while suspended.

All tuning is exposed as module parameters (mem_monitor_*) under
/sys/module/xe/parameters/. Defaults:

    mem_monitor_enabled                    false
    mem_monitor_growth_threshold_mb          250
    mem_monitor_poll_ms                     2000
    mem_monitor_debounce_count                 5
    mem_monitor_idle_timeout_ms            10000

S Sebinraj (3):
  RFC: ANDROID: drm/xe: add adaptive GPU memory growth monitor
    xe_mem_monitor
  RFC: ANDROID: drm/xe: register ttm page alloc vendor hooks internally
  RFC: ANDROID: drm/xe: Gate mem_monitor wake ups on order 9 external
    fragmentation

Ryan Neph (1):
  RFC: ANDROID: drm/xe: provide safe mem_monitor enable toggle via
    debugfs

 drivers/gpu/drm/BUILD.bazel              |   4 +
 drivers/gpu/drm/defconfig_xe             |   1 +
 drivers/gpu/drm/xe/Kconfig               |  14 +
 drivers/gpu/drm/xe/xe_bo.c               |   3 +
 drivers/gpu/drm/xe/xe_device.c           |  12 +
 drivers/gpu/drm/xe/xe_device_types.h     |  13 +
 drivers/gpu/drm/xe/xe_mem_monitor.c      | 727 +++++++++++++++++++++++
 drivers/gpu/drm/xe/xe_mem_monitor.h      |  95 +++
 drivers/gpu/drm/xe/xe_module.c           |  78 +++
 drivers/gpu/drm/xe/xe_module.h           |  15 +
 drivers/gpu/drm/xe/xe_pm.c               |   5 +
 drivers/gpu/drm/xe/xe_trace.h            |  16 +
 drivers/gpu/drm/xe/xe_ttm_vendor_hooks.c | 135 +++++
 drivers/gpu/drm/xe/xe_ttm_vendor_hooks.h |   9 +
 14 files changed, 1127 insertions(+)
 create mode 100644 drivers/gpu/drm/xe/xe_mem_monitor.c
 create mode 100644 drivers/gpu/drm/xe/xe_mem_monitor.h
 create mode 100644 drivers/gpu/drm/xe/xe_ttm_vendor_hooks.c
 create mode 100644 drivers/gpu/drm/xe/xe_ttm_vendor_hooks.h

-- 
2.34.1

[PATCH 1/4] RFC: ANDROID: drm/xe: add adaptive GPU memory growth monitor xe_mem_monitor

[S Sebinraj]

From: S Sebinraj <s.sebinraj@intel.corp-partner.google.com>

Introduce xe_mem_monitor, a periodic worker that samples
xe->global_total_pages (the same counter that feeds the
gpu_mem/gpu_mem_total tracepoint) and tracks GPU memory growth using an
exponential moving average (EMA) over per-interval deltas:

    delta(t) = max(0, mem(t) - mem(t-1))
    EMA(t)   = alpha * delta(t) + (1 - alpha) * EMA(t-1)

    where alpha = 0.8, applied to every real sample
    regardless of mode.

A single alpha and a single polling cadence (mem_monitor_poll_ms)
are used in both LATENCY and HIGH_THROUGHPUT mode, once
HIGH_THROUGHPUT mode is entered there is little benefit to re-checking
any faster, since mem_monitor_debounce_count consecutive
below-threshold samples are already required before switching back,
which itself provides the desired minimum dwell time
(debounce_count * poll_ms) at this single cadence.

When the EMA of growth exceeds a threshold, xe->gpu_mem_mode is switched
from XE_GPU_MEM_MODE_LATENCY to XE_GPU_MEM_MODE_HIGH_THROUGHPUT, and
back once mem_monitor_debounce_count consecutive below-threshold
samples are seen.

xe->gpu_mem_mode is intended to be read by the TTM page allocation path
to trade allocation latency for throughput during GPU memory growth
bursts; the GFP flag adjustment itself is added in a follow-up change.

For power, the worker sleeps after mem_monitor_idle_timeout_ms of no
significant activity in LATENCY mode. The worker is re-armed on
activity.

The worker is suspended/resumed across system and runtime PM
transitions with notify_activity() also disabled while suspended.

All tuning is exposed as module parameters (mem_monitor_*) under
/sys/module/xe/parameters/. Defaults:

    mem_monitor_enabled                    false
    mem_monitor_growth_threshold_mb          250
    mem_monitor_poll_ms                     2000
    mem_monitor_debounce_count                 5
    mem_monitor_idle_timeout_ms            10000

Guarded by new CONFIG_DRM_XE_MEM_MONITOR (depends on DRM_XE, selects
TRACE_GPU_MEM), enabled by default in defconfig_xe.

mem_monitor_enabled for fatcat is set via kernel arg xe.mem_monitor_enabled=1

Test: Verified via kernel logs while running WebGL-Aquarium / WLEU benchmark,
      mode switches to HIGH_THROUGHPUT on rapid allocation bursts and
      back once memory stabilizes. Confirmed idle-sleep/wake, EMA decay
      across sleep, clean suspend/resume behavior.

Cc: Carlos Santa <carlos.santa@intel.com>
Cc: Ryan Neph <ryanneph@google.com>
Cc: Renato Pereyra <renatopereyra@google.com>
Signed-off-by: S Sebinraj <s.sebinraj@intel.com>
---
 drivers/gpu/drm/BUILD.bazel          |   3 +
 drivers/gpu/drm/defconfig_xe         |   1 +
 drivers/gpu/drm/xe/Kconfig           |  14 +
 drivers/gpu/drm/xe/xe_bo.c           |   3 +
 drivers/gpu/drm/xe/xe_device.c       |   5 +
 drivers/gpu/drm/xe/xe_device_types.h |  12 +
 drivers/gpu/drm/xe/xe_mem_monitor.c  | 462 +++++++++++++++++++++++++++
 drivers/gpu/drm/xe/xe_mem_monitor.h  |  82 +++++
 drivers/gpu/drm/xe/xe_module.c       |  55 ++++
 drivers/gpu/drm/xe/xe_module.h       |  13 +
 drivers/gpu/drm/xe/xe_pm.c           |   5 +
 11 files changed, 655 insertions(+)
 create mode 100644 drivers/gpu/drm/xe/xe_mem_monitor.c
 create mode 100644 drivers/gpu/drm/xe/xe_mem_monitor.h

diff --git a/drivers/gpu/drm/BUILD.bazel b/drivers/gpu/drm/BUILD.bazel
index ac32f5cdecb9..274a1aa4ca93 100644
--- a/drivers/gpu/drm/BUILD.bazel
+++ b/drivers/gpu/drm/BUILD.bazel
@@ -154,6 +154,9 @@ ddk_module(
         "CONFIG_DRM_XE_GPUFREQTRACER": {
             True: ["xe/xe_gpufreqtracer.c"],
         },
+        "CONFIG_DRM_XE_MEM_MONITOR": {
+            True: ["xe/xe_mem_monitor.c"],
+        },
         "CONFIG_DRM_XE_GPUSVM": {
             True: ["xe/xe_svm.c"],
         },
diff --git a/drivers/gpu/drm/defconfig_xe b/drivers/gpu/drm/defconfig_xe
index c6e1c69fa663..4dc505580a37 100644
--- a/drivers/gpu/drm/defconfig_xe
+++ b/drivers/gpu/drm/defconfig_xe
@@ -3,3 +3,4 @@ CONFIG_DRM_XE_DISPLAY=y
 CONFIG_DRM_XE_DP_TUNNEL=y
 CONFIG_DRM_XE_FORCE_PROBE="*"
 CONFIG_DRM_XE_GPUFREQTRACER=y
+CONFIG_DRM_XE_MEM_MONITOR=y
diff --git a/drivers/gpu/drm/xe/Kconfig b/drivers/gpu/drm/xe/Kconfig
index 1521975c35fc..d0a0fcb33d64 100644
--- a/drivers/gpu/drm/xe/Kconfig
+++ b/drivers/gpu/drm/xe/Kconfig
@@ -150,6 +150,20 @@ config DRM_XE_GPUFREQTRACER
 
 	  If unsure, say N.
 
+config DRM_XE_MEM_MONITOR
+        bool "Enable XE GPU memory growth monitor"
+        depends on DRM_XE
+        select TRACE_GPU_MEM
+        default n
+        help
+          Enable adaptive GPU memory growth monitoring for the Intel XE driver.
+          Polls xe->global_total_pages at 500ms (active) or 2s (idle) intervals
+          and applies an exponential moving average to detect rapid allocation
+          bursts (>200 MB/interval).  When a burst is detected the TTM vendor
+          hook is disabled so that the kernel's normal direct-reclaim path can
+          apply backpressure on high-order page allocations, preventing stalls
+          from an unresponsive page cache.
+
 menu "drm/Xe Debugging"
 depends on DRM_XE
 depends on EXPERT
diff --git a/drivers/gpu/drm/xe/xe_bo.c b/drivers/gpu/drm/xe/xe_bo.c
index 493619229376..f2634d048cd4 100644
--- a/drivers/gpu/drm/xe/xe_bo.c
+++ b/drivers/gpu/drm/xe/xe_bo.c
@@ -27,6 +27,7 @@
 #include "xe_ggtt.h"
 #include "xe_gt.h"
 #include "xe_map.h"
+#include "xe_mem_monitor.h"
 #include "xe_migrate.h"
 #include "xe_pm.h"
 #include "xe_preempt_fence.h"
@@ -436,6 +437,8 @@ static void update_global_total_pages(struct ttm_device *ttm_dev,
 
 	trace_gpu_mem_total(xe->drm.primary->index, 0,
 			    global_total_pages << PAGE_SHIFT);
+
+	xe_mem_monitor_notify_activity(xe);
 #endif
 }
 
diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
index e405dbedd6b8..676ffc258b94 100644
--- a/drivers/gpu/drm/xe/xe_device.c
+++ b/drivers/gpu/drm/xe/xe_device.c
@@ -36,6 +36,7 @@
 #include "xe_force_wake.h"
 #include "xe_ggtt.h"
 #include "xe_gpufreqtracer.h"
+#include "xe_mem_monitor.h"
 #include "xe_gsc_proxy.h"
 #include "xe_gt.h"
 #include "xe_gt_mcr.h"
@@ -931,6 +932,10 @@ int xe_device_probe(struct xe_device *xe)
 	if (err)
 		return err;
 
+	err = xe_mem_monitor_init(xe);
+	if (err)
+		return err;
+
 	err = xe_late_bind_init(&xe->late_bind);
 	if (err)
 		return err;
diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
index 7b6c4e9a12b1..48aa9d8f6f56 100644
--- a/drivers/gpu/drm/xe/xe_device_types.h
+++ b/drivers/gpu/drm/xe/xe_device_types.h
@@ -39,6 +39,7 @@ struct intel_display;
 struct intel_dg_nvm_dev;
 struct xe_ggtt;
 struct xe_gpufreqtracer_data;
+struct xe_mem_monitor_data;
 struct xe_i2c;
 struct xe_pat_ops;
 struct xe_pxp;
@@ -583,6 +584,9 @@ struct xe_device {
 	/** @gpufreqtracer_data: GPU frequency tracer data */
 	struct xe_gpufreqtracer_data *gpufreqtracer_data;
 
+	/** @mem_monitor: GPU memory growth monitor data */
+	struct xe_mem_monitor_data *mem_monitor;
+
 	/** @needs_flr_on_fini: requests function-reset on fini */
 	bool needs_flr_on_fini;
 
@@ -630,6 +634,14 @@ struct xe_device {
 	 */
 	atomic64_t global_total_pages;
 #endif
+
+	/**
+	 * @gpu_mem_mode: current GPU memory allocation mode (enum xe_gpu_mem_mode),
+	 * switched between latency and high-throughput by xe_mem_monitor based on
+	 * GPU memory growth rate.
+	 */
+	atomic_t gpu_mem_mode;
+
 	/** @val: The domain for exhaustive eviction, which is currently per device. */
 	struct xe_validation_device val;
 
diff --git a/drivers/gpu/drm/xe/xe_mem_monitor.c b/drivers/gpu/drm/xe/xe_mem_monitor.c
new file mode 100644
index 000000000000..06fbd0624da7
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_mem_monitor.c
@@ -0,0 +1,462 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright © 2026 Intel Corporation
+ *
+ * xe_mem_monitor - Adaptive GPU memory growth monitor
+ *
+ * Periodically samples xe->global_total_pages (the same counter that feeds
+ * the gpu_mem/gpu_mem_total tracepoint) and applies an exponential moving
+ * average (EMA) over per-interval growth.  When growth exceeds a threshold
+ * xe->gpu_mem_mode is switched to XE_GPU_MEM_MODE_HIGH_THROUGHPUT, allowing
+ * the kernel's normal reclaim path to apply backpressure on TTM page
+ * allocations.  Once memory stabilises, the mode is switched back to
+ * XE_GPU_MEM_MODE_LATENCY.
+ *
+ * Algorithm:
+ *
+ *   delta(t)  = max(0, mem(t) - mem(t-1))          // clamp frees to zero
+ *   EMA(t)    = (ALPHA_NUM * delta(t) +
+ *               (2^ALPHA_SHIFT - ALPHA_NUM) * EMA(t-1) ) >> ALPHA_SHIFT
+ *
+ *   where ALPHA_NUM/2^ALPHA_SHIFT = alpha = 13/16 = 0.8125,
+ *   applied to every real sample regardless of mode, where ALPHA_SHIFT
+ *   = 4, (2^4 = 16). Both modes poll at the same mem_monitor_poll_ms
+ *   cadence and use the same smoothing weight.
+ *
+ * State machine (xe->gpu_mem_mode):
+ *   LATENCY mode, poll
+ *     -> EMA > mem_monitor_growth_threshold_mb: switch to HIGH_THROUGHPUT
+ *   HIGH_THROUGHPUT mode, poll (single polling rate - see
+ *   xe_mem_monitor_work()'s reschedule logic)
+ *     -> EMA below threshold for mem_monitor_debounce_count consecutive
+ *        samples -> switch back to LATENCY mode. Since samples are taken
+ *        every mem_monitor_poll_ms regardless of mode,
+ *        mem_monitor_debounce_count also acts as the implicit minimum dwell
+ *        time in HIGH_THROUGHPUT mode (debounce_count * poll_ms).
+ *
+ * Power saving: while in LATENCY mode, if no GPU allocation activity has been
+ * observed (see xe_mem_monitor_notify_activity()) for mem_monitor_idle_timeout_ms,
+ * the polling worker stops rescheduling itself entirely (goes to sleep, zero
+ * wakeups) rather than continuing to poll indefinitely. It is re-armed the
+ * next time real GPU memory growth is observed via
+ * xe_mem_monitor_notify_activity(). Since the state machine only ever reacts
+ * to growth (frees are clamped to zero), notify_activity() itself only
+ * resets the idle timer / wakes a sleeping worker when the live memory total
+ * has actually grown since the last real sample, a pure free or a
+ * populate/unpopulate pair that nets to no change, is ignored rather than
+ * forcing a full poll+decay cycle that could never affect the LATENCY/
+ * HIGH_THROUGHPUT decision.
+ *
+ * Since the EMA depends on samples being taken at a roughly regular cadence,
+ * and no samples are taken while asleep, the worker treats any skipped
+ * mem_monitor_poll_ms intervals since its last real sample as implicit
+ * zero-growth samples and fast-forwards the EMA decay for them before
+ * folding in the real delta observed on the sample that woke it up. This
+ * prevents stale, pre-sleep growth history from persisting indefinitely
+ * across arbitrarily long idle gaps.
+ */
+
+#include "xe_mem_monitor.h"
+
+#include <linux/atomic.h>
+#include <linux/jiffies.h>
+#include <linux/minmax.h>
+#include <linux/slab.h>
+#include <linux/workqueue.h>
+#include <drm/drm_managed.h>
+#include <drm/drm_print.h>
+
+#include "xe_device.h"
+#include "xe_device_types.h"
+#include "xe_module.h"
+
+/*
+ * EMA smoothing factor. alpha (13/16 = 0.8125) applies to every real
+ * sample, regardless of mode,
+ * since both LATENCY and HIGH_THROUGHPUT modes poll at the same
+ * mem_monitor_poll_ms cadence (see xe_mem_monitor_work()'s reschedule
+ * logic) and share the same smoothing weight.
+ */
+#define ALPHA_NUM		13U
+#define ALPHA_RETAINED		3U	/* 16 - 13 */
+#define ALPHA_SHIFT		4U	/* /16 */
+
+/*
+ * This is the max number of implicit zero-growth decay steps applied when waking
+ * from sleep (see xe_mem_monitor_work()). Each step degrades EMA by
+ * ALPHA_RETAINED/2^ALPHA_SHIFT = 3/16 per step,
+ * so after the number of steps below the EMA has already decayed to 0
+ * via integer truncation regardless of its starting value. No need to loop
+ * further for arbitrarily long sleeps.
+ */
+#define EMA_DECAY_SATURATE_PERIODS 32
+
+/*
+ * Minimum absolute change (bytes) since the worker's last real sample
+ * required for xe_mem_monitor_notify_activity() to treat it as activity
+ * worth acting on. Filters out negligible/noise-level trickle (a few KB of
+ * housekeeping churn) that would otherwise repeatedly wake the worker for no
+ * meaningful reason. Checked in both directions (growth or shrink): a
+ * significant free must also resync prev_mem_bytes, otherwise it can be left
+ * stuck at a stale high-water mark that later, real growth from a much lower
+ * baseline would never be able to cross.
+ * Current allocated bytes is compared to the prev bytes so we won't accumulate
+ * small errors less than 1 MB over multiple calls
+ */
+#define ACTIVITY_MIN_DELTA_BYTES (1ULL << 20)   /* 1 MB */
+
+/**
+ * struct xe_mem_monitor_data - per-device GPU memory monitor state
+ * @xe:                    back-pointer to the xe device
+ * @work:                  self-rescheduling delayed work
+ * @prev_mem_bytes:        GPU memory total from the previous poll. atomic64_t
+ *                         because it is also read from
+ *                         xe_mem_monitor_notify_activity(), which can run
+ *                         concurrently with the worker on another thread.
+ * @has_prev_sample:       false until the first sample is taken (baseline)
+ * @ema_growth_bytes:      EMA of per-interval memory growth (bytes)
+ * @is_high_throughput:    true while in HIGH_THROUGHPUT mode; only gates
+ *                         the idle-sleep check below (idle-sleep only
+ *                         applies in LATENCY mode).
+ * @below_threshold_count: consecutive below-threshold samples since entering
+ *                         HIGH_THROUGHPUT mode
+ * @last_activity_jiffies: jiffies value of the most recent GPU memory growth
+ *                         notification (see xe_mem_monitor_notify_activity())
+ * @last_poll_jiffies:     jiffies value of the most recent real sample taken
+ *                         by xe_mem_monitor_work(), used to detect and decay
+ *                         the EMA across skipped (slept-through) intervals
+ * @suspended:             set while the device is suspended (system or
+ *                         runtime PM); xe_mem_monitor_notify_activity() is a
+ *                         no-op while set, so that BO eviction/restore
+ *                         traffic generated by suspend/resume itself cannot
+ *                         re-arm the worker mid-transition. atomic_t because
+ *                         it is read from xe_mem_monitor_notify_activity() on
+ *                         arbitrary caller threads.
+ * @was_pending_before_suspend: whether the worker had a poll actually
+ *                         scheduled at the moment xe_mem_monitor_suspend()
+ *                         cancelled it; used by xe_mem_monitor_resume() to
+ *                         decide whether to reschedule, so that a monitor
+ *                         that was legitimately asleep before suspend stays
+ *                         asleep after resume instead of being woken
+ *                         unconditionally.
+ */
+struct xe_mem_monitor_data {
+	struct xe_device *xe;
+	struct delayed_work work;
+
+	atomic64_t prev_mem_bytes;
+	bool has_prev_sample;
+	u64 ema_growth_bytes;
+	int below_threshold_count;
+	unsigned long last_activity_jiffies;
+	unsigned long last_poll_jiffies;
+	bool is_high_throughput;
+	atomic_t suspended;
+	bool was_pending_before_suspend;
+};
+
+static void xe_mem_monitor_set_high_throughput_mode(struct xe_mem_monitor_data *mon)
+{
+	/* TODO: wire up to the actual vendor hook (adds __GFP_RETRY_MAYFAIL, TBD) */
+	drm_dbg(&mon->xe->drm,
+		 "xe_mem_monitor: switching to HIGH_THROUGHPUT mode (TBD)\n");
+	atomic_set(&mon->xe->gpu_mem_mode, XE_GPU_MEM_MODE_HIGH_THROUGHPUT);
+}
+
+static void xe_mem_monitor_set_latency_mode(struct xe_mem_monitor_data *mon)
+{
+	/* TODO: wire up to the actual vendor hook (TBD) */
+	drm_dbg(&mon->xe->drm,
+		 "xe_mem_monitor: switching to LATENCY mode (TBD)\n");
+	atomic_set(&mon->xe->gpu_mem_mode, XE_GPU_MEM_MODE_LATENCY);
+}
+
+static void xe_mem_monitor_work(struct work_struct *work)
+{
+	struct xe_mem_monitor_data *mon =
+		container_of(work, struct xe_mem_monitor_data, work.work);
+	struct xe_device *xe = mon->xe;
+	u64 current_bytes, clamped_delta;
+	u64 growth_threshold_bytes;
+	s64 delta_signed;
+	unsigned long now = jiffies;
+	unsigned long next_delay;
+
+	/*
+	 * Read the GPU memory total directly from Xe's own accounting counter.
+	 * This is the same value that drives the gpu_mem/gpu_mem_total
+	 * tracepoint, but read without any BPF or sysfs indirection.
+	 */
+	current_bytes = (u64)atomic64_read(&xe->global_total_pages) << PAGE_SHIFT;
+
+	if (!mon->has_prev_sample) {
+		drm_dbg(&xe->drm, "xe_mem_monitor: baseline = %llu MB\n",
+			current_bytes >> 20);
+		atomic64_set(&mon->prev_mem_bytes, current_bytes);
+		mon->has_prev_sample = true;
+		mon->last_poll_jiffies = now;
+		goto reschedule;
+	}
+
+	/*
+	 * If more than one nominal poll interval has elapsed since the
+	 * last real sample, the worker must have been asleep (or otherwise
+	 * delayed) for the extra time. Since no growth can occur unnoticed
+	 * while asleep (any GPU allocation activity would have woken it via
+	 * xe_mem_monitor_notify_activity()), treat each skipped interval as
+	 * an implicit zero-growth sample and fast-forward the EMA decay for
+	 * them before folding in the real delta observed below. Otherwise
+	 * stale, pre-sleep growth history would persist indefinitely across
+	 * arbitrarily long idle gaps.
+	 */
+	{
+		unsigned long poll_jiffies =
+			msecs_to_jiffies(xe_modparam.mem_monitor_poll_ms);
+		unsigned long skipped_periods = poll_jiffies ?
+			(now - mon->last_poll_jiffies) / poll_jiffies : 0;
+
+		if (skipped_periods > 1) {
+			unsigned long decay_periods = min_t(unsigned long,
+							    skipped_periods - 1,
+							    EMA_DECAY_SATURATE_PERIODS);
+
+			drm_dbg(&xe->drm,
+				"xe_mem_monitor: woke after %lu skipped poll interval(s), "
+				"fast-forwarding EMA decay by %lu\n",
+				skipped_periods - 1, decay_periods);
+
+			while (decay_periods-- > 0)
+				mon->ema_growth_bytes = (ALPHA_RETAINED *
+							 mon->ema_growth_bytes) >> ALPHA_SHIFT;
+		}
+	}
+
+	/* Clamp negative deltas (memory freed) to zero; only track growth. */
+	delta_signed = (s64)current_bytes - (s64)atomic64_read(&mon->prev_mem_bytes);
+	clamped_delta = delta_signed > 0 ? (u64)delta_signed : 0;
+
+	/* A single alpha smoothing weight is used regardless of mode. */
+	mon->ema_growth_bytes = (ALPHA_NUM * clamped_delta +
+				 ALPHA_RETAINED * mon->ema_growth_bytes)
+				>> ALPHA_SHIFT;
+
+	atomic64_set(&mon->prev_mem_bytes, current_bytes);
+	mon->last_poll_jiffies = now;
+
+	growth_threshold_bytes = (u64)xe_modparam.mem_monitor_growth_threshold_mb << 20;
+
+	if (atomic_read(&xe->gpu_mem_mode) == XE_GPU_MEM_MODE_LATENCY) {
+		if (mon->ema_growth_bytes > growth_threshold_bytes) {
+			drm_dbg(&xe->drm,
+				 "xe_mem_monitor: rapid GPU memory growth detected, delta=%llu MB, "
+				 "(EMA=%llu MB/interval > threshold=%llu MB)\n",
+				 clamped_delta >> 20,
+				 mon->ema_growth_bytes >> 20,
+				 growth_threshold_bytes >> 20);
+			xe_mem_monitor_set_high_throughput_mode(mon);
+			mon->is_high_throughput = true;
+			mon->below_threshold_count = 0;
+		}
+	} else {
+		/*
+		 * mem_monitor_debounce_count consecutive below-threshold
+		 * samples (taken at mem_monitor_poll_ms, same single
+		 * polling rate used in both modes) are required before
+		 * switching back to LATENCY mode. This also acts as the
+		 * implicit minimum dwell time in HIGH_THROUGHPUT mode.
+		 */
+		if (mon->ema_growth_bytes < growth_threshold_bytes) {
+			mon->below_threshold_count++;
+			drm_dbg(&xe->drm,
+				"xe_mem_monitor: GPU memory growth slowing "
+				"(EMA=%llu MB/interval), below-threshold sample %d/%u\n",
+				mon->ema_growth_bytes >> 20,
+				mon->below_threshold_count, xe_modparam.mem_monitor_debounce_count);
+			if (mon->below_threshold_count >= xe_modparam.mem_monitor_debounce_count) {
+				drm_dbg(&xe->drm,
+					 "xe_mem_monitor: GPU memory stable for %u "
+					 "consecutive samples\n",
+					 xe_modparam.mem_monitor_debounce_count);
+				xe_mem_monitor_set_latency_mode(mon);
+				mon->is_high_throughput = false;
+				mon->below_threshold_count = 0;
+			}
+		} else {
+			drm_dbg(&xe->drm,
+				"xe_mem_monitor: GPU memory still growing "
+				"(EMA=%llu MB/interval), resetting debounce counter\n",
+				mon->ema_growth_bytes >> 20);
+			mon->below_threshold_count = 0;
+		}
+	}
+
+reschedule:
+	if (!mon->is_high_throughput &&
+	    jiffies_to_msecs(jiffies - mon->last_activity_jiffies) >
+			xe_modparam.mem_monitor_idle_timeout_ms) {
+		drm_dbg(&xe->drm,
+			"xe_mem_monitor: no activity for %u ms, going to sleep\n",
+			xe_modparam.mem_monitor_idle_timeout_ms);
+		return;
+	}
+
+	/*
+	 * Single polling rate (mem_monitor_poll_ms) regardless of mode -
+	 * per review discussion, once HIGH_THROUGHPUT mode is entered we
+	 * don't need to re-check any faster, since mem_monitor_debounce_count
+	 * already provides the equivalent minimum dwell time at this cadence.
+	 */
+	next_delay = msecs_to_jiffies(xe_modparam.mem_monitor_poll_ms);
+	schedule_delayed_work(&mon->work, next_delay);
+}
+
+static void xe_mem_monitor_cleanup(struct drm_device *drm, void *arg)
+{
+	struct xe_mem_monitor_data *mon = arg;
+
+	cancel_delayed_work_sync(&mon->work);
+}
+
+/**
+ * xe_mem_monitor_notify_activity - notify the monitor of GPU memory changes
+ * @xe: the Xe device
+ *
+ * Called from the TTM populate/unpopulate path whenever GPU memory is
+ * allocated or freed. No-op while the device is suspended (see
+ * xe_mem_monitor_suspend()), since suspend/resume's own BO eviction/restore
+ * traffic would otherwise re-arm the worker mid-transition. Otherwise only
+ * acts when the live memory total has changed by at least
+ * ACTIVITY_MIN_DELTA_BYTES (in either direction) since the worker's last real
+ * sample, negligible trickle growth/shrink is ignored to avoid waking the
+ * worker for no meaningful reason. A significant free must still be acted on
+ * (not just growth): it resyncs prev_mem_bytes down to reality by letting the
+ * worker run once, which is required so that later real growth measured from
+ * that new, lower baseline can still be detected, otherwise prev_mem_bytes
+ * would stay stuck at a stale high-water mark that new growth might never
+ * cross. If the worker is already scheduled (delayed_work_pending()), it
+ * will read the live counter fresh when it runs and pick up whatever
+ * accumulated, so there's no need to re-schedule again here.
+ */
+void xe_mem_monitor_notify_activity(struct xe_device *xe)
+{
+	struct xe_mem_monitor_data *mon = xe->mem_monitor;
+	u64 current_bytes;
+	s64 delta_signed, abs_delta;
+
+	if (!mon)
+		return;
+
+	if (atomic_read(&mon->suspended))
+		return;
+
+	current_bytes = (u64)atomic64_read(&xe->global_total_pages) << PAGE_SHIFT;
+	delta_signed = (s64)current_bytes - (s64)atomic64_read(&mon->prev_mem_bytes);
+	abs_delta = delta_signed < 0 ? -delta_signed : delta_signed;
+
+	if (abs_delta < ACTIVITY_MIN_DELTA_BYTES)
+		return;
+
+	mon->last_activity_jiffies = jiffies;
+
+	if (delayed_work_pending(&mon->work))
+		return;
+
+	schedule_delayed_work(&mon->work, 0);
+}
+
+/**
+ * xe_mem_monitor_suspend - stop the polling worker for a PM transition
+ * @xe: the Xe device
+ *
+ * Sets the suspended flag first so any xe_mem_monitor_notify_activity()
+ * call racing with (or generated by) the suspend sequence itself (e.g.
+ * xe_bo_evict_all()) is a no-op, then synchronously cancels the worker.
+ * Whether a poll was actually pending at that point is recorded so
+ * xe_mem_monitor_resume() can decide whether to reschedule.
+ */
+void xe_mem_monitor_suspend(struct xe_device *xe)
+{
+	struct xe_mem_monitor_data *mon = xe->mem_monitor;
+
+	if (!mon)
+		return;
+
+	atomic_set(&mon->suspended, 1);
+	mon->was_pending_before_suspend = cancel_delayed_work_sync(&mon->work);
+}
+
+/**
+ * xe_mem_monitor_resume - resume the polling worker after a PM transition
+ * @xe: the Xe device
+ *
+ * Only reschedules the worker if it was actually pending at the time
+ * xe_mem_monitor_suspend() cancelled it. If the monitor had already gone to
+ * sleep (idle timeout) before suspend, it stays asleep across the
+ * transition rather than being unconditionally woken.
+ */
+void xe_mem_monitor_resume(struct xe_device *xe)
+{
+	struct xe_mem_monitor_data *mon = xe->mem_monitor;
+
+	if (!mon)
+		return;
+
+	atomic_set(&mon->suspended, 0);
+
+	if (mon->was_pending_before_suspend) {
+		unsigned long delay = msecs_to_jiffies(xe_modparam.mem_monitor_poll_ms);
+
+		schedule_delayed_work(&mon->work, delay);
+	}
+}
+
+/**
+ * xe_mem_monitor_init - initialise and start the GPU memory monitor
+ * @xe: the Xe device
+ *
+ * Allocates monitor state, registers a cleanup action via drmm, and
+ * schedules the first poll after one poll interval.
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+int xe_mem_monitor_init(struct xe_device *xe)
+{
+	struct xe_mem_monitor_data *mon;
+	int ret;
+
+	if (!xe_modparam.mem_monitor_enabled) {
+		drm_info(&xe->drm,
+			 "xe_mem_monitor: disabled via mem_monitor_enabled module parameter\n");
+		return 0;
+	}
+
+	mon = drmm_kzalloc(&xe->drm, sizeof(*mon), GFP_KERNEL);
+	if (!mon)
+		return -ENOMEM;
+
+	mon->xe = xe;
+	atomic_set(&xe->gpu_mem_mode, XE_GPU_MEM_MODE_LATENCY);
+	mon->last_activity_jiffies = jiffies;
+	INIT_DELAYED_WORK(&mon->work, xe_mem_monitor_work);
+
+	xe->mem_monitor = mon;
+
+	ret = drmm_add_action_or_reset(&xe->drm, xe_mem_monitor_cleanup, mon);
+	if (ret)
+		return ret;
+
+	schedule_delayed_work(&mon->work,
+			      msecs_to_jiffies(xe_modparam.mem_monitor_poll_ms));
+
+	drm_info(&xe->drm,
+		 "xe_mem_monitor: initialized: poll=%u ms, "
+		 "threshold=%u MB, alpha=0.%03u, "
+		 "debounce=%u, idle_timeout=%u ms\n",
+		 xe_modparam.mem_monitor_poll_ms,
+		 xe_modparam.mem_monitor_growth_threshold_mb,
+		 (ALPHA_NUM * 1000U + (1U << (ALPHA_SHIFT - 1))) >> ALPHA_SHIFT,
+		 xe_modparam.mem_monitor_debounce_count,
+		 xe_modparam.mem_monitor_idle_timeout_ms);
+
+	return 0;
+}
diff --git a/drivers/gpu/drm/xe/xe_mem_monitor.h b/drivers/gpu/drm/xe/xe_mem_monitor.h
new file mode 100644
index 000000000000..846c0628ed93
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_mem_monitor.h
@@ -0,0 +1,82 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright © 2026 Intel Corporation
+ */
+
+#ifndef _XE_MEM_MONITOR_H_
+#define _XE_MEM_MONITOR_H_
+
+struct xe_device;
+
+/**
+ * enum xe_gpu_mem_mode - GPU memory allocation mode controlled by xe_mem_monitor
+ * @XE_GPU_MEM_MODE_LATENCY: default mode; vendor hook strips reclaim flags
+ *                           from high-order TTM allocations for low latency.
+ * @XE_GPU_MEM_MODE_HIGH_THROUGHPUT: entered during rapid GPU memory growth;
+ *                           reclaim is allowed (and __GFP_RETRY_MAYFAIL added)
+ *                           so the kernel can shrink the page cache to satisfy
+ *                           high-order allocations.
+ */
+enum xe_gpu_mem_mode {
+	XE_GPU_MEM_MODE_LATENCY = 0,
+	XE_GPU_MEM_MODE_HIGH_THROUGHPUT,
+};
+
+/* Default values for module parameters, see xe_module.h/xe_module.c */
+#define XE_MEM_MONITOR_DEFAULT_ENABLED				false
+#define XE_MEM_MONITOR_DEFAULT_GROWTH_THRESHOLD_MB		250
+#define XE_MEM_MONITOR_DEFAULT_POLL_MS				2000
+#define XE_MEM_MONITOR_DEFAULT_DEBOUNCE_COUNT			5
+#define XE_MEM_MONITOR_DEFAULT_IDLE_TIMEOUT_MS			10000
+
+/*
+ * Minimum enforced value for mem_monitor_poll_ms (no maximum is
+ * enforced). Guards against a misconfigured/absurdly low value (e.g. 0)
+ * causing the delayed work to effectively busy-loop.
+ */
+#define XE_MEM_MONITOR_MIN_POLL_MS				500
+
+#ifdef CONFIG_DRM_XE_MEM_MONITOR
+
+int xe_mem_monitor_init(struct xe_device *xe);
+
+/*
+ * Notify the monitor that GPU allocation activity occurred (called from the
+ * TTM populate/unpopulate path). Updates the last-activity timestamp and
+ * re-arms the polling worker if it had gone to sleep due to inactivity.
+ */
+void xe_mem_monitor_notify_activity(struct xe_device *xe);
+
+/*
+ * Suspend/resume the polling worker across system and runtime PM
+ * transitions. Must be called from the same suspend/resume paths for
+ * the same reason: schedule_delayed_work() on system_wq is not freezable, so
+ * without this the worker could keep firing (or be re-armed by the BO
+ * eviction/restore traffic that suspend/resume itself generates) across the
+ * transition.
+ */
+void xe_mem_monitor_suspend(struct xe_device *xe);
+void xe_mem_monitor_resume(struct xe_device *xe);
+
+#else /* CONFIG_DRM_XE_MEM_MONITOR */
+
+static inline int xe_mem_monitor_init(struct xe_device *xe)
+{
+	return 0;
+}
+
+static inline void xe_mem_monitor_notify_activity(struct xe_device *xe)
+{
+}
+
+static inline void xe_mem_monitor_suspend(struct xe_device *xe)
+{
+}
+
+static inline void xe_mem_monitor_resume(struct xe_device *xe)
+{
+}
+
+#endif /* CONFIG_DRM_XE_MEM_MONITOR */
+
+#endif /* _XE_MEM_MONITOR_H_ */
diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
index 4878463734eb..a49b0c0f958b 100644
--- a/drivers/gpu/drm/xe/xe_module.c
+++ b/drivers/gpu/drm/xe/xe_module.c
@@ -15,6 +15,7 @@
 #include "xe_configfs.h"
 #include "xe_gpufreqtracer.h"
 #include "xe_hw_fence.h"
+#include "xe_mem_monitor.h"
 #include "xe_pci.h"
 #include "xe_pm.h"
 #include "xe_observation.h"
@@ -45,6 +46,13 @@ struct xe_modparam xe_modparam = {
 	.svm_notifier_size =	DEFAULT_SVM_NOTIFIER_SIZE,
 #ifdef CONFIG_DRM_XE_GPUFREQTRACER
 	.gpufreq_monitoring_interval_ms = XE_GPUFREQ_MONITORING_DEFAULT_INTERVAL_MS,
+#endif
+#ifdef CONFIG_DRM_XE_MEM_MONITOR
+	.mem_monitor_enabled =	XE_MEM_MONITOR_DEFAULT_ENABLED,
+	.mem_monitor_growth_threshold_mb = XE_MEM_MONITOR_DEFAULT_GROWTH_THRESHOLD_MB,
+	.mem_monitor_poll_ms = XE_MEM_MONITOR_DEFAULT_POLL_MS,
+	.mem_monitor_debounce_count = XE_MEM_MONITOR_DEFAULT_DEBOUNCE_COUNT,
+	.mem_monitor_idle_timeout_ms = XE_MEM_MONITOR_DEFAULT_IDLE_TIMEOUT_MS,
 #endif
 	/* the rest are 0 by default */
 };
@@ -108,6 +116,53 @@ MODULE_PARM_DESC(gpufreq_monitoring_interval_ms,
 		 __stringify(XE_GPUFREQ_MONITORING_DEFAULT_INTERVAL_MS) ")");
 #endif
 
+#ifdef CONFIG_DRM_XE_MEM_MONITOR
+module_param_named_unsafe(mem_monitor_enabled, xe_modparam.mem_monitor_enabled, bool, 0444);
+MODULE_PARM_DESC(mem_monitor_enabled,
+		 "Master enable switch for the GPU memory growth monitor and its "
+		 "vendor hooks. Intended to be set as a kernel boot argument"
+		 " [default=" __stringify(XE_MEM_MONITOR_DEFAULT_ENABLED) " (enabled)]");
+
+module_param_named(mem_monitor_growth_threshold_mb,
+		   xe_modparam.mem_monitor_growth_threshold_mb, uint, 0644);
+MODULE_PARM_DESC(mem_monitor_growth_threshold_mb,
+		 "GPU memory growth EMA threshold in MiB per interval that triggers "
+		 "HIGH_THROUGHPUT mode [default="
+		 __stringify(XE_MEM_MONITOR_DEFAULT_GROWTH_THRESHOLD_MB) "])");
+
+static int param_set_mem_monitor_poll_ms(const char *val, const struct kernel_param *kp)
+{
+	return param_set_uint_minmax(val, kp, XE_MEM_MONITOR_MIN_POLL_MS, UINT_MAX);
+}
+
+static const struct kernel_param_ops param_ops_mem_monitor_poll_ms = {
+	.set = param_set_mem_monitor_poll_ms,
+	.get = param_get_uint,
+};
+
+module_param_cb(mem_monitor_poll_ms, &param_ops_mem_monitor_poll_ms,
+		&xe_modparam.mem_monitor_poll_ms, 0644);
+MODULE_PARM_DESC(mem_monitor_poll_ms,
+		 "GPU memory monitor polling interval in milliseconds, minimum "
+		 __stringify(XE_MEM_MONITOR_MIN_POLL_MS) "ms "
+		 "[default=" __stringify(XE_MEM_MONITOR_DEFAULT_POLL_MS) "])");
+
+module_param_named(mem_monitor_debounce_count,
+		   xe_modparam.mem_monitor_debounce_count, uint, 0644);
+MODULE_PARM_DESC(mem_monitor_debounce_count,
+		 "Number of consecutive below-threshold samples required before "
+		 "switching back to LATENCY mode [default="
+		 __stringify(XE_MEM_MONITOR_DEFAULT_DEBOUNCE_COUNT) "])");
+
+module_param_named(mem_monitor_idle_timeout_ms,
+		   xe_modparam.mem_monitor_idle_timeout_ms, uint, 0644);
+MODULE_PARM_DESC(mem_monitor_idle_timeout_ms,
+		 "Time in milliseconds with no GPU allocation activity in LATENCY "
+		 "mode after which the monitor's polling worker goes to sleep "
+		 "(woken again on the next allocation) [default="
+		 __stringify(XE_MEM_MONITOR_DEFAULT_IDLE_TIMEOUT_MS) "])");
+#endif
+
 static int xe_check_nomodeset(void)
 {
 	if (drm_firmware_drivers_only())
diff --git a/drivers/gpu/drm/xe/xe_module.h b/drivers/gpu/drm/xe/xe_module.h
index 6a64ee221da6..24e0b52e48ca 100644
--- a/drivers/gpu/drm/xe/xe_module.h
+++ b/drivers/gpu/drm/xe/xe_module.h
@@ -26,6 +26,19 @@ struct xe_modparam {
 #ifdef CONFIG_DRM_XE_GPUFREQTRACER
 	u32 gpufreq_monitoring_interval_ms;
 #endif
+#ifdef CONFIG_DRM_XE_MEM_MONITOR
+	/*
+	 * Master enable switch for the GPU memory growth monitor and its
+	 * associated vendor hooks. Defaults is set in xe_mem_monitor.h
+	 * (XE_MEM_MONITOR_DEFAULT_ENABLED). Intended to be
+	 * settable as a kernel boot argument
+	 */
+	bool mem_monitor_enabled;
+	u32 mem_monitor_growth_threshold_mb;
+	u32 mem_monitor_poll_ms;
+	u32 mem_monitor_debounce_count;
+	u32 mem_monitor_idle_timeout_ms;
+#endif
 };
 
 extern struct xe_modparam xe_modparam;
diff --git a/drivers/gpu/drm/xe/xe_pm.c b/drivers/gpu/drm/xe/xe_pm.c
index b291265d18fc..a330a42ea187 100644
--- a/drivers/gpu/drm/xe/xe_pm.c
+++ b/drivers/gpu/drm/xe/xe_pm.c
@@ -24,6 +24,7 @@
 #include "xe_i2c.h"
 #include "xe_irq.h"
 #include "xe_late_bind_fw.h"
+#include "xe_mem_monitor.h"
 #include "xe_pcode.h"
 #include "xe_pxp.h"
 #include "xe_sriov_vf_ccs.h"
@@ -129,6 +130,7 @@ int xe_pm_suspend(struct xe_device *xe)
 	trace_xe_pm_suspend(xe, __builtin_return_address(0));
 
 	xe_gpufreqtracer_suspend_workers(xe);
+	xe_mem_monitor_suspend(xe);
 
 	err = xe_pxp_pm_suspend(xe->pxp);
 	if (err)
@@ -219,6 +221,7 @@ int xe_pm_resume(struct xe_device *xe)
 		goto err;
 
 	xe_gpufreqtracer_resume_workers(xe);
+	xe_mem_monitor_resume(xe);
 
 	xe_pxp_pm_resume(xe->pxp);
 
@@ -515,6 +518,7 @@ int xe_pm_runtime_suspend(struct xe_device *xe)
 	xe_rpm_lockmap_acquire(xe);
 
 	xe_gpufreqtracer_suspend_workers(xe);
+	xe_mem_monitor_suspend(xe);
 
 	err = xe_pxp_pm_suspend(xe->pxp);
 	if (err)
@@ -616,6 +620,7 @@ int xe_pm_runtime_resume(struct xe_device *xe)
 	}
 
 	xe_gpufreqtracer_resume_workers(xe);
+	xe_mem_monitor_resume(xe);
 
 	xe_pxp_pm_resume(xe->pxp);
 
-- 
2.34.1

[PATCH 2/4] RFC: ANDROID: drm/xe: register ttm page alloc vendor hooks internally

[S Sebinraj]

From: S Sebinraj <s.sebinraj@intel.corp-partner.google.com>

Register the TTM page allocation vendor hooks
(android_vh_ttm_pool_alloc_max_page_order,
android_vh_ttm_pool_alloc_page_flags) directly from Xe, rather than via
the standalone intel_ttm_vendor_hooks.ko module, for direct, low
overhead access to xe_device state.

xe_register_android_vendor_hooks()/xe_unregister_android_vendor_hooks()
are called from xe_device_create()/xe_device_destroy() respectively.
Both are gated on the xe_modparam.mem_monitor_enabled master switch
(shared with xe_mem_monitor): if disabled, the hooks are never
registered at all, leaving TTM's default GFP flag behavior completely
untouched.

The hook caps the max TTM page order at 9 (2M pages) and adjusts GFP
flags for that order only based on xe->gpu_mem_mode, which is maintained
by xe_mem_monitor:

  XE_GPU_MEM_MODE_LATENCY (default):
      strip __GFP_DIRECT_RECLAIM and __GFP_KSWAPD_RECLAIM so 2M page
      allocations fail fast rather than stalling on reclaim.
      __GFP_NORETRY (already set by TTM for order>0) is left untouched.

  XE_GPU_MEM_MODE_HIGH_THROUGHPUT:
      explicitly set __GFP_DIRECT_RECLAIM and __GFP_KSWAPD_RECLAIM, clear
      __GFP_NORETRY, and add __GFP_RETRY_MAYFAIL so the allocator retries
      harder (including shrinking the page cache) to satisfy 2M page
      allocations during bursts of GPU memory growth.

      __GFP_NORETRY and __GFP_RETRY_MAYFAIL represent mutually exclusive
      points on the same "costly allocation" retry spectrum (see
      Documentation / gfp_types.h); __GFP_NORETRY is cleared explicitly
      here since TTM sets it unconditionally for order>0 allocations and
      leaving it set would blunt the effect of __GFP_RETRY_MAYFAIL.

  - order-0: leave flags unmodified
  - order-9: select based on gpu_mem_mode
  - all others: ~gfp_flags &= ~(__GFP_DIRECT_RECLAIM | __GFP_KSWAPD_RECLAIM)

A message (rate-limited to mode transitions only, not per allocation) is
logged whenever the effective mode changes, to aid verification. A new
xe_gpu_mem_mode_switch tracepoint is also emitted on every mode
transition (from xe_mem_monitor, see xe_trace.h) so the switching
behavior can be recorded in Perfetto/ftrace traces for debugging and
parameter tuning.

Test: confirmed hook fires and GFP flags change correctly across mode
      transitions via dmesg while running WebGL Aquarium / WLEU
      benchmark. Confirmed xe_gpu_mem_mode_switch tracepoint fires on
      each transition via ftrace. Confirmed hooks are not registered
      when mem_monitor_enabled=0 is passed as a boot argument.

Cc: Carlos Santa <carlos.santa@intel.com>
Signed-off-by: Ryan Neph <ryanneph@google.com>
Signed-off-by: S Sebinraj <s.sebinraj@intel.com>
---
 drivers/gpu/drm/BUILD.bazel              |   1 +
 drivers/gpu/drm/xe/xe_device.c           |   6 +
 drivers/gpu/drm/xe/xe_mem_monitor.c      |  15 ++-
 drivers/gpu/drm/xe/xe_trace.h            |  16 +++
 drivers/gpu/drm/xe/xe_ttm_vendor_hooks.c | 135 +++++++++++++++++++++++
 drivers/gpu/drm/xe/xe_ttm_vendor_hooks.h |   9 ++
 6 files changed, 178 insertions(+), 4 deletions(-)
 create mode 100644 drivers/gpu/drm/xe/xe_ttm_vendor_hooks.c
 create mode 100644 drivers/gpu/drm/xe/xe_ttm_vendor_hooks.h

diff --git a/drivers/gpu/drm/BUILD.bazel b/drivers/gpu/drm/BUILD.bazel
index 274a1aa4ca93..f4fc4b94f71f 100644
--- a/drivers/gpu/drm/BUILD.bazel
+++ b/drivers/gpu/drm/BUILD.bazel
@@ -127,6 +127,7 @@ ddk_module(
         "xe/xe_trace_lrc.c",
         "xe/xe_ttm_stolen_mgr.c",
         "xe/xe_ttm_sys_mgr.c",
+        "xe/xe_ttm_vendor_hooks.c",
         "xe/xe_ttm_vram_mgr.c",
         "xe/xe_tuning.c",
         "xe/xe_uc.c",
diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
index 676ffc258b94..d21068c1effc 100644
--- a/drivers/gpu/drm/xe/xe_device.c
+++ b/drivers/gpu/drm/xe/xe_device.c
@@ -69,6 +69,7 @@
 #include "xe_tile.h"
 #include "xe_ttm_stolen_mgr.h"
 #include "xe_ttm_sys_mgr.h"
+#include "xe_ttm_vendor_hooks.h"
 #include "xe_vm.h"
 #include "xe_vm_madvise.h"
 #include "xe_vram.h"
@@ -419,6 +420,7 @@ static void xe_device_destroy(struct drm_device *dev, void *dummy)
 {
 	struct xe_device *xe = to_xe_device(dev);
 
+	xe_unregister_android_vendor_hooks(xe);
 	xe_bo_dev_fini(&xe->bo_device);
 
 	if (xe->preempt_fence_wq)
@@ -464,6 +466,10 @@ struct xe_device *xe_device_create(struct pci_dev *pdev,
 	if (err)
 		goto err;
 
+	err = xe_register_android_vendor_hooks(xe);
+	if (err)
+		goto err;
+
 	err = xe_shrinker_create(xe);
 	if (err)
 		goto err;
diff --git a/drivers/gpu/drm/xe/xe_mem_monitor.c b/drivers/gpu/drm/xe/xe_mem_monitor.c
index 06fbd0624da7..9205417bfb0b 100644
--- a/drivers/gpu/drm/xe/xe_mem_monitor.c
+++ b/drivers/gpu/drm/xe/xe_mem_monitor.c
@@ -69,6 +69,7 @@
 #include "xe_device.h"
 #include "xe_device_types.h"
 #include "xe_module.h"
+#include "xe_trace.h"
 
 /*
  * EMA smoothing factor. alpha (13/16 = 0.8125) applies to every real
@@ -157,18 +158,24 @@ struct xe_mem_monitor_data {
 
 static void xe_mem_monitor_set_high_throughput_mode(struct xe_mem_monitor_data *mon)
 {
-	/* TODO: wire up to the actual vendor hook (adds __GFP_RETRY_MAYFAIL, TBD) */
+	/*
+	 * The TTM vendor hook (xe_ttm_vendor_hooks.c) reads xe->gpu_mem_mode
+	 * directly on the next 2M page allocation, so updating the mode here
+	 * is sufficient - no separate hook call is needed.
+	 */
 	drm_dbg(&mon->xe->drm,
-		 "xe_mem_monitor: switching to HIGH_THROUGHPUT mode (TBD)\n");
+		 "xe_mem_monitor: switching to HIGH_THROUGHPUT mode\n");
 	atomic_set(&mon->xe->gpu_mem_mode, XE_GPU_MEM_MODE_HIGH_THROUGHPUT);
+	trace_xe_gpu_mem_mode_switch(XE_GPU_MEM_MODE_HIGH_THROUGHPUT, mon->ema_growth_bytes);
 }
 
 static void xe_mem_monitor_set_latency_mode(struct xe_mem_monitor_data *mon)
 {
-	/* TODO: wire up to the actual vendor hook (TBD) */
+	/* See xe_mem_monitor_set_high_throughput_mode() above. */
 	drm_dbg(&mon->xe->drm,
-		 "xe_mem_monitor: switching to LATENCY mode (TBD)\n");
+		 "xe_mem_monitor: switching to LATENCY mode\n");
 	atomic_set(&mon->xe->gpu_mem_mode, XE_GPU_MEM_MODE_LATENCY);
+	trace_xe_gpu_mem_mode_switch(XE_GPU_MEM_MODE_LATENCY, mon->ema_growth_bytes);
 }
 
 static void xe_mem_monitor_work(struct work_struct *work)
diff --git a/drivers/gpu/drm/xe/xe_trace.h b/drivers/gpu/drm/xe/xe_trace.h
index 314f42fcbcbd..77bfc08613b9 100644
--- a/drivers/gpu/drm/xe/xe_trace.h
+++ b/drivers/gpu/drm/xe/xe_trace.h
@@ -441,6 +441,22 @@ TRACE_EVENT(xe_eu_stall_data_read,
 		      __entry->read_size, __entry->total_size)
 );
 
+TRACE_EVENT(xe_gpu_mem_mode_switch,
+	    TP_PROTO(int mode, u64 growth_bytes),
+	    TP_ARGS(mode, growth_bytes),
+
+	    TP_STRUCT__entry(__field(int, mode)
+			     __field(u64, growth_bytes)
+			     ),
+
+	    TP_fast_assign(__entry->mode = mode;
+			   __entry->growth_bytes = growth_bytes;
+			   ),
+
+	    TP_printk("mode=%d, growth_bytes=%llu", __entry->mode,
+		      __entry->growth_bytes)
+);
+
 #endif
 
 /* This part must be outside protection */
diff --git a/drivers/gpu/drm/xe/xe_ttm_vendor_hooks.c b/drivers/gpu/drm/xe/xe_ttm_vendor_hooks.c
new file mode 100644
index 000000000000..b4136ac6cc88
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_ttm_vendor_hooks.c
@@ -0,0 +1,135 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2026 Google LLC.
+ *
+ * TTM page allocation vendor hooks, registered internally by Xe (rather than
+ * via the standalone intel_ttm_vendor_hooks.ko module) for direct, low
+ * overhead access to xe_device state.
+ *
+ * The GFP flags for 2M (order-9) page allocations are adjusted based on
+ * xe->gpu_mem_mode, which is maintained by xe_mem_monitor:
+ *
+ *   XE_GPU_MEM_MODE_LATENCY:          strip __GFP_DIRECT_RECLAIM and
+ *                                     __GFP_KSWAPD_RECLAIM for fast, low
+ *                                     latency allocation. __GFP_NORETRY (set
+ *                                     by TTM for order>0) is left untouched.
+ *   XE_GPU_MEM_MODE_HIGH_THROUGHPUT:  ensure __GFP_DIRECT_RECLAIM and
+ *                                     __GFP_KSWAPD_RECLAIM are set, clear
+ *                                     __GFP_NORETRY (set by TTM for order>0,
+ *                                     and mutually exclusive in intent with
+ *                                     __GFP_RETRY_MAYFAIL below), and add
+ *                                     __GFP_RETRY_MAYFAIL to maximize the
+ *                                     chance of a successful 2M page
+ *                                     allocation.
+ *
+ * All other orders (including order-0) are left completely unmodified.
+ */
+
+#include <linux/atomic.h>
+#include <linux/gfp.h>
+#include <linux/printk.h>
+#include <trace/hooks/ttm_pool.h>
+
+#include "xe_device_types.h"
+#include "xe_mem_monitor.h"
+#include "xe_module.h"
+#include "xe_ttm_vendor_hooks.h"
+
+/* 2M pages (order-9) are the only order whose GFP flags are adjusted. */
+#define XE_TTM_VENDOR_HOOKS_ORDER	9
+
+static void xe_android_vh_ttm_pool_alloc_max_page_order(void *data,
+							unsigned int *max_order)
+{
+	*max_order = XE_TTM_VENDOR_HOOKS_ORDER;
+}
+
+static void xe_android_vh_ttm_pool_alloc_page_flags(void *data,
+						    unsigned int order,
+						    gfp_t *gfp_flags)
+{
+	struct xe_device *xe = (struct xe_device *)data;
+	int mode;
+
+	/* order-0 must always be with __GFP_DIRECT_RECLAIM to avoid ENOMEM
+	 * unless it is absolutely necessary (system memory is completely
+	 * exhausted).
+	 */
+	if (!order)
+		return;
+
+	mode = atomic_read(&xe->gpu_mem_mode);
+
+	if (order == XE_TTM_VENDOR_HOOKS_ORDER &&
+	    mode == XE_GPU_MEM_MODE_HIGH_THROUGHPUT) {
+		/*
+		 * Ensure reclaim is allowed, clear __GFP_NORETRY (set by TTM for
+		 * order>0, but mutually exclusive in intent with
+		 * __GFP_RETRY_MAYFAIL) and add __GFP_RETRY_MAYFAIL so the
+		 * allocator retries harder for a 2M page instead of failing fast.
+		 * Order-9 (2MB) is specifically targeted here as the underlying IOMMU
+		 * hardware and Xe architecture operate most efficiently with 2MB alignments.
+		 */
+		*gfp_flags |= (__GFP_DIRECT_RECLAIM | __GFP_KSWAPD_RECLAIM | __GFP_RETRY_MAYFAIL);
+		*gfp_flags &= ~__GFP_NORETRY;
+	} else {
+		*gfp_flags &= ~(__GFP_DIRECT_RECLAIM | __GFP_KSWAPD_RECLAIM);
+	}
+}
+
+int xe_register_android_vendor_hooks(struct xe_device *xe)
+{
+	int ret;
+
+	/*
+	 * Gated by the same master switch as xe_mem_monitor: if the feature is
+	 * disabled, do not register the hooks at all so TTM's default GFP flag
+	 * behavior is left completely untouched.
+	 */
+	if (!xe_modparam.mem_monitor_enabled)
+		return 0;
+
+	ret = register_trace_android_vh_ttm_pool_alloc_max_page_order(
+		xe_android_vh_ttm_pool_alloc_max_page_order, (void *)xe);
+	if (ret) {
+		pr_warn("failed to register ttm_pool_alloc_max_page_order hook: %d\n",
+			ret);
+		goto fail;
+	}
+
+	ret = register_trace_android_vh_ttm_pool_alloc_page_flags(
+		xe_android_vh_ttm_pool_alloc_page_flags, (void *)xe);
+	if (ret) {
+		pr_warn("failed to register ttm_pool_alloc_page_flags hook: %d\n",
+			ret);
+		goto fail_alloc_page_flags;
+	}
+
+	return 0;
+
+fail_alloc_page_flags:
+	unregister_trace_android_vh_ttm_pool_alloc_max_page_order(
+		xe_android_vh_ttm_pool_alloc_max_page_order, (void *)xe);
+fail:
+	return ret;
+}
+
+void xe_unregister_android_vendor_hooks(struct xe_device *xe)
+{
+	int ret;
+
+	if (!xe_modparam.mem_monitor_enabled)
+		return;
+
+	ret = unregister_trace_android_vh_ttm_pool_alloc_max_page_order(
+		xe_android_vh_ttm_pool_alloc_max_page_order, (void *)xe);
+	if (ret)
+		pr_warn("failed to unregister ttm_pool_alloc_max_page_order hook: %d\n",
+			ret);
+
+	ret = unregister_trace_android_vh_ttm_pool_alloc_page_flags(
+		xe_android_vh_ttm_pool_alloc_page_flags, (void *)xe);
+	if (ret)
+		pr_warn("failed to unregister ttm_pool_alloc_page_flags hook: %d\n",
+			ret);
+}
diff --git a/drivers/gpu/drm/xe/xe_ttm_vendor_hooks.h b/drivers/gpu/drm/xe/xe_ttm_vendor_hooks.h
new file mode 100644
index 000000000000..f2059a3822ae
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_ttm_vendor_hooks.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2026 Google LLC.
+ */
+
+struct xe_device;
+
+int xe_register_android_vendor_hooks(struct xe_device *xe);
+void xe_unregister_android_vendor_hooks(struct xe_device *xe);
-- 
2.34.1

[PATCH 3/4] RFC: ANDROID: drm/xe: provide safe mem_monitor enable toggle via debugfs

[S Sebinraj]

From: Ryan Neph <ryanneph@google.com>

Provide a way to safely enable/disable mem_monitor at runtime and modify
its parameters with the sequence:

```
su
echo 0 > /sys/kernel/debug/dri/0/mem_monitor/enabled
; # modify params: /sys/module/xe/parameters/mem_monitor_*
echo 1 > /sys/kernel/debug/dri/0/mem_monitor/enabled
```

Also adds to /sys/kernel/debug/dri/0/mem_monitor/:
  - ./info: to show state of the mem_monitor
  - ./mode: to show/set the current mode

Set the mode by writing "latency" or "throughput" to the file.
If mem_monitor is:
  - disabled, this mode is persistent.
  - enabled, this mode may be changed by the mem_monitor

Test: echo 0 > /sys/kernel/dri/0/mem_monitor/enabled  # disables worker
Test: echo 1 > /sys/kernel/dri/0/mem_monitor/enabled  # re-enables worker

Signed-off-by: Ryan Neph <ryanneph@google.com>
Signed-off-by: S Sebinraj <s.sebinraj@intel.com>
---
 drivers/gpu/drm/xe/xe_device.c       |   1 +
 drivers/gpu/drm/xe/xe_device_types.h |   1 +
 drivers/gpu/drm/xe/xe_mem_monitor.c  | 153 ++++++++++++++++++++++++++-
 drivers/gpu/drm/xe/xe_mem_monitor.h  |  11 ++
 drivers/gpu/drm/xe/xe_module.c       |   1 +
 5 files changed, 166 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
index d21068c1effc..c8cec778cc97 100644
--- a/drivers/gpu/drm/xe/xe_device.c
+++ b/drivers/gpu/drm/xe/xe_device.c
@@ -941,6 +941,7 @@ int xe_device_probe(struct xe_device *xe)
 	err = xe_mem_monitor_init(xe);
 	if (err)
 		return err;
+	xe_mem_monitor_debugfs_register(xe);
 
 	err = xe_late_bind_init(&xe->late_bind);
 	if (err)
diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
index 48aa9d8f6f56..b920a412026b 100644
--- a/drivers/gpu/drm/xe/xe_device_types.h
+++ b/drivers/gpu/drm/xe/xe_device_types.h
@@ -586,6 +586,7 @@ struct xe_device {
 
 	/** @mem_monitor: GPU memory growth monitor data */
 	struct xe_mem_monitor_data *mem_monitor;
+	struct dentry *mem_monitor_debugfs_root;
 
 	/** @needs_flr_on_fini: requests function-reset on fini */
 	bool needs_flr_on_fini;
diff --git a/drivers/gpu/drm/xe/xe_mem_monitor.c b/drivers/gpu/drm/xe/xe_mem_monitor.c
index 9205417bfb0b..a34cafe56976 100644
--- a/drivers/gpu/drm/xe/xe_mem_monitor.c
+++ b/drivers/gpu/drm/xe/xe_mem_monitor.c
@@ -59,9 +59,12 @@
 #include "xe_mem_monitor.h"
 
 #include <linux/atomic.h>
+#include <linux/debugfs.h>
 #include <linux/jiffies.h>
 #include <linux/minmax.h>
+#include <linux/seq_file.h>
 #include <linux/slab.h>
+#include <linux/string.h>
 #include <linux/workqueue.h>
 #include <drm/drm_managed.h>
 #include <drm/drm_print.h>
@@ -154,6 +157,11 @@ struct xe_mem_monitor_data {
 	bool is_high_throughput;
 	atomic_t suspended;
 	bool was_pending_before_suspend;
+
+	struct {
+		atomic64_t current_bytes;
+		atomic64_t delta_signed;
+	} info;
 };
 
 static void xe_mem_monitor_set_high_throughput_mode(struct xe_mem_monitor_data *mon)
@@ -247,6 +255,9 @@ static void xe_mem_monitor_work(struct work_struct *work)
 				 ALPHA_RETAINED * mon->ema_growth_bytes)
 				>> ALPHA_SHIFT;
 
+	atomic64_set(&mon->info.current_bytes, current_bytes);
+	atomic64_set(&mon->info.delta_signed, delta_signed);
+
 	atomic64_set(&mon->prev_mem_bytes, current_bytes);
 	mon->last_poll_jiffies = now;
 
@@ -320,8 +331,21 @@ static void xe_mem_monitor_work(struct work_struct *work)
 static void xe_mem_monitor_cleanup(struct drm_device *drm, void *arg)
 {
 	struct xe_mem_monitor_data *mon = arg;
+	struct xe_device *xe = mon->xe;
 
+	WARN_ON_ONCE(!xe->mem_monitor);
 	cancel_delayed_work_sync(&mon->work);
+	kfree(mon);
+	xe->mem_monitor = NULL;
+}
+
+void xe_mem_monitor_fini(struct xe_device *xe)
+{
+	if (!xe->mem_monitor)
+		return;
+
+	drmm_release_action(&xe->drm, xe_mem_monitor_cleanup, (void *)xe->mem_monitor);
+	drm_info(&xe->drm, "xe_mem_monitor: stopped");
 }
 
 /**
@@ -437,7 +461,7 @@ int xe_mem_monitor_init(struct xe_device *xe)
 		return 0;
 	}
 
-	mon = drmm_kzalloc(&xe->drm, sizeof(*mon), GFP_KERNEL);
+	mon = kzalloc(sizeof(*mon), GFP_KERNEL);
 	if (!mon)
 		return -ENOMEM;
 
@@ -467,3 +491,130 @@ int xe_mem_monitor_init(struct xe_device *xe)
 
 	return 0;
 }
+
+static ssize_t enabled_write(struct file *file,
+			     const char __user *userbuf,
+			     size_t count, loff_t *ppos)
+{
+	struct seq_file *s = file->private_data;
+	struct xe_device *xe = s->private;
+	bool enable;
+	int ret;
+
+	ret = kstrtobool_from_user(userbuf, count, &enable);
+	if (ret)
+		return ret;
+
+	if (!enable && xe->mem_monitor)
+		xe_mem_monitor_fini(xe);
+	else if (enable && !xe->mem_monitor)
+		xe_mem_monitor_init(xe);
+
+	return count;
+}
+static int enabled_show(struct seq_file *s, void *data)
+{
+	struct xe_device *xe = s->private;
+
+	seq_printf(s, "%c\n", xe->mem_monitor ? 'Y' : 'N');
+	return 0;
+}
+DEFINE_SHOW_STORE_ATTRIBUTE(enabled);
+
+static int info_show(struct seq_file *s, void *data)
+{
+	struct xe_device *xe = s->private;
+	struct xe_mem_monitor_data *mon = xe->mem_monitor;
+
+	if (!mon)
+		return 1;
+
+	seq_printf(s, "current=%llu MB, delta=%lld MB, EMA=%llu MB mode=%d\n",
+		   atomic64_read(&mon->info.current_bytes) >> 20,
+		   atomic64_read(&mon->info.delta_signed) >> 20,
+		   mon->ema_growth_bytes >> 20,
+		   atomic_read(&xe->gpu_mem_mode));
+	return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(info);
+
+static ssize_t mode_write(struct file *file, const char __user *userbuf,
+			  size_t count, loff_t *ppos)
+{
+	struct seq_file *s = file->private_data;
+	struct xe_device *xe = s->private;
+	char buf[32];
+	size_t len;
+
+	len = min(count, sizeof(buf) - 1);
+	if (copy_from_user(buf, userbuf, len))
+		return -EFAULT;
+	buf[len] = '\0';
+
+	if (sysfs_streq(buf, "latency"))
+		atomic_set(&xe->gpu_mem_mode, XE_GPU_MEM_MODE_LATENCY);
+	else if (sysfs_streq(buf, "throughput"))
+		atomic_set(&xe->gpu_mem_mode, XE_GPU_MEM_MODE_HIGH_THROUGHPUT);
+
+	return count;
+}
+
+static int mode_show(struct seq_file *s, void *data)
+{
+	struct xe_device *xe = s->private;
+	enum xe_gpu_mem_mode mode = atomic_read(&xe->gpu_mem_mode);
+
+	/*
+	 * Print the options, placing brackets around the currently active mode.
+	 */
+	switch (mode) {
+	case XE_GPU_MEM_MODE_LATENCY:
+		seq_puts(s, "[latency] throughput\n");
+		break;
+	case XE_GPU_MEM_MODE_HIGH_THROUGHPUT:
+		seq_puts(s, "latency [throughput]\n");
+		break;
+	default:
+		seq_puts(s, "latency throughput\n");
+		break;
+	}
+
+	return 0;
+}
+DEFINE_SHOW_STORE_ATTRIBUTE(mode);
+
+static void xe_mem_monitor_debugfs_unregister(struct drm_device *dev,
+					      void *data)
+{
+	struct xe_device *xe = data;
+
+	debugfs_remove(xe->mem_monitor_debugfs_root);
+}
+
+/* Exposes controls through debugfs under the drm root for this device:
+ *   /sys/kernel/dri/0/mem_monitor/
+ */
+int xe_mem_monitor_debugfs_register(struct xe_device *xe)
+{
+	struct dentry *root;
+	int ret;
+
+	root = debugfs_create_dir("mem_monitor", xe->drm.debugfs_root);
+	if (IS_ERR(root)) {
+		drm_warn(&xe->drm, "failed to create mem_monitor debug directory");
+		return 1;
+	}
+	root->d_inode->i_private = xe;
+	xe->mem_monitor_debugfs_root = root;
+
+	if (xe_modparam.mem_monitor_enabled) {
+		WARN_ON(!xe->mem_monitor);
+		debugfs_create_file("enabled", 0600, root, xe, &enabled_fops);
+		debugfs_create_file("info", 0400, root, xe, &info_fops);
+	}
+	debugfs_create_file("mode", 0600, root, xe, &mode_fops);
+
+	ret = drmm_add_action_or_reset(&xe->drm,
+				       xe_mem_monitor_debugfs_unregister, xe);
+	return ret;
+}
diff --git a/drivers/gpu/drm/xe/xe_mem_monitor.h b/drivers/gpu/drm/xe/xe_mem_monitor.h
index 846c0628ed93..c617c10c1a12 100644
--- a/drivers/gpu/drm/xe/xe_mem_monitor.h
+++ b/drivers/gpu/drm/xe/xe_mem_monitor.h
@@ -39,6 +39,8 @@ enum xe_gpu_mem_mode {
 #ifdef CONFIG_DRM_XE_MEM_MONITOR
 
 int xe_mem_monitor_init(struct xe_device *xe);
+void xe_mem_monitor_fini(struct xe_device *xe);
+int xe_mem_monitor_debugfs_register(struct xe_device *xe);
 
 /*
  * Notify the monitor that GPU allocation activity occurred (called from the
@@ -77,6 +79,15 @@ static inline void xe_mem_monitor_resume(struct xe_device *xe)
 {
 }
 
+static inline void xe_mem_monitor_fini(struct xe_device *xe)
+{
+}
+
+static inline int xe_mem_monitor_debugfs_register(struct xe_device *xe)
+{
+	return 0;
+}
+
 #endif /* CONFIG_DRM_XE_MEM_MONITOR */
 
 #endif /* _XE_MEM_MONITOR_H_ */
diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
index a49b0c0f958b..a38776734544 100644
--- a/drivers/gpu/drm/xe/xe_module.c
+++ b/drivers/gpu/drm/xe/xe_module.c
@@ -5,6 +5,7 @@
 
 #include "xe_module.h"
 
+#include "linux/log2.h"
 #include <linux/init.h>
 #include <linux/module.h>
 #include <linux/stringify.h>
-- 
2.34.1

[PATCH 4/4] RFC: ANDROID: drm/xe: Gate mem_monitor wake ups on order 9 external fragmentation

[S Sebinraj]

From: S Sebinraj <s.sebinraj@intel.corp-partner.google.com>

xe_mem_monitor_notify_activity() previously woke the polling worker
based only on a raw byte-level delta (ACTIVITY_MIN_DELTA_BYTES), which
doesn't capture whether order 9 (2M) TTM allocations are actually
under any pressure.

Add xe_mem_monitor_zone_order9_free_bytes(), which reads the amount of
order 9 or larger free memory directly from struct zone's free_area
counts.

Add xe_mem_monitor_order9_supply_low(), which walks the populated zones
and returns true if any zone's order 9 free supply has dropped below
an adaptive floor. The floor is mon->max_delta_bytes, the largest
growth sample xe_mem_monitor_work() has ever observed on this device
clamped between the mem_monitor_order9_supply_min_mb and
mem_monitor_order9_supply_max_mb module parameters (in MiB).

Adapting the floor to the largest growth sample actually observed
avoids false positives from a single large but otherwise harmless
allocation burst when overall order 9 supply is still healthy.
The upper clamp (mem_monitor_order9_supply_max_mb) stops a single
anomalously large allocation from permanently pinning the floor too high.

Use this as an additional guard in xe_mem_monitor_notify_activity
both the existing byte-delta check and this free-supply check must
now indicate pressure before the worker is woken.

Test: Tested by running WLEU on a fragmented system and confirmed the
      worker is only woken when some populated zone's order-9 free
      supply drops below ORDER9_SUPPLY_MIN_PCT of total RAM.

Cc: Carlos Santa <carlos.santa@intel.com>
Cc: Erin Park <erin.park@intel.com>
Cc: Ryan Neph <ryanneph@google.com>
Cc: Renato Pereyra <renatopereyra@google.com>
Signed-off-by: S Sebinraj <s.sebinraj@intel.com>
---
 drivers/gpu/drm/xe/xe_mem_monitor.c | 115 +++++++++++++++++++++++++++-
 drivers/gpu/drm/xe/xe_mem_monitor.h |   2 +
 drivers/gpu/drm/xe/xe_module.c      |  22 ++++++
 drivers/gpu/drm/xe/xe_module.h      |   2 +
 4 files changed, 137 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/xe/xe_mem_monitor.c b/drivers/gpu/drm/xe/xe_mem_monitor.c
index a34cafe56976..24458af2700c 100644
--- a/drivers/gpu/drm/xe/xe_mem_monitor.c
+++ b/drivers/gpu/drm/xe/xe_mem_monitor.c
@@ -59,9 +59,11 @@
 #include "xe_mem_monitor.h"
 
 #include <linux/atomic.h>
+#include <linux/compiler.h>
 #include <linux/debugfs.h>
 #include <linux/jiffies.h>
 #include <linux/minmax.h>
+#include <linux/mmzone.h>
 #include <linux/seq_file.h>
 #include <linux/slab.h>
 #include <linux/string.h>
@@ -129,6 +131,13 @@
  * @last_poll_jiffies:     jiffies value of the most recent real sample taken
  *                         by xe_mem_monitor_work(), used to detect and decay
  *                         the EMA across skipped (slept-through) intervals
+ * @max_delta_bytes:       largest clamped_delta (growth-only, per real
+ *                         sample) ever observed by xe_mem_monitor_work(),
+ *                         used by xe_mem_monitor_order9_supply_low() to
+ *                         adapt its floor upward to this workload's
+ *                         demonstrated behaviour (clamped between
+ *                         mem_monitor_order9_supply_min_mb and
+ *                         mem_monitor_order9_supply_max_mb there).
  * @suspended:             set while the device is suspended (system or
  *                         runtime PM); xe_mem_monitor_notify_activity() is a
  *                         no-op while set, so that BO eviction/restore
@@ -155,6 +164,7 @@ struct xe_mem_monitor_data {
 	unsigned long last_activity_jiffies;
 	unsigned long last_poll_jiffies;
 	bool is_high_throughput;
+	u64 max_delta_bytes;
 	atomic_t suspended;
 	bool was_pending_before_suspend;
 
@@ -186,6 +196,26 @@ static void xe_mem_monitor_set_latency_mode(struct xe_mem_monitor_data *mon)
 	trace_xe_gpu_mem_mode_switch(XE_GPU_MEM_MODE_LATENCY, mon->ema_growth_bytes);
 }
 
+/*
+ * Returns the total amount of memory (bytes) available as order-9 (2M) or
+ * larger contiguous blocks in the given zone. Computed directly from
+ * struct zone's free_area[] counts (see <linux/mmzone.h>)
+ */
+static u64 xe_mem_monitor_zone_order9_free_bytes(struct zone *zone)
+{
+	unsigned long free_blocks_suitable = 0;
+	unsigned int order;
+
+	for (order = 9; order < NR_PAGE_ORDERS; order++) {
+		/* nr_free is lockless/diagnostic-only */
+		unsigned long blocks = data_race(zone->free_area[order].nr_free);
+
+		free_blocks_suitable += blocks << (order - 9);
+	}
+
+	return (u64)free_blocks_suitable << (9 + PAGE_SHIFT);
+}
+
 static void xe_mem_monitor_work(struct work_struct *work)
 {
 	struct xe_mem_monitor_data *mon =
@@ -255,6 +285,8 @@ static void xe_mem_monitor_work(struct work_struct *work)
 				 ALPHA_RETAINED * mon->ema_growth_bytes)
 				>> ALPHA_SHIFT;
 
+	mon->max_delta_bytes = max_t(u64, mon->max_delta_bytes, clamped_delta);
+
 	atomic64_set(&mon->info.current_bytes, current_bytes);
 	atomic64_set(&mon->info.delta_signed, delta_signed);
 
@@ -348,6 +380,58 @@ void xe_mem_monitor_fini(struct xe_device *xe)
 	drm_info(&xe->drm, "xe_mem_monitor: stopped");
 }
 
+/*
+ * Companion check to ACTIVITY_MIN_DELTA_BYTES for
+ * xe_mem_monitor_notify_activity(): a large byte-level delta doesn't
+ * necessarily mean order-9 (2M) allocations are under any real pressure, so
+ * this looks instead at whether any single populated zone's absolute
+ * order-9-and-larger free supply has dropped below the adaptive floor
+ * clamp_t(mon->max_delta_bytes, mem_monitor_order9_supply_min_mb,
+ * mem_monitor_order9_supply_max_mb) independent of the size of whatever
+ * delta triggered this call, since a single large delta is not itself
+ * evidence that supply is actually running low. Taking mon->max_delta_bytes
+ * (the largest growth sample ever observed on this device, see
+ * xe_mem_monitor_work()) as the basis, clamped between the two module
+ * parameters, means the floor adapts upward to this workload's own
+ * demonstrated behaviour without being able to grow unbounded from a
+ * single anomalous allocation.
+ */
+#if IS_ENABLED(CONFIG_COMPACTION)
+static bool xe_mem_monitor_order9_supply_low(struct xe_mem_monitor_data *mon)
+{
+	struct xe_device *xe = mon->xe;
+	struct pglist_data *pgdat = NODE_DATA(0);
+	u64 min_free_bytes = clamp_t(u64, mon->max_delta_bytes,
+				     (u64)xe_modparam.mem_monitor_order9_supply_min_mb << 20,
+				     (u64)xe_modparam.mem_monitor_order9_supply_max_mb << 20);
+	unsigned int z;
+
+	for (z = 0; z < MAX_NR_ZONES; z++) {
+		struct zone *zone = &pgdat->node_zones[z];
+
+		if (!populated_zone(zone))
+			continue;
+
+		if (xe_mem_monitor_zone_order9_free_bytes(zone) < min_free_bytes) {
+			drm_dbg(&xe->drm,
+				"xe_mem_monitor: zone=%s order9_free=%llu MB < min=%llu MB\n",
+				zone->name,
+				xe_mem_monitor_zone_order9_free_bytes(zone) >> 20,
+				min_free_bytes >> 20);
+			return true;
+		}
+	}
+
+	return false;
+}
+#else
+static inline bool xe_mem_monitor_order9_supply_low(struct xe_mem_monitor_data *mon)
+{
+	/* No zone data available - don't gate on it. */
+	return true;
+}
+#endif
+
 /**
  * xe_mem_monitor_notify_activity - notify the monitor of GPU memory changes
  * @xe: the Xe device
@@ -367,6 +451,12 @@ void xe_mem_monitor_fini(struct xe_device *xe)
  * cross. If the worker is already scheduled (delayed_work_pending()), it
  * will read the live counter fresh when it runs and pick up whatever
  * accumulated, so there's no need to re-schedule again here.
+ *
+ * As a further guard, even a significant byte-level change is ignored unless
+ * xe_mem_monitor_order9_supply_low() also reports that some populated
+ * zone's order-9-and-larger free supply has dropped below the adaptive
+ * floor described there. A byte-level delta alone doesn't imply 2M
+ * allocations are under any real pressure.
  */
 void xe_mem_monitor_notify_activity(struct xe_device *xe)
 {
@@ -387,11 +477,14 @@ void xe_mem_monitor_notify_activity(struct xe_device *xe)
 	if (abs_delta < ACTIVITY_MIN_DELTA_BYTES)
 		return;
 
-	mon->last_activity_jiffies = jiffies;
-
 	if (delayed_work_pending(&mon->work))
 		return;
 
+	if (!xe_mem_monitor_order9_supply_low(mon))
+		return;
+
+	mon->last_activity_jiffies = jiffies;
+
 	schedule_delayed_work(&mon->work, 0);
 }
 
@@ -525,15 +618,29 @@ static int info_show(struct seq_file *s, void *data)
 {
 	struct xe_device *xe = s->private;
 	struct xe_mem_monitor_data *mon = xe->mem_monitor;
+	int mode = atomic_read(&xe->gpu_mem_mode);
+	struct pglist_data *pgdat = NODE_DATA(0);
+	unsigned long long order9_free_bytes = 0;
 
 	if (!mon)
 		return 1;
 
-	seq_printf(s, "current=%llu MB, delta=%lld MB, EMA=%llu MB mode=%d\n",
+	for (int z = 0; z < MAX_NR_ZONES; z++) {
+		struct zone *zone = &pgdat->node_zones[z];
+
+		if (!populated_zone(zone))
+			continue;
+		order9_free_bytes += xe_mem_monitor_zone_order9_free_bytes(zone);
+	}
+
+	seq_printf(s, "current=%llu MB, delta=%lld MB, EMA=%llu MB mode=%s, alpha=0.%03u, "
+		   "order9_free_total=%llu MB\n",
 		   atomic64_read(&mon->info.current_bytes) >> 20,
 		   atomic64_read(&mon->info.delta_signed) >> 20,
 		   mon->ema_growth_bytes >> 20,
-		   atomic_read(&xe->gpu_mem_mode));
+		   mode == XE_GPU_MEM_MODE_LATENCY ? "latency" : "throughput",
+		   (ALPHA_NUM * 1000U + (1U << (ALPHA_SHIFT - 1))) >> ALPHA_SHIFT,
+		   order9_free_bytes >> 20);
 	return 0;
 }
 DEFINE_SHOW_ATTRIBUTE(info);
diff --git a/drivers/gpu/drm/xe/xe_mem_monitor.h b/drivers/gpu/drm/xe/xe_mem_monitor.h
index c617c10c1a12..a30fabd5a8ea 100644
--- a/drivers/gpu/drm/xe/xe_mem_monitor.h
+++ b/drivers/gpu/drm/xe/xe_mem_monitor.h
@@ -28,6 +28,8 @@ enum xe_gpu_mem_mode {
 #define XE_MEM_MONITOR_DEFAULT_POLL_MS				2000
 #define XE_MEM_MONITOR_DEFAULT_DEBOUNCE_COUNT			5
 #define XE_MEM_MONITOR_DEFAULT_IDLE_TIMEOUT_MS			10000
+#define XE_MEM_MONITOR_DEFAULT_ORDER9_SUPPLY_MIN_MB		512
+#define XE_MEM_MONITOR_DEFAULT_ORDER9_SUPPLY_MAX_MB		1024
 
 /*
  * Minimum enforced value for mem_monitor_poll_ms (no maximum is
diff --git a/drivers/gpu/drm/xe/xe_module.c b/drivers/gpu/drm/xe/xe_module.c
index a38776734544..69825baf100c 100644
--- a/drivers/gpu/drm/xe/xe_module.c
+++ b/drivers/gpu/drm/xe/xe_module.c
@@ -54,6 +54,8 @@ struct xe_modparam xe_modparam = {
 	.mem_monitor_poll_ms = XE_MEM_MONITOR_DEFAULT_POLL_MS,
 	.mem_monitor_debounce_count = XE_MEM_MONITOR_DEFAULT_DEBOUNCE_COUNT,
 	.mem_monitor_idle_timeout_ms = XE_MEM_MONITOR_DEFAULT_IDLE_TIMEOUT_MS,
+	.mem_monitor_order9_supply_min_mb = XE_MEM_MONITOR_DEFAULT_ORDER9_SUPPLY_MIN_MB,
+	.mem_monitor_order9_supply_max_mb = XE_MEM_MONITOR_DEFAULT_ORDER9_SUPPLY_MAX_MB,
 #endif
 	/* the rest are 0 by default */
 };
@@ -162,6 +164,26 @@ MODULE_PARM_DESC(mem_monitor_idle_timeout_ms,
 		 "mode after which the monitor's polling worker goes to sleep "
 		 "(woken again on the next allocation) [default="
 		 __stringify(XE_MEM_MONITOR_DEFAULT_IDLE_TIMEOUT_MS) "])");
+
+module_param_named(mem_monitor_order9_supply_min_mb,
+		   xe_modparam.mem_monitor_order9_supply_min_mb, uint, 0644);
+MODULE_PARM_DESC(mem_monitor_order9_supply_min_mb,
+		 "Minimum order-9-and-larger free supply, in MiB, required per "
+		 "populated zone before a significant GPU memory change wakes the "
+		 "monitor's polling worker. The effective floor actually used is "
+		 "the largest single growth sample ever observed by the worker, "
+		 "clamped between this value and mem_monitor_order9_supply_max_mb, "
+		 "so it adapts to this workload's own demonstrated behaviour without "
+		 "growing unbounded from a single anomalous allocation "
+		 "[default=" __stringify(XE_MEM_MONITOR_DEFAULT_ORDER9_SUPPLY_MIN_MB) "])");
+
+module_param_named(mem_monitor_order9_supply_max_mb,
+		   xe_modparam.mem_monitor_order9_supply_max_mb, uint, 0644);
+MODULE_PARM_DESC(mem_monitor_order9_supply_max_mb,
+		 "Upper bound, in MiB, on the adaptive order-9 free supply floor "
+		 "described under mem_monitor_order9_supply_min_mb - caps how high "
+		 "a single unusually large growth sample can permanently push that "
+		 "floor [default=" __stringify(XE_MEM_MONITOR_DEFAULT_ORDER9_SUPPLY_MAX_MB) "])");
 #endif
 
 static int xe_check_nomodeset(void)
diff --git a/drivers/gpu/drm/xe/xe_module.h b/drivers/gpu/drm/xe/xe_module.h
index 24e0b52e48ca..19d0de96e078 100644
--- a/drivers/gpu/drm/xe/xe_module.h
+++ b/drivers/gpu/drm/xe/xe_module.h
@@ -38,6 +38,8 @@ struct xe_modparam {
 	u32 mem_monitor_poll_ms;
 	u32 mem_monitor_debounce_count;
 	u32 mem_monitor_idle_timeout_ms;
+	u32 mem_monitor_order9_supply_min_mb;
+	u32 mem_monitor_order9_supply_max_mb;
 #endif
 };
 
-- 
2.34.1