Re: [PATCH v6 3/3] drm/xe: Add a minimum age gate to the shrinker

[Matthew Brost <matthew.brost@intel.com>]

On Fri, Oct 30, 2026 at 04:17:20PM +0100, Thomas Hellström wrote:
> On Tue, 2026-10-27 at 15:49 -0700, Matthew Brost wrote:
> > Backing up a BO costs a GPU blit plus a page copy, and if the BO is
> > part
> > of an active working set it is faulted straight back in on the next
> > exec.
> > Evicting a BO that was used moments ago is therefore almost always a
> > loss, yet the shrinker has no notion of recency beyond LRU order.
> > 
> > Timestamp BOs wherever their LRU position is updated, and read the
> > stamp
> > back from whatever owns that position. A BO in a VM's bulk move is
> > anchored inside its range, so it is aged by the VM-wide stamp taken
> > in
> > xe_vm_bump_lru() alongside the bulk move itself. Any other BO carries
> > its own, stamped there too and on a successful move into TT in
> > xe_bo_move(), where ttm_resource_alloc() has already placed the new
> > resource at the LRU tail.
> > 
> > Add a per-device shrinker_min_age_ms sysfs attribute, defaulting to
> > CONFIG_DRM_XE_SHRINKER_MIN_AGE_MS (0, disabled). While set, the
> > shrinker
> > won't back up BOs younger than the delay. Each LRU priority is
> > roughly
> > age ordered, so the gate is applied per priority: xe_shrinker_count()
> > snoops the first shrinkable BO of each priority and reports nothing
> > only
> > if all of them are young, and the backup walk skips to the next
> > priority
> > at the first young BO, both as a backstop for the count sample and so
> > a
> > hot priority doesn't shield colder BOs in another one.
> > 
> > Only BOs this shrinker would actually back up have a say. BOs in
> > XE_PL_SYSTEM, which are not GPU mapped and hence not part of a
> > working
> > set, and BOs xe_bo_shrink() rejects outright are never young.
> > Purgeable
> > BOs and those advised DONTNEED are neither: they are dropped rather
> > than
> > backed up and accounted to the purgeable shrinker, so they are never
> > young and the count snoop skips their XE_BO_PRIORITY_LOW band
> > entirely.
> > 
> > The gate is hard and applies at every reclaim priority, so a working
> > set
> > kept younger than the delay is never backed up and can lead to OOM.
> > This
> > is deliberate, and why it is off by default.
> 
> For completeness, Perhaps add to the commit message that
> the plan is to follow up with an OOM notifier?
> 

That is a possible option. Another idea I had was if we can get the
priority exported from the shrinker core - we have a config knob that
says 'at this priority or greater', shrink BOs which have been used
recently.

> I also wonder what happens during the PM notifier preparing for suspend
> on DGFX, Does that move mark bos accessed, hence possibly blocked by
> the shrinker at suspend time?
> 

I haven't tried this on DGFX, can give it try but this off by default.

I'm also a little confused by this - does the shrinker run at suspend
time? I thought once the memory in ram a hiberation image in created
from the that - i.e., there isn't a requirement the BOs to get shrunk
into shmem.

> > 
> > Assisted-by: LLM
> > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > ---
> >  drivers/gpu/drm/xe/Kconfig.profile   |  15 +++
> >  drivers/gpu/drm/xe/xe_bo.c           |  83 ++++++++++++++
> >  drivers/gpu/drm/xe/xe_bo.h           |   5 +
> >  drivers/gpu/drm/xe/xe_bo_types.h     |   8 ++
> >  drivers/gpu/drm/xe/xe_device_sysfs.c |  35 ++++++
> >  drivers/gpu/drm/xe/xe_pagefault.c    |   2 +-
> >  drivers/gpu/drm/xe/xe_shrinker.c     | 164
> > +++++++++++++++++++++++++--
> >  drivers/gpu/drm/xe/xe_shrinker.h     |   3 +
> >  drivers/gpu/drm/xe/xe_vm.c           |   4 +-
> >  drivers/gpu/drm/xe/xe_vm_types.h     |   8 ++
> >  10 files changed, 316 insertions(+), 11 deletions(-)
> > 
> > diff --git a/drivers/gpu/drm/xe/Kconfig.profile
> > b/drivers/gpu/drm/xe/Kconfig.profile
> > index 23045ed2aa94..dd3985e502b8 100644
> > --- a/drivers/gpu/drm/xe/Kconfig.profile
> > +++ b/drivers/gpu/drm/xe/Kconfig.profile
> > @@ -74,6 +74,7 @@ config DRM_XE_ENABLE_SCHEDTIMEOUT_LIMIT
> >  	  to apply to applicable user. For elevated user, all above
> > MIN
> >  	  and MAX values will apply when this configuration is
> > enable to
> >  	  apply limitation. By default limitation is applied.
> > +
> >  config DRM_XE_SHRINKER_NON_PURGEABLE_SEEKS
> >  	int "Default seeks value of the non-purgeable bo shrinker"
> >  	default 8
> > @@ -86,3 +87,17 @@ config DRM_XE_SHRINKER_NON_PURGEABLE_SEEKS
> >  	  reclaim pressure on non-purgeable buffer objects, which
> > need
> >  	  GPU idling and writeback to be reclaimed, in favour of
> > purgeable
> >  	  ones and other kernel caches.
> > +
> > +config DRM_XE_SHRINKER_MIN_AGE_MS
> > +	int "Default minimum BO age before shrinker backup (ms)"
> > +	default 0
> > +	range 0 1000000
> > +	help
> > +	  Default for the per-device shrinker_min_age_ms sysfs
> > attribute. The
> > +	  shrinker won't back up a BO that was moved into TT, or
> > bumped on the
> > +	  LRU by an exec, less than this many milliseconds ago. BOs
> > already
> > +	  evicted to system memory are not part of a working set and
> > are
> > +	  never protected. This applies at every reclaim priority: a
> > working
> > +	  set kept younger than this is never backed up and can
> > cause OOM.
> > +	  Only set if that risk is preferred over thrashing a hot
> > working
> > +	  set. 0 disables.
> > diff --git a/drivers/gpu/drm/xe/xe_bo.c b/drivers/gpu/drm/xe/xe_bo.c
> > index 42a6aa1a2e4a..a4c0c6284a24 100644
> > --- a/drivers/gpu/drm/xe/xe_bo.c
> > +++ b/drivers/gpu/drm/xe/xe_bo.c
> > @@ -1357,9 +1357,92 @@ static int xe_bo_move(struct ttm_buffer_object
> > *ttm_bo, bool evict,
> >  		xe_tt_unmap_sg(xe, ttm_bo->ttm);
> >  	}
> >  
> > +	/*
> > +	 * ttm_resource_alloc() already placed the new resource at
> > the tail of
> > +	 * its LRU, or of @bo's bulk move, so only record when that
> > happened.
> > +	 */
> > +	if (!ret && ttm_bo->resource && ttm_bo->resource->mem_type
> > == XE_PL_TT)
> > +		WRITE_ONCE(bo->lru_stamp, jiffies);
> > +
> >  	return ret;
> >  }
> >  
> > +/**
> > + * xe_bo_bump_lru_locked() - Bump a BO on the LRU and stamp its last
> > use
> > + * @bo: The buffer object, reserved by the caller.
> > + *
> > + * Move @bo to the tail of its LRU and record the time for the
> > shrinker's
> > + * minimum age gate. Only meaningful for BOs outside a bulk move,
> > since a
> > + * BO in one is anchored inside its range and is bumped, and aged,
> > with it.
> > + * The caller must hold the device LRU lock.
> > + */
> > +void xe_bo_bump_lru_locked(struct xe_bo *bo)
> > +{
> > +	WRITE_ONCE(bo->lru_stamp, jiffies);
> > +	ttm_bo_move_to_lru_tail(&bo->ttm);
> > +}
> > +
> > +/**
> > + * xe_bo_shrink_backup_candidate() - Whether the shrinker may back
> > up a BO
> > + * @bo: The TTM buffer object, reserved by the caller.
> > + *
> > + * A BO is a backup candidate if backing it up is the only way to
> > reclaim it.
> > + * BOs advised DONTNEED, which sit at XE_BO_PRIORITY_LOW, and
> > purgeable ones
> > + * are not: their pages are simply dropped, which is cheap, and they
> > are
> > + * accounted to the purgeable shrinker. Neither are the BOs
> > xe_bo_shrink()
> > + * hands to xe_bo_shrink_purge() outright: ghost BOs, which aren't
> > xe BOs,
> > + * and BOs already freed and waiting to be reaped. Nor are BOs whose
> > tt
> > + * isn't TTM_TT_FLAG_EXTERNAL_MAPPABLE, which xe_bo_shrink()
> > rejects.
> > + *
> > + * Return: true if @bo would be backed up by a non-purge shrink
> > pass.
> > + */
> > +bool xe_bo_shrink_backup_candidate(struct ttm_buffer_object *bo)
> > +{
> > +	struct xe_ttm_tt *xe_tt;
> > +
> > +	if (!xe_bo_is_xe_bo(bo) || !kref_read(&bo->base.refcount) ||
> > !bo->ttm ||
> > +	    !(bo->ttm->page_flags & TTM_TT_FLAG_EXTERNAL_MAPPABLE)
> > ||
> > +	    bo->priority <= XE_BO_PRIORITY_LOW)
> > +		return false;
> > +
> > +	xe_tt = container_of(bo->ttm, struct xe_ttm_tt, ttm);
> > +
> > +	return !xe_tt->purgeable;
> > +}
> > +
> > +/**
> > + * xe_bo_shrink_too_young() - Whether a BO was used too recently to
> > shrink
> > + * @bo: The TTM buffer object, reserved by the caller.
> > + * @min_age: Minimum age in jiffies. 0 disables the check.
> > + *
> > + * The age is taken from whatever owns @bo's LRU position:
> > &xe_vm.lru_stamp
> > + * if @bo rides along with a VM's bulk move, its own
> > &xe_bo.lru_stamp if not.
> > + *
> > + * BOs that the shrink walk would not back up anyway are never
> > considered
> > + * young, see xe_bo_shrink_backup_candidate(). Neither are BOs in
> > + * XE_PL_SYSTEM: they are not GPU mapped, hence not part of a
> > working set,
> > + * and must be moved back to TT, refreshing the stamp, before use.
> > + *
> > + * Return: true if @bo was touched less than @min_age jiffies ago.
> > + */
> > +bool xe_bo_shrink_too_young(struct ttm_buffer_object *bo, unsigned
> > long min_age)
> > +{
> > +	struct xe_bo *xe_bo;
> > +	unsigned long stamp;
> > +
> > +	if (!min_age || bo->resource->mem_type == XE_PL_SYSTEM ||
> > +	    !xe_bo_shrink_backup_candidate(bo))
> > +		return false;
> > +
> > +	xe_bo = ttm_to_xe_bo(bo);
> > +	if (bo->bulk_move && xe_bo->vm)
> > +		stamp = READ_ONCE(xe_bo->vm->lru_stamp);
> > +	else
> > +		stamp = READ_ONCE(xe_bo->lru_stamp);
> > +
> > +	return stamp && time_before(jiffies, stamp + min_age);
> > +}
> > +
> >  static long xe_bo_shrink_purge(struct ttm_operation_ctx *ctx,
> >  			       struct ttm_buffer_object *bo,
> >  			       unsigned long *scanned)
> > diff --git a/drivers/gpu/drm/xe/xe_bo.h b/drivers/gpu/drm/xe/xe_bo.h
> > index 77b3a02e9b8e..65d4c638e8d8 100644
> > --- a/drivers/gpu/drm/xe/xe_bo.h
> > +++ b/drivers/gpu/drm/xe/xe_bo.h
> > @@ -623,6 +623,11 @@ struct xe_bo_shrink_flags {
> >  	u32 writeback : 1;
> >  };
> >  
> > +void xe_bo_bump_lru_locked(struct xe_bo *bo);
> > +
> > +bool xe_bo_shrink_backup_candidate(struct ttm_buffer_object *bo);
> > +bool xe_bo_shrink_too_young(struct ttm_buffer_object *bo, unsigned
> > long min_age);
> > +
> >  long xe_bo_shrink(struct ttm_operation_ctx *ctx, struct
> > ttm_buffer_object *bo,
> >  		  const struct xe_bo_shrink_flags flags,
> >  		  unsigned long *scanned);
> > diff --git a/drivers/gpu/drm/xe/xe_bo_types.h
> > b/drivers/gpu/drm/xe/xe_bo_types.h
> > index 0265fb314c77..f63b668c7134 100644
> > --- a/drivers/gpu/drm/xe/xe_bo_types.h
> > +++ b/drivers/gpu/drm/xe/xe_bo_types.h
> > @@ -49,6 +49,14 @@ struct xe_bo {
> >  	u32 flags;
> >  	/** @vm: VM this BO is attached to, for extobj this will be
> > NULL */
> >  	struct xe_vm *vm;
> > +	/**
> > +	 * @lru_stamp: jiffies at the last LRU bump, done on a move
> > into TT
> > +	 * and at exec time. Only describes the LRU position of a BO
> > outside
> > +	 * a bulk move; BOs in one are aged by &xe_vm.lru_stamp
> > instead.
> > +	 * Consulted by the shrinker's minimum age gate. Accessed
> > with
> > +	 * READ_ONCE()/WRITE_ONCE().
> > +	 */
> > +	unsigned long lru_stamp;
> >  	/**
> >  	 * @q: Queue this BO is attached to, mostly for LRC BO, NULL
> > otherwise.
> >  	 * Protected by the BO dma_resv: readers must hold it across
> > both the
> > diff --git a/drivers/gpu/drm/xe/xe_device_sysfs.c
> > b/drivers/gpu/drm/xe/xe_device_sysfs.c
> > index ff5d25dd867c..26b1f71ed522 100644
> > --- a/drivers/gpu/drm/xe/xe_device_sysfs.c
> > +++ b/drivers/gpu/drm/xe/xe_device_sysfs.c
> > @@ -38,6 +38,12 @@
> >   * reclaimed by the shrinker. Higher values reduce reclaim pressure
> > on
> >   * non-purgeable buffer objects. Valid values are 1 to 64. Defaults
> > to
> >   * CONFIG_DRM_XE_SHRINKER_NON_PURGEABLE_SEEKS.
> > + *
> > + * shrinker_min_age_ms - Report/change the minimum time, in ms,
> > since a BO was
> > + * last moved to TT or bumped on the LRU by an exec before the
> > shrinker may
> > + * back it up. BOs evicted to system memory are never protected.
> > Defaults to
> > + * CONFIG_DRM_XE_SHRINKER_MIN_AGE_MS, 0 disables. This applies at
> > every
> > + * reclaim priority, so a non-zero value can lead to OOM.
> >   */
> >  
> >  static ssize_t
> > @@ -331,8 +337,37 @@ shrinker_non_purgeable_seeks_store(struct device
> > *dev,
> >  
> >  static DEVICE_ATTR_RW(shrinker_non_purgeable_seeks);
> >  
> > +static ssize_t
> > +shrinker_min_age_ms_show(struct device *dev, struct device_attribute
> > *attr,
> > +			 char *buf)
> > +{
> > +	struct xe_device *xe = pdev_to_xe_device(to_pci_dev(dev));
> > +
> > +	return sysfs_emit(buf, "%u\n", xe_shrinker_min_age_ms(xe-
> > >mem.shrinker));
> > +}
> > +
> > +static ssize_t
> > +shrinker_min_age_ms_store(struct device *dev, struct
> > device_attribute *attr,
> > +			  const char *buff, size_t count)
> > +{
> > +	struct xe_device *xe = pdev_to_xe_device(to_pci_dev(dev));
> > +	unsigned int ms;
> > +	int ret;
> > +
> > +	ret = kstrtouint(buff, 0, &ms);
> > +	if (ret)
> > +		return ret;
> > +
> > +	xe_shrinker_set_min_age_ms(xe->mem.shrinker, ms);
> > +
> > +	return count;
> > +}
> > +
> > +static DEVICE_ATTR_RW(shrinker_min_age_ms);
> > +
> >  static struct attribute *shrinker_attrs[] = {
> >  	&dev_attr_shrinker_non_purgeable_seeks.attr,
> > +	&dev_attr_shrinker_min_age_ms.attr,
> >  	NULL
> >  };
> >  
> > diff --git a/drivers/gpu/drm/xe/xe_pagefault.c
> > b/drivers/gpu/drm/xe/xe_pagefault.c
> > index 6828f81df50e..895866d639f8 100644
> > --- a/drivers/gpu/drm/xe/xe_pagefault.c
> > +++ b/drivers/gpu/drm/xe/xe_pagefault.c
> > @@ -226,7 +226,7 @@ static int xe_pagefault_handle_vma(struct xe_gt
> > *gt, struct xe_vma *vma,
> >  
> >  			if (bo) {
> >  				scoped_guard(spinlock, &vm->xe-
> > >ttm.lru_lock)
> > -					ttm_bo_move_to_lru_tail(&bo-
> > >ttm);
> > +					xe_bo_bump_lru_locked(bo);
> >  			}
> >  		}
> >  		xe_validation_ctx_fini(&ctx);
> > diff --git a/drivers/gpu/drm/xe/xe_shrinker.c
> > b/drivers/gpu/drm/xe/xe_shrinker.c
> > index c86455ea08d1..64e5e684f591 100644
> > --- a/drivers/gpu/drm/xe/xe_shrinker.c
> > +++ b/drivers/gpu/drm/xe/xe_shrinker.c
> > @@ -39,6 +39,8 @@
> >   * @shrink: Pointer to the mm shrinker handling non-purgeable
> > (shrinkable) objects.
> >   * @purge_shrink: Pointer to the mm shrinker handling purgeable
> > objects.
> >   * @pm_worker: Worker to wake up the device if required.
> > + * @min_age_ms: Minimum BO age before backup, in ms. 0 disables.
> > Defaults to
> > + * CONFIG_DRM_XE_SHRINKER_MIN_AGE_MS, overridable through sysfs.
> >   */
> >  struct xe_shrinker {
> >  	struct xe_device *xe;
> > @@ -50,6 +52,7 @@ struct xe_shrinker {
> >  	struct shrinker *shrink;
> >  	struct shrinker *purge_shrink;
> >  	struct work_struct pm_worker;
> > +	unsigned int min_age_ms;
> >  };
> >  
> >  static struct xe_shrinker *to_xe_shrinker(struct shrinker *shrink)
> > @@ -104,6 +107,7 @@ static void xe_shrinker_runtime_pm_put(struct
> > xe_shrinker *shrinker, bool runtim
> >  static int __xe_shrinker_walk(struct xe_shrinker *shrinker,
> >  			      struct ttm_operation_ctx *ctx,
> >  			      const struct xe_bo_shrink_flags flags,
> > +			      unsigned long min_age,
> >  			      unsigned long to_scan, unsigned long
> > *scanned,
> >  			      unsigned long *freed)
> >  {
> > @@ -136,6 +140,16 @@ static int __xe_shrinker_walk(struct xe_shrinker
> > *shrinker,
> >  			if (flags.purge && ttm_bo->priority >
> > XE_BO_PRIORITY_LOW)
> >  				break;
> >  
> > +			/*
> > +			 * Each LRU priority is roughly age ordered,
> > so the
> > +			 * rest of this priority is most likely
> > young too.
> > +			 * Move on to the next priority.
> > +			 */
> > +			if (xe_bo_shrink_too_young(ttm_bo, min_age))
> > {
> > +				ttm_bo_lru_cursor_skip_priority(&cur
> > s);
> > +				continue;
> > +			}
> > +
> >  			if (!ttm_bo_shrink_suitable(ttm_bo, ctx))
> >  				continue;
> >  
> > @@ -171,6 +185,7 @@ static int __xe_shrinker_walk(struct xe_shrinker
> > *shrinker,
> >  static int xe_shrinker_walk(struct xe_shrinker *shrinker,
> >  			    struct ttm_operation_ctx *ctx,
> >  			    const struct xe_bo_shrink_flags flags,
> > +			    unsigned long min_age,
> >  			    unsigned long to_scan, unsigned long
> > *scanned,
> >  			    unsigned long *freed)
> >  {
> > @@ -180,26 +195,126 @@ static int xe_shrinker_walk(struct xe_shrinker
> > *shrinker,
> >  
> >  	swap(no_wait_gpu, ctx->no_wait_gpu);
> >  	save_flags.writeback = false;
> > -	ret = __xe_shrinker_walk(shrinker, ctx, save_flags, to_scan,
> > scanned,
> > -				 freed);
> > +	ret = __xe_shrinker_walk(shrinker, ctx, save_flags, min_age,
> > +				 to_scan, scanned, freed);
> >  	swap(no_wait_gpu, ctx->no_wait_gpu);
> >  	if (ret || *scanned >= to_scan)
> >  		return ret;
> >  
> >  	if (!ctx->no_wait_gpu) {
> > -		ret = __xe_shrinker_walk(shrinker, ctx, save_flags,
> > to_scan, scanned,
> > -					 freed);
> > +		ret = __xe_shrinker_walk(shrinker, ctx, save_flags,
> > min_age,
> > +					 to_scan, scanned, freed);
> >  		if (ret || *scanned >= to_scan)
> >  			return ret;
> >  	}
> >  
> >  	if (flags.writeback)
> > -		ret = __xe_shrinker_walk(shrinker, ctx, flags,
> > to_scan, scanned,
> > -					 freed);
> > +		ret = __xe_shrinker_walk(shrinker, ctx, flags,
> > min_age,
> > +					 to_scan, scanned, freed);
> >  
> >  	return ret;
> >  }
> >  
> > +/**
> > + * DOC: Shrinker minimum age
> > + *
> > + * The per-device shrinker_min_age_ms sysfs attribute, defaulting to
> > + * CONFIG_DRM_XE_SHRINKER_MIN_AGE_MS, protects recently used BOs
> > from backup.
> > + * A BO's age is taken from whatever owns its position on the LRU,
> > and is
> > + * stamped wherever that position is updated: the VM-wide stamp for
> > a BO
> > + * riding along with a VM's bulk move, bumped on exec in
> > xe_vm_bump_lru(),
> > + * its own stamp otherwise, bumped there as well and on a move into
> > TT in
> > + * xe_bo_move(). BOs in XE_PL_SYSTEM are never young: they are not
> > GPU
> > + * mapped, so not part of a working set, and will be stamped again
> > when
> > + * moved back to TT for use.
> > + *
> > + * The gate is applied per LRU priority, each of which is roughly
> > age
> > + * ordered. xe_shrinker_count() snoops the first shrinkable BO of
> > each
> > + * priority on each TT-backed LRU. Only if all of them are young
> > does it stop
> > + * reporting shrinkable pages; anything else, including reaching no
> > verdict
> > + * at all, lets it report. The backup walk in xe_shrinker_scan()
> > skips to the
> > + * next priority at the first young BO, as a backstop since the
> > count snoop
> > + * is only a sample. A hot priority thus doesn't shield colder BOs
> > in another
> > + * priority. BOs that aren't backed up anyway have no say either
> > way: those
> > + * in XE_PL_SYSTEM are never young, and purgeable and DONTNEED ones,
> > which
> > + * belong to the purgeable shrinker, are ignored by the snoop
> > entirely.
> > + *
> > + * The gate is hard and applies at every reclaim priority. A working
> > set kept
> > + * younger than the delay is never backed up, which can lead to OOM.
> > This is
> > + * intended for users who prefer that risk over thrashing a hot
> > working set,
> > + * hence it is disabled by default.
> > + */
> > +static unsigned long xe_shrinker_min_age(struct xe_shrinker
> > *shrinker)
> > +{
> > +	unsigned int ms = READ_ONCE(shrinker->min_age_ms);
> > +
> > +	return ms ? msecs_to_jiffies(ms) : 0;
> > +}
> > +
> > +/* Max shrink-unsuitable BOs to skip per LRU before giving up the
> > snoop. */
> > +#define XE_SHRINKER_SNOOP_MAX	16
> > +
> > +/*
> > + * Whether the first BO of every priority on every TT-backed LRU
> > that this
> > + * shrinker would back up is younger than @min_age. BOs are
> > trylocked through
> > + * the reserved LRU walker since their tt may be torn down while
> > still on the
> > + * LRU. Contended BOs are skipped by the walker, and may be all of
> > them. If no
> > + * verdict can be reached, because nothing was inspected or the
> > snoop budget
> > + * ran out, report not young and leave it to the scan side check.
> > + */
> > +static bool xe_shrinker_lru_young(struct xe_shrinker *shrinker,
> > +				  unsigned long min_age)
> > +{
> > +	struct ttm_operation_ctx ctx = {
> > +		.interruptible = false,
> > +		.no_wait_gpu = false,
> > +	};
> > +	struct xe_device *xe = shrinker->xe;
> > +	unsigned int mem_type;
> > +	bool young = false;
> > +
> > +	for (mem_type = XE_PL_SYSTEM; mem_type <= XE_PL_TT;
> > ++mem_type) {
> > +		struct ttm_resource_manager *man =
> > ttm_manager_type(&xe->ttm, mem_type);
> > +		struct ttm_bo_lru_cursor curs;
> > +		struct ttm_buffer_object *ttm_bo;
> > +		struct ttm_lru_walk_arg arg = {
> > +			.ctx = &ctx,
> > +			.trylock_only = true,
> > +		};
> > +		unsigned int budget = XE_SHRINKER_SNOOP_MAX;
> > +
> > +		if (!man || !man->use_tt)
> > +			continue;
> > +
> > +		ttm_bo_lru_for_each_reserved_guarded(&curs, man,
> > &arg, ttm_bo) {
> > +			/*
> > +			 * Nothing in the DONTNEED band is backed
> > up, and it
> > +			 * is accounted to the purgeable shrinker,
> > so it has
> > +			 * no say here. Skip the whole band in one
> > go.
> > +			 */
> > +			if (ttm_bo->priority <= XE_BO_PRIORITY_LOW)
> > {
> > +				ttm_bo_lru_cursor_skip_priority(&cur
> > s);
> > +				continue;
> > +			}
> > +
> > +			if (!ttm_bo_shrink_suitable(ttm_bo, &ctx) ||
> > +			    !xe_bo_shrink_backup_candidate(ttm_bo))
> > {
> > +				if (!--budget)
> > +					return false;
> > +				continue;
> > +			}
> > +
> > +			if (!xe_bo_shrink_too_young(ttm_bo,
> > min_age))
> > +				return false;
> > +
> > +			young = true;
> > +			ttm_bo_lru_cursor_skip_priority(&curs);
> > +		}
> > +	}
> > +
> > +	return young;
> > +}
> > +
> >  static unsigned long
> >  xe_shrinker_count_purgeable(struct shrinker *shrink, struct
> > shrink_control *sc)
> >  {
> > @@ -239,7 +354,7 @@ static unsigned long
> >  xe_shrinker_count(struct shrinker *shrink, struct shrink_control
> > *sc)
> >  {
> >  	struct xe_shrinker *shrinker = to_xe_shrinker(shrink);
> > -	unsigned long num_pages;
> > +	unsigned long num_pages, min_age;
> >  	bool can_backup = !!(sc->gfp_mask & __GFP_FS);
> >  
> >  	if (!can_backup)
> > @@ -263,6 +378,10 @@ xe_shrinker_count(struct shrinker *shrink,
> > struct shrink_control *sc)
> >  
> >  	write_unlock(&shrinker->lock);
> >  
> > +	min_age = xe_shrinker_min_age(shrinker);
> > +	if (num_pages && min_age && xe_shrinker_lru_young(shrinker,
> > min_age))
> > +		return SHRINK_EMPTY;
> > +
> >  	return num_pages ? num_pages : SHRINK_EMPTY;
> >  }
> >  
> > @@ -297,7 +416,8 @@ static unsigned long
> > xe_shrinker_scan_purgeable(struct shrinker *shrink,
> >  		.purge = true,
> >  	};
> >  
> > -	xe_shrinker_walk(shrinker, &ctx, shrink_flags, nr_to_scan,
> > &nr_scanned, &freed);
> > +	xe_shrinker_walk(shrinker, &ctx, shrink_flags, 0,
> > nr_to_scan, &nr_scanned,
> > +			 &freed);
> >  
> >  	sc->nr_scanned = nr_scanned;
> >  
> > @@ -322,7 +442,9 @@ static unsigned long xe_shrinker_scan(struct
> > shrinker *shrink, struct shrink_con
> >  	/* Might need runtime PM to access backed-up or CCS-
> > compressed data. */
> >  	runtime_pm = xe_shrinker_runtime_pm_get(shrinker);
> >  
> > -	xe_shrinker_walk(shrinker, &ctx, shrink_flags, nr_to_scan,
> > &nr_scanned, &freed);
> > +	xe_shrinker_walk(shrinker, &ctx, shrink_flags,
> > +			 xe_shrinker_min_age(shrinker), nr_to_scan,
> > &nr_scanned,
> > +			 &freed);
> >  
> >  	sc->nr_scanned = nr_scanned;
> >  	xe_shrinker_runtime_pm_put(shrinker, runtime_pm);
> > @@ -387,6 +509,29 @@ int xe_shrinker_set_non_purgeable_seeks(struct
> > xe_shrinker *shrinker,
> >  	return 0;
> >  }
> >  
> > +/**
> > + * xe_shrinker_min_age_ms() - Get the shrinker minimum BO age
> > + * @shrinker: Pointer to the struct xe_shrinker.
> > + *
> > + * Return: Minimum age, in ms, a BO must reach before it may be
> > backed up.
> > + */
> > +unsigned int xe_shrinker_min_age_ms(struct xe_shrinker *shrinker)
> > +{
> > +	return READ_ONCE(shrinker->min_age_ms);
> > +}
> > +
> > +/**
> > + * xe_shrinker_set_min_age_ms() - Set the shrinker minimum BO age
> > + * @shrinker: Pointer to the struct xe_shrinker.
> > + * @ms: Minimum age, in ms. 0 disables the gate.
> > + *
> > + * See DOC: Shrinker minimum age. A non-zero value can lead to OOM.
> > + */
> > +void xe_shrinker_set_min_age_ms(struct xe_shrinker *shrinker,
> > unsigned int ms)
> > +{
> > +	WRITE_ONCE(shrinker->min_age_ms, ms);
> > +}
> > +
> >  /**
> >   * xe_shrinker_create() - Create an xe per-device shrinker
> >   * @xe: Pointer to the xe device.
> > @@ -415,6 +560,7 @@ int xe_shrinker_create(struct xe_device *xe)
> >  
> >  	INIT_WORK(&shrinker->pm_worker, xe_shrinker_pm);
> >  	shrinker->xe = xe;
> > +	shrinker->min_age_ms = CONFIG_DRM_XE_SHRINKER_MIN_AGE_MS;
> >  	rwlock_init(&shrinker->lock);
> >  
> >  	shrinker->purge_shrink->count_objects =
> > xe_shrinker_count_purgeable;
> > diff --git a/drivers/gpu/drm/xe/xe_shrinker.h
> > b/drivers/gpu/drm/xe/xe_shrinker.h
> > index 4b3c73689619..586d7772bd58 100644
> > --- a/drivers/gpu/drm/xe/xe_shrinker.h
> > +++ b/drivers/gpu/drm/xe/xe_shrinker.h
> > @@ -18,4 +18,7 @@ int xe_shrinker_non_purgeable_seeks(struct
> > xe_shrinker *shrinker);
> >  int xe_shrinker_set_non_purgeable_seeks(struct xe_shrinker
> > *shrinker,
> >  					unsigned int seeks);
> >  
> > +unsigned int xe_shrinker_min_age_ms(struct xe_shrinker *shrinker);
> > +void xe_shrinker_set_min_age_ms(struct xe_shrinker *shrinker,
> > unsigned int ms);
> > +
> >  #endif
> > diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
> > index 86c3b4fd232f..583c55b7f97d 100644
> > --- a/drivers/gpu/drm/xe/xe_vm.c
> > +++ b/drivers/gpu/drm/xe/xe_vm.c
> > @@ -463,13 +463,15 @@ void xe_vm_bump_lru(struct xe_vm *vm, struct
> > drm_exec *exec)
> >  	spinlock_t *lru_lock = &vm->xe->ttm.lru_lock;
> >  	struct drm_gem_object *obj;
> >  
> > +	WRITE_ONCE(vm->lru_stamp, jiffies);
> > +
> >  	spin_lock(lru_lock);
> >  	ttm_lru_bulk_move_tail(&vm->lru_bulk_move);
> >  	drm_exec_for_each_locked_object(exec, obj) {
> >  		if (!drm_gpuvm_is_extobj(&vm->gpuvm, obj))
> >  			continue;
> >  
> > -		ttm_bo_move_to_lru_tail(&gem_to_xe_bo(obj)->ttm);
> > +		xe_bo_bump_lru_locked(gem_to_xe_bo(obj));
> >  		cond_resched_lock(lru_lock);
> >  	}
> >  	spin_unlock(lru_lock);
> > diff --git a/drivers/gpu/drm/xe/xe_vm_types.h
> > b/drivers/gpu/drm/xe/xe_vm_types.h
> > index fb9305124679..046505a285e9 100644
> > --- a/drivers/gpu/drm/xe/xe_vm_types.h
> > +++ b/drivers/gpu/drm/xe/xe_vm_types.h
> > @@ -267,6 +267,14 @@ struct xe_vm {
> >  	/** @lru_bulk_move: Bulk LRU move list for this VM's BOs */
> >  	struct ttm_lru_bulk_move lru_bulk_move;
> >  
> > +	/**
> > +	 * @lru_stamp: jiffies at the last xe_vm_bump_lru(), which
> > is where
> > +	 * @lru_bulk_move is moved to the LRU tail. Ages every BO in
> > that
> > +	 * bulk move for the shrinker's minimum age gate. Accessed
> > with
> > +	 * READ_ONCE()/WRITE_ONCE().
> > +	 */
> > +	unsigned long lru_stamp;
> 
> Isn't the bo resv / vm resv always held then this is accessed?
>

xe_bo_shrink_too_young is the lockless read side.

Matt
 
> Thanks,
> Thomas
> 
> 
> 
> > +
> >  	u64 size;
> >  
> >  	struct xe_pt *pt_root[XE_MAX_TILES_PER_DEVICE];