Re: [PATCH v2 2/2] drm/xe: Update shrinker batch size based on average BO size

["Thomas Hellström" <thomas.hellstrom@linux.intel.com>]

On Tue, 2026-08-18 at 14:32 -0700, Matthew Brost wrote:
> On Tue, Aug 18, 2026 at 02:10:35PM +0200, Thomas Hellström wrote:
> > On Fri, 2026-08-14 at 16:24 -0700, Matthew Brost wrote:
> > > On Fri, Aug 14, 2026 at 04:37:37PM +0200, Thomas Hellström wrote:
> > > > Update our preferred vmscan batch size on each count pass to
> > > > avoid
> > > > invoking scan_objects for requests too small to free even a
> > > > single
> > > > average-sized GEM object. Our rough estimate for an effective
> > > > batch
> > > > is twice the average number of pages per populated ttm_tt
> > > > across
> > > > all
> > > > shrinkable and purgeable objects. The factor of two provides
> > > > headroom
> > > > so that most scan invocations can free at least one GEM object
> > > > despite
> > > > variability in object sizes.
> > > > 
> > > > The batch value is updated as an exponential moving average,
> > > > (old_batch + avg) / 2, to smooth out sudden changes in the
> > > > object population. It is floored at 128 pages, the kernel
> > > > default
> > > > SHRINK_BATCH, to ensure the shrinker remains responsive when
> > > > there
> > > > are very few objects.
> > > > 
> > > > The populated_tts counter introduced in the previous commit
> > > > provides
> > > > the object count needed for the average. We inherit the same
> > > > justification as the analogous mechanism in i915: shrinking a
> > > > GEM
> > > > object has non-trivial locking overhead, so firing the shrinker
> > > > for
> > > > requests smaller than a single object is wasteful.
> > > > 
> > > > v2:
> > > > - Fix the average object size estimate to account for the full
> > > >   shrinkable and purgeable population.
> > > > 
> > > > Assisted-by: GitHub_Copilot:claude-sonnet-4.6
> > > > Assisted-by: GitHub_Copilot:claude-sonnet-5
> > > 
> > > This is probably the right direction given what we currently have
> > > in
> > > terms of shrinker control, but the core heuristic is still a
> > > pretty
> > > poor
> > > one. My understanding is that it combines batch and seek values
> > > using
> > > some odd math to determine whether a scan is worthwhile at a
> > > given
> > > priority level. We probably want to avoid shrinking at the
> > > initial
> > > scan
> > > priorities, and I believe this change accomplishes that.
> > 
> > Yes, but I think that's a side-effect not to be fully relied upon.
> > 
> > The meaning of this value IMO is to tell the core how many objects
> > to
> > expect for a scan request, so that the core can hold off shrinking
> > until that many objects is actually this shrinker's fair share of
> > its
> > available objects. So the side effect would be that this shrinker's
> > fair share of shrinking may not trigger a scan request if shrinking
> > is
> > triggered by compacting?
> > 
> 
> I think you mean higher order allocations, not compaction. Reclaim is
> the input to compaction - see compaction_ready, compact_gap usage in
> vmscan.c

Yes, I meant shrinking triggered by higher order allocations, but used
compaction as a term for shrinking any order page in order to be able
to coalesce memory into higher order. That might not be the correct
terminology, though.

> 
> So I think a side affect could be higher order allocation never enter
> our shrinker if compaction_ready flips to true before our batch size
> /
> seek values are asked for (total_scan math in do_shrink_slab).

Yes, that's a possible side-effect.

> 
> > > 
> > > That said, I think we really want two shrinkers instead: one with
> > > the
> > > default settings (or perhaps even a reduced seek value) for
> > > purgeable
> > > BOs, and another for BOs that we legitimately need to back up.
> > > The
> > > purgeable one should be favored to run eariler, likewise the TTM
> > > pool
> > > shrinker should be favored run before our shrinker too.
> > 
> > I don't think we can or should use the batch size to decide which
> > shrinker should be prioritized. IIRC one of the comments to
> > previous
> 
> It probably isn't the right approach, but my concern is that our
> shrinker
> won't run at higher orders when there are cheap reclaimable pages
> (i.e.,
> we have purged BOs that can immediately make higher-order pages
> available
> or allow compaction to do its job of forming higher-order pages). I
> have
> already seen shrinker backoff being too aggressive when
> compaction_ready()
> returns true, resulting in virtually zero THP availability because
> shrinkers hold onto enough non-movable pages scattered throughout
> memory
> to prevent successful compaction (I have a local core MM patch that
> fixes
> this issue).
> 
> Purgable and non-purgable pages have fundamentally different
> shrinking
> costs, and that distinction needs to be expressed somehow. The
> opportunistic compaction (wrongly named) shrinker series attempts to
> capture this.

But since the core attempts to be fair poking shrinkers, and that's not
really what we want (we want it to shrink purgeable stuff first, and
avoid shrinking non-purgeable stuff). Then we could return the default
batch size when we have purgeable bos to shrink and stop the scan after
those. If nothing purgeable, we return the average bo size as batch
size. At least for now until we have core MM functionality in place for
that.

/Thomas

> 
> > series what that shrinkers should appear similar to the core,
> > unless
> > some form of differentiation is implemented in the core. If we were
> > to
> > add two shrinkers it would mean that purgeable objects would get
> > its
> > fair share of shrinking and so would also active / live objects.
> > With
> > the current design we prioritize internally to make sure we target
> > purgeable objects first.
> > 
> 
> One shrinker could scan only purgable pages, while the other could
> scan
> both purgable pages and those in the working set. But maybe that
> isn't
> the right answer either.
> 
> So I'm torn on this. I think this series will help with the higher-
> order
> eviction feedback loop but, at the same time, may make higher-order
> availability worse in certain cases. I think the proper solution for
> both issues is core shrinker work, but it has been hard to gain any
> traction there.
> 
> Matt
> 
> > > 
> > > Also we really should look at getting priority based shrinking in
> > > too, I
> > > have follow up there too which disconnects purgable / not in
> > > working
> > > set
> > > from shared VM dma-resv also, further prioritizing though
> > > shrinks.
> > > 
> > > > Signed-off-by: Thomas Hellström
> > > > <thomas.hellstrom@linux.intel.com>
> > > > ---
> > > >  drivers/gpu/drm/xe/xe_shrinker.c | 26
> > > > ++++++++++++++++++++++++++
> > > >  1 file changed, 26 insertions(+)
> > > > 
> > > > diff --git a/drivers/gpu/drm/xe/xe_shrinker.c
> > > > b/drivers/gpu/drm/xe/xe_shrinker.c
> > > > index cded230f5459..284fce207705 100644
> > > > --- a/drivers/gpu/drm/xe/xe_shrinker.c
> > > > +++ b/drivers/gpu/drm/xe/xe_shrinker.c
> > > > @@ -146,6 +146,8 @@ xe_shrinker_count(struct shrinker *shrink,
> > > > struct shrink_control *sc)
> > > >  {
> > > >  	struct xe_shrinker *shrinker = to_xe_shrinker(shrink);
> > > >  	unsigned long num_pages;
> > > > +	unsigned long total_pages;
> > > > +	unsigned long populated_tts;
> > > >  	bool can_backup = !!(sc->gfp_mask & __GFP_FS);
> > > >  
> > > >  	num_pages = ttm_backup_bytes_avail() >> PAGE_SHIFT;
> > > > @@ -157,8 +159,32 @@ xe_shrinker_count(struct shrinker *shrink,
> > > > struct shrink_control *sc)
> > > >  		num_pages = 0;
> > > >  
> > > >  	num_pages += shrinker->purgeable_pages;
> > > > +	total_pages = shrinker->shrinkable_pages + shrinker-
> > > > > purgeable_pages;
> > > > +	populated_tts = shrinker->populated_tts;
> > > >  	read_unlock(&shrinker->lock);
> > > >  
> > > > +	/*
> > > > +	 * Update our preferred vmscan batch size for the next
> > > > pass.
> > > > +	 * Our rough guess for an effective batch size is
> > > > twice
> > > > the average
> > > > +	 * number of pages per GEM object. That is, we don't
> > > > want
> > > > the
> > > > +	 * shrinker to fire until the request is large enough
> > > > to
> > > > justify
> > > > +	 * the overhead of freeing at least one GEM object.
> > > > +	 *
> > > > +	 * Base the average on the full shrinkable + purgeable
> > > > population
> > > > +	 * (total_pages), not on num_pages, which is reduced
> > > > to
> > > > just the
> > > > +	 * purgeable pages whenever the gfp mask disallows
> > > > backup
> > > > (can_backup
> > > > +	 * false). Otherwise the estimate would systematically
> > > > undershoot in
> > > > +	 * exactly the GFP_NOFS / GFP_NOIO reclaim paths where
> > > > avoiding
> > > > +	 * excessive scan_objects() calls matters most.
> > > > +	 */
> > > > +	if (populated_tts) {
> > > > +		unsigned long avg = 2 * total_pages /
> > > > populated_tts;
> > > > +
> > > > +		shrinker->shrink->batch =
> > > > +			max((shrinker->shrink->batch + avg) >>
> > > > 1,
> > > > +			    128UL /* default SHRINK_BATCH */);
> > > 
> > > I think 128UL should be at least whatever TTM pool batch is?
> > 
> > Again, we should not use this to attempt to prioritize between
> > shrinkers. Just to ensure that we give a fair estimate of the
> > actual
> > batch size.
> > 
> > Thanks,
> > Thomas
> > 
> > 
> > > 
> > > Matt
> > > 
> > > > +	}
> > > > +
> > > >  	return num_pages ? num_pages : SHRINK_EMPTY;
> > > >  }
> > > >  
> > > > -- 
> > > > 2.55.0
> > > >