Re: [RFC PATCH v2 1/1] drm/xe: keep VM-bound WC BOs resident during reclaim

[Matthew Brost <matthew.brost@intel.com>]

On Sat, Aug 08, 2026 at 05:34:04PM +0800, Neil Zhong wrote:
> On Fri, Jul 31, 2026 at 11:00:49PM -0700, Matthew Brost wrote:
> > This actually roughly what downstream customers are carrying :).
> >
> > It is basically these 3 patches [9] [10] [11] implemented directly in
> > the Xe shrinker code to avoid touching the MM or TTM...
> >
> > Alas I got nack'd by someone outside my subsystem on this approach.
> >
> > I think if you use the reference patches above to implement a heuristic
> > in Xe, or come up with a similar one, it will likely solve this issue.
> >
> > Since you're on 6.18, you may also be missing some Xe/TTM changes
> > related to this problem that have already been merged into drm-tip.
> > There are a couple of one-line fixes that should help somewhat, but the
> > heuristic is what I think will actually address the root cause.
> >
> > As heads up, I've started looking at this again and pushing to get
> > something upstream as this at least 5th time someone or org has flagged
> > this as a problem. Any data you can provide will help us push towards a
> > solution.
> 
> Hi Matt,
> 

Thanks for all the details here, very helpful.

> Thanks. I tested [9]-[13] on the same machine and the fragmentation
> heuristic does reduce the frequency of the problem. However, after these
> tests I would like to clarify my actual requirement, since my previous
> watermark-based proposal did not express it correctly.
> 
> For BOs that belong to a latency-critical visual processing working set,
> I think userspace should be able to mark them as non-shrinkable, and Xe
> should not back them up under any memory-reclaim condition while that mark
> is held. This would be a hard residency contract, not another reclaim
> priority or a fragmentation hint.

This is roughly what customers have indicated to us for laptop-type
products: anything displayed on the screen should avoid eviction or
shrinking at all costs. This series came out of that discussion:

https://patchwork.freedesktop.org/series/170454/

This customer, in particular, utilizes priority bands to express this
heuristic (e.g., the compositor is the highest priority, any
non-privileged UI-related content is normal priority, and everything
else is low priority). I'm not sure if stock distros do anything like
this.

Pinning would take this even further, allowing the compositor (or anyone
really) to effectively say, "Don't shrink this".

> 
> Why a hard contract is useful for visual workloads
> --------------------------------------------------
> 
> The reproducer is continuous 4K60 HDR playback. Every decoded frame is

Can you give me instructions on how to recreate this on our end and your
machine, memory details? I have a bunch of various reproducers which I
have been using for shrinker work and the more the better.

> imported through DMA-BUF and processed by libplacebo/OpenGL for HDR tone
> mapping before presentation. The active BO set contains decoded video
> surfaces, intermediate render targets and presentation-related surfaces.
> 

DMA-BUF in priority series moves to the prior band that is least likely
to be shrunk.

> At 60 Hz the complete frame interval is 16.667 ms. If glFlush() is blocked
> for longer than this, the application misses at least one presentation
> deadline. Repeated missed deadlines are perceived directly as dropped
> frames or visible stutter. A 100-500 ms reclaim/restore storm is an
> obvious freeze, even if the system eventually recovers and its average
> throughput looks normal.
> 
> Fence-idle is not equivalent to cold for this workload. A video surface
> can have no active fence in the small gap between two frames and still be
> part of the application's current visual working set. The next frame can
> need the same BO immediately.
> 

Ok, I think I see a potential problem here with priorities. If, for
example, a buffer is assigned a priority indicating that it is unlikely
to be evicted but has no active fences, it could be chosen for shrinking
before buffers whose priorities indicate "shrink this first" if those
buffers have active fences.

> The trace demonstrates exactly this case. In one sequence, kswapd
> completed backup of an 8,208-page BO and the rendering thread started
> restoring the exact same ttm_tt about 33 microseconds later. The kernel
> copied about 32 MiB to shmem, dropped the WC pages, then immediately had
> to allocate pages, copy the data back and reapply WC.
> 
> In the ten-minute default-watermark capture:
> 
>   successful ttm_tt_backup:                 4,853
>   ttm_tt_restore:                           4,810
>   minimum backup+restore copy traffic: 50,436.105 MiB
>   Flush > 16.667 ms:                         209
>   Flush > 100 ms:                             65
>   maximum trace-aligned Flush:           223.195 ms

Also a quick write up how you extracted these numbers from reproducer so
I can recreate on my end.

> 
> Of 4,791 restores matched to the same preceding backup, 4,279 happened
> within 100 ms and 4,753 within one second. kswapd0 performed 4,839 of the
> 4,853 backups, while the player's rendering thread performed most of the
> restores. Of the 209 Flush calls over one frame interval, 205 contained
> ttm_tt_restore() and all 209 contained set_pages_array_wc().
> 
> This is not useful recovery of cold memory. It is destruction and
> immediate reconstruction of the visible working set.
> 

Yes, indeed. We really don't want to destroy a working set unless the
core system genuinely needs memory and doing so is the only option.
Even then, there may be parts of the working set that simply cannot be
shrunk, as you are suggesting.

> What the heuristic test showed
> ------------------------------
> 
> I backported [9]-[13], extended the fragmentation check to direct reclaim,
> and tested a `high + min` watermark threshold. In a follow-up run with the
> same kernel, video and playback configuration, I set vm.min_free_kbytes to
> 50000. This lowered the Normal-zone `high + min` threshold from about
> 379.8 MiB to 255.1 MiB.
> 
> With the lower threshold, the fragmentation helper returned true more
> often and Xe working-set churn fell by about 90%:
> 
>   successful ttm_tt_backup:        4,853 -> 443
>   ttm_tt_restore:                  4,810 -> 435
>   Flush > 16.667 ms:                 209 -> 19
>   Flush > 100 ms:                     65 -> 3
> 
> This confirms that preventing working-set backup prevents the visual
> stalls. It does not make the individual restore path cheaper. When the
> heuristic still allowed a storm, the maximum Flush was 191.655 ms and
> contained 25 restores and 23 WC conversions.
> 
> I do not think tuning global watermarks is the right solution. More

Nor do I. [9]-[13] were Xe replacement for what is IMO a proper solution
in the core MM: https://patchwork.freedesktop.org/series/168651/ I'm
pushing on this patch a bit more with the core MM maintainers and have
another shrinker locally that is semi-related to this as well.

I guess I'd like numbers with the patch above + priority bands to see if
that is enough prevent working set shrinking of valuable buffers +
spikes in flush times.

> importantly, I no longer think that protection for explicitly identified
> visual BOs should depend on whether reclaim was caused by fragmentation
> or genuine low memory. Once userspace has declared a bounded set as
> presentation-critical, violating that residency guarantee produces an
> immediate and user-visible failure.
> 

To be clear - this would be an addition to fixes discussed above, right?

> Possible explicit marking mechanism
> -----------------------------------
> 
> Could Xe provide an opt-in, mlock-like mechanism for this purpose?
>

Yes, we could implement something like this, but we'd need buy-in across
the entire stack (i.e., from user space as well). I'll run this by the
internal team too to see if anyone can immediately poke holes in it,
because I don't currently see any obvious issues.
 
> One possible interface would be a new DRM_IOCTL_XE_MADVISE VMA attribute,
> for example DRM_XE_VMA_ATTR_RECLAIM_POLICY, with states similar to:
> 
>   DRM_XE_VMA_RECLAIM_DEFAULT
>   DRM_XE_VMA_RECLAIM_NO_SHRINK

This seems like a reasonable API.

> 
> NO_SHRINK would mean that the backing BO is excluded from the Xe shrinker
> while at least one protected VMA holds the attribute. Userspace would set
> it when a video/render surface enters the active visual pipeline and clear
> it after the surface leaves that working set. Unbind, VM destruction or
> file close would also release the holder automatically.
> 
> For a BO shared by multiple VMAs, Xe could maintain a BO-level
> no_shrink_count, similar to the holder accounting already used for
> purgeable state. The shrinker would skip a BO with a non-zero count. I
> would prefer a separate shrinker-specific count rather than exposing TTM
> pin_count, because pinning also affects placement and migration, which is
> broader than the requested guarantee.
> 
> The existing WILLNEED state does not provide this contract: it prevents
> purging of the contents, but the non-purge shrinker may still back up and
> unpopulate the BO. It also cannot simply be redefined because WILLNEED is
> the default state for all VMAs. SCANOUT is not sufficient either, since
> many HDR intermediate and imported video surfaces are not scanout BOs.
> 
> I understand that an unprivileged client must not be allowed to make an
> unbounded amount of memory unreclaimable. Like mlock, this could be
> controlled by an explicit per-file, per-client or cgroup byte limit, and

I think we could just hook into mlock accounting. There is an exported
function for exactly this purpose:

https://elixir.bootlin.com/linux/v7.1.7/source/mm/util.c#L549

You'd have to deal with multiple VMAs (from the same or different MMs in
a dma-buf) aliasing the same BO and ensure that accounting remains
consistent everywhere, but it shouldn't be too difficult. We already
have this problem WILLNEED/WONTNEED and solved it.

Ofc, this only works for system memory buffers so we'd some VRAM type
accounting too. iirc Thomas was working on cgroups for that part in a
slightly different context though.

> possibly by a privilege check. If the requested protected set exceeds the
> configured limit, the madvise should fail rather than silently accepting
> the mark and later violating it under pressure. The application or system
> service would then decide which visual surfaces to protect or release.
> 
> Within that bounded contract, however, I think NO_SHRINK should remain a
> hard guarantee even in genuine low-memory reclaim. Under pressure the
> kernel may reclaim unmarked BOs and other memory, reject additional
> NO_SHRINK requests, or require the application/service to release part of
> its protected set. Backing up an already accepted presentation-critical
> BO and stalling a frame by hundreds of milliseconds defeats the purpose
> of the interface.
> 
> For comparison, my original workaround approximated such a hard contract
> by excluding VM-bound WC BOs from non-purge shrinking. In a 21-minute
> capture, no XE_EXEC, VM_BIND or DMA-BUF ioctl exceeded the 16.7 ms frame
> interval; their maxima were 348 us, 136 us and 20 us. The call rate of
> set_pages_array_wc fell to 0.179/s. That automatic VM-bound WC rule is too
> broad, but an explicit and bounded userspace mark could provide the same
> latency guarantee only for the BOs that actually need it.
> 
> Would an explicit, bounded NO_SHRINK/latency-critical VMA attribute be a
> reasonable Xe UAPI direction? If so, I can prototype the BO holder
> accounting and shrinker exclusion, then modify the video/Mesa path to mark
> only the active visual working set and collect another strict A/B trace.

No issue if you want to prototype this, but as mentioned above, this
would require buy-in from user space (which is not under my control) and
at least one other person on the KMD team (most likely Thomas). So I
can't guarantee that it won't be rejected by someone. 

Matt

> [9] https://patchwork.freedesktop.org/patch/720030/?series=165329&rev=1
> [10] https://patchwork.freedesktop.org/patch/720036/?series=165329&rev=1
> [11] https://patchwork.freedesktop.org/patch/720031/?series=165329&rev=1
> [12] https://patchwork.freedesktop.org/patch/732698/?series=168466&rev=1
> [13] https://patchwork.freedesktop.org/patch/732420/?series=168389&rev=1
> 
> Thanks,
> Neil