Re: [PATCH v1 2/2] mm/memory: add anonymous mTHP folios to deferred split list

[Johannes Weiner <hannes@cmpxchg.org>]

On Thu, Jul 30, 2026 at 03:46:25PM +0200, David Hildenbrand (Arm) wrote:
> On 7/29/26 17:04, Johannes Weiner wrote:
> > (2) With a mix of basepages and THPs, there could indeed be a lot of
> >     basepages ahead of underused THPs. That means swapping before
> >     getting to space that is much cheaper to reclaim.
> > 
> >     The current setup isn't perfect in that regard, as the shrinker
> >     runs simultaneously as the LRU. But it's making guaranteed forward
> >     progress through the THPs, even as the first LRU pages are scanned.
> 
> The shrinker would obviously remain and scan the list for candidates.

That's slightly different from what I had pictured. But I don't think
it changes my arguments much.

> We might want to remember how man / if any such entries we have on
> the list.

Right. The question is how does the shrinker actually find them:

> > (3) The anon LRU has folio lifetime, but the splitqueue is one-shot:
> >     we scan each THP once, and then it's either split and dropped, or
> >     found full and dropped. That THP never needs to be revisited. The
> >     queue actually empties as the workload establishes itself.
> > 
> >     The anon LRU ~ splitqueue argument is only true around startup.
> > 
> >     If we used the anon LRU, we'd need per-page state to avoid repeat
> >     underused checks. And we need external state to not scan the anon
> >     LRU at all if there are no new THPs (and no swap). And if that's
> >     just a counter for "new, not yet scanned THPs", a single fault
> >     will cause you to walk the entire anon LRU before you get to it.
> 
> Remembering "not yet scanned" through a pageflag (for large folios) is indeed
> very easy.
> 
> I don't quite understand the "a single fault", can you elaborate?

Let's say you have a 1TB host with 800G anon populated.

The oldest folios on the list might be THP. The newest ones might
be. You could have a mix of basepages and THPs. The ordering
constantly changes as the folios are aged, rotated, reclaimed.

How can it find a handful of unscanned THPs in an ocean of folios?
Even if you mark the folio state, that's hundreds of millions of
entries whose state you have to check in the worst case?

The lru lock is one of the most congested MM locks on large
machines. *Maybe* you can do it locklessly.

Maybe you can add thresholds where you don't scan if there aren't
"that many" new THPs just yet. That means magic numbers and reduced
predictability.

Maybe you can be clever and scan from the head of the inactive list
where (most) new folios start. You still need to skip over basepages
that faulted after. Skip over the referenced pages that have been
rotated around concurrently. That could mitigate some common cases,
but not the worst case.

The search pool stays enormous for the entire runtime of the
workload. It never gets better, never converges. It continues to
include every other irrelevant anon page, and every THP that you've
previously scanned already. A single new THP fault and the search
problem starts over.

I just don't see how that's algorithmically sound.

> > So I think reusing the anon LRU is flawed. It's fundamentally
> > different needles in fundamentally different haystacks.
> > 
> > If we can agree on that, then the lock contention problem has a
> > different scope as well: it's a simple optimization issue, not a
> > fundamental data structure arrangement issue.
> 
> I don't agree yet :) But maybe I am missing something important.
> 
> Note that the "simple optimization issue" is not so simple once you
> realize what kind of a pain the batched LRU already creates us when it
> comes to predicting the number of expected folio references.
> 
> It's a pain I don't want to extend to other areas.

Since we already need to do it for the LRU pages anyway, isn't it a "+
in_deferred_cache(folio) extension to existing refcount checks?

I don't want to sound dismissive at all. It's a problem. However,

- it seems way more tractable than the shared list,
- nobody has produced hard data to show that either is justified.

> > If I understand you correctly, the concern is that people will enable
> > all manner of mTHP orders, and 99% of the anon faults, including all
> > the order-3, order-4 pagelets, will go through the list_lru lock on
> > fault, with no batching.
> 
> Yes. See Barry's LRU cache change I linked as reply to Usama who is looking for
> example at a system that mostly just uses order-2 anon folios.

I took a look, but I just see a microbenchmark. That doesn't seem
enough to make a proper cost-benefit analysis on the complexity that's
being proposed - whether that's a shared list design, or a splitqueue
cache.

(As opposed to the patch of hooking into the existing LRU cache infra,
which is kind of a no-brainer.)

So I still think somebody who actually cares needs to show that it's a
practical problem, propose a solution and show hard numbers to justify
the engineering tradeoff.