Re: [LSF/MM/BPF TOPIC] Towards Unified and Extensible Memory Reclaim (reclaim_ext)

[Johannes Weiner <hannes@cmpxchg.org>]

On Thu, Mar 26, 2026 at 03:35:28PM +0000, Lorenzo Stoakes (Oracle) wrote:
> On Thu, Mar 26, 2026 at 10:24:28AM -0500, Gregory Price wrote:
> > ... snip snip snip ...
> >
> > > >
> > > > How Do We Get There
> > > > -------------------
> > > >
> > > > Do we merge the two mechanisms feature by feature, or do we prioritize
> > > > moving MGLRU to the pluggable model then follow with LRU once we are
> > > > happy with the result?
> > >
> > > Absolutely by a distance the first is preferable. The pluggability is
> > > controversial here and needs careful consideration.
> > >
> >
> > Pluggability asside - I do not think merging these two things "feature
> > by feature" is actually feasible (I would be delighted to be wrong).
> >
> > Many MGLRU "features" solve problems that MGLRU invents for itself.
> >
> > Take MGLRU's PID controller - its entire purpose is to try to smooth out
> > refault rates and "learn" from prior mistakes - but it's fundamentally
> > tied to MGLRU's aging system, and the aging systems differ greatly.
> >
> >   - LRU:   actual lists - active/inactive - that maintain ordering
> >   - MGLRU: "generations", "inter-generation tiers", aging-in-place
> >
> > "Merging" this is essentially inventing something completely new - or
> > more reasonably just migrating everyone to MGLRU.
> >
> > In terms of managing risk, it seems far more reasonable to either split
> > MGLRU off into its own file and formalize the interface (ops), or simply
> > rip it out and let each individual feature fight its way back in.
> 
> But _surely_ (and Shakeel can come back on this I guess) there are things that
> are commonalities.

There are some commonalities, but MGLRU was almost maximalist in its
approach to finding parallel solutions and reinventing various wheels
with little commentary, explanations or isolated testing.

For example, MGLRU took a totally different, ad-hoc approach to
dealing with dirty and writeback pages. It's been converging on the
LRU mechanism. This process has been stretching out for years, with
users eventually running into all the same problems that shaped the
LRU implementation to begin with. Yes, you need to wake flushers from
reclaim. Yes, you will OOM if you don't throttle on writeback.

There are many other divergences like this that complicate the picture:
- Cgroup tree iteration, per-zone lists to implement node reclaim.
- Divergent anon/file balancing policies.
- A notably different approach to scan resistance.

Many of these were not part of the main pitch at the time, but they’ve
created sizable technical debt that we’re now forced to reconcile.

I think MGLRU's NIH-attitude towards the problem space set it up for
running into past lessons again and learning the hard way, just like
with writeback.

The good thing is that there are some integration efforts now, even if
they don't come from the people that promised them. And some of them
do exactly the targeted, rigorous tests on a per-component basis that
is needed to sort it out (and was asked for back then).

But there are many workloads, many hardware configurations, and many
cornercases to cover, so this will take time. The end result doesn't
just need to be fast for some workloads, it also needs to be
universal, robust, easy to reason about and predictable.

Based on the current differences and how unification has been going so
far, I think it's premature to claim that we're close to deleting one.

And the current code structure makes it difficult to whittle down the
differences methodically.

IMO modularization is the best path forward. Giving people the ability
to experiment with a la carte combinations of features would make it
much easier to actually production test and prove individual ideas.

A nice side effect of this is that entirely new ideas would also be
easier to try out.

I think a good start would be to keep the common bits - "library" code
like shrink_folio_list() list and shared facilities like kswapd - in
vmscan.c. Move LRU and MGLRU specifics to their own files.

Then as much as possible extract and generalize functionality into the
common code so it can plug into both. For example, collecting accessed
bits from page tables instead of rmap chains should really not have to
be specific to one. Nor how the cgroup tree is iterated.

It might be possible to make N lists a natural extension of 2 lists,
so that the tracking datastructures themselves can be shared. With
minimal parameterization from the policy engines.

If we can get to a place where the only difference is how reference
data is interpreted and causes the lists to be sorted - you know, the
actual replacement policy - that is a much more manageable gap to
evaluate and argue about. Or swap out to try entirely new ones.