Re: [LSF/MM/BPF TOPIC] Towards Unified and Extensible Memory Reclaim (reclaim_ext)

[Shakeel Butt <shakeel.butt@linux.dev>]

On Thu, Mar 26, 2026 at 11:43:46AM +0000, Lorenzo Stoakes (Oracle) wrote:
> On Wed, Mar 25, 2026 at 02:06:37PM -0700, Shakeel Butt wrote:
[...]
> >
> > The Fix: One Reclaim, Pluggable and Extensible
> > -----------------------------------------------
> >
> > We need one reclaim system, not two. One code path that everyone
> > maintains, everyone tests, and everyone benefits from. But it needs to
> > be pluggable as there will always be cases where someone wants some
> > customization for their specialized workload or wants to explore some
> > new techniques/ideas, and we do not want to get into the current mess
> > again.
> 
> OK so I was with you up until the pluggable bit :) it's like you're
> combining two things here, obviously - unification and pluggability.
> 
> I think we should consider both separately.

Yes, I should be more explicit that these are two different steps. First
unification and then provide a framework for extensibility.

[...]

> 
> > New ideas get implemented as new policies, not as 3,000-line forks. Good
> > mechanisms from MGLRU (page table scanning, Bloom filters, lookaround)
> > become shared infrastructure available to any policy. And if someone
> > comes up with a better eviction algorithm tomorrow, they plug it in
> > without touching the core.
> >
> > Making reclaim pluggable implies we define it as a set of function
> > methods (let's call them reclaim_ops) hooking into a stable codebase we
> > rarely modify. We then have two big questions to answer: how do these
> > reclaim ops look, and how do we move the existing code to the new model?
> 
> Hmm, I'm not so sure about that. But it depends really on who has access to
> these operations.
> 
> The issue with operations in general is that they eliminate the possibility
> of the general code being able to make assumptions about what's happening.
> 
> For instance, the .mmap f_op callback meant that we had to account for any
> possible thing being done by a driver. You couldn't make assumptions about
> vma state, page table state, etc. and of course things happened that we
> didn't anticipate, leading to bugs.
> 
> So I guess it's less 'no ops' more so 'what do we actually expose to the
> ops', 'what assumptions do we bake in about how the ops are used' and very
> importantly - 'who gets to populate them'.
> 
> If they're _exclusively_ mm-internal then that's fine.
> 
> Reclaim is a _very_ _very_ sensitive part of mm. At the point it's being
> activated you may be under extreme memory pressure, so a hook even
> allocating at all may either fail or enter infinite loops.
> 
> We are also very sensitive on things like rmap locks and also, of course, -
> timing.
> 
> It's not just a perf concern, if we are too slow, we might end up thrashing
> when we could otherwise not have.
> 
> Also there ends up being a question of how much now-internal functionality
> we end up exposing to users.
> 
> So we really need a good definition of who we intend should use this stuff,
> and how any such interface should be designed.
> 
> I mean, if sufficiently abstracted, and with very carefully restricted
> constrainst perhaps we could work around a lot of this but we have to tread
> _very_ carefully here.

Good points and I think we are still at the early stage of defining what
operations these would be. One of the complain during MGLRU upstream effort was
that the traditional LRU is too rigid and is very hard to experiment new ideas.
I want to eliminate such future complains. If you want to experiment some new
algorithm or new heuristic, experiment using the new framework.

I think once we start unifying the reclaim mechanisms, these operations will
start becoming more clear.