Re: [LSF/MM/BPF BoF Session] Numa-Aware Placement for Page Cache Pages

[Gregory Price <gourry@gourry.net>]

On Sun, May 03, 2026 at 09:48:01PM +0530, Ritesh Harjani wrote:
> Gregory Price <gourry@gourry.net> writes:
> 
> MADV_POPULATE_READ_NOIO should ensure that only the cached folios
> belonging to that file are mapped into the process address space w/o
> doing any extra disk I/Os. The subsequent mbind call with MPOL_MF_MOVE,
> will then ensure that all the existing mapped folios are migrated into the
> chosen numa node. And also that any new pages which gets faulted in will
> get allocated onto the chose numa node because of MPOL_BIND policy. 
> 

This all gets rather racy with buffered I/O, I'm not sure we can make
this work the way either one of us want.  I need to chew on this.

> I believe there might be existing applications which might be facing
> this problem today. This can happen, for instance, when there is a
> workload which can run multiple times and may run across different NUMA
> nodes. Our internal test team once reported a similar performance
> regression with llama-bench on subsequent runs when running it across
> different NUMA nodes. The reason this happened was that the existing
> page cache folios of model weight file (from the previous run on a
> separate NUMA node) were not getting migrated (because they were not
> calling MADV_POPULATE_READ since it can cause a read of a large model
> weight file into the page cache all at once).
> 

There's a long standing issue of unmapped page cache files getting
trapped on lower-tier memory, i think that's an orthoganol issue to the
discussion here.

IIRC DAMON can migrate them, i think, and vmscan.c will happily demote
these folios.  So at a minimum we know they're "migratable".

> With that in mind, do we think having something like
> MADV_POPULATE_READ_NOIO make sense to address such problems? Do we have
> any other usecases of this too?
> Or do we see any problems with this, due to which it never existed?
> 
> (Note that I haven't yet given a thought for how it should behave for
> anon memory).
>

I'm not sure why this would apply to anon?  Unless the issue is
specifically anon MAP_SHARED.

~Gregory