Re: [PATCH v2 3/4] mm/mglru: skip cold PUD subtrees during aging

[Baoquan He <baoquan.he@linux.dev>]

On 09/02/26 at 07:53am, Barry Song wrote:
> On Tue, Sep 1, 2026 at 2:38 PM Baoquan He <hebaoquan@kylinos.cn> wrote:
> >
> > The aging walks into every PUD and runs the PMD-level Bloom filter
> > on each PMD. Add a coarser PUD-level filter (pud_filters) one level
> > up:
> >  - walk_pmd_range() now reports whether it found any young leaf entries,
> >  - walk_pud_range() records that in the PUD filter, and
> >  - on subsequent generations, skips the whole 1GB subtree when the filter
> >    says it had none last generation.
> >
> > The double-buffered filter flips with each new iteration, so newly hot or
> > migrated-in pages are re-checked promptly rather than suppressed
> > indefinitely. force_scan walks bypass the PUD test, so manual aging and
> > newly added mm's always rescan and re-populate the filter.
> >
> > To keep hot regions marked, also report the covering PUD from the rmap
> > feedback path (lru_gen_look_around()), so regions whose hotness is only
> > observed by eviction will be re-scanned.
> >
> > Signed-off-by: Baoquan He <hebaoquan@kylinos.cn>
> > ---
> >  mm/vmscan.c | 52 +++++++++++++++++++++++++++++++++++++++++++++++-----
> >  1 file changed, 47 insertions(+), 5 deletions(-)
> [...]
> > @@ -4397,8 +4431,14 @@ bool lru_gen_look_around(struct page_vma_mapped_walk *pvmw, unsigned int nr)
> >         lazy_mmu_mode_disable();
> >
> >         /* feedback from rmap walkers to page table walkers */
> > -       if (mm_state && suitable_to_scan(i, young))
> > +       if (mm_state && suitable_to_scan(i, young)) {
> > +               /* the PUD entry covering the young PTEs scanned above */
> > +               pud_t *pud_p = pud_offset(p4d_offset(pgd_offset(vma->vm_mm, pvmw->address),
> > +                                                    pvmw->address), pvmw->address);
> > +
> >                 update_pmd_bloom_filter(mm_state, max_seq, pvmw->pmd);
> > +               update_pud_bloom_filter(mm_state, max_seq, pud_p);
> > +       }
> 
> 
> Hi Baoquan,
> 
> I am really not against this idea, and I believe it can benefit
> NUMA cases.
> 
> That said, I might be being overly cautious, but I'm a bit concerned
> that this re-walk of the page tables could slightly hurt machines that
> don't benefit from the PUD filter at all. For example, Android devices
> typically have relatively small amounts of memory, so a PUD covers a
> really large range of their address space, which is unlikely to be
> useful on an 8 GB Android device. Also, `lru_gen_look_around()` is a
> really hot path. On Android, we've observed that it can consume a
> significant amount of CPU.
> 
> Is there any possibility of implementing this in a low-cost way?

Hi Barry,

Thanks for the careful review, and both concerns are fair. Let me address
them separately.

(1) Aging walker overhead

The added cost in the aging walker is one PUD-level bloom test (2 bit
lookups) before walk_pmd_range(), plus a bool return from walk_pmd_range()
and a bloom update when young. This runs once per PUD per aging pass, not
per page fault, and is a tiny fraction of what the aging already does per
PUD. The test is also what lets the filter discover cold subtrees, so it
cannot simply be disabled.

(2) lru_gen_look_around() hot path

Agreed - this is the genuinely hot path (every fault), and on a
small-memory system where the PUD granularity is too coarse to be useful,
the pud_offset() + bloom update would be pure overhead. So I'll gate it
on a per-lruvec flag that tracks whether the aging walker is currently
skipping PUDs:

    /* iterate_mm_list(), at the start of a new generation's walk: */
    WRITE_ONCE(mm_state->pud_filter_used, false);

    /* walk_pud_range(), when a PUD is actually skipped: */
    WRITE_ONCE(mm_state->pud_filter_used, true);

    /* lru_gen_look_around(): */
    if (mm_state && suitable_to_scan(i, young) &&
        READ_ONCE(mm_state->pud_filter_used)) {
        pud_t *pud_p = pud_offset(...);
        update_pmd_bloom_filter(mm_state, max_seq, pvmw->pmd);
        update_pud_bloom_filter(mm_state, max_seq, pud_p);
    }

The flag is per-generation: cleared at the start of each aging generation
and set again when a PUD is skipped, so the feedback tracks whether the
filter is currently engaging. Before any PUD is skipped in a generation,
look_around() pays only a single flag loading.

I'll measure both on the 8GB test VM with a clean (fully unpatched)
baseline vs the patched kernel, and on a 3-node / 1.1TiB bare-metal
machine.

Thanks
Baoquan