Re: [PATCH v2 3/4] mm/mglru: skip cold PUD subtrees during aging

[Baoquan He <baoquan.he@linux.dev>]

On 09/02/26 at 07:53am, Barry Song wrote:
> On Tue, Sep 1, 2026 at 2:38 PM Baoquan He <hebaoquan@kylinos.cn> wrote:
> >
> > The aging walks into every PUD and runs the PMD-level Bloom filter
> > on each PMD. Add a coarser PUD-level filter (pud_filters) one level
> > up:
> >  - walk_pmd_range() now reports whether it found any young leaf entries,
> >  - walk_pud_range() records that in the PUD filter, and
> >  - on subsequent generations, skips the whole 1GB subtree when the filter
> >    says it had none last generation.
> >
> > The double-buffered filter flips with each new iteration, so newly hot or
> > migrated-in pages are re-checked promptly rather than suppressed
> > indefinitely. force_scan walks bypass the PUD test, so manual aging and
> > newly added mm's always rescan and re-populate the filter.
> >
> > To keep hot regions marked, also report the covering PUD from the rmap
> > feedback path (lru_gen_look_around()), so regions whose hotness is only
> > observed by eviction will be re-scanned.
> >
> > Signed-off-by: Baoquan He <hebaoquan@kylinos.cn>
> > ---
> >  mm/vmscan.c | 52 +++++++++++++++++++++++++++++++++++++++++++++++-----
> >  1 file changed, 47 insertions(+), 5 deletions(-)
> [...]
> > @@ -4397,8 +4431,14 @@ bool lru_gen_look_around(struct page_vma_mapped_walk *pvmw, unsigned int nr)
> >         lazy_mmu_mode_disable();
> >
> >         /* feedback from rmap walkers to page table walkers */
> > -       if (mm_state && suitable_to_scan(i, young))
> > +       if (mm_state && suitable_to_scan(i, young)) {
> > +               /* the PUD entry covering the young PTEs scanned above */
> > +               pud_t *pud_p = pud_offset(p4d_offset(pgd_offset(vma->vm_mm, pvmw->address),
> > +                                                    pvmw->address), pvmw->address);
> > +
> >                 update_pmd_bloom_filter(mm_state, max_seq, pvmw->pmd);
> > +               update_pud_bloom_filter(mm_state, max_seq, pud_p);
> > +       }
> 
> 
> Hi Baoquan,
> 
> I am really not against this idea, and I believe it can benefit
> NUMA cases.
> 
> That said, I might be being overly cautious, but I'm a bit concerned
> that this re-walk of the page tables could slightly hurt machines that
> don't benefit from the PUD filter at all. For example, Android devices
> typically have relatively small amounts of memory, so a PUD covers a
> really large range of their address space, which is unlikely to be
> useful on an 8 GB Android device. Also, `lru_gen_look_around()` is a
> really hot path. On Android, we've observed that it can consume a
> significant amount of CPU.
> 
> Is there any possibility of implementing this in a low-cost way?

I made a draft patch, could you help check and test if it's performing
better on Andriod device?

From 5b92a88925a7fe1733ca54595ece7be1bd122909 Mon Sep 17 00:00:00 2001
From: Baoquan He <hebaoquan@kylinos.cn>
Date: Wed, 2 Sep 2026 11:07:10 +0800
Subject: [PATCH] mm/mglru: gate the PUD rmap feedback on filter engagement
Content-type: text/plain

lru_gen_look_around() updates the PUD-level Bloom filter so that regions
whose hotness is only observed by eviction stay marked and are re-scanned
by the aging walker. But on systems where the PUD filter never skips
(e.g. small-memory devices where a PUD covers too much address space to
be useful), this is pure overhead on the page-fault hot path.

Gate the PUD feedback on a per-lruvec flag that is set when the aging
walker actually skips a PUD and cleared at the start of each generation.
look_around() then pays only a single flag load while no PUD is being
skipped, and the feedback engages exactly when the filter is active.
PUDs are only marked from the eviction side when the aging is really
skipping them. So hot regions stay protected where the filter is active,
and machines that never benefit get zero extra cost.

Signed-off-by: Baoquan He <hebaoquan@kylinos.cn>
---
 include/linux/mmzone.h | 2 ++
 mm/vmscan.c            | 9 +++++++--
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 276d0ca8d1c1..803431dfac80 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -617,6 +617,8 @@ struct lru_gen_mm_state {
 	unsigned long *pmd_filters[NR_BLOOM_FILTERS];
 	/* PUD-level Bloom filters flip after each iteration */
 	unsigned long *pud_filters[NR_BLOOM_FILTERS];
+	/* whether the aging has skipped a PUD this generation */
+	bool pud_filter_used;
 	/* the mm stats for debugging */
 	unsigned long stats[NR_HIST_GENS][NR_MM_STATS];
 };
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 035bd9ea49aa..22851b02264f 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -3143,8 +3143,11 @@ static bool iterate_mm_list(struct lru_gen_mm_walk *walk, struct mm_struct **ite
 	if (!mm_state->head)
 		mm_state->head = &mm_list->fifo;
 
-	if (mm_state->head == &mm_list->fifo)
+	if (mm_state->head == &mm_list->fifo) {
+		/* new generation: reset the PUD-filter engagement flag */
+		WRITE_ONCE(mm_state->pud_filter_used, false);
 		first = true;
+	}
 
 	do {
 		mm_state->head = mm_state->head->next;
@@ -3858,6 +3861,7 @@ static int walk_pud_range(p4d_t *p4d, unsigned long start, unsigned long end,
 
 		/* Skip a subtree whose 512 PMDs all failed the PMD-level filter last gen */
 		if (!walk->force_scan && !test_pud_bloom_filter(mm_state, walk->seq, pud + i)) {
+			WRITE_ONCE(mm_state->pud_filter_used, true);
 			walk->mm_stats[MM_PUD_EMPTY_SKIPPED]++;
 			continue;
 		}
@@ -4434,7 +4438,8 @@ bool lru_gen_look_around(struct page_vma_mapped_walk *pvmw, unsigned int nr)
 	lazy_mmu_mode_disable();
 
 	/* feedback from rmap walkers to page table walkers */
-	if (mm_state && suitable_to_scan(i, young)) {
+	if (mm_state && suitable_to_scan(i, young) &&
+	    READ_ONCE(mm_state->pud_filter_used)) {
 		/* the PUD entry covering the young PTEs scanned above */
 		pud_t *pud_p = pud_offset(p4d_offset(pgd_offset(vma->vm_mm, pvmw->address),
 						     pvmw->address), pvmw->address);
-- 
2.54.0