From: Baoquan He <baoquan.he@linux.dev>
To: Kairui Song <ryncsn@gmail.com>
Cc: linux-mm@kvack.org, akpm@linux-foundation.org, david@kernel.org,
kasong@tencent.com, baohua@kernel.org,
baolin.wang@linux.alibaba.com, qi.zheng@linux.dev,
shakeel.butt@linux.dev, axelrasmussen@google.com,
yuanchu@google.com, weixugc@google.com,
Baoquan He <hebaoquan@kylinos.cn>
Subject: Re: [PATCH v3 2/4] mm/mglru: add PUD-level Bloom filter state and generic helpers
Date: Thu, 19 Nov 2026 17:24:43 +0800 [thread overview]
Message-ID: <av7AW0kaJTUnucuC@fedora> (raw)
In-Reply-To: <avq-hc6E4Urpf6x4@KASONG-MC4>
On 11/17/26 at 12:24am, Kairui Song wrote:
> On Fri, Nov 13, 2026 at 04:49:51PM +0800, Baoquan He wrote:
> > From: Baoquan He <hebaoquan@kylinos.cn>
> >
> > Generalize the Bloom filter state and helpers so that another filter
> > level can reuse the same hash, double-buffering and reset logic.
> >
> > Replace the PMD-level filter array with a two-dimensional filters array
> > indexed by level, and add BLOOM_FILTER_PUD and BLOOM_FILTER_PMD. The
> > PUD-level filters operate at PUD granularity and generalize the
> > suppression one level up: when all PMDs under a PUD fail the PMD-level
> > filter, the whole PMD iteration can be skipped. Cross-node empty walks
> > are the most visible case: a foreign mm's PUDs have no young entries for
> > this lruvec, so the entire PMD iteration is pure waste.
> >
> > The existing PMD-level code now goes through the generic
> > test/update/reset_bloom_filter() helpers.
> >
> > No behavior change yet; the PUD-level filters are populated and used by
> > the next patch.
> >
> > Signed-off-by: Baoquan He <hebaoquan@kylinos.cn>
> > Signed-off-by: Baoquan He <baoquan.he@linux.dev>
>
> ...
>
> > diff --git a/mm/vmscan.c b/mm/vmscan.c
> > index 98037a562ab6..9dbb3f477d7d 100644
> > --- a/mm/vmscan.c
> > +++ b/mm/vmscan.c
> > @@ -2798,6 +2798,10 @@ static bool __maybe_unused seq_is_valid(struct lruvec *lruvec)
> > * walk_pmd_range(); the eviction also report them when walking the rmap
> > * in lru_gen_look_around().
> > *
>
> Not a problem for this patch, but the comment above may want an update
> in the next commit. This describes things only at the PMD level, while
> the PUD level, the ~1/5 and ~1/2 FP figures still hold for (same m, k
> and n). But the PMD false positive only causes a wasted PTE walk if
> the PUD gate also passes, so the effective FP rate is weighted by
> the PUD part too. The PUD filter has its own FP, which is rarer but
> costs a whole 1 GiB subtree, so maybe the 4 KiB bitmap for PUD
> is not that obviously oversized (despite the PUD number are much
> lower than PMD we are using same sized bitmaps).
OK, makes sense, will update.
>
> > + * A second, coarser pair of filters sits one level up. The filters are
> > + * indexed by level: BLOOM_FILTER_PMD for the PMD-level filters above, and
> > + * BLOOM_FILTER_PUD for the PUD-level pair.
> > + *
> > * For future optimizations:
> > * 1. It's not necessary to keep both filters all the time. The spare one can be
> > * freed after the RCU grace period and reallocated if needed again.
>
> Still not a problem but this optimization is more important now. We have a
> extra 8KB peak overhead per lruvec if I did the math right, not too bad
> as there won't be too many of them, even on large machines (memcg number
> is currently hard limited to 64k and they are lazily allocated). But
> worth mentioning and optimized later.
Agree but don't worry, I have considered this and queued patch in my
local branch. Will post them after this pud filter series.
>
> > @@ -2823,14 +2827,13 @@ static void get_item_key(void *item, int *key)
> > key[1] = hash >> BLOOM_FILTER_SHIFT;
> > }
> >
> > -static bool test_bloom_filter(struct lru_gen_mm_state *mm_state, unsigned long seq,
> > - void *item)
> > +static bool __test_bloom_filter(unsigned long **filters, unsigned long seq, void *item)
>
> There is only one caller of this (test_bloom_filter), do we really
> need the __ version?
OK, will remove them.
>
> > {
> > int key[2];
> > unsigned long *filter;
> > int gen = filter_gen_from_seq(seq);
> >
> > - filter = READ_ONCE(mm_state->filters[gen]);
> > + filter = READ_ONCE(filters[gen]);
> > if (!filter)
> > return true;
> >
> > @@ -2839,14 +2842,13 @@ static bool test_bloom_filter(struct lru_gen_mm_state *mm_state, unsigned long s
> > return test_bit(key[0], filter) && test_bit(key[1], filter);
> > }
> >
> > -static void update_bloom_filter(struct lru_gen_mm_state *mm_state, unsigned long seq,
> > - void *item)
> > +static void __update_bloom_filter(unsigned long **filters, unsigned long seq, void *item)
>
> Also this one, only one caller, the __ intermediate layer seems unnecessary.
Ditto.
>
> > {
> > int key[2];
> > unsigned long *filter;
> > int gen = filter_gen_from_seq(seq);
> >
> > - filter = READ_ONCE(mm_state->filters[gen]);
> > + filter = READ_ONCE(filters[gen]);
> > if (!filter)
> > return;
> >
> > @@ -2858,12 +2860,12 @@ static void update_bloom_filter(struct lru_gen_mm_state *mm_state, unsigned long
> > set_bit(key[1], filter);
> > }
> >
> > -static void reset_bloom_filter(struct lru_gen_mm_state *mm_state, unsigned long seq)
> > +static void __reset_bloom_filter(unsigned long **filters, unsigned long seq)
> > {
> > unsigned long *filter;
> > int gen = filter_gen_from_seq(seq);
> >
> > - filter = mm_state->filters[gen];
> > + filter = filters[gen];
> > if (filter) {
> > bitmap_clear(filter, 0, BIT(BLOOM_FILTER_SHIFT));
> > return;
> > @@ -2871,7 +2873,25 @@ static void reset_bloom_filter(struct lru_gen_mm_state *mm_state, unsigned long
> >
> > filter = bitmap_zalloc(BIT(BLOOM_FILTER_SHIFT),
> > __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN);
> > - WRITE_ONCE(mm_state->filters[gen], filter);
> > + WRITE_ONCE(filters[gen], filter);
> > +}
> > +
> > +static bool test_bloom_filter(struct lru_gen_mm_state *mm_state, int level,
> > + unsigned long seq, void *item)
> > +{
> > + return __test_bloom_filter(mm_state->filters[level], seq, item);
> > +}
> > +
> > +static void update_bloom_filter(struct lru_gen_mm_state *mm_state, int level,
> > + unsigned long seq, void *item)
> > +{
> > + __update_bloom_filter(mm_state->filters[level], seq, item);
> > +}
> > +
> > +static void reset_bloom_filter(struct lru_gen_mm_state *mm_state, int level,
> > + unsigned long seq)
> > +{
> > + __reset_bloom_filter(mm_state->filters[level], seq);
> > }
> >
> > /******************************************************************************
> > @@ -3114,7 +3134,7 @@ static bool iterate_mm_list(struct lru_gen_mm_walk *walk, struct mm_struct **ite
> > spin_unlock(&mm_list->lock);
> >
> > if (mm && first)
> > - reset_bloom_filter(mm_state, walk->seq + 1);
> > + reset_bloom_filter(mm_state, BLOOM_FILTER_PMD, walk->seq + 1);
> >
> > if (*iter)
> > mmdrop(*iter);
> > @@ -3771,7 +3791,8 @@ static void walk_pmd_range(pud_t *pud, unsigned long start, unsigned long end,
> > walk_pmd_range_locked(pud, addr, vma, args, bitmap, &first);
> > }
> >
> > - if (!walk->force_scan && !test_bloom_filter(mm_state, walk->seq, pmd + i))
> > + if (!walk->force_scan &&
> > + !test_bloom_filter(mm_state, BLOOM_FILTER_PMD, walk->seq, pmd + i))
>
> This looks nice.
>
> > continue;
> >
> > walk->mm_stats[MM_NONLEAF_FOUND]++;
> > @@ -3782,7 +3803,7 @@ static void walk_pmd_range(pud_t *pud, unsigned long start, unsigned long end,
> > walk->mm_stats[MM_NONLEAF_ADDED]++;
> >
> > /* carry over to the next generation */
> > - update_bloom_filter(mm_state, walk->seq + 1, pmd + i);
> > + update_bloom_filter(mm_state, BLOOM_FILTER_PMD, walk->seq + 1, pmd + i);
> > }
> >
> > walk_pmd_range_locked(pud, -1, vma, args, bitmap, &first);
> > @@ -4417,8 +4438,8 @@ bool lru_gen_look_around(struct page_vma_mapped_walk *pvmw, unsigned int nr)
> >
> > /* feedback from rmap walkers to page table walkers */
> > if (mm_state && suitable_to_scan(i, young))
> > - update_bloom_filter(mm_state, READ_ONCE(lruvec->lrugen.max_seq),
> > - pvmw->pmd);
> > + update_bloom_filter(mm_state, BLOOM_FILTER_PMD,
> > + READ_ONCE(lruvec->lrugen.max_seq), pvmw->pmd);
> >
> > mem_cgroup_put(memcg);
> >
> > @@ -5981,6 +6002,7 @@ void lru_gen_init_memcg(struct mem_cgroup *memcg)
> > void lru_gen_exit_memcg(struct mem_cgroup *memcg)
> > {
> > int i;
> > + int level;
> > int nid;
> > struct lru_gen_mm_list *mm_list = get_mm_list(memcg);
> >
> > @@ -5998,9 +6020,11 @@ void lru_gen_exit_memcg(struct mem_cgroup *memcg)
> > if (!mm_state)
> > continue;
> >
> > - for (i = 0; i < NR_BLOOM_FILTERS; i++) {
> > - bitmap_free(mm_state->filters[i]);
> > - mm_state->filters[i] = NULL;
> > + for (level = 0; level < BLOOM_FILTER_LEVELS; level++) {
> > + for (i = 0; i < NR_BLOOM_FILTERS; i++) {
> > + bitmap_free(mm_state->filters[level][i]);
> > + mm_state->filters[level][i] = NULL;
> > + }
>
> So just a few nit style and comment issue from me, rest looks good, thanks!
>
> Reviewed-by: Kairui Song <kasong@tencent.com>
Thanks for careful reviewing and great suggestion.
next prev parent reply other threads:[~2026-11-19 9:24 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-13 8:49 [PATCH v3 0/4] skip empty PUD subtrees during aging with a PUD-level Bloom filter Baoquan He
2026-11-13 8:49 ` [PATCH v3 1/4] mm/mglru: add MM_WALK_EMPTY stats and tracepoint Baoquan He
2026-11-13 8:53 ` Baoquan He
2026-11-14 3:42 ` sashiko-bot
2026-11-16 7:24 ` Baoquan He
2026-11-13 8:49 ` [PATCH v3 2/4] mm/mglru: add PUD-level Bloom filter state and generic helpers Baoquan He
2026-11-16 16:24 ` Kairui Song
2026-11-19 9:24 ` Baoquan He [this message]
2026-11-13 8:49 ` [PATCH v3 3/4] mm/mglru: skip cold PUD subtrees during aging Baoquan He
2026-11-14 3:42 ` sashiko-bot
2026-11-16 9:13 ` Baoquan He
2026-11-13 8:49 ` [PATCH v3 4/4] mm/mglru: count PUD subtrees skipped by the PUD-level filter Baoquan He
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=av7AW0kaJTUnucuC@fedora \
--to=baoquan.he@linux.dev \
--cc=akpm@linux-foundation.org \
--cc=axelrasmussen@google.com \
--cc=baohua@kernel.org \
--cc=baolin.wang@linux.alibaba.com \
--cc=david@kernel.org \
--cc=hebaoquan@kylinos.cn \
--cc=kasong@tencent.com \
--cc=linux-mm@kvack.org \
--cc=qi.zheng@linux.dev \
--cc=ryncsn@gmail.com \
--cc=shakeel.butt@linux.dev \
--cc=weixugc@google.com \
--cc=yuanchu@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox