Linux-mm Archive on lore.kernel.org
 help / color / mirror / Atom feed
From: Baoquan He <baoquan.he@linux.dev>
To: Kairui Song <ryncsn@gmail.com>
Cc: linux-mm@kvack.org, akpm@linux-foundation.org, david@kernel.org,
	kasong@tencent.com, baohua@kernel.org,
	baolin.wang@linux.alibaba.com, qi.zheng@linux.dev,
	shakeel.butt@linux.dev, axelrasmussen@google.com,
	yuanchu@google.com, weixugc@google.com,
	Baoquan He <hebaoquan@kylinos.cn>
Subject: Re: [PATCH v3 2/4] mm/mglru: add PUD-level Bloom filter state and generic helpers
Date: Thu, 19 Nov 2026 17:24:43 +0800	[thread overview]
Message-ID: <av7AW0kaJTUnucuC@fedora> (raw)
In-Reply-To: <avq-hc6E4Urpf6x4@KASONG-MC4>

On 11/17/26 at 12:24am, Kairui Song wrote:
> On Fri, Nov 13, 2026 at 04:49:51PM +0800, Baoquan He wrote:
> > From: Baoquan He <hebaoquan@kylinos.cn>
> > 
> > Generalize the Bloom filter state and helpers so that another filter
> > level can reuse the same hash, double-buffering and reset logic.
> > 
> > Replace the PMD-level filter array with a two-dimensional filters array
> > indexed by level, and add BLOOM_FILTER_PUD and BLOOM_FILTER_PMD. The
> > PUD-level filters operate at PUD granularity and generalize the
> > suppression one level up: when all PMDs under a PUD fail the PMD-level
> > filter, the whole PMD iteration can be skipped. Cross-node empty walks
> > are the most visible case: a foreign mm's PUDs have no young entries for
> > this lruvec, so the entire PMD iteration is pure waste.
> > 
> > The existing PMD-level code now goes through the generic
> > test/update/reset_bloom_filter() helpers.
> > 
> > No behavior change yet; the PUD-level filters are populated and used by
> > the next patch.
> > 
> > Signed-off-by: Baoquan He <hebaoquan@kylinos.cn>
> > Signed-off-by: Baoquan He <baoquan.he@linux.dev>
> 
> ...
> 
> > diff --git a/mm/vmscan.c b/mm/vmscan.c
> > index 98037a562ab6..9dbb3f477d7d 100644
> > --- a/mm/vmscan.c
> > +++ b/mm/vmscan.c
> > @@ -2798,6 +2798,10 @@ static bool __maybe_unused seq_is_valid(struct lruvec *lruvec)
> >   * walk_pmd_range(); the eviction also report them when walking the rmap
> >   * in lru_gen_look_around().
> >   *
> 
> Not a problem for this patch, but the comment above may want an update
> in the next commit. This describes things only at the PMD level, while
> the PUD level, the ~1/5 and ~1/2 FP figures still hold for (same m, k
> and n). But the PMD false positive only causes a wasted PTE walk if
> the PUD gate also passes, so the effective FP rate is weighted by
> the PUD part too.  The PUD filter has its own FP, which is rarer but
> costs a whole 1 GiB subtree, so maybe the 4 KiB bitmap for PUD
> is not that obviously oversized (despite the PUD number are much
> lower than PMD we are using same sized bitmaps).

OK, makes sense, will update.

> 
> > + * A second, coarser pair of filters sits one level up. The filters are
> > + * indexed by level: BLOOM_FILTER_PMD for the PMD-level filters above, and
> > + * BLOOM_FILTER_PUD for the PUD-level pair.
> > + *
> >   * For future optimizations:
> >   * 1. It's not necessary to keep both filters all the time. The spare one can be
> >   *    freed after the RCU grace period and reallocated if needed again.
> 
> Still not a problem but this optimization is more important now. We have a
> extra 8KB peak overhead per lruvec if I did the math right, not too bad
> as there won't be too many of them, even on large machines (memcg number
> is currently hard limited to 64k and they are lazily allocated). But
> worth mentioning and optimized later.

Agree but don't worry, I have considered this and queued patch in my
local branch. Will post them after this pud filter series.

> 
> > @@ -2823,14 +2827,13 @@ static void get_item_key(void *item, int *key)
> >  	key[1] = hash >> BLOOM_FILTER_SHIFT;
> >  }
> >  
> > -static bool test_bloom_filter(struct lru_gen_mm_state *mm_state, unsigned long seq,
> > -			      void *item)
> > +static bool __test_bloom_filter(unsigned long **filters, unsigned long seq, void *item)
> 
> There is only one caller of this (test_bloom_filter), do we really
> need the __ version?

OK, will remove them.

> 
> >  {
> >  	int key[2];
> >  	unsigned long *filter;
> >  	int gen = filter_gen_from_seq(seq);
> >  
> > -	filter = READ_ONCE(mm_state->filters[gen]);
> > +	filter = READ_ONCE(filters[gen]);
> >  	if (!filter)
> >  		return true;
> >  
> > @@ -2839,14 +2842,13 @@ static bool test_bloom_filter(struct lru_gen_mm_state *mm_state, unsigned long s
> >  	return test_bit(key[0], filter) && test_bit(key[1], filter);
> >  }
> >  
> > -static void update_bloom_filter(struct lru_gen_mm_state *mm_state, unsigned long seq,
> > -				void *item)
> > +static void __update_bloom_filter(unsigned long **filters, unsigned long seq, void *item)
> 
> Also this one, only one caller, the __ intermediate layer seems unnecessary.

Ditto.

> 
> >  {
> >  	int key[2];
> >  	unsigned long *filter;
> >  	int gen = filter_gen_from_seq(seq);
> >  
> > -	filter = READ_ONCE(mm_state->filters[gen]);
> > +	filter = READ_ONCE(filters[gen]);
> >  	if (!filter)
> >  		return;
> >  
> > @@ -2858,12 +2860,12 @@ static void update_bloom_filter(struct lru_gen_mm_state *mm_state, unsigned long
> >  		set_bit(key[1], filter);
> >  }
> >  
> > -static void reset_bloom_filter(struct lru_gen_mm_state *mm_state, unsigned long seq)
> > +static void __reset_bloom_filter(unsigned long **filters, unsigned long seq)
> >  {
> >  	unsigned long *filter;
> >  	int gen = filter_gen_from_seq(seq);
> >  
> > -	filter = mm_state->filters[gen];
> > +	filter = filters[gen];
> >  	if (filter) {
> >  		bitmap_clear(filter, 0, BIT(BLOOM_FILTER_SHIFT));
> >  		return;
> > @@ -2871,7 +2873,25 @@ static void reset_bloom_filter(struct lru_gen_mm_state *mm_state, unsigned long
> >  
> >  	filter = bitmap_zalloc(BIT(BLOOM_FILTER_SHIFT),
> >  			       __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN);
> > -	WRITE_ONCE(mm_state->filters[gen], filter);
> > +	WRITE_ONCE(filters[gen], filter);
> > +}
> > +
> > +static bool test_bloom_filter(struct lru_gen_mm_state *mm_state, int level,
> > +			      unsigned long seq, void *item)
> > +{
> > +	return __test_bloom_filter(mm_state->filters[level], seq, item);
> > +}
> > +
> > +static void update_bloom_filter(struct lru_gen_mm_state *mm_state, int level,
> > +				unsigned long seq, void *item)
> > +{
> > +	__update_bloom_filter(mm_state->filters[level], seq, item);
> > +}
> > +
> > +static void reset_bloom_filter(struct lru_gen_mm_state *mm_state, int level,
> > +			       unsigned long seq)
> > +{
> > +	__reset_bloom_filter(mm_state->filters[level], seq);
> >  }
> >  
> >  /******************************************************************************
> > @@ -3114,7 +3134,7 @@ static bool iterate_mm_list(struct lru_gen_mm_walk *walk, struct mm_struct **ite
> >  	spin_unlock(&mm_list->lock);
> >  
> >  	if (mm && first)
> > -		reset_bloom_filter(mm_state, walk->seq + 1);
> > +		reset_bloom_filter(mm_state, BLOOM_FILTER_PMD, walk->seq + 1);
> >  
> >  	if (*iter)
> >  		mmdrop(*iter);
> > @@ -3771,7 +3791,8 @@ static void walk_pmd_range(pud_t *pud, unsigned long start, unsigned long end,
> >  			walk_pmd_range_locked(pud, addr, vma, args, bitmap, &first);
> >  		}
> >  
> > -		if (!walk->force_scan && !test_bloom_filter(mm_state, walk->seq, pmd + i))
> > +		if (!walk->force_scan &&
> > +		    !test_bloom_filter(mm_state, BLOOM_FILTER_PMD, walk->seq, pmd + i))
> 
> This looks nice.
> 
> >  			continue;
> >  
> >  		walk->mm_stats[MM_NONLEAF_FOUND]++;
> > @@ -3782,7 +3803,7 @@ static void walk_pmd_range(pud_t *pud, unsigned long start, unsigned long end,
> >  		walk->mm_stats[MM_NONLEAF_ADDED]++;
> >  
> >  		/* carry over to the next generation */
> > -		update_bloom_filter(mm_state, walk->seq + 1, pmd + i);
> > +		update_bloom_filter(mm_state, BLOOM_FILTER_PMD, walk->seq + 1, pmd + i);
> >  	}
> >  
> >  	walk_pmd_range_locked(pud, -1, vma, args, bitmap, &first);
> > @@ -4417,8 +4438,8 @@ bool lru_gen_look_around(struct page_vma_mapped_walk *pvmw, unsigned int nr)
> >  
> >  	/* feedback from rmap walkers to page table walkers */
> >  	if (mm_state && suitable_to_scan(i, young))
> > -		update_bloom_filter(mm_state, READ_ONCE(lruvec->lrugen.max_seq),
> > -				    pvmw->pmd);
> > +		update_bloom_filter(mm_state, BLOOM_FILTER_PMD,
> > +				    READ_ONCE(lruvec->lrugen.max_seq), pvmw->pmd);
> >  
> >  	mem_cgroup_put(memcg);
> >  
> > @@ -5981,6 +6002,7 @@ void lru_gen_init_memcg(struct mem_cgroup *memcg)
> >  void lru_gen_exit_memcg(struct mem_cgroup *memcg)
> >  {
> >  	int i;
> > +	int level;
> >  	int nid;
> >  	struct lru_gen_mm_list *mm_list = get_mm_list(memcg);
> >  
> > @@ -5998,9 +6020,11 @@ void lru_gen_exit_memcg(struct mem_cgroup *memcg)
> >  		if (!mm_state)
> >  			continue;
> >  
> > -		for (i = 0; i < NR_BLOOM_FILTERS; i++) {
> > -			bitmap_free(mm_state->filters[i]);
> > -			mm_state->filters[i] = NULL;
> > +		for (level = 0; level < BLOOM_FILTER_LEVELS; level++) {
> > +			for (i = 0; i < NR_BLOOM_FILTERS; i++) {
> > +				bitmap_free(mm_state->filters[level][i]);
> > +				mm_state->filters[level][i] = NULL;
> > +			}
> 
> So just a few nit style and comment issue from me, rest looks good, thanks!
> 
> Reviewed-by: Kairui Song <kasong@tencent.com>

Thanks for careful reviewing and great suggestion.




  reply	other threads:[~2026-11-19  9:24 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-13  8:49 [PATCH v3 0/4] skip empty PUD subtrees during aging with a PUD-level Bloom filter Baoquan He
2026-11-13  8:49 ` [PATCH v3 1/4] mm/mglru: add MM_WALK_EMPTY stats and tracepoint Baoquan He
2026-11-13  8:53   ` Baoquan He
2026-11-14  3:42   ` sashiko-bot
2026-11-16  7:24     ` Baoquan He
2026-11-13  8:49 ` [PATCH v3 2/4] mm/mglru: add PUD-level Bloom filter state and generic helpers Baoquan He
2026-11-16 16:24   ` Kairui Song
2026-11-19  9:24     ` Baoquan He [this message]
2026-11-13  8:49 ` [PATCH v3 3/4] mm/mglru: skip cold PUD subtrees during aging Baoquan He
2026-11-14  3:42   ` sashiko-bot
2026-11-16  9:13     ` Baoquan He
2026-11-13  8:49 ` [PATCH v3 4/4] mm/mglru: count PUD subtrees skipped by the PUD-level filter Baoquan He

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=av7AW0kaJTUnucuC@fedora \
    --to=baoquan.he@linux.dev \
    --cc=akpm@linux-foundation.org \
    --cc=axelrasmussen@google.com \
    --cc=baohua@kernel.org \
    --cc=baolin.wang@linux.alibaba.com \
    --cc=david@kernel.org \
    --cc=hebaoquan@kylinos.cn \
    --cc=kasong@tencent.com \
    --cc=linux-mm@kvack.org \
    --cc=qi.zheng@linux.dev \
    --cc=ryncsn@gmail.com \
    --cc=shakeel.butt@linux.dev \
    --cc=weixugc@google.com \
    --cc=yuanchu@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox