Re: [PATCH 0/9] mm/mglru: suppress empty page table walks during aging

[Baoquan He <baoquan.he@linux.dev>]

On 08/24/26 at 03:37pm, Baoquan He wrote:
> In the current MGLRU, lru_gen_use_mm() will mark an mm for all nodes at
> each context switch, so on each node aging will walk into each mm's page
> table independently. On a multi-NUMA node system, one process launched
> on one or a subset of nodes, its mm is walked by all other nodes's aging
> while finds on pages for their lruvec. This is pure waste (100% walks
> are empty on those other nodes)
> 
> This patch series suppresses these empty page table walks with two
> complementary mechanisms at different levels:
> 
>   - empty_map (cross-node specifc). At mm granularity, one process's mm
>     whose walk found no pages for one node's lruvec is skipped on that
>     node between re-scan passes.
> 
>   - PUD-level Bloom filter (general). One level up from the existing PMD
>     filters, it skips any 1GB PUD that had no young entries last
>     generation - namely whose 512 PMDs all failed PMD test. Cross-node
>     empty walks can prove its effect the best, but it also suppresses
>     purely code PUD inside local mms. So it reduces unnecessary walking
>     in any workload with cold areas.
> 
> The two complement each other: empty_map reduces the *number* of
> cross-node walks, the PUD-level filter reduces the *cost* of the walks
> that remain.
> 
> Measurement test (3-socket Xeon 6766E, 32 x 256MB workers pinned to node 0):
> 
>   node 1 (foreign mm walks):
>     empty_map off (PUD filter on): 3300 walks, all empty, ~1238 PTE scans/walk
>     both on:                       900 walks (-73%), ~0 PTE scans/walk

node 1 (foreign mm walks):
  no (PUD filter|empty_map) (baseline): 3300 walks, all empty, ~1238 PTE scans/walk
  empty_map off (PUD filter on):        3300 walks, all empty, ~0 PTE scans/walk
  both on:                              900 walks (-73%), ~0 PTE scans/walk

Sorry, double check here and found I pasted the wrong content. The above
table is the right one. The 1st case means on baseline kernel, no pud
filter and no empty_map, on node 1 3300 walks are all empty walks, and
each walk scans ~1238 PTE.

> 
>   - empty_map cuts node 1's walk count by ~73% (every 4th pass re-scans
>     to close migration/fault windows).
>   - the PUD-level filter makes the remaining walks nearly free: the 1GB
>     subtrees are skipped instead of iterating their 512 PMDs
>     (leaf_total drops from ~1238 to ~0 per walk).
>   - worker RSS is unchanged in both modes - no premature reclaim. A
>     force_scan pass confirms the local pages are found young and eligible
>     (leaf_eligible == young == 21320 on node 0) when the filters are
>     bypassed, i.e. the pages are hot and the aging walker can find them.
> 
> Performance regression test (make -j4 in a 3G memory cgroup, 4 vCPU / 8GB
> 2-NUMA VM, median of 3 runs):
> 
>                   baseline   patch    diff
>     build time     10m01s    9m41s    -3.4%
>     pgpgin        136868    138080   +0.9%
>     pgmajfault      1832      1862   +1.6%
> 
> Build time, page-in and major-fault counts are within run-to-run
> variance of a no-patch baseline - no measurable regression. The build
> time is if anything slightly lower, consistent with the suppression
> reducing reclaim overhead (fewer cross-node empty walks) during a real
> memory-pressure workload.
> 
> The test codes/scripts (numa_workload + run_test_v4.sh) are available at:
> https://github.com/baoquan-he/mglru-empty-walk-test
> 
> RFC-v1:
>   - Bloom filter helpers are named symmetrically -
>     test/update/reset_pmd_bloom_filter() alongside the PUD-level ones, and
>     the struct field filters -> pmd_filters (Barry).
> 
>   - An mm is marked empty only when its page tables were actually walked
>     (a failed mmap_read_trylock() or a stale seq is not empty), and the skip
>     is invalidated on the major page-fault and migration paths. (Sashiko)
> 
>   - Reworked the re-scan: the old shared counter across nodes got stuck at
>     the slowest node, so the skip never really engaged. Now each node re-scans
>     its empty mms every N passes on its own clock. N is the skip_empty knob
>     (default 4, proper read/write with input validation). (Sashiko)
> 
>   - Added measurement counters/tracepoint, and a kernel-build regression
>     test (no measurable impact).
> 
> 
> Baoquan He (9):
>   mm/mglru: add MM_WALK_EMPTY stats and tracepoint
>   mm/mglru: suppress cross-node empty page table walks
>   mm/mglru: add debugfs knob for the empty-walk skip threshold
>   mm/mglru: invalidate empty-walk skip on page fault and migration
>   mm/mglru: add PUD-level Bloom filter state
>   mm/mglru: refactor Bloom filter helpers for two filter levels
>   mm/mglru: skip PUD subtrees during aging
>   mm/mglru: report hot PUDs from the rmap feedback path
>   mm/mglru: count PUD subtrees skipped by the PUD-level filter
> 
>  include/linux/mm_types.h      |  22 ++++
>  include/linux/mmzone.h        |  13 ++-
>  include/trace/events/vmscan.h |  30 +++++
>  mm/huge_memory.c              |   3 +
>  mm/memory.c                   |  15 +++
>  mm/migrate.c                  |   4 +
>  mm/vmscan.c                   | 205 +++++++++++++++++++++++++++++-----
>  7 files changed, 264 insertions(+), 28 deletions(-)
> 
> 
> base-commit: efecab401cb15fd3bb9bc05990609acb6b267ff2
> -- 
> 2.54.0
> 
>