* [RFC PATCH 0/1] mm/mglru: MADV_RANDOM page cache never ages and thrashes the oldest generation
@ 2026-11-01 16:04 Maxim Muzafarov
2026-11-01 16:04 ` [RFC PATCH 1/1] mm/mglru: age when the oldest generation is thrashing Maxim Muzafarov
2026-11-01 18:02 ` [RFC PATCH 0/1] mm/mglru: MADV_RANDOM page cache never ages and thrashes the oldest generation Kairui Song
0 siblings, 2 replies; 3+ messages in thread
From: Maxim Muzafarov @ 2026-11-01 16:04 UTC (permalink / raw)
To: linux-mm
Cc: Maxim Muzafarov, Andrew Morton, Kairui Song, Qi Zheng,
Shakeel Butt, Barry Song, Axel Rasmussen, Yuanchu Xie, Wei Xu,
Baoquan He, Baolin Wang, Yu Zhao, Johannes Weiner,
David Hildenbrand, Michal Hocko, Lorenzo Stoakes, linux-kernel
Hi,
It looks like I've stumbled upon an awkward behaviour. When MGLRU is
enabled, page cache faulted in through an MADV_RANDOM mapping goes into
the oldest generation, and the refaults land there at the same rate as
well. In my tests bpftrace shows the generations stay at seq 0-3, with
no aging attempts and no new generations during the workload run, and
/sys/kernel/debug/lru_gen_full shows for generation seq 0 15758245
pages evicted and 15580028 pages refaulted, which is 98.87%.
At the same time, a file read through a normal mmap lands one
generation higher and none of its pages are evicted. It stays cached
until the end of the workload, and the randomly read working set
shrinks. With lru_gen disabled, the CLRU evicts the file
instead, and throughput recovers 20-30 seconds after the file was read.
I would expect the same behaviour for MGLRU.
I first saw this slowdown on a single-node OpenSearch 3.6 (Lucene 10.4)
HNSW vector search workload [1]. The node had about 2.8 GiB of page
cache for a 2.9 GiB index, and Lucene maps the .vec and .vex files with
MADV_RANDOM. Kernel 7.2.10-200.fc44.aarch64.
Search latency and throughput on a single-node OpenSearch:
MGLRU on MGLRU off
p50, ms 286.0 73.6
p90, ms 512.1 135.6
p99, ms 687.0 191.0
queries/s 27.4 87.1
Disk reads, same runs:
MGLRU on MGLRU off
major faults/query 1484 325
disk reads/query 1484 325
read size, KiB 4.0 4.0
disk read, GiB/run 509.6 111.5
This is clearly visible on the page cache chart [2], where "blue" are
page cache pages mapped into at least one process's address space and
"orange" the rest of the page cache.
Then I reproduced it with a synthetic HNSW-like query script in python,
and later with fio alone, on 7.2.10-200.fc44.aarch64, 7.2.14, 7.2.14
with the MGLRU-FG v4 series [3][4], and on mainline v7.3-rc7.
Kernel and setup
----------------
Arm64 Lima VM (vz) on an Apple M1 Max host, 8 vCPUs, 16 GiB RAM, virtio
disk, btrfs, 4K pages, THP madvise. Tests run in a 2 GiB memcg with
swap disabled.
CONFIG_LRU_GEN=y, CONFIG_LRU_GEN_ENABLED=y, CONFIG_LRU_GEN_WALKS_MMU=y.
Reproducer
----------
The script below reproduces the stall with fio, MGLRU on (y) or off
(n). 4 fio jobs read the vector file at random through an mmap in a
2 GiB memcg, with MADV_RANDOM or no advice (normal).
At t=90 a second fio reads a 600 MB cold file twice through a normal
mmap. With MGLRU on and MADV_RANDOM, the cold file stays cached and
IOPS doesn't recover.
./run.sh <y|n> [random|normal]
-------
#!/usr/bin/env bash
set -u
usage="usage: sudo $0 <y|n> [random|normal]"
lru=${1:?$usage}
advice=${2:-random}
case $advice in random) hint=1 ;; normal) hint=0 ;; *) echo "$usage" >&2; exit 1 ;; esac
out=$(dirname "$(readlink -f "$0")")/results/$(uname -r)/$lru-$advice.txt
[ ! -e "$out" ] || { echo "$out exists, move it aside first" >&2; exit 1; }
vec=/var/tmp/mglru/vs-vec.bin
cold=/var/tmp/mglru/cold.bin
unit=mglru-06-$$
cg=/sys/fs/cgroup/system.slice/$unit.scope
log=/tmp/mglru-06-$$.fio
workload() {
fio --name=vec --filename=$vec --ioengine=mmap --rw=randread --bs=4k \
--fadvise_hint=$hint --random_distribution=zipf:0.8 --numjobs=4 \
--group_reporting --time_based --runtime=270 \
--status-interval=1 --output-format=terse --output=$log &
sleep 90
fio --name=cold --filename=$cold --ioengine=mmap --rw=read --bs=4k \
--fadvise_hint=0 --loops=2 > /dev/null
wait
}
memstat() {
awk -v key=$1 '$1 == key { print $2 }' $cg/memory.stat
}
cached_mb() {
fincore -nb -o RES $1 | awk '{ print int($1 / 2^20) }'
}
render() {
local t=$1 kb faults refaults activated
kb=$(tail -n 1 $log | cut -d';' -f6)
faults=$(memstat pgmajfault)
refaults=$(memstat workingset_refault_file)
activated=$(memstat workingset_activate_file)
printf '%4s %9s %9s %9s %9s %7s %7s\n' $t \
$(( (kb - prev[0]) / 4 / 5 )) \
$(( (faults - prev[1]) / 5 )) \
$(( (refaults - prev[2]) / 5 )) \
$(( (activated - prev[3]) / 5 )) \
$(cached_mb $vec) $(cached_mb $cold)
prev=($kb $faults $refaults $activated)
}
mkdir -p /var/tmp/mglru
[ -f $vec ] || dd if=/dev/urandom of=$vec bs=1M count=1905 status=none
[ -f $cold ] || dd if=/dev/urandom of=$cold bs=1M count=600 status=none
echo $lru > /sys/kernel/mm/lru_gen/enabled
sync
echo 3 > /proc/sys/vm/drop_caches
mkdir -p "$(dirname "$out")"
exec > >(tee "$out")
echo "MGLRU $(cat /sys/kernel/mm/lru_gen/enabled), kernel $(uname -r), $advice advice, cold file read at t=90 s"
printf '%4s %9s %9s %9s %9s %7s %7s\n' t IOPS majflt/s refault/s activ/s vec_MB cold_MB
export -f workload
export vec cold hint log
systemd-run --scope --quiet --unit=$unit -p MemoryMax=2G -p MemorySwapMax=0 bash -c workload &
until [ -e $cg/memory.stat ]; do kill -0 $! 2>/dev/null || exit 1; sleep 0.1; done
prev=(0 $(memstat pgmajfault) $(memstat workingset_refault_file) $(memstat workingset_activate_file))
for t in $(seq 5 5 270); do
sleep 5
render $t
done
wait
-------
Results
-------
v7.3-rc7, cold file read at t=90
lru_gen on, random:
t IOPS majflt/s refault/s activ/s vec_MB cold_MB
85 4966675 0 0 0 1904 0
90 4863829 1150 1158 0 1729 277
95 174236 30664 30669 0 1408 600
120 202080 28161 28163 0 1408 600
180 207005 26665 26667 0 1408 600
270 226341 22677 22669 0 1409 600
lru_gen off, random:
t IOPS majflt/s refault/s activ/s vec_MB cold_MB
85 5007909 0 0 0 1904 0
90 5150972 140 228 51 1873 139
95 188806 29509 29504 9283 1413 600
110 342921 31803 31802 23547 1697 313
115 2563389 15857 15849 10540 1903 107
120 5157256 31 31 27 1904 107
270 5179181 2 28 27 1904 107
lru_gen on, normal:
t IOPS majflt/s refault/s activ/s vec_MB cold_MB
85 5093044 0 0 0 1904 0
90 4887136 0 0 0 1904 39
95 2612769 3665 88721 6458 1904 109
100 4997923 0 0 0 1904 106
180 5134344 0 0 0 1904 104
270 5025222 0 0 0 1904 104
lru_gen off, normal:
t IOPS majflt/s refault/s activ/s vec_MB cold_MB
85 4866859 0 0 0 1904 0
90 4466107 0 0 0 1904 103
95 2378250 3027 34135 33956 1885 127
100 2734588 5431 5520 5500 1903 109
105 4933296 62 69 67 1904 104
270 4822353 2 17 14 1904 104
As you can see, only MGLRU with MADV_RANDOM stalls, at about 220k IOPS
until the end. It activates no refaults, and cold.bin stays at 600 MB
while the vector file stays near 1408 MB. With MGLRU off, the same
workload is back to ~5M IOPS 30 seconds after the cold read. With
normal advice, both LRUs recover within 15 seconds.
With MGLRU and MADV_RANDOM, vs-vec.bin reads in my python reproducer
drop from ~2.5M/s to ~300K/s after the cold read and never recover
(chart [5]). On the same kernel with normal advice, MGLRU recovers
normally (chart [6]).
What I traced
-------------
On 7.2.14 with bpftrace, every MADV_RANDOM fault and refault is
inserted in the old gen. lru_gen_enter_fault() sets in_lru_fault from
vma_has_recency(), which is false for VM_RAND_READ, so refaults are
never activated. The aging walk and the look-around on eviction skip
these mappings too.
On 7.2.14 with bpftrace, the oldest generation is evicted and refilled
at the same rate, and never empties. min_seq doesn't advance,
fentry:try_to_inc_max_seq is never called, and the window stays at
seq 0-3.
On 7.2.10 (fio, no tracing), one manual aging step, "+ memcg_id 0
max_seq" written to /sys/kernel/debug/lru_gen, evicts the cold file
within 10 seconds.
I also tested the MGLRU-FG v4 series [3][4] on top of 7.2.14. The stall
is the same, refaults on the MADV_RANDOM mapping are still not
activated. The series cover letter [3] describes this as intended,
so I'm not sure what the right fix is here.
RFC patch
---------
Patch 1/1 is a draft against v7.3-rc7. I'm still not sure it is the
right fix and would appreciate feedback. The simplest thing I found is
one more condition in should_run_aging(), which does the same as the
manual aging step above, at the cost of a few more reads and a
comparison.
In short:
1. Run aging when eviction only recycles the oldest generation
2. Pass the evictable size that get_nr_to_scan() already computes down
to should_run_aging(), so lruvec_evictable_size() is still called
once per pass and not on every batch
The module parameter lru_gen_age_on_thrash and noinline was only for
A/B runs and bpftrace and comparing results.
Results with the patch
----------------------
The fio reproducer from above on v7.3-rc7 with the patch. "checks" is
the number of oldest_gen_thrashing() calls in the last 5 seconds, and
"gens" the number of generations added by aging, both from bpftrace.
lru_gen on, random, lru_gen_age_on_thrash=Y:
t IOPS majflt/s refault/s activ/s vec_MB cold_MB checks gens
85 5019235 0 0 0 1904 0 0 0
90 5073570 377 384 0 1764 254 0 0
95 130923 20534 20527 0 1418 600 2441 0
105 218624 23462 23456 0 1418 600 1785 0
115 178889 26035 26043 0 1418 600 1338 0
120 175238 26918 26910 0 1418 600 2086 1
125 256621 27773 27780 0 1418 600 1959 0
130 253813 24509 24501 0 1677 342 1910 0
135 2678888 11649 11649 0 1904 116 1914 0
140 5064423 2 2 0 1904 116 297 0
180 4872187 0 0 0 1904 116 0 0
265 4939524 0 0 0 1904 115 0 0
The check returned true 3 times, and aging added one generation, at
t=117 (seq 0-3 to 1-4).
lru_gen on, random, lru_gen_age_on_thrash=N:
t IOPS majflt/s refault/s activ/s vec_MB cold_MB checks gens
85 5127299 0 0 0 1904 0 0 0
90 5290342 409 417 0 1822 196 0 0
95 151082 27372 27372 0 1418 600 2739 0
120 251683 33784 33789 0 1418 600 2425 0
180 252607 31539 31542 0 1418 600 2314 0
265 223364 28211 28220 0 1418 600 2290 0
The recovery is still slower than with CLRU. With the patch IOPS
recovers at t=135-140, with CLRU at t=115-120. I think the patch adds
this delay, as the first aging step needs up to one lruvec worth of
refaults after the stall starts.
In the OpenSearch workload reclaim runs from the start, so the counter
is already close to the threshold when the cold file is read, and the
recovery is comparable to CLRU (chart [7]).
The threshold could be lowered, but it's not clear to me yet how to
pick it right. The patch is a draft to get feedback on the approach.
Any thoughts?
Thanks,
Maxim Muzafarov
Links
-----
[1] https://docs.opensearch.org/latest/benchmark/workloads/vectorsearch/
[2] https://gist.github.com/user-attachments/assets/cbfa5c17-0a89-458f-b816-4d1bb72cb97c
[3] https://lore.kernel.org/all/20261016-mglru-fg-v4-0-2106750c0bfa@tencent.com/
[4] https://github.com/zen-kernel/zen-kernel/commits/7.2/mglru-fg-v4/
[5] https://gist.github.com/user-attachments/assets/f65fbbf3-a30a-464f-8535-29d39e0dff43
[6] https://gist.github.com/user-attachments/assets/5d28512d-3a1d-4475-b20e-3690a8c9e196
[7] https://gist.github.com/user-attachments/assets/41b7aaa1-5e9c-43dc-8fc1-09d2df52f8d7
Maxim Muzafarov (1):
mm/mglru: age when the oldest generation is thrashing
mm/vmscan.c | 32 ++++++++++++++++++++++++++++----
1 file changed, 28 insertions(+), 4 deletions(-)
base-commit: 24a19b953e2a3d0ea5a8c4951a0ab8f32720c907
--
2.56.0
^ permalink raw reply [flat|nested] 3+ messages in thread
* [RFC PATCH 1/1] mm/mglru: age when the oldest generation is thrashing
2026-11-01 16:04 [RFC PATCH 0/1] mm/mglru: MADV_RANDOM page cache never ages and thrashes the oldest generation Maxim Muzafarov
@ 2026-11-01 16:04 ` Maxim Muzafarov
2026-11-01 18:02 ` [RFC PATCH 0/1] mm/mglru: MADV_RANDOM page cache never ages and thrashes the oldest generation Kairui Song
1 sibling, 0 replies; 3+ messages in thread
From: Maxim Muzafarov @ 2026-11-01 16:04 UTC (permalink / raw)
To: linux-mm
Cc: Maxim Muzafarov, Andrew Morton, Kairui Song, Qi Zheng,
Shakeel Butt, Barry Song, Axel Rasmussen, Yuanchu Xie, Wei Xu,
Baoquan He, Baolin Wang, Yu Zhao, Johannes Weiner,
David Hildenbrand, Michal Hocko, Lorenzo Stoakes, linux-kernel
With four generations, should_run_aging() only runs when the window
shrinks. With every fault on MADV_RANDOM mapping landing in min_seq,
and the oldest generation refilling as fast as it is evicted,
min_seq never advances and pages in the generations above it are
never evicted.
Run aging when the oldest file generation has refaulted more pages than
the lruvec holds since it became the oldest.
Signed-off-by: Maxim Muzafarov <maxmuzaf@gmail.com>
---
mm/vmscan.c | 32 ++++++++++++++++++++++++++++----
1 file changed, 28 insertions(+), 4 deletions(-)
diff --git a/mm/vmscan.c b/mm/vmscan.c
index f11491ee9ed5..f59b29723ec9 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -4966,8 +4966,26 @@ static int evict_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
return scanned;
}
+static bool oldest_gen_thrashing(struct lruvec *lruvec, unsigned long *min_seq,
+ int swappiness, unsigned long evictable)
+{
+ int tier;
+ unsigned long refaulted = 0;
+ struct lru_gen_folio *lrugen = &lruvec->lrugen;
+ int hist = lru_hist_from_seq(min_seq[LRU_GEN_FILE]);
+
+ if (swappiness == SWAPPINESS_ANON_ONLY)
+ return false;
+
+ for (tier = 0; tier < MAX_NR_TIERS; tier++)
+ refaulted += atomic_long_read(&lrugen->refaulted[hist][LRU_GEN_FILE][tier]);
+
+ return refaulted > evictable;
+}
+
static bool should_run_aging(struct lruvec *lruvec, unsigned long max_seq,
- struct scan_control *sc, int swappiness)
+ struct scan_control *sc, int swappiness,
+ unsigned long evictable)
{
DEFINE_MIN_SEQ(lruvec);
@@ -4975,6 +4993,9 @@ static bool should_run_aging(struct lruvec *lruvec, unsigned long max_seq,
if (evictable_min_seq(min_seq, swappiness) + MIN_NR_GENS > max_seq)
return true;
+ if (oldest_gen_thrashing(lruvec, min_seq, swappiness, evictable))
+ return true;
+
/* try to avoid aging, do gentle reclaim at the default priority */
if (sc->priority == DEF_PRIORITY)
return false;
@@ -4984,7 +5005,8 @@ static bool should_run_aging(struct lruvec *lruvec, unsigned long max_seq,
}
static long get_nr_to_scan(struct lruvec *lruvec, struct scan_control *sc,
- struct mem_cgroup *memcg, int swappiness)
+ struct mem_cgroup *memcg, int swappiness,
+ unsigned long *nr_evictable)
{
unsigned long nr_to_scan, evictable;
struct pglist_data *pgdat = lruvec_pgdat(lruvec);
@@ -5003,6 +5025,7 @@ static long get_nr_to_scan(struct lruvec *lruvec, struct scan_control *sc,
}
evictable = lruvec_evictable_size(lruvec, swappiness);
+ *nr_evictable = evictable;
/* try to scrape all its memory if this memcg was deleted */
if (!mem_cgroup_online(memcg))
@@ -5053,10 +5076,11 @@ static bool try_to_shrink_lruvec(struct lruvec *lruvec, struct scan_control *sc)
{
bool need_rotate = false, should_age = false;
long nr_batch, nr_to_scan;
+ unsigned long evictable = 0;
int swappiness = get_swappiness(lruvec, sc);
struct mem_cgroup *memcg = lruvec_memcg(lruvec);
- nr_to_scan = get_nr_to_scan(lruvec, sc, memcg, swappiness);
+ nr_to_scan = get_nr_to_scan(lruvec, sc, memcg, swappiness, &evictable);
while (nr_to_scan > 0) {
int delta;
DEFINE_MAX_SEQ(lruvec);
@@ -5066,7 +5090,7 @@ static bool try_to_shrink_lruvec(struct lruvec *lruvec, struct scan_control *sc)
break;
}
- if (should_run_aging(lruvec, max_seq, sc, swappiness)) {
+ if (should_run_aging(lruvec, max_seq, sc, swappiness, evictable)) {
if (try_to_inc_max_seq(lruvec, max_seq, swappiness, false))
need_rotate = true;
should_age = true;
base-commit: 24a19b953e2a3d0ea5a8c4951a0ab8f32720c907
--
2.56.0
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [RFC PATCH 0/1] mm/mglru: MADV_RANDOM page cache never ages and thrashes the oldest generation
2026-11-01 16:04 [RFC PATCH 0/1] mm/mglru: MADV_RANDOM page cache never ages and thrashes the oldest generation Maxim Muzafarov
2026-11-01 16:04 ` [RFC PATCH 1/1] mm/mglru: age when the oldest generation is thrashing Maxim Muzafarov
@ 2026-11-01 18:02 ` Kairui Song
1 sibling, 0 replies; 3+ messages in thread
From: Kairui Song @ 2026-11-01 18:02 UTC (permalink / raw)
To: Maxim Muzafarov
Cc: linux-mm, Andrew Morton, Qi Zheng, Shakeel Butt, Barry Song,
Axel Rasmussen, Yuanchu Xie, Wei Xu, Baoquan He, Baolin Wang,
Yu Zhao, Johannes Weiner, David Hildenbrand, Michal Hocko,
Lorenzo Stoakes, linux-kernel
On Mon, Nov 2, 2026 at 12:04 AM Maxim Muzafarov <maxmuzaf@gmail.com> wrote:
>
> Hi,
Hello Maxim!
> It looks like I've stumbled upon an awkward behaviour. When MGLRU is
You are not alone, I encountered same issue too :)
> enabled, page cache faulted in through an MADV_RANDOM mapping goes into
> the oldest generation, and the refaults land there at the same rate as
> well. In my tests bpftrace shows the generations stay at seq 0-3, with
> no aging attempts and no new generations during the workload run, and
> /sys/kernel/debug/lru_gen_full shows for generation seq 0 15758245
> pages evicted and 15580028 pages refaulted, which is 98.87%.
>
> At the same time, a file read through a normal mmap lands one
> generation higher and none of its pages are evicted. It stays cached
> until the end of the workload, and the randomly read working set
> shrinks. With lru_gen disabled, the CLRU evicts the file
> instead, and throughput recovers 20-30 seconds after the file was read.
> I would expect the same behaviour for MGLRU.
>
> I first saw this slowdown on a single-node OpenSearch 3.6 (Lucene 10.4)
> HNSW vector search workload [1]. The node had about 2.8 GiB of page
> cache for a 2.9 GiB index, and Lucene maps the .vec and .vex files with
> MADV_RANDOM. Kernel 7.2.10-200.fc44.aarch64.
>
> Search latency and throughput on a single-node OpenSearch:
>
> MGLRU on MGLRU off
> p50, ms 286.0 73.6
> p90, ms 512.1 135.6
> p99, ms 687.0 191.0
> queries/s 27.4 87.1
>
> Disk reads, same runs:
>
> MGLRU on MGLRU off
> major faults/query 1484 325
> disk reads/query 1484 325
> read size, KiB 4.0 4.0
> disk read, GiB/run 509.6 111.5
>
> This is clearly visible on the page cache chart [2], where "blue" are
> page cache pages mapped into at least one process's address space and
> "orange" the rest of the page cache.
>
> Then I reproduced it with a synthetic HNSW-like query script in python,
> and later with fio alone, on 7.2.10-200.fc44.aarch64, 7.2.14, 7.2.14
> with the MGLRU-FG v4 series [3][4], and on mainline v7.3-rc7.
The missing piece here is refault distance. Both MGLRU and CLRU ignore
hotness on eviction for vma_has_recency() == false case, however, CLRU
still has refault distance protection, while MGLRU doesn't.
I observed a similar issue, which disappeared after we added refault
distance for MGLRU.
There is an early version of FG that have it:
https://lore.kernel.org/linux-mm/20260502-mglru-fg-v1-0-913619b014d9@tencent.com/
See this patch, it too long so I have to split it out and dropped it
for later version:
https://lore.kernel.org/linux-mm/20260502-mglru-fg-v1-30-913619b014d9@tencent.com/
This code snip:
+ /* Set refault as active. */
+ if (distance < total / 2) {
+ folio_set_active(folio);
I may change it to something like
folio_set_lru_refs(LRU_REFS_WORKINGSET) later for non in-fault case
cause this kind of break FG for other cases.
That way, even a random folio will be put to the head directly instead
of keep thrasing on tail. And placing on head or newer gen will trigger aging.
I think I got the problem right? I'll run your benchmark too a bit
later, still tweaking it for some other issues. The refault distance
part needs some rebasing and tweaking as it will have chemical
reactions with many other MGLRU components and it may not work as
expected yet (I also need to use it for an MRU fallback, because, in
theory a pure random case may benifit more from fallback to MRU
stragegy, but your case is a hybrid of hot random and cold non-random,
so a bit tricky to get it right)...
(And we may also loose lru_gen_in_fault for refault case? Not sure...)
But I think we are getting there :), and your report is a really nice case!
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2026-11-01 18:03 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-01 16:04 [RFC PATCH 0/1] mm/mglru: MADV_RANDOM page cache never ages and thrashes the oldest generation Maxim Muzafarov
2026-11-01 16:04 ` [RFC PATCH 1/1] mm/mglru: age when the oldest generation is thrashing Maxim Muzafarov
2026-11-01 18:02 ` [RFC PATCH 0/1] mm/mglru: MADV_RANDOM page cache never ages and thrashes the oldest generation Kairui Song
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox