* [PATCH v6 0/2] mm/vmscan: fix NR_ISOLATED accounting and throttle MGLRU eviction
@ 2026-10-14 9:00 Hui Zhu
2026-10-14 9:00 ` [PATCH v6 1/2] mm/vmscan: fix missing NR_ISOLATED counter update in MGLRU reclaim path Hui Zhu
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Hui Zhu @ 2026-10-14 9:00 UTC (permalink / raw)
To: Andrew Morton, Kairui Song, Qi Zheng, Shakeel Butt, Barry Song,
Axel Rasmussen, Yuanchu Xie, Wei Xu, Johannes Weiner,
David Hildenbrand, Michal Hocko, Lorenzo Stoakes, Baolin Wang,
linux-mm, linux-kernel
Cc: Hui Zhu
From: Hui Zhu <zhuhui@kylinos.cn>
Barry asked for a real-world workload that suffers from the missing
throttling. This version adds one: a CI-style kernel build inside a
memory-limited cgroup with slow storage (mglru_build_repro.sh),
measured at three pressure levels, plus redis-based cache service runs
as no-impact data points.
The series is also rebased onto the latest mm-stable; patch 2 is
adapted to the current isolate_folios() type-selection context
(is_single_type_reclaim()), with no semantic change.
The legacy reclaim path has two mechanisms around isolated folios that
MGLRU lacks:
1. NR_ISOLATED_ANON/FILE counters are updated when folios are isolated
from the inactive lists. Compaction's too_many_isolated() relies on
them to decide when to back off. The MGLRU reclaim path never
updates them, so compaction cannot see MGLRU's in-flight isolation.
2. shrink_inactive_list() throttles direct reclaim via
too_many_isolated() when isolated folios pile up. MGLRU's
evict_folios() isolates folios without any such check, so many
concurrent reclaimers can over-isolate the same (oldest) generation,
leading to unnecessary swapping, thrashing and premature memcg OOM.
Patch 1 fixes the counter accounting in the MGLRU isolation path.
Patch 2 adds a per-lruvec throttle, mirroring the legacy behavior but
adapted to MGLRU's per-lruvec contention and its dynamic type
selection/fallback.
Testing
=======
Five test scripts are provided: the two synthetic reproducers from v5
and three real-world workload harnesses new in v6. All are available
at:
https://gist.github.com/teawater/3ef51251f2e91a5a600e3d26bb477e34
Common environment: 10 CPU / 8GB QEMU guest, cgroup v2, MGLRU enabled
(0x0007). "unpatched" and "patched" are the same tree built
without/with this series; every configuration is run with identical
parameters on both kernels. Each cell below is one run; the build
metrics are cumulative counters over a 1800s window, which smooths
run-to-run noise.
Real workload 1: CI kernel build (mglru_build_repro.sh)
-------------------------------------------------------
Models a CI build pod (k8s / OBS / gitlab-runner): repeated
from-scratch "make -j20 vmlinux" rounds inside a memory-limited
cgroup, with the object tree on ext4 over dm-delay (10ms) to model
slow/congested storage. 20 concurrent compilers charge into one memcg
and enter direct reclaim at the same time, while dirty object files
stretch the isolate->shrink->putback window - the workload shape
too_many_isolated() was invented for. Each kernel runs 1800s per
pressure level; the headline metric is the number of objects actually
compiled in that window (a full round is ~3,800 objects).
memory.max = 1024MB (extreme pressure):
unpatched patched
objects compiled 4,744 7,211 (+52%)
pgscan_direct 2,901,083,521 2,228,776,807 (-23%)
scan efficiency 0.195 0.242
workingset_refault 565,169,241 537,686,479
mm_vmscan_throttled 0 511,441 (all
VMSCAN_THROTTLE_ISOLATED)
nr_isolated peak 0 (invisible) 1,152
memory.max = 1536MB (heavy pressure):
unpatched patched
objects compiled 18,969 25,124 (+32%)
pgscan_direct 1,223,905,508 816,812,074 (-33%)
scan efficiency 0.320 0.338
workingset_refault 390,048,986 273,442,445 (-30%)
memcg charge fails 8,344,775 5,868,167 (-30%)
round wall time 199-391s 182-275s
mm_vmscan_throttled 0 503,117
nr_isolated peak 0 (invisible) 1,053
At both levels -j20 is oversubscribed (20 cc1 processes need ~3GB of
anon in total), so rounds end with a cc1 being memcg-OOM-killed on
either kernel. Normalized per completed round the kill rate is the
same (13.2 vs 13.1 at 1024MB, 5.8 vs 6.1 at 1536MB); the patched
kernel simply reaches that point sooner because less time is burned
thrashing, so it cycles through more rounds. The patches do not claim
to avoid these OOMs - the point is that the patched kernel performs
23-33% fewer futile LRU scans and gets 32-52% more build work done in
the same wall time.
memory.max = 2048MB (light pressure - no-harm control):
unpatched patched
rounds completed 6 6
avg wall time/round 271s 270s
scan efficiency 0.984 0.978
mm_vmscan_throttled 0 0
nr_isolated peak 0 (invisible) 144
When reclaim keeps up, the throttle never fires and both kernels
perform identically. Patch 1 still shows: the unpatched kernel reports
nr_isolated=0 even while isolation is in flight, which is exactly the
blind spot it fixes.
Real workload 2: redis cache service (mglru_redis_repro.sh,
mglru_redis_shards_repro.sh)
-------------------------------------------------------------
A redis-server serving a 1KB-value dataset larger than its 96MB
cgroup, with swap on dm-delay (50ms): first as a single instance
(100k keys, 200 clients), then as 8 shards with AOF persistence on a
second dm-delay device sharing one cgroup (8 concurrent chargers).
The bottleneck here is swap-in I/O serialized on redis's single
command thread, not isolation contention: charge-induced reclaim is a
trickle (~400 scans/s vs 10^5-10^6 scans/s in the build runs),
isolation never piles up, and the throttle correctly stays dormant on
both kernels:
unpatched patched
single instance 41.6 ops/s 41.2 ops/s
(avg 2999ms) (avg 2999ms)
8 shards aggregate 333-419 ops/s 310-358 ops/s
(mean 376, (mean 335,
4 runs) (*) 4 runs)
mm_vmscan_throttled 0 0 (all runs)
nr_isolated peak 0 0
(*) the aggregate is dominated by a preload-end residency lottery
(which shards' datasets happen to stay hot) and swings by up to 26%
between runs of the SAME kernel; with 4 runs per kernel the
between-kernel difference of the means is within that noise (Welch
t=1.95, p~0.11). The throttle never fired in any patched run (the
tracepoint covers preload as well), so both kernels execute the same
reclaim path here, and the kernel-side counters agree across all 8
runs: steal/scan 0.59-0.63, swap-ins == refaults, zero OOM. These
runs document the no-impact side: the check is dormant unless
isolation actually piles up.
Synthetic reproducers (unchanged from v5)
-----------------------------------------
mglru_iso_repro.sh (64 threads, 48MB working set, 60s):
Drives concurrent direct reclaim inside the memcg and measures scan
efficiency, throttle events, in-flight isolation and throughput.
Neither kernel OOMs at this concurrency; the value of the patch shows
in reclaim quality:
unpatched patched
OOM kills 0 0
mm_vmscan_throttled 0 35645 (all
VMSCAN_THROTTLE_ISOLATED)
nr_isolated peak 0 (invisible) 230
total touches 246,499,132,369 301,456,426,692
scan efficiency 0.0261 0.0194
The patched kernel completes ~22% more work in the same 60s: the
throttle keeps concurrent reclaimers from trampling the same
generation, so less CPU is burned in reclaim. Note nr_isolated is
always 0 on the unpatched kernel - the over-isolation is invisible
there, which is exactly what patch 1 fixes.
mglru_iso_repro_v2.sh (192 threads, 48MB working set, 60s):
Raises concurrency to the point where over-isolation becomes fatal:
unpatched patched
OOM kills 1 (task killed) 0
memcg oom events 51 0
mm_vmscan_throttled 0 512292 (all
VMSCAN_THROTTLE_ISOLATED)
total touches 0 (killed) 682,300,000,972
With 192 threads the unpatched kernel cannot keep reclaim ahead of
allocation and the task is OOM-killed; the patched kernel survives the
full run and keeps reclaim making progress.
Changelog:
v6:
According to the comments of Barry, add real-world workload tests:
a CI-style kernel build (repeated "make -j20 vmlinux") inside a
memory-limited cgroup with its object tree on dm-delay-backed storage,
measured at three pressure levels (1024/1536/2048MB), plus redis-based
cache service runs (single instance and 8 shards sharing one cgroup)
as no-impact data points. New and updated test scripts are in the
gist. Rebased onto the latest mm-stable; patch 2 is adapted to the
current isolate_folios() type-selection context (is_single_type_
reclaim()), with no semantic change. Re-run the tests and update the
test logs.
v5:
According to the comments of Kairun and feedback from the test scripts,
make the throttle check per lruvec (new nr_isolated counter) and skip
empty types in the allowed mask.
v4:
According to the commens of Baolin and Barry, rework patch 2:
drop the throttle_is_throttled() helper extracted
from shrink_inactive_list() and leave the legacy path untouched.
The new MGLRU-only throttle_evictable_types() only sleeps when all
evictable types are over-isolated - v3 throttled as soon as any of
them was, which unnecessarily blocked the reclaim of the other type
and passes the mask of the remaining types to isolate_folios(),
which restricts both its initial choice and its fallback, so that
isolation never lands on a throttled type. The v3 gate did not
constrain the type actually isolated, so the fallback could still
pick the over-isolated one.
Re-run the tests and update the test log.
v3:
According to the commens of Baolin, remove the redundant nr_isolated
check before restoring the NR_ISOLATED_* counters in evict_folios().
rename the extracted helper to throttle_is_throttled() to avoid
confusion with the existing wake_throttle_isolated() naming space.
Use for_each_evictable_type() in the MGLRU throttle to check each
evictable type's isolation instead of only the type returned by
get_type_to_scan(), since isolate_folios() may fall back to the
other type.
Re-run the tests and update the test log.
v2:
According to the commens of Kairun, Rebased on mm-unstable.
Split into two patches; patch 2 is new and adds the
too_many_isolated() throttling to the MGLRU eviction path, which v1
did not cover.
Add test infomations.
Hui Zhu (2):
mm/vmscan: fix missing NR_ISOLATED counter update in MGLRU reclaim
path
mm/vmscan: throttle MGLRU eviction when isolated folios pile up
include/linux/mmzone.h | 2 +
mm/vmscan.c | 174 ++++++++++++++++++++++++++++++++++++++---
2 files changed, 163 insertions(+), 13 deletions(-)
--
2.43.0
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH v6 1/2] mm/vmscan: fix missing NR_ISOLATED counter update in MGLRU reclaim path 2026-10-14 9:00 [PATCH v6 0/2] mm/vmscan: fix NR_ISOLATED accounting and throttle MGLRU eviction Hui Zhu @ 2026-10-14 9:00 ` Hui Zhu 2026-10-14 9:00 ` [PATCH v6 2/2] mm/vmscan: throttle MGLRU eviction when isolated folios pile up Hui Zhu 2026-10-14 23:09 ` [PATCH v6 0/2] mm/vmscan: fix NR_ISOLATED accounting and throttle MGLRU eviction Andrew Morton 2 siblings, 0 replies; 6+ messages in thread From: Hui Zhu @ 2026-10-14 9:00 UTC (permalink / raw) To: Andrew Morton, Kairui Song, Qi Zheng, Shakeel Butt, Barry Song, Axel Rasmussen, Yuanchu Xie, Wei Xu, Johannes Weiner, David Hildenbrand, Michal Hocko, Lorenzo Stoakes, Baolin Wang, linux-mm, linux-kernel Cc: Hui Zhu From: Hui Zhu <zhuhui@kylinos.cn> MGLRU evict_folios() isolates folios from the LRU without updating the NR_ISOLATED_ANON/FILE counters, unlike the legacy shrink_inactive_list() path. This causes compaction's too_many_isolated() check to under-count isolated pages when MGLRU reclaim is active. Add NR_ISOLATED counter updates in evict_folios(): increment after isolate_folios() and decrement after all retry passes complete, using the existing nr_isolated which holds the original isolated count. Signed-off-by: Hui Zhu <zhuhui@kylinos.cn> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com> Reviewed-by: Barry Song <baohua@kernel.org> --- mm/vmscan.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/mm/vmscan.c b/mm/vmscan.c index 245f68c75b28..3bd604204ec7 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4982,6 +4982,9 @@ static int evict_folios(unsigned long nr_to_scan, struct lruvec *lruvec, scanned = isolate_folios(nr_to_scan, lruvec, sc, swappiness, &list, &isolated, &type, &type_scanned); nr_isolated = isolated; + if (nr_isolated) + __mod_node_page_state(pgdat, NR_ISOLATED_ANON + type, + nr_isolated); /* Scanning may have emptied the oldest gen, flush it */ if (scanned) @@ -5044,6 +5047,8 @@ static int evict_folios(unsigned long nr_to_scan, struct lruvec *lruvec, goto retry; } + mod_node_page_state(pgdat, NR_ISOLATED_ANON + type, -nr_isolated); + if (nr_isolated > total_reclaimed) mod_lruvec_state(lruvec, PGROTATE_ANON + type, nr_isolated - total_reclaimed); -- 2.43.0 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v6 2/2] mm/vmscan: throttle MGLRU eviction when isolated folios pile up 2026-10-14 9:00 [PATCH v6 0/2] mm/vmscan: fix NR_ISOLATED accounting and throttle MGLRU eviction Hui Zhu 2026-10-14 9:00 ` [PATCH v6 1/2] mm/vmscan: fix missing NR_ISOLATED counter update in MGLRU reclaim path Hui Zhu @ 2026-10-14 9:00 ` Hui Zhu 2026-10-14 23:09 ` [PATCH v6 0/2] mm/vmscan: fix NR_ISOLATED accounting and throttle MGLRU eviction Andrew Morton 2 siblings, 0 replies; 6+ messages in thread From: Hui Zhu @ 2026-10-14 9:00 UTC (permalink / raw) To: Andrew Morton, Kairui Song, Qi Zheng, Shakeel Butt, Barry Song, Axel Rasmussen, Yuanchu Xie, Wei Xu, Johannes Weiner, David Hildenbrand, Michal Hocko, Lorenzo Stoakes, Baolin Wang, linux-mm, linux-kernel Cc: Hui Zhu From: Hui Zhu <zhuhui@kylinos.cn> The legacy path throttles direct reclaim via too_many_isolated() when isolated folios pile up, but MGLRU's evict_folios() isolates folios without this check, which can lead to unnecessary swapping, thrashing and OOM. Add lru_gen_too_many_isolated() and throttle_evictable_types(), called from evict_folios() before the lruvec lock is taken since throttling sleeps. The check is per lruvec rather than node-wide: reclaimers contend per lruvec, and a node-wide check would let isolation in one memcg throttle reclaim in another. The threshold is the evictable size of the type divided by MIN_NR_GENS, approximating one generation. Since isolate_folios() picks the type from refault feedback and may fall back to the other one, throttle_evictable_types() collects the types that are not over-isolated and have evictable folios, and sleeps only when all of them are over-isolated. The mask is passed to isolate_folios() to restrict its choice and fallback, so an over-isolated type never blocks the other, and isolation never lands on a throttled or empty type. Like the legacy path, kswapd and !writeback_throttling_sane() reclaim are exempt, the threshold is raised for GFP_NOIO/GFP_NOFS callers, and a dying task fakes progress to exit reclaim quickly. Tested with threads write-touching a 48MB anonymous working set in a 16MB memcg backed by swap delayed 50ms via dm-delay. With 192 threads the task is OOM-killed without this patch and survives with it. With 64 threads both kernels survive the 60s run, and the patched kernel completes ~22% more touches (301B vs 246B). Signed-off-by: Hui Zhu <zhuhui@kylinos.cn> --- include/linux/mmzone.h | 2 + mm/vmscan.c | 171 +++++++++++++++++++++++++++++++++++++---- 2 files changed, 159 insertions(+), 14 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index c070b867e2f3..e8fc002c98c0 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -583,6 +583,8 @@ struct lru_gen_folio { /* can be modified without holding the LRU lock */ atomic_long_t evicted[NR_HIST_GENS][ANON_AND_FILE][MAX_NR_TIERS]; atomic_long_t refaulted[NR_HIST_GENS][ANON_AND_FILE][MAX_NR_TIERS]; + /* the number of folios isolated from this lruvec and being reclaimed */ + atomic_long_t nr_isolated[ANON_AND_FILE]; /* whether the multi-gen LRU is enabled */ bool enabled; /* the memcg generation this lru_gen_folio belongs to */ diff --git a/mm/vmscan.c b/mm/vmscan.c index 3bd604204ec7..2d551a1f91c7 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4232,25 +4232,34 @@ static void set_initial_priority(struct pglist_data *pgdat, struct scan_control sc->priority = clamp(priority, DEF_PRIORITY / 2, DEF_PRIORITY); } -static unsigned long lruvec_evictable_size(struct lruvec *lruvec, int swappiness) +static unsigned long lruvec_type_evictable_size(struct lruvec *lruvec, int type) { - int gen, type, zone; + int gen, zone; unsigned long seq, total = 0; struct lru_gen_folio *lrugen = &lruvec->lrugen; DEFINE_MAX_SEQ(lruvec); DEFINE_MIN_SEQ(lruvec); - for_each_evictable_type(type, swappiness) { - for (seq = min_seq[type]; seq <= max_seq; seq++) { - gen = lru_gen_from_seq(seq); - for (zone = 0; zone < MAX_NR_ZONES; zone++) - total += max(READ_ONCE(lrugen->nr_pages[gen][type][zone]), 0L); - } + for (seq = min_seq[type]; seq <= max_seq; seq++) { + gen = lru_gen_from_seq(seq); + for (zone = 0; zone < MAX_NR_ZONES; zone++) + total += max(READ_ONCE(lrugen->nr_pages[gen][type][zone]), 0L); } return total; } +static unsigned long lruvec_evictable_size(struct lruvec *lruvec, int swappiness) +{ + unsigned long total = 0; + int type; + + for_each_evictable_type(type, swappiness) + total += lruvec_type_evictable_size(lruvec, type); + + return total; +} + static bool lruvec_is_sizable(struct lruvec *lruvec, struct scan_control *sc) { unsigned long total; @@ -4913,16 +4922,129 @@ static inline bool is_single_type_reclaim(int swappiness) swappiness == SWAPPINESS_ANON_ONLY; } +/* + * The MGLRU counterpart to too_many_isolated(). + * + * too_many_isolated() compares node-wide isolated counts against + * node-wide inactive sizes. It predates per-memcg LRU lists, which + * moved isolation to per-lruvec granularity. MGLRU reclaimers contend + * per lruvec, so this check is per lruvec as well, and isolation in + * one memcg does not throttle reclaim in another. + * + * The legacy threshold, the size of the inactive list, is not usable + * here either: in MGLRU the inactive counters are only a compatibility + * shim - they jump around when swap runs out or a memcg hits its swap + * limit, and proactive aging can inflate them. Compare against the + * total number of evictable pages of the type divided by MIN_NR_GENS, + * approximating the size of one generation - the same unit the + * original MGLRU aging heuristics used. + */ +static bool lru_gen_too_many_isolated(struct lruvec *lruvec, int type, + struct scan_control *sc) +{ + unsigned long isolated, evictable; + + if (current_is_kswapd()) + return false; + + if (!writeback_throttling_sane(sc)) + return false; + + isolated = atomic_long_read(&lruvec->lrugen.nr_isolated[type]); + evictable = lruvec_type_evictable_size(lruvec, type); + + /* + * GFP_NOIO/GFP_NOFS callers are allowed to isolate more pages, so + * they won't be blocked by normal direct-reclaimers, forming a + * circular deadlock. + */ + if (gfp_has_io_fs(sc->gfp_mask)) + evictable >>= 3; + + return isolated > evictable / MIN_NR_GENS; +} + +/* + * Unlike the legacy path, where the LRU list to isolate from is known + * before isolation, isolate_folios() picks the type from the refault + * feedback and may fall back to the other one. Therefore, instead of + * throttling on a single type, collect the evictable types that do not + * have too many isolated folios, and only sleep when all of them do. + * + * Returns the mask of the types isolate_folios() may isolate from, or + * 0 if reclaim should stop. Also sets @fatal to tell the caller that + * the task received a fatal signal while waiting. + */ +static unsigned int throttle_evictable_types(struct lruvec *lruvec, + int swappiness, + struct scan_control *sc, + bool *fatal) +{ + unsigned int allowed; + bool stalled = false; + struct pglist_data *pgdat = lruvec_pgdat(lruvec); + int i; + + *fatal = false; + + for (;;) { + allowed = 0; + for_each_evictable_type(i, swappiness) { + /* + * A type with no evictable folios has nothing to + * isolate. Don't allow it, otherwise reclaim can be + * redirected to it and spin making no progress, e.g. + * when the only type with folios is over-isolated. + */ + if (!lruvec_type_evictable_size(lruvec, i)) + continue; + + if (!lru_gen_too_many_isolated(lruvec, i, sc)) + allowed |= BIT(i); + } + + if (allowed) { + /* Wake up reclaimers waiting on the isolation to go down. */ + wake_throttle_isolated(pgdat); + return allowed; + } + + /* + * All evictable types are over-isolated. Like the legacy + * path, wait once for concurrent reclaimers to put their + * isolated folios back; give up if that makes no progress. + */ + if (stalled) + return 0; + + stalled = true; + reclaim_throttle(pgdat, VMSCAN_THROTTLE_ISOLATED); + + /* We are about to die and free our memory. Return now. */ + if (fatal_signal_pending(current)) { + *fatal = true; + return 0; + } + } +} + static int isolate_folios(unsigned long nr_to_scan, struct lruvec *lruvec, struct scan_control *sc, int swappiness, - struct list_head *list, int *isolated, - int *isolate_type, int *isolate_scanned) + unsigned int allowed, struct list_head *list, + int *isolated, int *isolate_type, int *isolate_scanned) { bool type_fallback_allowed = !is_single_type_reclaim(swappiness); int type = get_type_to_scan(lruvec, swappiness); int total_scanned = 0, scanned, tier; bool tried = false; + /* + * The preferred type may have been excluded by + * throttle_evictable_types(); start from the other one. + */ + if (!(allowed & BIT(type))) + type = !type; + retry: tier = get_tier_idx(lruvec, type); scanned = scan_folios(nr_to_scan, lruvec, sc, @@ -4937,9 +5059,11 @@ static int isolate_folios(unsigned long nr_to_scan, struct lruvec *lruvec, /* * We are running out of the current reclaim type. Fall back to - * the other type if allowed. + * the other type if allowed. Only fall back to a type that is + * not throttled. */ - if (!scanned && !tried && type_fallback_allowed) { + if (!scanned && !tried && type_fallback_allowed && + (allowed & BIT(!type))) { type = !type; tried = true; type_fallback_allowed = false; @@ -4973,18 +5097,36 @@ static int evict_folios(unsigned long nr_to_scan, struct lruvec *lruvec, bool skip_retry = false; struct mem_cgroup *memcg = lruvec_memcg(lruvec); struct pglist_data *pgdat = lruvec_pgdat(lruvec); + unsigned int allowed; + bool fatal; + + allowed = throttle_evictable_types(lruvec, swappiness, sc, &fatal); + if (!allowed) { + /* + * We are about to die and free our memory. Like the legacy + * path, pretend some pages were reclaimed so reclaim + * unwinds quickly instead of looping back into the + * throttle. + */ + if (fatal) + sc->nr_reclaimed += SWAP_CLUSTER_MAX; + + return 0; + } lruvec_lock_irq(lruvec); /* In case folio deletion left empty old gens, flush them */ try_to_inc_min_seq(lruvec, swappiness); - scanned = isolate_folios(nr_to_scan, lruvec, sc, swappiness, + scanned = isolate_folios(nr_to_scan, lruvec, sc, swappiness, allowed, &list, &isolated, &type, &type_scanned); nr_isolated = isolated; - if (nr_isolated) + if (nr_isolated) { __mod_node_page_state(pgdat, NR_ISOLATED_ANON + type, nr_isolated); + atomic_long_add(nr_isolated, &lruvec->lrugen.nr_isolated[type]); + } /* Scanning may have emptied the oldest gen, flush it */ if (scanned) @@ -5047,6 +5189,7 @@ static int evict_folios(unsigned long nr_to_scan, struct lruvec *lruvec, goto retry; } + atomic_long_sub(nr_isolated, &lruvec->lrugen.nr_isolated[type]); mod_node_page_state(pgdat, NR_ISOLATED_ANON + type, -nr_isolated); if (nr_isolated > total_reclaimed) -- 2.43.0 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH v6 0/2] mm/vmscan: fix NR_ISOLATED accounting and throttle MGLRU eviction 2026-10-14 9:00 [PATCH v6 0/2] mm/vmscan: fix NR_ISOLATED accounting and throttle MGLRU eviction Hui Zhu 2026-10-14 9:00 ` [PATCH v6 1/2] mm/vmscan: fix missing NR_ISOLATED counter update in MGLRU reclaim path Hui Zhu 2026-10-14 9:00 ` [PATCH v6 2/2] mm/vmscan: throttle MGLRU eviction when isolated folios pile up Hui Zhu @ 2026-10-14 23:09 ` Andrew Morton 2026-10-15 6:24 ` Hui Zhu 2026-10-15 6:45 ` Barry Song 2 siblings, 2 replies; 6+ messages in thread From: Andrew Morton @ 2026-10-14 23:09 UTC (permalink / raw) To: Hui Zhu Cc: Kairui Song, Qi Zheng, Shakeel Butt, Barry Song, Axel Rasmussen, Yuanchu Xie, Wei Xu, Johannes Weiner, David Hildenbrand, Michal Hocko, Lorenzo Stoakes, Baolin Wang, linux-mm, linux-kernel, Hui Zhu On Wed, 14 Oct 2026 17:00:22 +0800 Hui Zhu <hui.zhu@linux.dev> wrote: > The legacy reclaim path has two mechanisms around isolated folios that > MGLRU lacks: > > 1. NR_ISOLATED_ANON/FILE counters are updated when folios are isolated > from the inactive lists. Compaction's too_many_isolated() relies on > them to decide when to back off. The MGLRU reclaim path never > updates them, so compaction cannot see MGLRU's in-flight isolation. > > 2. shrink_inactive_list() throttles direct reclaim via > too_many_isolated() when isolated folios pile up. MGLRU's > evict_folios() isolates folios without any such check, so many > concurrent reclaimers can over-isolate the same (oldest) generation, > leading to unnecessary swapping, thrashing and premature memcg OOM. > > Patch 1 fixes the counter accounting in the MGLRU isolation path. > Patch 2 adds a per-lruvec throttle, mirroring the legacy behavior but > adapted to MGLRU's per-lruvec contention and its dynamic type > selection/fallback. Thanks. Questions for yourself and maintainers: [1/2] is nice and simple and reviewed. How about I queue this for second week of the upcoming merge window? [2/2] is much more code, isn't reviewed and has a Sashiko whine[*]. So I'm thinking this is 7.4-rcX material. [*] https://sashiko.dev/#/patchset/cover.1791968227.git.zhuhui@kylinos.cn ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v6 0/2] mm/vmscan: fix NR_ISOLATED accounting and throttle MGLRU eviction 2026-10-14 23:09 ` [PATCH v6 0/2] mm/vmscan: fix NR_ISOLATED accounting and throttle MGLRU eviction Andrew Morton @ 2026-10-15 6:24 ` Hui Zhu 2026-10-15 6:45 ` Barry Song 1 sibling, 0 replies; 6+ messages in thread From: Hui Zhu @ 2026-10-15 6:24 UTC (permalink / raw) To: Andrew Morton Cc: Kairui Song, Qi Zheng, Shakeel Butt, Barry Song, Axel Rasmussen, Yuanchu Xie, Wei Xu, Johannes Weiner, David Hildenbrand, Michal Hocko, Lorenzo Stoakes, Baolin Wang, linux-mm, linux-kernel, Hui Zhu 在 2026/10/15 07:09, Andrew Morton: > On Wed, 14 Oct 2026 17:00:22 +0800 Hui Zhu <hui.zhu@linux.dev> wrote: > >> The legacy reclaim path has two mechanisms around isolated folios that >> MGLRU lacks: >> >> 1. NR_ISOLATED_ANON/FILE counters are updated when folios are isolated >> from the inactive lists. Compaction's too_many_isolated() relies on >> them to decide when to back off. The MGLRU reclaim path never >> updates them, so compaction cannot see MGLRU's in-flight isolation. >> >> 2. shrink_inactive_list() throttles direct reclaim via >> too_many_isolated() when isolated folios pile up. MGLRU's >> evict_folios() isolates folios without any such check, so many >> concurrent reclaimers can over-isolate the same (oldest) generation, >> leading to unnecessary swapping, thrashing and premature memcg OOM. >> >> Patch 1 fixes the counter accounting in the MGLRU isolation path. >> Patch 2 adds a per-lruvec throttle, mirroring the legacy behavior but >> adapted to MGLRU's per-lruvec contention and its dynamic type >> selection/fallback. > Thanks. Questions for yourself and maintainers: > > [1/2] is nice and simple and reviewed. How about I queue this for > second week of the upcoming merge window? > > [2/2] is much more code, isn't reviewed and has a Sashiko whine[*]. So > I'm thinking this is 7.4-rcX material. > > > [*] https://sashiko.dev/#/patchset/cover.1791968227.git.zhuhui@kylinos.cn > Hi Andrew, Sounds good to me. Thanks. Best, Hui ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v6 0/2] mm/vmscan: fix NR_ISOLATED accounting and throttle MGLRU eviction 2026-10-14 23:09 ` [PATCH v6 0/2] mm/vmscan: fix NR_ISOLATED accounting and throttle MGLRU eviction Andrew Morton 2026-10-15 6:24 ` Hui Zhu @ 2026-10-15 6:45 ` Barry Song 1 sibling, 0 replies; 6+ messages in thread From: Barry Song @ 2026-10-15 6:45 UTC (permalink / raw) To: Andrew Morton Cc: Hui Zhu, Kairui Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie, Wei Xu, Johannes Weiner, David Hildenbrand, Michal Hocko, Lorenzo Stoakes, Baolin Wang, linux-mm, linux-kernel, Hui Zhu On Thu, Oct 15, 2026 at 7:09 AM Andrew Morton <akpm@linux-foundation.org> wrote: > > On Wed, 14 Oct 2026 17:00:22 +0800 Hui Zhu <hui.zhu@linux.dev> wrote: > > > The legacy reclaim path has two mechanisms around isolated folios that > > MGLRU lacks: > > > > 1. NR_ISOLATED_ANON/FILE counters are updated when folios are isolated > > from the inactive lists. Compaction's too_many_isolated() relies on > > them to decide when to back off. The MGLRU reclaim path never > > updates them, so compaction cannot see MGLRU's in-flight isolation. > > > > 2. shrink_inactive_list() throttles direct reclaim via > > too_many_isolated() when isolated folios pile up. MGLRU's > > evict_folios() isolates folios without any such check, so many > > concurrent reclaimers can over-isolate the same (oldest) generation, > > leading to unnecessary swapping, thrashing and premature memcg OOM. > > > > Patch 1 fixes the counter accounting in the MGLRU isolation path. > > Patch 2 adds a per-lruvec throttle, mirroring the legacy behavior but > > adapted to MGLRU's per-lruvec contention and its dynamic type > > selection/fallback. > > Thanks. Questions for yourself and maintainers: > > [1/2] is nice and simple and reviewed. How about I queue this for > second week of the upcoming merge window? > +1 for this. > [2/2] is much more code, isn't reviewed and has a Sashiko whine[*]. So > I'm thinking this is 7.4-rcX material. > > > [*] https://sashiko.dev/#/patchset/cover.1791968227.git.zhuhui@kylinos.cn Best Regards Barry ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2026-10-15 6:45 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-10-14 9:00 [PATCH v6 0/2] mm/vmscan: fix NR_ISOLATED accounting and throttle MGLRU eviction Hui Zhu 2026-10-14 9:00 ` [PATCH v6 1/2] mm/vmscan: fix missing NR_ISOLATED counter update in MGLRU reclaim path Hui Zhu 2026-10-14 9:00 ` [PATCH v6 2/2] mm/vmscan: throttle MGLRU eviction when isolated folios pile up Hui Zhu 2026-10-14 23:09 ` [PATCH v6 0/2] mm/vmscan: fix NR_ISOLATED accounting and throttle MGLRU eviction Andrew Morton 2026-10-15 6:24 ` Hui Zhu 2026-10-15 6:45 ` Barry Song
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.