* [RFC PATCH 1/4] mm: mglru: only fall back when reclaim is running at high priority
2026-07-26 1:29 [RFC PATCH 0/4] mm: mglru: fix swappiness behavior Barry Song (Xiaomi)
@ 2026-07-26 1:29 ` Barry Song (Xiaomi)
2026-07-26 1:29 ` [RFC PATCH 2/4] mm: mglru: do try_to_inc_min_seq if scanned==0 Barry Song (Xiaomi)
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Barry Song (Xiaomi) @ 2026-07-26 1:29 UTC (permalink / raw)
To: akpm, linux-mm
Cc: hannes, david, mhocko, qi.zheng, shakeel.butt, ljs, kasong,
axelrasmussen, yuanchu, weixugc, linux-kernel, lyugaofei,
stevensd, Barry Song (Xiaomi)
Respect the type selected by positive_ctrl_err(), where swappiness
controls the relative weights of SP and PV. Falling back too readily
violates that bias.
Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
---
mm/vmscan.c | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 566c4e837c7d..f79402760776 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -4855,9 +4855,12 @@ static int isolate_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
* If scanned > 0 and isolated == 0, avoid falling back to the
* other type, as this type remains sufficient. Falling back
* too readily can disrupt the positive_ctrl_err() bias.
+ * Only fall back when reclaim is running at high priority.
*/
- if (!scanned)
- type = !type;
+ if (!scanned) {
+ if (!sc->priority)
+ type = !type;
+ }
}
return total_scanned;
--
2.39.3 (Apple Git-146)
^ permalink raw reply related [flat|nested] 6+ messages in thread* [RFC PATCH 2/4] mm: mglru: do try_to_inc_min_seq if scanned==0
2026-07-26 1:29 [RFC PATCH 0/4] mm: mglru: fix swappiness behavior Barry Song (Xiaomi)
2026-07-26 1:29 ` [RFC PATCH 1/4] mm: mglru: only fall back when reclaim is running at high priority Barry Song (Xiaomi)
@ 2026-07-26 1:29 ` Barry Song (Xiaomi)
2026-07-26 1:29 ` [RFC PATCH 3/4] mm: mglru: run aging when pages are severely imbalanced across gens Barry Song (Xiaomi)
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Barry Song (Xiaomi) @ 2026-07-26 1:29 UTC (permalink / raw)
To: akpm, linux-mm
Cc: hannes, david, mhocko, qi.zheng, shakeel.butt, ljs, kasong,
axelrasmussen, yuanchu, weixugc, linux-kernel, lyugaofei,
stevensd, Barry Song (Xiaomi)
If scanned == 0, we are likely running out of folios in the
reclaimable gens. Advance min_seq to keep it in sync with the
current reclaimable state.
Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
---
mm/vmscan.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/mm/vmscan.c b/mm/vmscan.c
index f79402760776..4f3a375de86e 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -4858,6 +4858,7 @@ static int isolate_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
* Only fall back when reclaim is running at high priority.
*/
if (!scanned) {
+ try_to_inc_min_seq(lruvec, swappiness);
if (!sc->priority)
type = !type;
}
--
2.39.3 (Apple Git-146)
^ permalink raw reply related [flat|nested] 6+ messages in thread* [RFC PATCH 3/4] mm: mglru: run aging when pages are severely imbalanced across gens
2026-07-26 1:29 [RFC PATCH 0/4] mm: mglru: fix swappiness behavior Barry Song (Xiaomi)
2026-07-26 1:29 ` [RFC PATCH 1/4] mm: mglru: only fall back when reclaim is running at high priority Barry Song (Xiaomi)
2026-07-26 1:29 ` [RFC PATCH 2/4] mm: mglru: do try_to_inc_min_seq if scanned==0 Barry Song (Xiaomi)
@ 2026-07-26 1:29 ` Barry Song (Xiaomi)
2026-07-26 1:29 ` [RFC PATCH 4/4] mm: mglru: run aging if the preferred type has no folios in reclaimable gens Barry Song (Xiaomi)
2026-07-26 3:41 ` [RFC PATCH 0/4] mm: mglru: fix swappiness behavior Barry Song
4 siblings, 0 replies; 6+ messages in thread
From: Barry Song (Xiaomi) @ 2026-07-26 1:29 UTC (permalink / raw)
To: akpm, linux-mm
Cc: hannes, david, mhocko, qi.zheng, shakeel.butt, ljs, kasong,
axelrasmussen, yuanchu, weixugc, linux-kernel, lyugaofei,
stevensd, Barry Song
From: lyugaofei <lyugaofei@xiaomi.com>
This partially restores the reclaim behavior introduced in Yu
Zhao's initial MGLRU commit, ac35a4902370 ("mm: multi-gen LRU:
minimal implementation"):
/*
* It's also ideal to spread pages out evenly, i.e., 1/(MIN_NR_GENS+1)
* of the total number of pages for each generation. A reasonable range
* for this average portion is [1/MIN_NR_GENS, 1/(MIN_NR_GENS+2)]. The
* aging cares about the upper bound of hot pages, while the eviction
* cares about the lower bound of cold pages.
*/
if (young * MIN_NR_GENS > total)
return true;
if (old * (MIN_NR_GENS + 2) < total)
return true;
But with a stricter condition: the youngest gen must have at
least four times as many folios as the oldest. This helps keep
folios of the preferred type distributed across the reclaimable
gens.
Signed-off-by: lyugaofei <lyugaofei@xiaomi.com>
Co-developed-by: Barry Song (Xiaomi) <baohua@kernel.org>
Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
---
mm/vmscan.c | 34 +++++++++++++++++++++++++++++++++-
1 file changed, 33 insertions(+), 1 deletion(-)
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 4f3a375de86e..9aac4febc206 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -4963,9 +4963,37 @@ static int evict_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
return scanned;
}
+static bool lru_gen_imbalanced(struct lruvec *lruvec, int type,
+ unsigned long max_seq, unsigned long min_seq,
+ int swappiness)
+{
+ struct lru_gen_folio *lrugen = &lruvec->lrugen;
+ unsigned long young = 0, old = 0, seq;
+
+ if (swappiness == MIN_SWAPPINESS || swappiness > MAX_SWAPPINESS)
+ return false;
+
+ for (seq = min_seq; seq <= max_seq; seq++) {
+ int gen = lru_gen_from_seq(seq);
+ unsigned long size = 0;
+ int zone;
+
+ gen = lru_gen_from_seq(seq);
+ for (zone = 0; zone < MAX_NR_ZONES; zone++)
+ size += max(READ_ONCE(lrugen->nr_pages[gen][type][zone]), 0L);
+
+ if (seq + MIN_NR_GENS > max_seq)
+ young += size;
+ else
+ old += size;
+ }
+ return young > old * 4;
+}
+
static bool should_run_aging(struct lruvec *lruvec, unsigned long max_seq,
struct scan_control *sc, int swappiness)
{
+ int type = get_type_to_scan(lruvec, swappiness);
DEFINE_MIN_SEQ(lruvec);
/* have to run aging, since eviction is not possible anymore */
@@ -4977,7 +5005,11 @@ static bool should_run_aging(struct lruvec *lruvec, unsigned long max_seq,
return false;
/* better to run aging even though eviction is still possible */
- return evictable_min_seq(min_seq, swappiness) + MIN_NR_GENS == max_seq;
+ if (evictable_min_seq(min_seq, swappiness) + MIN_NR_GENS == max_seq)
+ return true;
+
+ /* Run aging if the preferred type is severely imbalanced across gens */
+ return lru_gen_imbalanced(lruvec, type, max_seq, min_seq[type], swappiness);
}
static long get_nr_to_scan(struct lruvec *lruvec, struct scan_control *sc,
--
2.39.3 (Apple Git-146)
^ permalink raw reply related [flat|nested] 6+ messages in thread* [RFC PATCH 4/4] mm: mglru: run aging if the preferred type has no folios in reclaimable gens
2026-07-26 1:29 [RFC PATCH 0/4] mm: mglru: fix swappiness behavior Barry Song (Xiaomi)
` (2 preceding siblings ...)
2026-07-26 1:29 ` [RFC PATCH 3/4] mm: mglru: run aging when pages are severely imbalanced across gens Barry Song (Xiaomi)
@ 2026-07-26 1:29 ` Barry Song (Xiaomi)
2026-07-26 3:41 ` [RFC PATCH 0/4] mm: mglru: fix swappiness behavior Barry Song
4 siblings, 0 replies; 6+ messages in thread
From: Barry Song (Xiaomi) @ 2026-07-26 1:29 UTC (permalink / raw)
To: akpm, linux-mm
Cc: hannes, david, mhocko, qi.zheng, shakeel.butt, ljs, kasong,
axelrasmussen, yuanchu, weixugc, linux-kernel, lyugaofei,
stevensd, Barry Song (Xiaomi)
Respect the type selected by positive_ctrl_err(). If there are no
reclaimable gens left for that type, run aging.
Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
---
mm/vmscan.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 9aac4febc206..be34b8b946ad 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -5000,6 +5000,10 @@ static bool should_run_aging(struct lruvec *lruvec, unsigned long max_seq,
if (evictable_min_seq(min_seq, swappiness) + MIN_NR_GENS > max_seq)
return true;
+ /* run aging if the preferred type is exhausted */
+ if (min_seq[type] + MIN_NR_GENS > max_seq)
+ return true;
+
/* try to avoid aging, do gentle reclaim at the default priority */
if (sc->priority == DEF_PRIORITY)
return false;
--
2.39.3 (Apple Git-146)
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [RFC PATCH 0/4] mm: mglru: fix swappiness behavior
2026-07-26 1:29 [RFC PATCH 0/4] mm: mglru: fix swappiness behavior Barry Song (Xiaomi)
` (3 preceding siblings ...)
2026-07-26 1:29 ` [RFC PATCH 4/4] mm: mglru: run aging if the preferred type has no folios in reclaimable gens Barry Song (Xiaomi)
@ 2026-07-26 3:41 ` Barry Song
4 siblings, 0 replies; 6+ messages in thread
From: Barry Song @ 2026-07-26 3:41 UTC (permalink / raw)
To: akpm, linux-mm
Cc: hannes, david, mhocko, qi.zheng, shakeel.butt, ljs, kasong,
axelrasmussen, yuanchu, weixugc, linux-kernel, lyugaofei,
stevensd
On Sun, Jul 26, 2026 at 9:29 AM Barry Song (Xiaomi) <baohua@kernel.org> wrote:
>
> The active/inactive LRU respects swappiness well. Anonymous page
> scanning and reclamation increase roughly linearly with
> swappiness, while file page scanning and reclamation decrease
> accordingly.
> For example, when swappiness reaches 200, both pgsteal_file and
> pgscan_file drop to zero while building the kernel in a 1 GB
> memcg. In contrast, MGLRU shows almost no change across different
> swappiness values.
>
> pgsteal_file
>
> Swappiness LRU MGLRU
> --------------------------------
> 1 10567455 763612
> 36 990706 480205
> 71 688170 415848
> 106 446294 386164
> 141 286307 359196
> 176 201733 351686
> 200 0 330093
>
>
> pgsteal_anon
>
> Swappiness LRU MGLRU
> --------------------------------
> 1 4410548 2726362
> 36 2465268 2762859
> 71 2677908 2885124
> 106 2737227 2841796
> 141 2984276 3035015
> 176 3381338 2938302
> 200 13116359 3113499
>
>
> pgscan_file
>
> Swappiness LRU MGLRU
> --------------------------------
> 1 17997223 923094
> 36 1325674 539571
> 71 852345 464222
> 106 538207 464477
> 141 357253 412277
> 176 217536 399446
> 200 0 375902
>
>
> pgscan_anon
>
> Swappiness LRU MGLRU
> --------------------------------
> 1 31639423 5987136
> 36 23441521 5753224
> 71 26067110 6101780
> 106 25619448 5782919
> 141 26842088 6234264
> 176 29200021 5980292
> 200 62193924 6413125
>
> This patchset respects the type selected by positive_ctrl_err(),
> which uses swappiness as its gain. It does so by:
>
> 1. Avoiding premature fallback to the other type. Only fall back
> when reclaim is running at high priority.
>
> 2. Running aging when the preferred type has few or no
> reclaimable folios, so more folios of that type become
> reclaimable.
>
> With this patchset, swappiness starts to behave similarly to the
> active/inactive LRU.
>
> pgsteal_file
>
> Swappiness LRU MGLRU MGLRU+Patch
> -------------------------------------------------
> 1 10567455 763612 3507619
> 36 990706 480205 528781
> 71 688170 415848 478478
> 106 446294 386164 411632
> 141 286307 359196 375137
> 176 201733 351686 298586
> 200 0 330093 0
>
>
> pgsteal_anon
>
> Swappiness LRU MGLRU MGLRU+Patch
> -------------------------------------------------
> 1 4410548 2726362 2876888
> 36 2465268 2762859 3022062
> 71 2677908 2885124 3129555
> 106 2737227 2841796 2974192
> 141 2984276 3035015 3092280
> 176 3381338 2938302 3353395
> 200 13116359 3113499 14348518
>
>
> pgscan_file
>
> Swappiness LRU MGLRU MGLRU+Patch
> -------------------------------------------------
> 1 17997223 923094 5842416
> 36 1325674 539571 648111
> 71 852345 464222 569930
> 106 538207 464477 480767
> 141 357253 412277 431911
> 176 217536 399446 337768
> 200 0 375902 0
>
>
> pgscan_anon
>
> Swappiness LRU MGLRU MGLRU+Patch
> -------------------------------------------------
> 1 31639423 5987136 4658593
> 36 23441521 5753224 5444726
> 71 26067110 6101780 5755804
> 106 25619448 5782919 5872415
> 141 26842088 6234264 6144096
> 176 29200021 5980292 6615500
> 200 62193924 6413125 22984986
>
> Another possible approach is to decouple anonymous and file-backed
> aging by maintaining separate max_seq values for each type. This
> allows anonymous and file-backed memory to age and be reclaimed
> independently, enabling the swappiness-preferred type to be reclaimed
> more aggressively while allowing the other type to lag behind. This
> approach has already been adopted by projects such as CachyOS [1] and
> Chromium [2].
>
> However, this approach requires substantial changes to MGLRU and
> fundamentally alters its design by breaking the shared aging timeline
> between anonymous and file-backed memory. This timeline is the
> foundation for mechanisms such as the PID controller and the
> min_ttl_ms thrashing protection.
>
> That is why this patchset aims to fix the swappiness behavior
> without fundamentally changing MGLRU's design, with minimal changes.
>
> [1] https://github.com/firelzrd/re-swappiness
> [2] https://chromium.googlesource.com/chromiumos/third_party/kernel/+log/929932351492d01f0aee37a0ac3be8c7bd88f80d
>
> Barry Song (Xiaomi) (3):
> mm: mglru: only fall back when reclaim is running at high priority
> mm: mglru: do try_to_inc_min_seq if scanned==0
> mm: mglru: run aging if the preferred type has no folios in
> reclaimable gens
>
> lyugaofei (1):
> mm: mglru: run aging when pages are severely imbalanced across gens
I checked Sashiki's comment:
https://sashiko.dev/#/patchset/20260726012946.18684-1-baohua@kernel.org
1. Sashiko says
"
> @@ -4858,6 +4858,7 @@ static int isolate_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
> * Only fall back when reclaim is running at high priority.
> */
> if (!scanned) {
> + try_to_inc_min_seq(lruvec, swappiness);
Is this call to try_to_inc_min_seq() redundant?
The caller evict_folios() acquires the LRU lock and calls
try_to_inc_min_seq() just before calling isolate_folios().
If scanned == 0 on the first iteration of the loop in isolate_folios(),
the lists haven't changed since the previous try_to_inc_min_seq() call.
This means get_nr_gens == MIN_NR_GENS, making this new call a strict
no-op.
If scanned == 0 happens on the second iteration (because the first
iteration processed folios without isolating any), try_to_inc_min_seq()
might advance min_seq. However, the for_each_evictable_type() loop
only iterates twice, so it terminates immediately without scanning the
newly advanced generation.
After isolate_folios() returns, evict_folios() unconditionally calls
try_to_inc_min_seq() again anyway. Does this logic successfully retry
with new folios, or does it only perform unnecessary work?
"
I observed that scan_folios() always returns 0 because the list is
empty, making the following early-return check appear to be
ineffective:
if (get_nr_gens(lruvec, type) == MIN_NR_GENS)
return 0;
So I thought min_seq should be advanced when scanned == 0, since
there are no folios left in the reclaimable generation.
After taking another look, I realized that isn't the real issue.
The problem is that try_to_inc_min_seq() doesn't always reflect the
actual oldest reclaimable generation. It can leave min_seq pointing
to an older generation even after that generation has no folios
remaining:
static void try_to_inc_min_seq(struct lruvec *lruvec, int swappiness)
{
...
/* see the comment on lru_gen_folio */
if (swappiness && swappiness <= MAX_SWAPPINESS) {
unsigned long seq = lrugen->max_seq - MIN_NR_GENS;
if (min_seq[LRU_GEN_ANON] > seq && min_seq[LRU_GEN_FILE] < seq)
min_seq[LRU_GEN_ANON] = seq;
else if (min_seq[LRU_GEN_FILE] > seq &&
min_seq[LRU_GEN_ANON] < seq)
min_seq[LRU_GEN_FILE] = seq;
}
}
I think the function itself should be fixed instead. Otherwise, I can
leave it as is and simply drop the newly added call to
try_to_inc_min_seq().
2. Sashiko says
"
> + gen = lru_gen_from_seq(seq);
This isn't a bug, but is this second assignment of gen redundant since it was
already initialized to the exact same value in its declaration?
"
This is right. I will fix it in RFC v2.
3. Sashiko says
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index 566c4e837c7d5..f794027607767 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -4855,9 +4855,12 @@ static int isolate_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
> * If scanned > 0 and isolated == 0, avoid falling back to the
> * other type, as this type remains sufficient. Falling back
> * too readily can disrupt the positive_ctrl_err() bias.
> + * Only fall back when reclaim is running at high priority.
> */
> - if (!scanned)
> - type = !type;
> + if (!scanned) {
Does this logic completely prevent kswapd from falling back to the other LRU
type?
Looking at kswapd's reclaim loop in balance_pgdat(), it explicitly exits
before sc.priority reaches 0:
mm/vmscan.c:balance_pgdat() {
...
if (raise_priority || !nr_reclaimed)
sc.priority--;
} while (sc.priority >= 1);
Because sc.priority never reaches 0 inside the kswapd reclaim path, kswapd
will never satisfy this fallback condition. It will repeatedly scan the
empty type, fail to make progress, drop its priority, and eventually give up
and sleep.
For direct reclaim, this also means processes will stall fruitlessly through
multiple priority drops (triggering VMSCAN_THROTTLE_NOPROGRESS delays)
before finally reaching priority 0 and falling back, which could lead to
severe stalling or premature OOM killer invocation.
> + if (!sc->priority)
> + type = !type;
> + }
> }
This seems to be a valid concern. I could address it by:
diff --git a/mm/vmscan.c b/mm/vmscan.c
index be34b8b946ad..d69a0ed26f8c 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -4859,7 +4859,7 @@ static int isolate_folios(unsigned long
nr_to_scan, struct lruvec *lruvec,
*/
if (!scanned) {
try_to_inc_min_seq(lruvec, swappiness);
- if (!sc->priority)
+ if (sc->priority <= 2)
type = !type;
}
}
4. Sashiko says
"> But with a stricter condition: the youngest gen must have at
> least four times as many folios as the oldest. This helps keep
> folios of the preferred type distributed across the reclaimable
> gens.
Is the commit message description accurate regarding "the youngest gen"?
Looking at lru_gen_imbalanced(), the code checks:
if (seq + MIN_NR_GENS > max_seq)
Since MIN_NR_GENS is 2, wouldn't this condition group the two youngest
generations into the young variable rather than just the single youngest
generation?"
Yes, I'll update the description.
5. Sashiko says
"
> +static bool lru_gen_imbalanced(struct lruvec *lruvec, int type,
> + unsigned long max_seq, unsigned long min_seq,
> + int swappiness)
> +{
> + struct lru_gen_folio *lrugen = &lruvec->lrugen;
> + unsigned long young = 0, old = 0, seq;
> +
> + if (swappiness == MIN_SWAPPINESS || swappiness > MAX_SWAPPINESS)
> + return false;
Does this unexpectedly disable the imbalance heuristic for systems where
swappiness is configured to 0?
When swappiness is 0 (MIN_SWAPPINESS), get_type_to_scan() correctly selects
LRU_GEN_FILE, but this check causes an immediate return of false. Wouldn't
file pages still need to be balanced across generations on these systems?
"
Technically, we should maintain the balance for swappiness values of
0 and 201 as well. However, in those cases we always reclaim a single
type, so the swappiness bias is irrelevant. We can therefore skip
this.
Best Regards
Barry
^ permalink raw reply related [flat|nested] 6+ messages in thread