* [RFC PATCH v2 1/5] mm: mglru: avoid scanning empty generations in scan_folios()
2026-07-26 12:21 [RFC PATCH v2 0/5] mm: mglru: fix swappiness behavior Barry Song (Xiaomi)
@ 2026-07-26 12:21 ` Barry Song (Xiaomi)
2026-07-26 12:21 ` [RFC PATCH v2 2/5] mm: mglru: only fall back when reclaim is running at high priority Barry Song (Xiaomi)
` (3 subsequent siblings)
4 siblings, 0 replies; 7+ messages in thread
From: Barry Song (Xiaomi) @ 2026-07-26 12:21 UTC (permalink / raw)
To: akpm, linux-mm
Cc: hannes, david, mhocko, qi.zheng, shakeel.butt, ljs, kasong,
axelrasmussen, yuanchu, weixugc, linux-kernel, lyugaofei,
stevensd, Barry Song (Xiaomi)
Commit 16b475d2ac3c ("mm/mglru: avoid reclaim type fall back when
isolation makes no progress") only falls back to the other type when
scanned == 0. However, I have frequently observed cases where
scanned > 0, but the older reclaimable generation becomes empty
after the first scan_folios(). As a result, the second
scan_folios() for the same type performs a redundant scan over an
empty generation.
We can avoid this by checking whether the reclaimable generation has
become empty when scanned < nr_to_scan and we still have fewer than
MIN_LRU_BATCH isolated folios after scan_folios().
Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
---
mm/vmscan.c | 11 +++++++----
1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 566c4e837c7d..babbce4bbfe8 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -4852,11 +4852,14 @@ static int isolate_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
break;
}
/*
- * If scanned > 0 and isolated == 0, avoid falling back to the
- * other type, as this type remains sufficient. Falling back
- * too readily can disrupt the positive_ctrl_err() bias.
+ * If scanned >= nr_to_scan or isolated >= MIN_LRU_BATCH,
+ * avoid falling back to the other type. The preferred
+ * type is still reclaimable; otherwise, it would have
+ * already run out of reclaimable generations. Falling
+ * back too readily can disrupt the positive_ctrl_err()
+ * bias.
*/
- if (!scanned)
+ if (scanned < nr_to_scan && *isolated < MIN_LRU_BATCH)
type = !type;
}
--
2.39.3 (Apple Git-146)
^ permalink raw reply related [flat|nested] 7+ messages in thread* [RFC PATCH v2 2/5] mm: mglru: only fall back when reclaim is running at high priority
2026-07-26 12:21 [RFC PATCH v2 0/5] mm: mglru: fix swappiness behavior Barry Song (Xiaomi)
2026-07-26 12:21 ` [RFC PATCH v2 1/5] mm: mglru: avoid scanning empty generations in scan_folios() Barry Song (Xiaomi)
@ 2026-07-26 12:21 ` Barry Song (Xiaomi)
2026-07-26 22:08 ` Barry Song
2026-07-26 12:21 ` [RFC PATCH v2 3/5] mm: mglru: prevent min_seq[type] from pointing to an empty generation Barry Song (Xiaomi)
` (2 subsequent siblings)
4 siblings, 1 reply; 7+ messages in thread
From: Barry Song (Xiaomi) @ 2026-07-26 12:21 UTC (permalink / raw)
To: akpm, linux-mm
Cc: hannes, david, mhocko, qi.zheng, shakeel.butt, ljs, kasong,
axelrasmussen, yuanchu, weixugc, linux-kernel, lyugaofei,
stevensd, Barry Song (Xiaomi)
Respect the type selected by positive_ctrl_err(), where swappiness
controls the relative weights of SP and PV. Falling back too readily
undermines that bias. Only fall back after making a sufficient effort
to reclaim from the preferred type.
Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
---
mm/vmscan.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/mm/vmscan.c b/mm/vmscan.c
index babbce4bbfe8..4a387cc4145a 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -4857,10 +4857,14 @@ static int isolate_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
* type is still reclaimable; otherwise, it would have
* already run out of reclaimable generations. Falling
* back too readily can disrupt the positive_ctrl_err()
- * bias.
+ * bias. Also, only fall back when reclaim is running at
+ * a high priority.
*/
- if (scanned < nr_to_scan && *isolated < MIN_LRU_BATCH)
+ if (scanned < nr_to_scan && *isolated < MIN_LRU_BATCH) {
+ if (sc->priority > 2)
+ break;
type = !type;
+ }
}
return total_scanned;
--
2.39.3 (Apple Git-146)
^ permalink raw reply related [flat|nested] 7+ messages in thread* Re: [RFC PATCH v2 2/5] mm: mglru: only fall back when reclaim is running at high priority
2026-07-26 12:21 ` [RFC PATCH v2 2/5] mm: mglru: only fall back when reclaim is running at high priority Barry Song (Xiaomi)
@ 2026-07-26 22:08 ` Barry Song
0 siblings, 0 replies; 7+ messages in thread
From: Barry Song @ 2026-07-26 22:08 UTC (permalink / raw)
To: akpm, linux-mm
Cc: hannes, david, mhocko, qi.zheng, shakeel.butt, ljs, kasong,
axelrasmussen, yuanchu, weixugc, linux-kernel, lyugaofei,
stevensd
On Sun, Jul 26, 2026 at 8:21 PM Barry Song (Xiaomi) <baohua@kernel.org> wrote:
>
> Respect the type selected by positive_ctrl_err(), where swappiness
> controls the relative weights of SP and PV. Falling back too readily
> undermines that bias. Only fall back after making a sufficient effort
> to reclaim from the preferred type.
>
> Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
> ---
> mm/vmscan.c | 8 ++++++--
> 1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index babbce4bbfe8..4a387cc4145a 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -4857,10 +4857,14 @@ static int isolate_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
> * type is still reclaimable; otherwise, it would have
> * already run out of reclaimable generations. Falling
> * back too readily can disrupt the positive_ctrl_err()
> - * bias.
> + * bias. Also, only fall back when reclaim is running at
> + * a high priority.
> */
> - if (scanned < nr_to_scan && *isolated < MIN_LRU_BATCH)
> + if (scanned < nr_to_scan && *isolated < MIN_LRU_BATCH) {
> + if (sc->priority > 2)
> + break;
> type = !type;
> + }
https://sashiko.dev/#/patchset/20260726122123.7614-1-baohua%40kernel.org
"> diff --git a/mm/vmscan.c b/mm/vmscan.c
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -4867,10 +4867,14 @@ static int isolate_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
[ ... ]
> - if (scanned < nr_to_scan && *isolated < MIN_LRU_BATCH)
> + if (scanned < nr_to_scan && *isolated < MIN_LRU_BATCH) {
> + if (sc->priority > 2)
> + break;
Does this logic artificially drop the scan priority and cause false memory
pressure events when the preferred LRU type is empty?
By breaking the isolation loop instead of falling back to the populated
LRU list, the direct reclaim path might falsely conclude that no pages can
be isolated at the current priority. This forces the VM to artificially drop
the scanning priority from the default priority down to 2.
This rapid priority drop could have two severe consequences. First, it
aggressively invokes shrink_slab() at artificially high priorities, causing
unnecessary eviction of active slab objects (like dentries and inodes)
which results in extreme latency spikes.
Second, when sc->priority drops to 3, vmpressure_prio() triggers a critical
memory pressure event. Userspace daemons (like lmkd or oomd) might respond
by killing background apps, falsely assuming the system is completely out
of memory, even though the non-preferred LRU list could be full of clean,
evictable pages.
> type = !type;
> + }
> }
>
> return total_scanned;"
I can change the priority to "DEF_PRIORITY - 2":
diff --git a/mm/vmscan.c b/mm/vmscan.c
index d63322cdb1bc..12eba6216fde 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -4851,7 +4851,7 @@ static int isolate_folios(unsigned long
nr_to_scan, struct lruvec *lruvec,
* a high priority.
*/
if (scanned < nr_to_scan && *isolated < MIN_LRU_BATCH) {
- if (sc->priority > 2)
+ if (sc->priority > DEF_PRIORITY - 2)
break;
type = !type;
}
And the result is still fine:
pgsteal_file
Swappiness LRU MGLRU MGLRU+Patch
-------------------------------------------------
1 10567455 763612 2191652
36 990706 480205 414483
71 688170 415848 386966
106 446294 386164 354976
141 286307 359196 348493
176 201733 351686 297113
200 0 330093 2544
pgsteal_anon
Swappiness LRU MGLRU MGLRU+Patch
-------------------------------------------------
1 4410548 2726362 3511241
36 2465268 2762859 3027445
71 2677908 2885124 3359352
106 2737227 2841796 3224762
141 2984276 3035015 3221214
176 3381338 2938302 3446206
200 13116359 3113499 14700981
pgscan_file
Swappiness LRU MGLRU MGLRU+Patch
-------------------------------------------------
1 17997223 923094 3178493
36 1325674 539571 517029
71 852345 464222 460869
106 538207 464477 411314
141 357253 412277 402429
176 217536 399446 333375
200 0 375902 3083
pgscan_anon
Swappiness LRU MGLRU MGLRU+Patch
-------------------------------------------------
1 31639423 5987136 6184980
36 23441521 5753224 5727338
71 26067110 6101780 5932826
106 25619448 5782919 6105922
141 26842088 6234264 6147981
176 29200021 5980292 6707570
200 62193924 6413125 23288983
If I drop this patch entirely, swappiness still functions:
pgsteal_file
Swappiness LRU MGLRU MGLRU+Patch
-------------------------------------------------
1 10567455 763612 2118900
36 990706 480205 426026
71 688170 415848 396647
106 446294 386164 376780
141 286307 359196 334243
176 201733 351686 287544
200 0 330093 309
pgsteal_anon
Swappiness LRU MGLRU MGLRU+Patch
-------------------------------------------------
1 4410548 2726362 3579570
36 2465268 2762859 3300899
71 2677908 2885124 3183903
106 2737227 2841796 3419872
141 2984276 3035015 3121920
176 3381338 2938302 3198154
200 13116359 3113499 14348490
pgscan_file
Swappiness LRU MGLRU MGLRU+Patch
-------------------------------------------------
1 17997223 923094 2898754
36 1325674 539571 509825
71 852345 464222 470746
106 538207 464477 434145
141 357253 412277 378242
176 217536 399446 322824
200 0 375902 694
pgscan_anon
Swappiness LRU MGLRU MGLRU+Patch
-------------------------------------------------
1 31639423 5987136 6594803
36 23441521 5753224 5979158
71 26067110 6101780 6200677
106 25619448 5782919 6480037
141 26842088 6234264 6106924
176 29200021 5980292 6553534
200 62193924 6413125 22763932
So I guess the other aging patches have a larger impact on
improving swappiness behavior than this one.
Best Regards
Barry
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [RFC PATCH v2 3/5] mm: mglru: prevent min_seq[type] from pointing to an empty generation
2026-07-26 12:21 [RFC PATCH v2 0/5] mm: mglru: fix swappiness behavior Barry Song (Xiaomi)
2026-07-26 12:21 ` [RFC PATCH v2 1/5] mm: mglru: avoid scanning empty generations in scan_folios() Barry Song (Xiaomi)
2026-07-26 12:21 ` [RFC PATCH v2 2/5] mm: mglru: only fall back when reclaim is running at high priority Barry Song (Xiaomi)
@ 2026-07-26 12:21 ` Barry Song (Xiaomi)
2026-07-26 12:21 ` [RFC PATCH v2 4/5] mm: mglru: run aging when pages are severely imbalanced across gens Barry Song (Xiaomi)
2026-07-26 12:21 ` [RFC PATCH v2 5/5] mm: mglru: run aging if the preferred type has no folios in reclaimable gens Barry Song (Xiaomi)
4 siblings, 0 replies; 7+ messages in thread
From: Barry Song (Xiaomi) @ 2026-07-26 12:21 UTC (permalink / raw)
To: akpm, linux-mm
Cc: hannes, david, mhocko, qi.zheng, shakeel.butt, ljs, kasong,
axelrasmussen, yuanchu, weixugc, linux-kernel, lyugaofei,
stevensd, Barry Song (Xiaomi)
In try_to_inc_min_seq(), min_seq[LRU_GEN_ANON] and
min_seq[LRU_GEN_FILE] can be fixed up to point to empty generations
in order to keep their gap within one. As a result, scan_folios()
may repeatedly scan empty generations. This is confusing, as I
observed scan_folios() returning 0 even though the following check in
scan_folios() doesn't take effect:
if (get_nr_gens(lruvec, type) == MIN_NR_GENS)
return 0;
There is no need to adjust min_seq[], since the reclaim logic already
triggers aging when the number of generations reaches
MIN_NR_GENS, and reclaim never reduces it below MIN_NR_GENS.
Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
---
include/linux/mmzone.h | 6 ++----
mm/vmscan.c | 10 ----------
2 files changed, 2 insertions(+), 14 deletions(-)
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index a26c8b855222..233d2006a541 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -552,10 +552,8 @@ enum {
* The youngest generation number is stored in max_seq for both anon and file
* types as they are aged on an equal footing. The oldest generation numbers are
* stored in min_seq[] separately for anon and file types so that they can be
- * incremented independently. Ideally min_seq[] are kept in sync when both anon
- * and file types are evictable. However, to adapt to situations like extreme
- * swappiness, they are allowed to be out of sync by at most
- * MAX_NR_GENS-MIN_NR_GENS-1.
+ * incremented independently. For both file and anonymous memory, the minimum
+ * generation must be at least MIN_NR_GENS.
*
* The number of pages in each generation is eventually consistent and therefore
* can be transiently negative when reset_batch_size() is pending.
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 4a387cc4145a..d6bd64b4dced 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -3972,16 +3972,6 @@ static void try_to_inc_min_seq(struct lruvec *lruvec, int swappiness)
if (!seq_inc_flag)
return;
- /* see the comment on lru_gen_folio */
- if (swappiness && swappiness <= MAX_SWAPPINESS) {
- unsigned long seq = lrugen->max_seq - MIN_NR_GENS;
-
- if (min_seq[LRU_GEN_ANON] > seq && min_seq[LRU_GEN_FILE] < seq)
- min_seq[LRU_GEN_ANON] = seq;
- else if (min_seq[LRU_GEN_FILE] > seq && min_seq[LRU_GEN_ANON] < seq)
- min_seq[LRU_GEN_FILE] = seq;
- }
-
for_each_evictable_type(type, swappiness) {
if (min_seq[type] <= lrugen->min_seq[type])
continue;
--
2.39.3 (Apple Git-146)
^ permalink raw reply related [flat|nested] 7+ messages in thread* [RFC PATCH v2 4/5] mm: mglru: run aging when pages are severely imbalanced across gens
2026-07-26 12:21 [RFC PATCH v2 0/5] mm: mglru: fix swappiness behavior Barry Song (Xiaomi)
` (2 preceding siblings ...)
2026-07-26 12:21 ` [RFC PATCH v2 3/5] mm: mglru: prevent min_seq[type] from pointing to an empty generation Barry Song (Xiaomi)
@ 2026-07-26 12:21 ` Barry Song (Xiaomi)
2026-07-26 12:21 ` [RFC PATCH v2 5/5] mm: mglru: run aging if the preferred type has no folios in reclaimable gens Barry Song (Xiaomi)
4 siblings, 0 replies; 7+ messages in thread
From: Barry Song (Xiaomi) @ 2026-07-26 12:21 UTC (permalink / raw)
To: akpm, linux-mm
Cc: hannes, david, mhocko, qi.zheng, shakeel.butt, ljs, kasong,
axelrasmussen, yuanchu, weixugc, linux-kernel, lyugaofei,
stevensd, Barry Song
From: lyugaofei <lyugaofei@xiaomi.com>
This partially restores the reclaim behavior introduced in Yu
Zhao's initial MGLRU commit, ac35a4902370 ("mm: multi-gen LRU:
minimal implementation"):
/*
* It's also ideal to spread pages out evenly, i.e., 1/(MIN_NR_GENS+1)
* of the total number of pages for each generation. A reasonable range
* for this average portion is [1/MIN_NR_GENS, 1/(MIN_NR_GENS+2)]. The
* aging cares about the upper bound of hot pages, while the eviction
* cares about the lower bound of cold pages.
*/
if (young * MIN_NR_GENS > total)
return true;
if (old * (MIN_NR_GENS + 2) < total)
return true;
But with a stricter condition: the younger generations must contain
at least four times as many folios as the older generations. This
allows aging to keep folios of the preferred type spread across the
reclaimable generations.
Signed-off-by: lyugaofei <lyugaofei@xiaomi.com>
Co-developed-by: Barry Song (Xiaomi) <baohua@kernel.org>
Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
---
mm/vmscan.c | 37 ++++++++++++++++++++++++++++++++++++-
1 file changed, 36 insertions(+), 1 deletion(-)
diff --git a/mm/vmscan.c b/mm/vmscan.c
index d6bd64b4dced..7c13dedb0b1f 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -4956,9 +4956,40 @@ static int evict_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
return scanned;
}
+static bool lru_gen_imbalanced(struct lruvec *lruvec, int type,
+ unsigned long max_seq, unsigned long min_seq,
+ int swappiness)
+{
+ struct lru_gen_folio *lrugen = &lruvec->lrugen;
+ unsigned long young = 0, old = 0, seq;
+
+ /*
+ * reclaim is forced to a single type in those cases, so there is
+ * no need to consider the swappiness bias
+ */
+ if (swappiness == MIN_SWAPPINESS || swappiness > MAX_SWAPPINESS)
+ return false;
+
+ for (seq = min_seq; seq <= max_seq; seq++) {
+ int gen = lru_gen_from_seq(seq);
+ unsigned long size = 0;
+ int zone;
+
+ for (zone = 0; zone < MAX_NR_ZONES; zone++)
+ size += max(READ_ONCE(lrugen->nr_pages[gen][type][zone]), 0L);
+
+ if (seq + MIN_NR_GENS > max_seq)
+ young += size;
+ else
+ old += size;
+ }
+ return young > old * 4;
+}
+
static bool should_run_aging(struct lruvec *lruvec, unsigned long max_seq,
struct scan_control *sc, int swappiness)
{
+ int type = get_type_to_scan(lruvec, swappiness);
DEFINE_MIN_SEQ(lruvec);
/* have to run aging, since eviction is not possible anymore */
@@ -4970,7 +5001,11 @@ static bool should_run_aging(struct lruvec *lruvec, unsigned long max_seq,
return false;
/* better to run aging even though eviction is still possible */
- return evictable_min_seq(min_seq, swappiness) + MIN_NR_GENS == max_seq;
+ if (evictable_min_seq(min_seq, swappiness) + MIN_NR_GENS == max_seq)
+ return true;
+
+ /* Run aging if the preferred type is severely imbalanced across gens */
+ return lru_gen_imbalanced(lruvec, type, max_seq, min_seq[type], swappiness);
}
static long get_nr_to_scan(struct lruvec *lruvec, struct scan_control *sc,
--
2.39.3 (Apple Git-146)
^ permalink raw reply related [flat|nested] 7+ messages in thread* [RFC PATCH v2 5/5] mm: mglru: run aging if the preferred type has no folios in reclaimable gens
2026-07-26 12:21 [RFC PATCH v2 0/5] mm: mglru: fix swappiness behavior Barry Song (Xiaomi)
` (3 preceding siblings ...)
2026-07-26 12:21 ` [RFC PATCH v2 4/5] mm: mglru: run aging when pages are severely imbalanced across gens Barry Song (Xiaomi)
@ 2026-07-26 12:21 ` Barry Song (Xiaomi)
4 siblings, 0 replies; 7+ messages in thread
From: Barry Song (Xiaomi) @ 2026-07-26 12:21 UTC (permalink / raw)
To: akpm, linux-mm
Cc: hannes, david, mhocko, qi.zheng, shakeel.butt, ljs, kasong,
axelrasmussen, yuanchu, weixugc, linux-kernel, lyugaofei,
stevensd, Barry Song (Xiaomi)
Respect the type selected by positive_ctrl_err(). If there are no
reclaimable gens left for that type, run aging.
Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
---
mm/vmscan.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 7c13dedb0b1f..d63322cdb1bc 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -4996,6 +4996,10 @@ static bool should_run_aging(struct lruvec *lruvec, unsigned long max_seq,
if (evictable_min_seq(min_seq, swappiness) + MIN_NR_GENS > max_seq)
return true;
+ /* run aging if the preferred type is exhausted */
+ if (min_seq[type] + MIN_NR_GENS > max_seq)
+ return true;
+
/* try to avoid aging, do gentle reclaim at the default priority */
if (sc->priority == DEF_PRIORITY)
return false;
--
2.39.3 (Apple Git-146)
^ permalink raw reply related [flat|nested] 7+ messages in thread