[RFC PATCH 0/4] mm: mglru: fix swappiness behavior

[RFC PATCH 0/4] mm: mglru: fix swappiness behavior

[Barry Song (Xiaomi)]

The active/inactive LRU respects swappiness well. Anonymous page
scanning and reclamation increase roughly linearly with
swappiness, while file page scanning and reclamation decrease
accordingly.
For example, when swappiness reaches 200, both pgsteal_file and
pgscan_file drop to zero while building the kernel in a 1 GB
memcg. In contrast, MGLRU shows almost no change across different
swappiness values.

                   pgsteal_file

Swappiness    LRU         MGLRU
--------------------------------
1             10567455      763612
36              990706      480205
71              688170      415848
106             446294      386164
141             286307      359196
176             201733      351686
200                  0      330093


                   pgsteal_anon

Swappiness    LRU         MGLRU
--------------------------------
1              4410548     2726362
36             2465268     2762859
71             2677908     2885124
106            2737227     2841796
141            2984276     3035015
176            3381338     2938302
200           13116359     3113499


                   pgscan_file

Swappiness    LRU         MGLRU
--------------------------------
1             17997223      923094
36             1325674      539571
71              852345      464222
106             538207      464477
141             357253      412277
176             217536      399446
200                  0      375902


                   pgscan_anon

Swappiness    LRU         MGLRU
--------------------------------
1             31639423     5987136
36            23441521     5753224
71            26067110     6101780
106           25619448     5782919
141           26842088     6234264
176           29200021     5980292
200           62193924     6413125

This patchset respects the type selected by positive_ctrl_err(),
which uses swappiness as its gain. It does so by:

1. Avoiding premature fallback to the other type. Only fall back
   when reclaim is running at high priority.

2. Running aging when the preferred type has few or no
   reclaimable folios, so more folios of that type become
   reclaimable.

With this patchset, swappiness starts to behave similarly to the
active/inactive LRU.

                         pgsteal_file

Swappiness      LRU        MGLRU      MGLRU+Patch
-------------------------------------------------
1             10567455      763612        3507619
36              990706      480205         528781
71              688170      415848         478478
106             446294      386164         411632
141             286307      359196         375137
176             201733      351686         298586
200                  0      330093              0


                         pgsteal_anon

Swappiness      LRU        MGLRU      MGLRU+Patch
-------------------------------------------------
1              4410548     2726362        2876888
36             2465268     2762859        3022062
71             2677908     2885124        3129555
106            2737227     2841796        2974192
141            2984276     3035015        3092280
176            3381338     2938302        3353395
200           13116359     3113499       14348518


                         pgscan_file

Swappiness      LRU        MGLRU      MGLRU+Patch
-------------------------------------------------
1             17997223      923094        5842416
36             1325674      539571         648111
71              852345      464222         569930
106             538207      464477         480767
141             357253      412277         431911
176             217536      399446         337768
200                  0      375902              0


                         pgscan_anon

Swappiness      LRU        MGLRU      MGLRU+Patch
-------------------------------------------------
1             31639423     5987136        4658593
36            23441521     5753224        5444726
71            26067110     6101780        5755804
106           25619448     5782919        5872415
141           26842088     6234264        6144096
176           29200021     5980292        6615500
200           62193924     6413125       22984986

Another possible approach is to decouple anonymous and file-backed
aging by maintaining separate max_seq values for each type. This
allows anonymous and file-backed memory to age and be reclaimed
independently, enabling the swappiness-preferred type to be reclaimed
more aggressively while allowing the other type to lag behind. This
approach has already been adopted by projects such as CachyOS [1] and
Chromium [2].

However, this approach requires substantial changes to MGLRU and
fundamentally alters its design by breaking the shared aging timeline
between anonymous and file-backed memory. This timeline is the
foundation for mechanisms such as the PID controller and the
min_ttl_ms thrashing protection.

That is why this patchset aims to fix the swappiness behavior
without fundamentally changing MGLRU's design, with minimal changes.

[1] https://github.com/firelzrd/re-swappiness
[2] https://chromium.googlesource.com/chromiumos/third_party/kernel/+log/929932351492d01f0aee37a0ac3be8c7bd88f80d

Barry Song (Xiaomi) (3):
  mm: mglru: only fall back when reclaim is running at high priority
  mm: mglru: do try_to_inc_min_seq if scanned==0
  mm: mglru: run aging if the preferred type has no folios in
    reclaimable gens

lyugaofei (1):
  mm: mglru: run aging when pages are severely imbalanced across gens

 mm/vmscan.c | 46 +++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 43 insertions(+), 3 deletions(-)

-- 
2.39.3 (Apple Git-146)

[RFC PATCH 1/4] mm: mglru: only fall back when reclaim is running at high priority

[Barry Song (Xiaomi)]

Respect the type selected by positive_ctrl_err(), where swappiness
controls the relative weights of SP and PV. Falling back too readily
violates that bias.

Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
---
 mm/vmscan.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/mm/vmscan.c b/mm/vmscan.c
index 566c4e837c7d..f79402760776 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -4855,9 +4855,12 @@ static int isolate_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
 		 * If scanned > 0 and isolated == 0, avoid falling back to the
 		 * other type, as this type remains sufficient. Falling back
 		 * too readily can disrupt the positive_ctrl_err() bias.
+		 * Only fall back when reclaim is running at high priority.
 		 */
-		if (!scanned)
-			type = !type;
+		if (!scanned) {
+			if (!sc->priority)
+				type = !type;
+		}
 	}
 
 	return total_scanned;
-- 
2.39.3 (Apple Git-146)

[RFC PATCH 2/4] mm: mglru: do try_to_inc_min_seq if scanned==0

[Barry Song (Xiaomi)]

If scanned == 0, we are likely running out of folios in the
reclaimable gens. Advance min_seq to keep it in sync with the
current reclaimable state.

Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
---
 mm/vmscan.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/mm/vmscan.c b/mm/vmscan.c
index f79402760776..4f3a375de86e 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -4858,6 +4858,7 @@ static int isolate_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
 		 * Only fall back when reclaim is running at high priority.
 		 */
 		if (!scanned) {
+			try_to_inc_min_seq(lruvec, swappiness);
 			if (!sc->priority)
 				type = !type;
 		}
-- 
2.39.3 (Apple Git-146)

[RFC PATCH 3/4] mm: mglru: run aging when pages are severely imbalanced across gens

[Barry Song (Xiaomi)]

From: lyugaofei <lyugaofei@xiaomi.com>

This partially restores the reclaim behavior introduced in Yu
Zhao's initial MGLRU commit, ac35a4902370 ("mm: multi-gen LRU:
minimal implementation"):
	/*
	 * It's also ideal to spread pages out evenly, i.e., 1/(MIN_NR_GENS+1)
	 * of the total number of pages for each generation. A reasonable range
	 * for this average portion is [1/MIN_NR_GENS, 1/(MIN_NR_GENS+2)]. The
	 * aging cares about the upper bound of hot pages, while the eviction
	 * cares about the lower bound of cold pages.
	 */
	if (young * MIN_NR_GENS > total)
		return true;
	if (old * (MIN_NR_GENS + 2) < total)
		return true;

But with a stricter condition: the youngest gen must have at
least four times as many folios as the oldest. This helps keep
folios of the preferred type distributed across the reclaimable
gens.

Signed-off-by: lyugaofei <lyugaofei@xiaomi.com>
Co-developed-by: Barry Song (Xiaomi) <baohua@kernel.org>
Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
---
 mm/vmscan.c | 34 +++++++++++++++++++++++++++++++++-
 1 file changed, 33 insertions(+), 1 deletion(-)

diff --git a/mm/vmscan.c b/mm/vmscan.c
index 4f3a375de86e..9aac4febc206 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -4963,9 +4963,37 @@ static int evict_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
 	return scanned;
 }
 
+static bool lru_gen_imbalanced(struct lruvec *lruvec, int type,
+		unsigned long max_seq, unsigned long min_seq,
+		int swappiness)
+{
+	struct lru_gen_folio *lrugen = &lruvec->lrugen;
+	unsigned long young = 0, old = 0, seq;
+
+	if (swappiness == MIN_SWAPPINESS || swappiness > MAX_SWAPPINESS)
+		return false;
+
+	for (seq = min_seq; seq <= max_seq; seq++) {
+		int gen = lru_gen_from_seq(seq);
+		unsigned long size = 0;
+		int zone;
+
+		gen = lru_gen_from_seq(seq);
+		for (zone = 0; zone < MAX_NR_ZONES; zone++)
+			size += max(READ_ONCE(lrugen->nr_pages[gen][type][zone]), 0L);
+
+		if (seq + MIN_NR_GENS > max_seq)
+			young += size;
+		else
+			old += size;
+	}
+	return young > old * 4;
+}
+
 static bool should_run_aging(struct lruvec *lruvec, unsigned long max_seq,
 			     struct scan_control *sc, int swappiness)
 {
+	int type = get_type_to_scan(lruvec, swappiness);
 	DEFINE_MIN_SEQ(lruvec);
 
 	/* have to run aging, since eviction is not possible anymore */
@@ -4977,7 +5005,11 @@ static bool should_run_aging(struct lruvec *lruvec, unsigned long max_seq,
 		return false;
 
 	/* better to run aging even though eviction is still possible */
-	return evictable_min_seq(min_seq, swappiness) + MIN_NR_GENS == max_seq;
+	if (evictable_min_seq(min_seq, swappiness) + MIN_NR_GENS == max_seq)
+		return true;
+
+	/* Run aging if the preferred type is severely imbalanced across gens */
+	return lru_gen_imbalanced(lruvec, type, max_seq, min_seq[type], swappiness);
 }
 
 static long get_nr_to_scan(struct lruvec *lruvec, struct scan_control *sc,
-- 
2.39.3 (Apple Git-146)

[RFC PATCH 4/4] mm: mglru: run aging if the preferred type has no folios in reclaimable gens

[Barry Song (Xiaomi)]

Respect the type selected by positive_ctrl_err(). If there are no
reclaimable gens left for that type, run aging.

Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
---
 mm/vmscan.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/mm/vmscan.c b/mm/vmscan.c
index 9aac4febc206..be34b8b946ad 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -5000,6 +5000,10 @@ static bool should_run_aging(struct lruvec *lruvec, unsigned long max_seq,
 	if (evictable_min_seq(min_seq, swappiness) + MIN_NR_GENS > max_seq)
 		return true;
 
+	/* run aging if the preferred type is exhausted */
+	if (min_seq[type] + MIN_NR_GENS > max_seq)
+		return true;
+
 	/* try to avoid aging, do gentle reclaim at the default priority */
 	if (sc->priority == DEF_PRIORITY)
 		return false;
-- 
2.39.3 (Apple Git-146)

Re: [RFC PATCH 0/4] mm: mglru: fix swappiness behavior

[Barry Song]

On Sun, Jul 26, 2026 at 9:29 AM Barry Song (Xiaomi) <baohua@kernel.org> wrote:
>
> The active/inactive LRU respects swappiness well. Anonymous page
> scanning and reclamation increase roughly linearly with
> swappiness, while file page scanning and reclamation decrease
> accordingly.
> For example, when swappiness reaches 200, both pgsteal_file and
> pgscan_file drop to zero while building the kernel in a 1 GB
> memcg. In contrast, MGLRU shows almost no change across different
> swappiness values.
>
>                    pgsteal_file
>
> Swappiness    LRU         MGLRU
> --------------------------------
> 1             10567455      763612
> 36              990706      480205
> 71              688170      415848
> 106             446294      386164
> 141             286307      359196
> 176             201733      351686
> 200                  0      330093
>
>
>                    pgsteal_anon
>
> Swappiness    LRU         MGLRU
> --------------------------------
> 1              4410548     2726362
> 36             2465268     2762859
> 71             2677908     2885124
> 106            2737227     2841796
> 141            2984276     3035015
> 176            3381338     2938302
> 200           13116359     3113499
>
>
>                    pgscan_file
>
> Swappiness    LRU         MGLRU
> --------------------------------
> 1             17997223      923094
> 36             1325674      539571
> 71              852345      464222
> 106             538207      464477
> 141             357253      412277
> 176             217536      399446
> 200                  0      375902
>
>
>                    pgscan_anon
>
> Swappiness    LRU         MGLRU
> --------------------------------
> 1             31639423     5987136
> 36            23441521     5753224
> 71            26067110     6101780
> 106           25619448     5782919
> 141           26842088     6234264
> 176           29200021     5980292
> 200           62193924     6413125
>
> This patchset respects the type selected by positive_ctrl_err(),
> which uses swappiness as its gain. It does so by:
>
> 1. Avoiding premature fallback to the other type. Only fall back
>    when reclaim is running at high priority.
>
> 2. Running aging when the preferred type has few or no
>    reclaimable folios, so more folios of that type become
>    reclaimable.
>
> With this patchset, swappiness starts to behave similarly to the
> active/inactive LRU.
>
>                          pgsteal_file
>
> Swappiness      LRU        MGLRU      MGLRU+Patch
> -------------------------------------------------
> 1             10567455      763612        3507619
> 36              990706      480205         528781
> 71              688170      415848         478478
> 106             446294      386164         411632
> 141             286307      359196         375137
> 176             201733      351686         298586
> 200                  0      330093              0
>
>
>                          pgsteal_anon
>
> Swappiness      LRU        MGLRU      MGLRU+Patch
> -------------------------------------------------
> 1              4410548     2726362        2876888
> 36             2465268     2762859        3022062
> 71             2677908     2885124        3129555
> 106            2737227     2841796        2974192
> 141            2984276     3035015        3092280
> 176            3381338     2938302        3353395
> 200           13116359     3113499       14348518
>
>
>                          pgscan_file
>
> Swappiness      LRU        MGLRU      MGLRU+Patch
> -------------------------------------------------
> 1             17997223      923094        5842416
> 36             1325674      539571         648111
> 71              852345      464222         569930
> 106             538207      464477         480767
> 141             357253      412277         431911
> 176             217536      399446         337768
> 200                  0      375902              0
>
>
>                          pgscan_anon
>
> Swappiness      LRU        MGLRU      MGLRU+Patch
> -------------------------------------------------
> 1             31639423     5987136        4658593
> 36            23441521     5753224        5444726
> 71            26067110     6101780        5755804
> 106           25619448     5782919        5872415
> 141           26842088     6234264        6144096
> 176           29200021     5980292        6615500
> 200           62193924     6413125       22984986
>
> Another possible approach is to decouple anonymous and file-backed
> aging by maintaining separate max_seq values for each type. This
> allows anonymous and file-backed memory to age and be reclaimed
> independently, enabling the swappiness-preferred type to be reclaimed
> more aggressively while allowing the other type to lag behind. This
> approach has already been adopted by projects such as CachyOS [1] and
> Chromium [2].
>
> However, this approach requires substantial changes to MGLRU and
> fundamentally alters its design by breaking the shared aging timeline
> between anonymous and file-backed memory. This timeline is the
> foundation for mechanisms such as the PID controller and the
> min_ttl_ms thrashing protection.
>
> That is why this patchset aims to fix the swappiness behavior
> without fundamentally changing MGLRU's design, with minimal changes.
>
> [1] https://github.com/firelzrd/re-swappiness
> [2] https://chromium.googlesource.com/chromiumos/third_party/kernel/+log/929932351492d01f0aee37a0ac3be8c7bd88f80d
>
> Barry Song (Xiaomi) (3):
>   mm: mglru: only fall back when reclaim is running at high priority
>   mm: mglru: do try_to_inc_min_seq if scanned==0
>   mm: mglru: run aging if the preferred type has no folios in
>     reclaimable gens
>
> lyugaofei (1):
>   mm: mglru: run aging when pages are severely imbalanced across gens

I checked Sashiki's comment:
https://sashiko.dev/#/patchset/20260726012946.18684-1-baohua@kernel.org

1. Sashiko says
"
> @@ -4858,6 +4858,7 @@ static int isolate_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
> * Only fall back when reclaim is running at high priority.
> */
> if (!scanned) {
> + try_to_inc_min_seq(lruvec, swappiness);
Is this call to try_to_inc_min_seq() redundant?
The caller evict_folios() acquires the LRU lock and calls
try_to_inc_min_seq() just before calling isolate_folios().
If scanned == 0 on the first iteration of the loop in isolate_folios(),
the lists haven't changed since the previous try_to_inc_min_seq() call.
This means get_nr_gens == MIN_NR_GENS, making this new call a strict
no-op.
If scanned == 0 happens on the second iteration (because the first
iteration processed folios without isolating any), try_to_inc_min_seq()
might advance min_seq. However, the for_each_evictable_type() loop
only iterates twice, so it terminates immediately without scanning the
newly advanced generation.
After isolate_folios() returns, evict_folios() unconditionally calls
try_to_inc_min_seq() again anyway. Does this logic successfully retry
with new folios, or does it only perform unnecessary work?
"

I observed that scan_folios() always returns 0 because the list is
empty, making the following early-return check appear to be
ineffective:

        if (get_nr_gens(lruvec, type) == MIN_NR_GENS)
                return 0;

So I thought min_seq should be advanced when scanned == 0, since
there are no folios left in the reclaimable generation.

After taking another look, I realized that isn't the real issue.
The problem is that try_to_inc_min_seq() doesn't always reflect the
actual oldest reclaimable generation. It can leave min_seq pointing
to an older generation even after that generation has no folios
remaining:

static void try_to_inc_min_seq(struct lruvec *lruvec, int swappiness)
{
        ...
        /* see the comment on lru_gen_folio */
        if (swappiness && swappiness <= MAX_SWAPPINESS) {
                unsigned long seq = lrugen->max_seq - MIN_NR_GENS;

                if (min_seq[LRU_GEN_ANON] > seq && min_seq[LRU_GEN_FILE] < seq)
                        min_seq[LRU_GEN_ANON] = seq;
                else if (min_seq[LRU_GEN_FILE] > seq &&
min_seq[LRU_GEN_ANON] < seq)
                        min_seq[LRU_GEN_FILE] = seq;
        }
}

I think the function itself should be fixed instead. Otherwise, I can
leave it as is and simply drop the newly added call to
try_to_inc_min_seq().

2. Sashiko says

"
> + gen = lru_gen_from_seq(seq);
This isn't a bug, but is this second assignment of gen redundant since it was
already initialized to the exact same value in its declaration?
"

This is right. I will fix it in RFC v2.

3. Sashiko says
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index 566c4e837c7d5..f794027607767 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -4855,9 +4855,12 @@ static int isolate_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
> * If scanned > 0 and isolated == 0, avoid falling back to the
> * other type, as this type remains sufficient. Falling back
> * too readily can disrupt the positive_ctrl_err() bias.
> + * Only fall back when reclaim is running at high priority.
> */
> - if (!scanned)
> - type = !type;
> + if (!scanned) {
Does this logic completely prevent kswapd from falling back to the other LRU
type?
Looking at kswapd's reclaim loop in balance_pgdat(), it explicitly exits
before sc.priority reaches 0:
mm/vmscan.c:balance_pgdat() {
...
if (raise_priority || !nr_reclaimed)
sc.priority--;
} while (sc.priority >= 1);
Because sc.priority never reaches 0 inside the kswapd reclaim path, kswapd
will never satisfy this fallback condition. It will repeatedly scan the
empty type, fail to make progress, drop its priority, and eventually give up
and sleep.
For direct reclaim, this also means processes will stall fruitlessly through
multiple priority drops (triggering VMSCAN_THROTTLE_NOPROGRESS delays)
before finally reaching priority 0 and falling back, which could lead to
severe stalling or premature OOM killer invocation.
> + if (!sc->priority)
> + type = !type;
> + }
> }

This seems to be a valid concern. I could address it by:
diff --git a/mm/vmscan.c b/mm/vmscan.c
index be34b8b946ad..d69a0ed26f8c 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -4859,7 +4859,7 @@ static int isolate_folios(unsigned long
nr_to_scan, struct lruvec *lruvec,
                 */
                if (!scanned) {
                        try_to_inc_min_seq(lruvec, swappiness);
-                       if (!sc->priority)
+                       if (sc->priority <= 2)
                                type = !type;
                }
        }

4. Sashiko says

"> But with a stricter condition: the youngest gen must have at
> least four times as many folios as the oldest. This helps keep
> folios of the preferred type distributed across the reclaimable
> gens.
Is the commit message description accurate regarding "the youngest gen"?
Looking at lru_gen_imbalanced(), the code checks:
if (seq + MIN_NR_GENS > max_seq)
Since MIN_NR_GENS is 2, wouldn't this condition group the two youngest
generations into the young variable rather than just the single youngest
generation?"

Yes, I'll update the description.

5. Sashiko says
"
> +static bool lru_gen_imbalanced(struct lruvec *lruvec, int type,
> + unsigned long max_seq, unsigned long min_seq,
> + int swappiness)
> +{
> + struct lru_gen_folio *lrugen = &lruvec->lrugen;
> + unsigned long young = 0, old = 0, seq;
> +
> + if (swappiness == MIN_SWAPPINESS || swappiness > MAX_SWAPPINESS)
> + return false;
Does this unexpectedly disable the imbalance heuristic for systems where
swappiness is configured to 0?
When swappiness is 0 (MIN_SWAPPINESS), get_type_to_scan() correctly selects
LRU_GEN_FILE, but this check causes an immediate return of false. Wouldn't
file pages still need to be balanced across generations on these systems?
"

Technically, we should maintain the balance for swappiness values of
0 and 201 as well. However, in those cases we always reclaim a single
type, so the swappiness bias is irrelevant. We can therefore skip
this.

Best Regards
Barry