[PATCH 0/3] mm/mglru: clean up isolate_folios and scan_folios for readability and clarity

[PATCH 0/3] mm/mglru: clean up isolate_folios and scan_folios for readability and clarity

[Barry Song (Xiaomi)]

This is a cleanup series split out from the MGLRU swappiness series [1],
with the cleanup changes separated to make them easier to review.

Right now, isolate_folios() is quite difficult to follow:

1. It uses for_each_evictable_type(i, swappiness) to iterate over the
types, but i is not actually used as the type within the loop body.

2. It uses scanned == 0 to detect whether the current reclaim type is
exhausted, but this is not an accurate indication.

3. It has an internal retry when no folios can be isolated after scanning
some folios, but the retry is implemented in a way nobody can understand.

This patchset makes these behaviors explicit and much easier to follow.

Run kernel builds for several rounds in a 1 GB memcg and take the
average build time. The patchset shows almost no performance impact,
with a very small improvement that could simply be noise:

                w/o patch   w/patch      Delta

real             109.149    108.990     -0.15%
sys              223.213    222.592     -0.28%

pgscan_file      498,208    486,994     -2.25%
refault_file     262,696    254,688     -3.05%

Thanks very much to Baolin and Lian for their previous reviews of the
original RFC patchset for this part.

[1] https://lore.kernel.org/linux-mm/20260812121658.69965-1-baohua@kernel.org/

Barry Song (Xiaomi) (2):
  mm/mglru: improve scan_folios() exhaustion detection
  mm/mglru: retry the same type once if isolation fails due to races

Ridong Chen (1):
  mm/mglru: improve readability of isolate_folios()

 mm/vmscan.c | 78 ++++++++++++++++++++++++++++++++++++-----------------
 1 file changed, 54 insertions(+), 24 deletions(-)

-- 
2.34.1

[PATCH 1/3] mm/mglru: improve readability of isolate_folios()

[Barry Song (Xiaomi)]

From: Ridong Chen <chenridong@xiaomi.com>

The for_each_evictable_type() loop in isolate_folios()
is misleading: it does not actually iterate over each
evictable type. Instead, get_type_to_scan() selects the
type to scan, while the iterator `i` merely bounds the
number of attempts.

Make the fallback behavior explicit in the code and remove the
opaque for_each_evictable_type(i, swappiness).

Signed-off-by: Ridong Chen <chenridong@xiaomi.com>
Co-developed-by: Barry Song (Xiaomi) <baohua@kernel.org>
Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
---
 mm/vmscan.c | 46 ++++++++++++++++++++++++++--------------------
 1 file changed, 26 insertions(+), 20 deletions(-)

diff --git a/mm/vmscan.c b/mm/vmscan.c
index c1404a59523d..d5cc30b667ad 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -4833,35 +4833,41 @@ static int get_type_to_scan(struct lruvec *lruvec, int swappiness)
 	return positive_ctrl_err(&sp, &pv);
 }
 
+static inline bool is_single_type_reclaim(int swappiness)
+{
+	return swappiness == MIN_SWAPPINESS ||
+	       swappiness == SWAPPINESS_ANON_ONLY;
+}
+
 static int isolate_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
 			  struct scan_control *sc, int swappiness,
 			  struct list_head *list, int *isolated,
 			  int *isolate_type, int *isolate_scanned)
 {
-	int i;
-	int total_scanned = 0;
+	bool type_fallback_allowed = !is_single_type_reclaim(swappiness);
 	int type = get_type_to_scan(lruvec, swappiness);
+	int total_scanned = 0, scanned, tier;
 
-	for_each_evictable_type(i, swappiness) {
-		int scanned;
-		int tier = get_tier_idx(lruvec, type);
+retry:
+	tier = get_tier_idx(lruvec, type);
+	scanned = scan_folios(nr_to_scan, lruvec, sc,
+			      type, tier, list, isolated);
 
-		scanned = scan_folios(nr_to_scan, lruvec, sc,
-				      type, tier, list, isolated);
+	total_scanned += scanned;
+	if (*isolated) {
+		*isolate_type = type;
+		*isolate_scanned = scanned;
+		return total_scanned;
+	}
 
-		total_scanned += scanned;
-		if (*isolated) {
-			*isolate_type = type;
-			*isolate_scanned = scanned;
-			break;
-		}
-		/*
-		 * If scanned > 0 and isolated == 0, avoid falling back to the
-		 * other type, as this type remains sufficient. Falling back
-		 * too readily can disrupt the positive_ctrl_err() bias.
-		 */
-		if (!scanned)
-			type = !type;
+	/*
+	 * We are running out of the current reclaim type. Fall back to
+	 * the other type if allowed.
+	 */
+	if (!scanned && type_fallback_allowed) {
+		type = !type;
+		type_fallback_allowed = false;
+		goto retry;
 	}
 
 	return total_scanned;
-- 
2.34.1

[PATCH 2/3] mm/mglru: improve scan_folios() exhaustion detection

[Barry Song (Xiaomi)]

Commit 16b475d2ac3c ("mm/mglru: avoid reclaim type fall back when
isolation makes no progress") uses scanned == 0 to determine
whether scan_folios() has exhausted a reclaim type. However,
this is not always sufficient. It is possible for scanned > 0,
while the oldest reclaimable generation is exhausted after the
first scan_folios() call.

We detect early_stop in scan_folios(). If we stop early for any reason,
it means the current reclaim type is not exhausted yet. If early_stop is
never reached, it means we have exhausted the current oldest generation
without hitting any scanning limit.

Another issue is that if the lruvec has 4 generations, we might have
exhausted the oldest generation while the second oldest generation is
still reclaimable. In that case, this type is not exhausted yet.

Add an exhausted output argument to scan_folios() so it can
explicitly report whether the reclaimable lists for the current
type have been exhausted.

Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
---
 mm/vmscan.c | 28 ++++++++++++++++++++++------
 1 file changed, 22 insertions(+), 6 deletions(-)

diff --git a/mm/vmscan.c b/mm/vmscan.c
index d5cc30b667ad..1f2e574b0061 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -4721,7 +4721,8 @@ static bool isolate_folio(struct lruvec *lruvec, struct folio *folio, struct sca
 
 static int scan_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
 		       struct scan_control *sc, int type, int tier,
-		       struct list_head *list, int *isolatedp)
+		       struct list_head *list, int *isolatedp,
+		       bool *exhausted)
 {
 	int i;
 	int gen;
@@ -4732,12 +4733,15 @@ static int scan_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
 	int skipped = 0;
 	unsigned long remaining = nr_to_scan;
 	struct lru_gen_folio *lrugen = &lruvec->lrugen;
+	bool early_stop = false;
 
 	VM_WARN_ON_ONCE(nr_to_scan > MAX_LRU_BATCH);
 	VM_WARN_ON_ONCE(!list_empty(list));
 
-	if (get_nr_gens(lruvec, type) == MIN_NR_GENS)
+	if (get_nr_gens(lruvec, type) == MIN_NR_GENS) {
+		*exhausted = true;
 		return 0;
+	}
 
 	gen = lru_gen_from_seq(lrugen->min_seq[type]);
 
@@ -4768,8 +4772,10 @@ static int scan_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
 				skipped_zone += delta;
 			}
 
-			if (!--remaining || max(isolated, skipped_zone) >= MIN_LRU_BATCH)
+			if (!--remaining || max(isolated, skipped_zone) >= MIN_LRU_BATCH) {
+				early_stop = true;
 				break;
+			}
 		}
 
 		if (skipped_zone) {
@@ -4778,8 +4784,10 @@ static int scan_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
 			skipped += skipped_zone;
 		}
 
-		if (!remaining || isolated >= MIN_LRU_BATCH)
+		if (!remaining || isolated >= MIN_LRU_BATCH) {
+			early_stop = true;
 			break;
+		}
 	}
 
 	item = PGSCAN_KSWAPD + reclaimer_offset(sc);
@@ -4790,6 +4798,13 @@ static int scan_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
 				scanned, skipped, isolated,
 				type ? LRU_INACTIVE_FILE : LRU_INACTIVE_ANON);
 
+	/*
+	 * If we didn't stop early, all reclaimable folios in the current
+	 * generation have been scanned. We are exhausted if this is the last
+	 * reclaimable generation.
+	 */
+	*exhausted = !early_stop &&
+		     lrugen->min_seq[type] + MIN_NR_GENS == lrugen->max_seq;
 	*isolatedp = isolated;
 	return scanned;
 }
@@ -4847,11 +4862,12 @@ static int isolate_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
 	bool type_fallback_allowed = !is_single_type_reclaim(swappiness);
 	int type = get_type_to_scan(lruvec, swappiness);
 	int total_scanned = 0, scanned, tier;
+	bool exhausted;
 
 retry:
 	tier = get_tier_idx(lruvec, type);
 	scanned = scan_folios(nr_to_scan, lruvec, sc,
-			      type, tier, list, isolated);
+			      type, tier, list, isolated, &exhausted);
 
 	total_scanned += scanned;
 	if (*isolated) {
@@ -4864,7 +4880,7 @@ static int isolate_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
 	 * We are running out of the current reclaim type. Fall back to
 	 * the other type if allowed.
 	 */
-	if (!scanned && type_fallback_allowed) {
+	if (exhausted && type_fallback_allowed) {
 		type = !type;
 		type_fallback_allowed = false;
 		goto retry;
-- 
2.34.1

[PATCH 3/3] mm/mglru: retry the same type once if isolation fails due to races

[Barry Song (Xiaomi)]

If we are not exhausted (i.e., there are still folios in the
reclaimable generations) but fail to isolate any folios due to
promotions, protections, or races, retry once more to avoid going
through the outer loop again.

Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
---
 mm/vmscan.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/mm/vmscan.c b/mm/vmscan.c
index 1f2e574b0061..1f302386d8ab 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -4862,7 +4862,7 @@ static int isolate_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
 	bool type_fallback_allowed = !is_single_type_reclaim(swappiness);
 	int type = get_type_to_scan(lruvec, swappiness);
 	int total_scanned = 0, scanned, tier;
-	bool exhausted;
+	bool exhausted, tried = false;
 
 retry:
 	tier = get_tier_idx(lruvec, type);
@@ -4885,6 +4885,14 @@ static int isolate_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
 		type_fallback_allowed = false;
 		goto retry;
 	}
+	/*
+	 * We are not exhausted, but failed to isolate any folios due to
+	 * promotions, protections, or races. Retry once to avoid a larger loop.
+	 */
+	if (!exhausted && !tried) {
+		tried = true;
+		goto retry;
+	}
 
 	return total_scanned;
 }
-- 
2.34.1

Re: [PATCH 0/3] mm/mglru: clean up isolate_folios and scan_folios for readability and clarity

[Lian Wang (ProcessMission)]

From: Lian Wang <lianux.mm@gmail.com>

Hi Barry,

I went through all three patches. The explicit fallback, exhaustion
reporting, and bounded retry make the control flow much clearer.

In particular, the exhaustion check preserves the early-stop cases and
avoids falling back after only the oldest generation has been drained when
another generation remains reclaimable. The retry is still bounded to one
extra attempt.

No issues from my side.

For the series:

Reviewed-by: Lian Wang <lianux.mm@gmail.com>

Thanks,
Lian

On Thu, 20 Aug 2026 12:56:00 +0800 "Barry Song (Xiaomi)" <baohua@kernel.org> wrote:

> This is a cleanup series split out from the MGLRU swappiness series [1],
> with the cleanup changes separated to make them easier to review.
>
> This patchset makes these behaviors explicit and much easier to follow.
>
> Thanks very much to Baolin and Lian for their previous reviews of the
> original RFC patchset for this part.