[PATCH mm-unstable v3 0/2] mm/vmscan: fix NR_ISOLATED accounting and throttling for MGLRU

[PATCH mm-unstable v3 0/2] mm/vmscan: fix NR_ISOLATED accounting and throttling for MGLRU

[Hui Zhu]

From: Hui Zhu <zhuhui@kylinos.cn>

The legacy reclaim path updates the NR_ISOLATED_ANON/FILE node
counters around isolation and throttles direct reclaimers via
too_many_isolated() when isolated folios pile up.  The MGLRU eviction
path does neither: evict_folios() isolates folios without touching
the counters and never consults too_many_isolated().

Patch 1 updates NR_ISOLATED_ANON/FILE around isolation in
evict_folios(), reusing the existing nr_isolated.  Without this the
counters stay at zero while MGLRU reclaim is active, so compaction's
too_many_isolated() cannot see the pages MGLRU has isolated.

Patch 2 extracts the throttling loop from shrink_inactive_list() into
throttle_is_throttled() and calls it from evict_folios() as well, keeping
the two reclaim paths unified, in line with what was done previously
for writeback reclaim.  This way the MGLRU eviction path is throttled
when isolated folios pile up instead of thrashing the shrinking LRU
lists - the scenario the too_many_isolated() check exists for.  A
dying task fakes reclaim progress exactly like the legacy path so it
exits reclaim quickly.

Testing
=======

Test on 8G RAM qemu.
The reproducer confines stress-ng workers in a 192M memcg and swaps
through dm-delay (300ms write latency) so pageout is slow and isolated
folios pile up; the workload is intentionally extreme to force the
throttle path.
Throttle events are counted via the mm_vmscan_throttled tracepoint.
The test scripts and test log is in [1].

Test 1, reclaim throttling, parallel direct reclaim in the memcg:

                             before        after
throttle events (ISOLATED)       0       512488
  - from kswapd                  0            0
nr_isolated_anon peak            0         1450
  - samples above the
    too_many_isolated threshold  0/1086    293/1057
pgscan_direct            1253539938    148299282
pswpout                   3383673      6502879

Without the series MGLRU reclaim spins on the shrinking LRU lists
while nr_isolated_* stays at 0 and nothing is throttled: pgscan_direct
runs to 1.25 billion with only 3.3M pages swapped out.  With the
series the counters are updated, isolated folios cross the
too_many_isolated threshold (293 of 1057 samples) and direct
reclaimers are throttled; scanning drops by an order of magnitude
while pswpout goes up, i.e. the reclaimers back off and let the slow
swap device finish writing out the isolated folios instead of
thrashing.  kswapd stays exempt.

Test 2, counters visible to compaction, same pressure plus
compact_memory in parallel:

                             before        after
nr_isolated peak                59         2271

The "before" 59 is compaction's own transient isolation; reclaim's
isolation is invisible.  With patch 1 it becomes visible to
compaction's too_many_isolated().  (Compaction's own throttling
threshold, (inactive + active) / 16, is ~23k pages on this box and
needs more pile-up than the box can generate; test 1 exercises the
same throttling mechanism end to end on the reclaim side.)

[1] https://gist.github.com/teawater/d3968aac92eb6bd1378beb54a82933f4

Changelog:
v3:
Accoding to the commens of Baolin, remove the redundant nr_isolated
check before restoring the NR_ISOLATED_* counters in evict_folios().
rename the extracted helper to throttle_is_throttled() to avoid
confusion with the existing wake_throttle_isolated() naming space.
Use for_each_evictable_type() in the MGLRU throttle to check each
evictable type's isolation instead of only the type returned by
get_type_to_scan(), since isolate_folios() may fall back to the
other type.
Re-run the tests and update the test log.
v2:
Accoding to the commens of Kairun, Rebased on mm-unstable.
Split into two patches; patch 2 is new and adds the
too_many_isolated() throttling to the MGLRU eviction path, which v1
did not cover.
Add test infomations.

Hui Zhu (2):
  mm/vmscan: fix missing NR_ISOLATED counter update in MGLRU reclaim
    path
  mm/vmscan: apply too_many_isolated() throttling to MGLRU eviction

 mm/vmscan.c | 76 ++++++++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 64 insertions(+), 12 deletions(-)

-- 
2.53.0

[PATCH mm-unstable v3 1/2] mm/vmscan: fix missing NR_ISOLATED counter update in MGLRU reclaim path

[Hui Zhu]

From: Hui Zhu <zhuhui@kylinos.cn>

MGLRU evict_folios() isolates folios from the LRU without updating
the NR_ISOLATED_ANON/FILE counters, unlike the legacy
shrink_inactive_list() path. This causes compaction's
too_many_isolated() check to under-count isolated pages when MGLRU
reclaim is active.

Add NR_ISOLATED counter updates in evict_folios(): increment after
isolate_folios() and decrement after all retry passes complete, using
the existing nr_isolated which holds the original isolated count.

Signed-off-by: Hui Zhu <zhuhui@kylinos.cn>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
---
 mm/vmscan.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/mm/vmscan.c b/mm/vmscan.c
index c1404a59523d..98226bb021f3 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -4892,6 +4892,9 @@ static int evict_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
 	scanned = isolate_folios(nr_to_scan, lruvec, sc, swappiness,
 				 &list, &isolated, &type, &type_scanned);
 	nr_isolated = isolated;
+	if (nr_isolated)
+		__mod_node_page_state(pgdat, NR_ISOLATED_ANON + type,
+				      nr_isolated);
 
 	/* Scanning may have emptied the oldest gen, flush it */
 	if (scanned)
@@ -4954,6 +4957,8 @@ static int evict_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
 		goto retry;
 	}
 
+	mod_node_page_state(pgdat, NR_ISOLATED_ANON + type, -nr_isolated);
+
 	if (nr_isolated > total_reclaimed)
 		mod_lruvec_state(lruvec, PGROTATE_ANON + type,
 				 nr_isolated - total_reclaimed);
-- 
2.53.0

[PATCH mm-unstable v3 2/2] mm/vmscan: apply too_many_isolated() throttling to MGLRU eviction

[Hui Zhu]

From: Hui Zhu <zhuhui@kylinos.cn>

The legacy path throttles direct reclaim in shrink_inactive_list()
when too many isolated folios pile up, but MGLRU's evict_folios()
isolates folios without this check, which can lead to unnecessary
swapping, thrashing and OOM.

With the NR_ISOLATED counters now updated in evict_folios(), extract
the throttling loop from shrink_inactive_list() into
throttle_is_throttled() and reuse it in evict_folios(). Since the
type to isolate is unknown until isolation and isolate_folios() may
fall back to the other type, check all evictable types with
for_each_evictable_type() and throttle if any of them has too many
isolated folios.

If a fatal signal is pending, fake reclaim progress the same way the
legacy path does, so the dying task exits reclaim quickly instead of
being held in the throttle.

Signed-off-by: Hui Zhu <zhuhui@kylinos.cn>
---
 mm/vmscan.c | 71 ++++++++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 59 insertions(+), 12 deletions(-)

diff --git a/mm/vmscan.c b/mm/vmscan.c
index 98226bb021f3..6fe8824430ac 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -1819,7 +1819,7 @@ bool folio_isolate_lru(struct folio *folio)
  * the LRU list will go small and be scanned faster than necessary, leading to
  * unnecessary swapping, thrashing and OOM.
  */
-static bool too_many_isolated(struct pglist_data *pgdat, int file,
+static bool too_many_isolated(struct pglist_data *pgdat, bool file,
 		struct scan_control *sc)
 {
 	unsigned long inactive, isolated;
@@ -1856,6 +1856,37 @@ static bool too_many_isolated(struct pglist_data *pgdat, int file,
 	return too_many;
 }
 
+/*
+ * Throttle reclaim if too many isolated folios are piling up. If this makes
+ * no progress, the caller is probably looping on unevictable folios, so give
+ * up. Returns true to tell the caller to stop reclaiming, and sets @fatal
+ * if the task received a fatal signal while waiting, so that the caller can
+ * bail out faster.
+ */
+static bool throttle_is_throttled(struct pglist_data *pgdat, bool file,
+				  struct scan_control *sc, bool *fatal)
+{
+	bool stalled = false;
+
+	*fatal = false;
+	while (unlikely(too_many_isolated(pgdat, file, sc))) {
+		if (stalled)
+			return true;
+
+		/* wait a bit for the reclaimer. */
+		stalled = true;
+		reclaim_throttle(pgdat, VMSCAN_THROTTLE_ISOLATED);
+
+		/* We are about to die and free our memory. Return now. */
+		if (fatal_signal_pending(current)) {
+			*fatal = true;
+			return true;
+		}
+	}
+
+	return false;
+}
+
 /*
  * move_folios_to_lru() moves folios from private @list to appropriate LRU list.
  *
@@ -1992,19 +2023,14 @@ static unsigned long shrink_inactive_list(unsigned long nr_to_scan,
 	bool file = is_file_lru(lru);
 	enum node_stat_item item;
 	struct pglist_data *pgdat = lruvec_pgdat(lruvec);
-	bool stalled = false;
-
-	while (unlikely(too_many_isolated(pgdat, file, sc))) {
-		if (stalled)
-			return 0;
-
-		/* wait a bit for the reclaimer. */
-		stalled = true;
-		reclaim_throttle(pgdat, VMSCAN_THROTTLE_ISOLATED);
+	bool fatal;
 
+	if (throttle_is_throttled(pgdat, file, sc, &fatal)) {
 		/* We are about to die and free our memory. Return now. */
-		if (fatal_signal_pending(current))
+		if (fatal)
 			return SWAP_CLUSTER_MAX;
+
+		return 0;
 	}
 
 	lru_add_drain();
@@ -4877,12 +4903,33 @@ static int evict_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
 	enum node_stat_item item;
 	struct reclaim_stat stat;
 	struct lru_gen_mm_walk *walk;
-	int scanned, reclaimed;
+	int i, scanned, reclaimed;
 	int isolated = 0, nr_isolated = 0, type, type_scanned;
 	unsigned long total_reclaimed = 0;
 	bool skip_retry = false;
 	struct mem_cgroup *memcg = lruvec_memcg(lruvec);
 	struct pglist_data *pgdat = lruvec_pgdat(lruvec);
+	bool fatal;
+
+	/*
+	 * The type to isolate is unknown until isolation, and
+	 * isolate_folios() may fall back to the other type. Throttle if
+	 * any evictable type has too many isolated folios.
+	 */
+	for_each_evictable_type(i, swappiness) {
+		if (throttle_is_throttled(pgdat, i, sc, &fatal)) {
+			/*
+			 * We are about to die and free our memory. Like the
+			 * legacy path, pretend some pages were reclaimed so
+			 * reclaim unwinds quickly instead of looping back
+			 * into the throttle.
+			 */
+			if (fatal)
+				sc->nr_reclaimed += SWAP_CLUSTER_MAX;
+
+			return 0;
+		}
+	}
 
 	lruvec_lock_irq(lruvec);
 
-- 
2.53.0