[RFC PATCH 2/4] mm/vmscan: only reclaim file pages in node reclaim when over min_unmapped_pages

[Ridong Chen <ridong.chen@linux.dev>]

From: Ridong Chen <chenridong@xiaomi.com>

Node reclaim enters shrink_node() when unmapped page cache is over
min_unmapped_pages OR reclaimable slab is over min_slab_pages, but the
threshold only decides whether to enter shrink_node(), not what it
reclaims.

min_unmapped_pages is documented to keep a small amount of unmapped page
cache around so that file I/O is not immediately thrown out: "Zone
reclaim will only occur if more than this percentage of pages are in a
state that zone_reclaim_mode allows to be reclaimed."  Yet once slab
alone trips the gate, shrink_node() still reclaims file pages and can
drive unmapped page cache below min_unmapped_pages, defeating the
protection.

Carry the decision into the reclaim path via a scan_control flag set
only on the node reclaim path, and honour it on both reclaim
implementations:

 - traditional LRU: get_scan_count() forces SCAN_ANON, or scans nothing
   when anon cannot be reclaimed (e.g. no swap), rather than falling
   back to SCAN_FILE and breaching the floor;

 - MGLRU: scan_folios() leaves the file type alone, and isolate_folios()
   falls back to anon.

The flag defaults to zero, so kswapd, direct reclaim, memcg reclaim,
proactive reclaim and drop_caches are unaffected; only the node reclaim
path sets it.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ridong Chen <chenridong@xiaomi.com>
---
 mm/vmscan.c | 38 +++++++++++++++++++++++++++++++++++---
 1 file changed, 35 insertions(+), 3 deletions(-)

diff --git a/mm/vmscan.c b/mm/vmscan.c
index 7e65d0ba4a96..1e56973ceb73 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -120,6 +120,12 @@ struct scan_control {
 	 */
 	unsigned int skip_slab_reclaim:1;
 
+	/*
+	 * When set, file pages are not reclaimed because unmapped page cache
+	 * is already at or below min_unmapped_pages.
+	 */
+	unsigned int skip_file_reclaim:1;
+
 	/* Not allow cache_trim_mode to be turned on as part of reclaim? */
 	unsigned int no_cache_trim_mode:1;
 
@@ -2581,6 +2587,21 @@ static void get_scan_count(struct lruvec *lruvec, struct scan_control *sc,
 		goto out;
 	}
 
+	/*
+	 * node_reclaim protects unmapped page cache down to
+	 * min_unmapped_pages: skip file pages and reclaim anon only.  As with
+	 * the anon-only case above, if anon cannot be reclaimed there is
+	 * nothing to do without breaching the floor, so scan nothing.
+	 */
+	if (sc->skip_file_reclaim) {
+		if (!can_reclaim_anon_pages(memcg, pgdat->node_id, sc)) {
+			memset(nr, 0, sizeof(*nr) * NR_LRU_LISTS);
+			return;
+		}
+		scan_balance = SCAN_ANON;
+		goto out;
+	}
+
 	/* If we have no swap space, do not bother scanning anon folios. */
 	if (!sc->may_swap || !can_reclaim_anon_pages(memcg, pgdat->node_id, sc)) {
 		scan_balance = SCAN_FILE;
@@ -4749,6 +4770,14 @@ static int scan_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
 	VM_WARN_ON_ONCE(nr_to_scan > MAX_LRU_BATCH);
 	VM_WARN_ON_ONCE(!list_empty(list));
 
+	/*
+	 * node_reclaim protects unmapped page cache down to min_unmapped_pages,
+	 * so leave the file type alone; isolate_folios() then falls back to
+	 * anon.
+	 */
+	if (sc->skip_file_reclaim && type == LRU_GEN_FILE)
+		return 0;
+
 	if (get_nr_gens(lruvec, type) == MIN_NR_GENS)
 		return 0;
 
@@ -7949,13 +7978,16 @@ unsigned long node_reclaim(struct pglist_data *pgdat, gfp_t gfp_mask, unsigned i
 		return 0;
 
 	/*
-	 * min_slab_pages only gates slab reclaim: when reclaimable slab is
-	 * already at or below the limit, leave the shrinkers alone even if we
-	 * entered node reclaim to trim unmapped page cache.
+	 * Each limit only gates its own type of reclaim.  When reclaimable
+	 * slab or unmapped page cache is already at or below its limit, leave
+	 * that type alone even if the other type tripped the gate and brought
+	 * us into node reclaim.
 	 */
 	sc.skip_slab_reclaim =
 		node_page_state_pages(pgdat, NR_SLAB_RECLAIMABLE_B) <=
 		pgdat->min_slab_pages;
+	sc.skip_file_reclaim =
+		node_pagecache_reclaimable(pgdat) <= pgdat->min_unmapped_pages;
 
 	ret = __node_reclaim(pgdat, nr_pages, &sc);
 	clear_bit_unlock(PGDAT_RECLAIM_LOCKED, &pgdat->flags);
-- 
2.34.1