From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=100.103.45.18 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1794446232; b=JzQTUSmPV503RR3FIdQnhA98ePD+bpZXVFSMDnQXPb8OZGSWx+UTOzQdaWGGi2/EnRfM vlYCHFjE/4DTOZ/qY196tAIGC/obVUEuq0ecYmMFfvEAmU/kDJPs/qqOSTkhG6kSJaN6M sYgJAETpKCuB1l7IBRJs774Jv/yCjOTNjUQ/gPiv4G/OE7vizi4igpqxsrk+jR2cNkKEP r5Aty23TXEJNSo4HiaHYMTeH9QJQgHzu1FAEYfXWoMOf/2CfyrYb9JxkRulfISVCop9mN A2+jKHdQCVvyIZ6GbI8AZKiN+HnxBuuysA1BFFhkV6G2Em5Ppew+AqzS1JrTiCymjfw== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1794446232; h=Received:DKIM-Signature:Date:To:From:Subject:Message-Id; bh=LJubBKKzYYVjdBOtv+eA8AZ9tcVmyi9lSZxVEe4QdSM=; b=IpmrtswvLA761LpKykR4MOZyUkb9ugg8Y6SqDj4IBxbC/DRgVNOFBx+5SUHXSasMhtx9 xIusNJP0bTkawnNEl1jk/XwKtdvNpVsbwUXncqxN+WojfHaaWxxS5MwZEDEganCQQnyDD VEGX8wti2LR0vgCww+tgZYbopXHI1Rj/dS6MXQ/KFZEtFEzXOjy+qfN+16CJoDYR3ltFM oIwFOdZyuaRg9ZEUZn5v3mNoiWkg/8zRJ7ppT0XM6COHwmgQd2Bqttem6cAQXhzNVtFYe ibE4Vvaiy9eY8W9b8XyUHwRYirHPTgGuy0gSeA/Gy5k6ISjoVFisS97KAPFBNIutMHQ== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass header.d=linux-foundation.org header.i=@linux-foundation.org header.a=rsa-sha256 header.s=korg header.b=Oj9Reges; arc=none smtp.remote-ip=100.103.45.18 Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9F5BD606A31C for ; Thu, 12 Nov 2026 01:17:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.a=rsa-sha256 header.s=korg header.b=Oj9Reges Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6C0011F00A05; Thu, 12 Nov 2026 01:17:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=korg; t=1794446232; bh=LJubBKKzYYVjdBOtv+eA8AZ9tcVmyi9lSZxVEe4QdSM=; h=Date:To:From:Subject; b=Oj9RegesumZR2Ay0TSanEun0X20b4aWoPhVQeHDcqg8hROjgj88oA6aFoguF4hxZJ Xedj9HEB2ZBO11PNbDUnXdJAcwhUNNg07xYKOzZ35wsNu4uYXHbbU4MZal0grZwIpD 8tVu68IOtNiACvLBL0YLDG9Zvp+lBGop0UK+58GQ= Date: Wed, 11 Nov 2026 17:17:12 -0800 To: mm-commits@vger.kernel.org,zhuhui@kylinos.cn,zhangbo56@xiaomi.com, yuanchu@google.com,weixugc@google.com,wangzicheng@honor.com, surenb@google.com,sidchintamaneni@gmail.com,shakeel.butt@linux.dev, ryncsn@gmail.com,rppt@kernel.org,ridong.chen@linux.dev,qi.zheng@linux.dev, mhocko@suse.com,lyugaofei@xiaomi.com,liuxiang.277@bytedance.com, lianux.mm@gmail.com,liam@infradead.org,kunwu.chan@gmail.com, hui.zhu@linux.dev,hannes@cmpxchg.org,baoquan.he@linux.dev, baolin.wang@linux.alibaba.com,baohua@kernel.org,axelrasmussen@google.com, kasong@tencent.com,akpm@linux-foundation.org From: Andrew Morton Subject: + mm-mglru-skip-zones-above-reclaim_idx-instead-of-rotating-them.patch added to mm-new branch Message-Id: <20261112011712.6C0011F00A05@smtp.kernel.org> X-Mailing-List: mm-commits@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list The patch titled Subject: mm/mglru: skip zones above reclaim_idx instead of rotating them has been added to the -mm mm-new branch. Its filename is mm-mglru-skip-zones-above-reclaim_idx-instead-of-rotating-them.patch This patch will shortly appear at https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/mm-mglru-skip-zones-above-reclaim_idx-instead-of-rotating-them.patch This patch will later appear in the mm-new branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm Note, mm-new is a provisional staging ground for work-in-progress patches, and acceptance into mm-new is a notification for others take notice and to finish up reviews. Please do not hesitate to respond to review feedback and post updated versions to replace or incrementally fixup patches in mm-new. The mm-new branch of mm.git is not included in linux-next If a few days of testing in mm-new is successful, the patch will me moved into mm.git's mm-unstable branch, which is included in linux-next Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's *** Remember to use Documentation/process/submit-checklist.rst when testing your code *** The -mm tree is included into linux-next via various branches at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm and is updated there most days ------------------------------------------------------ From: Kairui Song Subject: mm/mglru: skip zones above reclaim_idx instead of rotating them Date: Thu, 12 Nov 2026 07:25:52 +0800 scan_folios() used to visit every zone. Folios in zones above sc->reclaim_idx can never be isolated, so sort_folio() rotated them to a younger generation, the "ineligible" branch, or, when only one gen was left, skipped them. This wasted scan work on every pass, and the rotation kept the protected folios from aging. Restrict both the scan and the gen selection to the eligible zones, like the legacy LRU, which only isolates folios in zones up to reclaim_idx: - lruvec_populated_min_seq() takes a zone_limit, so the oldest populated gen is searched only among the zones the scan can reclaim from. A gen populated only in protected zones no longer stalls the scan. - scan_folios() iterates the eligible zones in the descending order of the legacy zonelist walk, draining the zone at reclaim_idx first. - The ineligible branch of sort_folio() is now dead and removed, with a VM_WARN_ON_ONCE_FOLIO() guarding the new invariant. try_to_inc_min_seq() still considers all zones, so min_seq[type] is only bumped once the oldest gen is empty everywhere. At DEF_PRIORITY the search bound stays min_seq[type], so the scan still starts from the oldest gen regardless of whether it is populated. Link: https://lore.kernel.org/20261111232603.9934-6-baohua@kernel.org Signed-off-by: Kairui Song Signed-off-by: Barry Song (Xiaomi) Signed-off-by: Andrew Morton Cc: Axel Rasmussen Cc: Baolin Wang Cc: Baoquan He Cc: Johannes Weiner Cc: Cc: Kunwu Chan Cc: Liam R. Howlett Cc: wang lian Cc: Cc: Cc: Michal Hocko Cc: Qi Zheng Cc: Ridong Chen Cc: Mike Rapoport Cc: Kairui Song Cc: Shakeel Butt Cc: Suren Baghdasaryan Cc: wangzicheng Cc: Wei Xu Cc: Yuanchu Xie Cc: Bo Zhang Cc: Hui Zhu Cc: Siddharth Chintamaneni --- mm/vmscan.c | 47 +++++++++++++++++++---------------------------- 1 file changed, 19 insertions(+), 28 deletions(-) --- a/mm/vmscan.c~mm-mglru-skip-zones-above-reclaim_idx-instead-of-rotating-them +++ a/mm/vmscan.c @@ -3992,7 +3992,7 @@ done: /* Similar to lruvec_evictable_size, return the oldest populated seq of the type */ static unsigned long lruvec_populated_min_seq(struct lruvec *lruvec, int type, - unsigned long max_seq) + int zone_limit, unsigned long max_seq) { int gen, zone; unsigned long min_seq; @@ -4003,7 +4003,7 @@ static unsigned long lruvec_populated_mi min_seq = READ_ONCE(lrugen->min_seq[type]); while (min_seq <= max_seq) { gen = lru_gen_from_seq(min_seq); - for (zone = 0; zone < MAX_NR_ZONES; zone++) { + for (zone = 0; zone < zone_limit; zone++) { if (!list_empty(&lrugen->folios[gen][type][zone])) return min_seq; } @@ -4026,7 +4026,7 @@ static void try_to_inc_min_seq(struct lr /* Requires at least MIN_NR_GENS gens for basic functionality */ seq = lrugen->max_seq - MIN_NR_GENS; for_each_evictable_type(type, swappiness) { - min_seq[type] = lruvec_populated_min_seq(lruvec, type, seq + 1); + min_seq[type] = lruvec_populated_min_seq(lruvec, type, MAX_NR_ZONES, seq + 1); if (min_seq[type] > lrugen->min_seq[type]) seq_inc_flag = true; } @@ -4691,6 +4691,7 @@ static bool sort_folio(struct lruvec *lr struct lru_gen_folio *lrugen = &lruvec->lrugen; VM_WARN_ON_ONCE_FOLIO(gen >= MAX_NR_GENS, folio); + VM_WARN_ON_ONCE_FOLIO(zone > sc->reclaim_idx, folio); /* unevictable: let it through and the generic path will cull it */ if (!folio_evictable(folio)) @@ -4707,10 +4708,8 @@ static bool sort_folio(struct lruvec *lr * longer be promoted or protected, so just isolate them. The one * exception is lru_gen_reparent_memcg(), which is handled above. */ - if (min_gen == max_gen) { - VM_WARN_ON_ONCE(zone > sc->reclaim_idx); + if (min_gen == max_gen) return false; - } /* protected */ if (tier > tier_idx || refs + workingset == BIT(LRU_REFS_WIDTH) + 1) { @@ -4727,13 +4726,6 @@ static bool sort_folio(struct lruvec *lr return true; } - /* ineligible */ - if (zone > sc->reclaim_idx) { - gen = folio_inc_gen(lruvec, folio, min_gen); - list_move_tail(&folio->lru, &lrugen->folios[gen][type][zone]); - return true; - } - return false; } @@ -4765,16 +4757,17 @@ static int scan_folios(unsigned long nr_ struct scan_control *sc, int type, int tier, struct list_head *list, int *isolatedp) { - int i; enum node_stat_item item; + int zone_idx; int sorted = 0; int scanned = 0; int isolated = 0; int skipped = 0; int gen, max_gen; - unsigned long min_seq; + unsigned long seq; unsigned long remaining = nr_to_scan; struct lru_gen_folio *lrugen = &lruvec->lrugen; + struct pglist_data *pgdat = lruvec_pgdat(lruvec); DEFINE_MAX_SEQ(lruvec); VM_WARN_ON_ONCE(nr_to_scan > MAX_LRU_BATCH); @@ -4786,24 +4779,22 @@ static int scan_folios(unsigned long nr_ * when under pressure to respect swappiness more and reduce IO cost. */ if (sc->priority == DEF_PRIORITY) - min_seq = lrugen->min_seq[type]; + seq = lrugen->min_seq[type]; else - min_seq = lruvec_populated_min_seq(lruvec, type, lrugen->max_seq); + seq = lrugen->max_seq; - gen = lru_gen_from_seq(min_seq); + seq = lruvec_populated_min_seq(lruvec, type, sc->reclaim_idx + 1, seq); + gen = lru_gen_from_seq(seq); max_gen = lru_gen_from_seq(max_seq); - for (i = MAX_NR_ZONES; i > 0; i--) { + /* Scan eligible zones in descending order. */ + for (zone_idx = sc->reclaim_idx; zone_idx >= 0; zone_idx--) { LIST_HEAD(moved); int skipped_zone = 0; - int zone = (sc->reclaim_idx + i) % MAX_NR_ZONES; - struct list_head *head = &lrugen->folios[gen][type][zone]; + struct zone *zone = &pgdat->node_zones[zone_idx]; + struct list_head *head = &lrugen->folios[gen][type][zone_idx]; - /* - * If there is only one gen left, rotating the unreclaimable - * zone become meaningless, just skip them. - */ - if (gen == max_gen && zone > sc->reclaim_idx) + if (!managed_zone(zone)) continue; while (!list_empty(head)) { @@ -4813,7 +4804,7 @@ static int scan_folios(unsigned long nr_ VM_WARN_ON_ONCE_FOLIO(folio_test_unevictable(folio), folio); VM_WARN_ON_ONCE_FOLIO(folio_test_active(folio), folio); VM_WARN_ON_ONCE_FOLIO(folio_is_file_lru(folio) != type, folio); - VM_WARN_ON_ONCE_FOLIO(folio_zonenum(folio) != zone, folio); + VM_WARN_ON_ONCE_FOLIO(folio_zonenum(folio) != zone_idx, folio); scanned += delta; if (sort_folio(lruvec, folio, sc, gen, max_gen, tier)) @@ -4832,7 +4823,7 @@ static int scan_folios(unsigned long nr_ if (skipped_zone) { list_splice(&moved, head); - __count_zid_vm_events(PGSCAN_SKIP, zone, skipped_zone); + __count_zid_vm_events(PGSCAN_SKIP, zone_idx, skipped_zone); skipped += skipped_zone; } _ Patches currently in -mm which might be from kasong@tencent.com are mm-mglru-report-evictable-sizes-per-type.patch mm-mglru-extract-lruvec_populated_min_seq-helper.patch mm-mglru-reclaim-from-the-oldest-populated-gen.patch mm-mglru-lift-the-forced-protection-of-reclaimer.patch mm-mglru-skip-zones-above-reclaim_idx-instead-of-rotating-them.patch mm-mglru-return-for-unavailable-anon-reclaim-earlier.patch mm-mglru-balance-anon-and-file-scan-budget-by-refault-rate.patch mm-mglru-lift-youngest-gen-protection-for-higher-priority.patch