From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=100.103.45.18 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1794446237; b=mJxtpuGtu1yZWacq8PdYIhDSJow4G9cIMu4u3pbFRZnD+JgC9rg5cIzx7Fh+jpT7jyD/ AHIP9coPfULaAwnkQoyLpOjBqPhHKAUfrmTkhKQ2f6naWFm9fVQBedHIYGjG/Eld3lPGz SCjnyDrl2yOtqa16zXVIwJOmXe++oA9hHv96QbXzn23yiGDPxv+bWH7TUfgBS58cqudsQ P7IOMH5mEBimuUI+9wvp6oBpbDxCSRH6DKqyOGVBW+KLh2+QPf9XMf1oYPE9Xnpc72IGX +9/iWydX84CDwTZXHGZ7kCnbg4ZYHTErGURmn7CG0F5N7E9gYZyEGvU5OPcWWhdmh0g== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1794446237; h=Received:DKIM-Signature:Date:To:From:Subject:Message-Id; bh=zHdreTQ4R6EqQ4zWcJWrSEBF5WrJHznKGa+rK2Emnws=; b=sjyOylWfuuVmf4/dFpoKyqZzB5iMV+Op2azgHqDl0cFdtxS6wYPxlbzAC7MnFCFvzKA4 4OogYH6EBLGd5VXScw1QAD4OzXa1VNn5DX3SNz7vY4pXpoXH4UHVGMlsqbb6jW7cCLSsw qsa7291fJC6FwJfdrT3cXlBDbyt/NNrpAdKzvnUngsV9RLz+pKfeQDRKSmCGZEXQv6wLS Ulr1Bx76cNeuLhtheIfdY8k/12caEVV7puxLutdno9i4cbMQPqHFTFfvZjrAdR6OM2vd8 IriF0I6oESjufw7qZqg82IsMVBy8MUAAJgcmpd/KTPZlxH50OKmQchnC53HaKiQZDUw== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass header.d=linux-foundation.org header.i=@linux-foundation.org header.a=rsa-sha256 header.s=korg header.b=ecpgfSeS; arc=none smtp.remote-ip=100.103.45.18 Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D7820606A336 for ; Thu, 12 Nov 2026 01:17:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.a=rsa-sha256 header.s=korg header.b=ecpgfSeS Received: by smtp.kernel.org (Postfix) with ESMTPSA id A3CB31F00A05; Thu, 12 Nov 2026 01:17:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=korg; t=1794446237; bh=zHdreTQ4R6EqQ4zWcJWrSEBF5WrJHznKGa+rK2Emnws=; h=Date:To:From:Subject; b=ecpgfSeS2V/FpflR9XwVKnHXxBN7EuDOVEFrdKWrlMkTOUy7Z1Hz48pObS+u65KQ+ T/Yv67HpC3JdizbC6+Eyk3fQwDzUKpzEsDu35aCogG28Pf78aKatCA6/7CTBWQ3uVj 9Aj1fHxO3Ygw+grAjffTsVAMjevLqp3O4x7LnAtY= Date: Wed, 11 Nov 2026 17:17:17 -0800 To: mm-commits@vger.kernel.org,zhuhui@kylinos.cn,zhangbo56@xiaomi.com, yuanchu@google.com,weixugc@google.com,wangzicheng@honor.com, surenb@google.com,sidchintamaneni@gmail.com,shakeel.butt@linux.dev, rppt@kernel.org,ridong.chen@linux.dev,qi.zheng@linux.dev,mhocko@suse.com, lyugaofei@xiaomi.com,liuxiang.277@bytedance.com,lianux.mm@gmail.com, liam@infradead.org,kunwu.chan@gmail.com,kasong@tencent.com,hui.zhu@linux.dev, hannes@cmpxchg.org,chentao@kylinos.cn,baoquan.he@linux.dev, baolin.wang@linux.alibaba.com,axelrasmussen@google.com,baohua@kernel.org, akpm@linux-foundation.org From: Andrew Morton Subject: + mm-mglru-avoid-placing-rejected-folios-below-populated-gen.patch added to mm-new branch Message-Id: <20261112011717.A3CB31F00A05@smtp.kernel.org> X-Mailing-List: mm-commits@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list The patch titled Subject: mm/mglru: avoid placing rejected folios below populated gen has been added to the -mm mm-new branch. Its filename is mm-mglru-avoid-placing-rejected-folios-below-populated-gen.patch This patch will shortly appear at https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/mm-mglru-avoid-placing-rejected-folios-below-populated-gen.patch This patch will later appear in the mm-new branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm Note, mm-new is a provisional staging ground for work-in-progress patches, and acceptance into mm-new is a notification for others take notice and to finish up reviews. Please do not hesitate to respond to review feedback and post updated versions to replace or incrementally fixup patches in mm-new. The mm-new branch of mm.git is not included in linux-next If a few days of testing in mm-new is successful, the patch will me moved into mm.git's mm-unstable branch, which is included in linux-next Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's *** Remember to use Documentation/process/submit-checklist.rst when testing your code *** The -mm tree is included into linux-next via various branches at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm and is updated there most days ------------------------------------------------------ From: "Barry Song (Xiaomi)" Subject: mm/mglru: avoid placing rejected folios below populated gen Date: Thu, 12 Nov 2026 07:25:56 +0800 We used to scan only the oldest generation in scan_folios(). In evict_folios(), we checked whether lru_gen_folio_seq(lruvec, folio, false) == min_seq[type]. If so, the folio was marked active and moved to younger generations. This was itself buggy, as folios in generation min_seq[type] - 1 would not be marked active by folio_set_active(), potentially causing a cold/hot inversion. Besides this issue, the old approach also differs from the current behavior, where we can scan fewer than two generations. Now that we can scan younger generations, including the youngest generation, this no longer prevents rejected folios from being placed in a generation below or at the populated generation. Such rejected folios would then be scanned again and again, resulting in extremely low `pg_steal/pg_scan` ratios. For example: pgsteal_file: 2312605 pgscan_file: 36465693 reclamation efficiency: 6.34% We pass `scanned_seq` to `move_folios_to_lru()` and make sure we either promote the folio or place it in the youngest generation when the populated generation is already the youngest one. Also, folio_set_active() is no longer needed. With this change, the results become: pgsteal_file: 2683815 pgscan_file: 4224244 reclamation efficiency: 63.53% The sys time for building the kernel also decreased from 4m30.532s to 3m50.188s while swappiness==1. Link: https://lore.kernel.org/20261111232603.9934-10-baohua@kernel.org Signed-off-by: Barry Song (Xiaomi) Signed-off-by: Andrew Morton Reviewed-by: Kunwu Chan Cc: Axel Rasmussen Cc: Baolin Wang Cc: Baoquan He Cc: Johannes Weiner Cc: Cc: Kairui Song Cc: Liam R. Howlett Cc: wang lian Cc: Cc: Cc: Michal Hocko Cc: Qi Zheng Cc: Ridong Chen Cc: Mike Rapoport Cc: Shakeel Butt Cc: Suren Baghdasaryan Cc: wangzicheng Cc: Wei Xu Cc: Yuanchu Xie Cc: Bo Zhang Cc: Hui Zhu Cc: Siddharth Chintamaneni Cc: Kunwu Chan --- include/linux/mm_inline.h | 22 +++++++++++++++++++++- mm/vmscan.c | 24 ++++++++++++------------ 2 files changed, 33 insertions(+), 13 deletions(-) --- a/include/linux/mm_inline.h~mm-mglru-avoid-placing-rejected-folios-below-populated-gen +++ a/include/linux/mm_inline.h @@ -332,7 +332,8 @@ static inline unsigned long lru_gen_foli return max(READ_ONCE(lrugen->max_seq) - gen + 1, READ_ONCE(lrugen->min_seq[type])); } -static inline bool lru_gen_add_folio(struct lruvec *lruvec, struct folio *folio, bool reclaiming) +static inline bool __lru_gen_add_folio(struct lruvec *lruvec, struct folio *folio, + unsigned long scanned_seq, bool reclaiming) { unsigned long seq; unsigned long flags; @@ -347,6 +348,8 @@ static inline bool lru_gen_add_folio(str return false; seq = lru_gen_folio_seq(lruvec, folio, reclaiming); + if (scanned_seq != -1UL) + seq = max(seq, min(scanned_seq + 1, lrugen->max_seq)); gen = lru_gen_from_seq(seq); flags = (gen + 1UL) << LRU_GEN_PGOFF; /* see the comment on MIN_NR_GENS about PG_active */ @@ -362,6 +365,17 @@ static inline bool lru_gen_add_folio(str return true; } +static inline bool lru_gen_add_folio(struct lruvec *lruvec, struct folio *folio, bool reclaiming) +{ + return __lru_gen_add_folio(lruvec, folio, -1UL, reclaiming); +} + +static inline bool lru_gen_add_rejected_folio(struct lruvec *lruvec, struct folio *folio, + unsigned long scanned_seq) +{ + return __lru_gen_add_folio(lruvec, folio, scanned_seq, false); +} + static inline bool lru_gen_del_folio(struct lruvec *lruvec, struct folio *folio, bool reclaiming) { unsigned long flags; @@ -419,6 +433,12 @@ static inline bool lru_gen_add_folio(str { return false; } + +static inline bool lru_gen_add_rejected_folio(struct lruvec *lruvec, struct folio *folio, + unsigned long scanned_seq) +{ + return false; +} static inline bool lru_gen_del_folio(struct lruvec *lruvec, struct folio *folio, bool reclaiming) { --- a/mm/vmscan.c~mm-mglru-avoid-placing-rejected-folios-below-populated-gen +++ a/mm/vmscan.c @@ -1865,7 +1865,8 @@ static bool too_many_isolated(struct pgl * * Note: The caller must not hold any lruvec lock. */ -static unsigned int move_folios_to_lru(struct list_head *list, bool do_rotate) +static unsigned int move_folios_to_lru(struct list_head *list, unsigned long seq, + bool do_rotate) { int nr_pages, nr_moved = 0; struct lruvec *lruvec = NULL; @@ -1922,7 +1923,7 @@ static unsigned int move_folios_to_lru(s !folio_test_referenced(folio) && folio_ref_count(folio) == folio_expected_ref_count(folio)) lruvec_add_folio_tail(lruvec, folio); - else + else if (seq == -1UL || !lru_gen_add_rejected_folio(lruvec, folio, seq)) lruvec_add_folio(lruvec, folio); nr_pages = folio_nr_pages(folio); @@ -2041,7 +2042,7 @@ static unsigned long shrink_inactive_lis nr_reclaimed = shrink_folio_list(&folio_list, pgdat, sc, &stat, false, lruvec_memcg(lruvec)); - move_folios_to_lru(&folio_list, true); + move_folios_to_lru(&folio_list, -1UL, true); mod_lruvec_state(lruvec, PGDEMOTE_KSWAPD + reclaimer_offset(sc), stat.nr_demoted); @@ -2152,8 +2153,8 @@ static void shrink_active_list(unsigned /* * Move folios back to the lru list. */ - nr_activate = move_folios_to_lru(&l_active, false); - nr_deactivate = move_folios_to_lru(&l_inactive, false); + nr_activate = move_folios_to_lru(&l_active, -1UL, false); + nr_deactivate = move_folios_to_lru(&l_inactive, -1UL, false); count_vm_events(PGDEACTIVATE, nr_deactivate); count_memcg_events(lruvec_memcg(lruvec), PGDEACTIVATE, nr_deactivate); @@ -4755,7 +4756,8 @@ static bool isolate_folio(struct lruvec static int scan_folios(unsigned long nr_to_scan, struct lruvec *lruvec, struct scan_control *sc, int type, int tier, - struct list_head *list, int *isolatedp) + struct list_head *list, int *isolatedp, + unsigned long *scanned_seq) { enum node_stat_item item; int zone_idx; @@ -4786,6 +4788,7 @@ static int scan_folios(unsigned long nr_ seq = lrugen->max_seq; seq = lruvec_populated_min_seq(lruvec, type, sc->reclaim_idx + 1, seq); + *scanned_seq = seq; gen = lru_gen_from_seq(seq); max_gen = lru_gen_from_seq(max_seq); @@ -4982,6 +4985,7 @@ static int evict_folios(unsigned long nr int tier, scanned, reclaimed; int isolated = 0, nr_isolated = 0; unsigned long total_reclaimed = 0; + unsigned long scanned_seq; bool skip_retry = false; struct mem_cgroup *memcg = lruvec_memcg(lruvec); struct pglist_data *pgdat = lruvec_pgdat(lruvec); @@ -4993,7 +4997,7 @@ static int evict_folios(unsigned long nr tier = get_tier_idx(lruvec, type); scanned = scan_folios(nr_to_scan, lruvec, sc, - type, tier, &list, &isolated); + type, tier, &list, &isolated, &scanned_seq); nr_isolated = isolated; if (nr_isolated) __mod_node_page_state(pgdat, NR_ISOLATED_ANON + type, @@ -5019,8 +5023,6 @@ retry: type ? LRU_INACTIVE_FILE : LRU_INACTIVE_ANON); list_for_each_entry_safe_reverse(folio, next, &list, lru) { - DEFINE_MIN_SEQ(lruvec); - /* move_folios_to_lru() culls unevictable folios via folio_putback_lru() */ if (!folio_evictable(folio)) continue; @@ -5039,11 +5041,9 @@ retry: * so this effectively promotes them by at least one generation. */ folio_set_lru_refs(folio, 0); - if (lru_gen_folio_seq(lruvec, folio, false) == min_seq[type]) - folio_set_active(folio); } - move_folios_to_lru(&list, false); + move_folios_to_lru(&list, scanned_seq, false); walk = current->reclaim_state->mm_walk; if (walk && walk->batched) { _ Patches currently in -mm which might be from baohua@kernel.org are arm64-vmalloc-allow-arch_vmap_pte_range_map_size-to-batch-multiple-cont_pte.patch mm-vmalloc-extend-page-table-walk-to-support-larger-page_shift-sizes-and-eliminate-page-table-rewalk.patch mm-vmalloc-map-contiguous-pages-in-batches-for-vmap-if-possible.patch mm-vmalloc-align-vm_area-so-vmap-can-batch-mappings.patch mm-mglru-avoid-placing-rejected-folios-below-populated-gen.patch mm-mglru-run-aging-when-one-type-has-no-populatable-folios.patch mm-mglru-restrict-eviction-accounting-to-eligible-zones.patch mm-mglru-reduce-lru-lock-contention-by-shrinking-folio-batches-in-inc_min_seq.patch mm-mglru-make-swappiness=1-file-reclaim-more-aggressive.patch