From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 27B3C47CA64 for ; Fri, 21 Aug 2026 10:26:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787308010; cv=none; b=Gob9KtaTnXz39SEctXDrVUZHnMo4iLoPeFj9LsHaL6q4V/Z1O1+1pQy4VjxZ9qUlbIhdpXKB+WUMCFxYHWJO44/OO0W0PbH/hiPTYQ7bvP3XOnzMzRYGoHKphSMLxz6hmsXdG5rg6ZmZq5GLilH8Ki98aodaI5u+1nFvW7Zavl8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787308010; c=relaxed/simple; bh=YF8lnrLkcB0qrQEcSpeDBGFiBxT7QOoiq2glgdnd18U=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=QJpggIBt7Y49bXBbdGLYfbvV5QITO1FJXOZlvyt963Qsw2G60jhmssi13nRn+RXW6kOarKxBt1af/Y5w1AR4LeXvsBnboOamYfSDOkjllcuYwK2edGNjfMXRq0AR+CpkiWnCFp1kscTVLpz4R9NXIZIogYXWqrIq3NVg+V0IC64= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ielwHHfe; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ielwHHfe" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B5A2B1F00A3A; Fri, 21 Aug 2026 10:26:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1787307998; bh=DUPbWhkO9uqnpeFMVIsJBlpt8zN0k3f8YPV9jGEypkU=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=ielwHHfesCgnNzots4rpLldqZiC88Hm8FP2b2C/4oMMSgPDfDTgfH2gjNE0Ii18zO lLNNSfD0C5FObB2dU+gKoa+DrgFLYnRJOA7kCygQeBw0rr9nBnQOkQqrQJoJ/N8Gl+ hhFEkgRcxVUSfLbyr6uaiXv3+PGa/vVZly6xlnW5G1OHLybXzeuTPMmFVxxHCZTYEO GjsYecacC08cdmzbdAHnkQXRIwmVziWgZQPAS1a+Ddw/SZsP5V9rZ661O76OcEc1gb XCfUP4/Y01cqNsITdHx6okr4MIiBDwQvxRxeeLg+59W9QqAD4q1PAFs8txYCv0AkRc 29SoD1XBg71HQ== From: "Barry Song (Xiaomi)" To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: axelrasmussen@google.com, baolin.wang@linux.alibaba.com, baoquan.he@linux.dev, chenridong@xiaomi.com, david@kernel.org, hannes@cmpxchg.org, kasong@tencent.com, lianux.mm@gmail.com, linux-kernel@vger.kernel.org, ljs@kernel.org, lyugaofei@xiaomi.com, mhocko@kernel.org, qi.zheng@linux.dev, shakeel.butt@linux.dev, stevensd@chromium.org, wangzicheng@honor.com, weixugc@google.com, yuanchu@google.com, zhangbo56@xiaomi.com, "Barry Song (Xiaomi)" Subject: [PATCH 5/6] mm/mglru: move folios from oldest gen to second-oldest gen from head to tail Date: Fri, 21 Aug 2026 18:25:37 +0800 Message-Id: <20260821102538.22642-6-baohua@kernel.org> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20260821102538.22642-1-baohua@kernel.org> References: <20260821102538.22642-1-baohua@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit For reclamation, it makes sense to reclaim folios from tail to head, as folios near the head are relatively hot. However, when moving folios from the oldest generation to the second-oldest generation, using the tail-to-head order would effectively cause a cold/hot inversion. Signed-off-by: Barry Song (Xiaomi) --- mm/vmscan.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 7bd01875fade..2fd82b2ca4d1 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -191,8 +191,20 @@ struct scan_control { prefetchw(&prev->_field); \ } \ } while (0) +#define prefetchw_next_lru_folio(_folio, _base, _field) \ + do { \ + if ((_folio)->lru.next != _base) { \ + struct folio *next; \ + \ + next = list_entry((_folio)->lru.next, \ + struct folio, lru); \ + prefetchw(&next->_field); \ + } \ + } while (0) + #else #define prefetchw_prev_lru_folio(_folio, _base, _field) do { } while (0) +#define prefetchw_next_lru_folio(_folio, _base, _field) do { } while (0) #endif /* @@ -3932,9 +3944,10 @@ static bool inc_min_seq(struct lruvec *lruvec, int type, int swappiness) for (zone = 0; zone < MAX_NR_ZONES; zone++) { struct list_head *head = &lrugen->folios[old_gen][type][zone]; unsigned long protected[MAX_NR_TIERS] = {}, delta = 0; + struct list_head *pos = head->next; - while (!list_empty(head)) { - struct folio *folio = lru_to_folio(head); + while (pos != head) { + struct folio *folio = list_entry(pos, struct folio, lru); long nr_pages = folio_nr_pages(folio); int refs = folio_lru_refs(folio); bool workingset = folio_test_workingset(folio); @@ -3945,6 +3958,8 @@ static bool inc_min_seq(struct lruvec *lruvec, int type, int swappiness) VM_WARN_ON_ONCE_FOLIO(folio_is_file_lru(folio) != type, folio); VM_WARN_ON_ONCE_FOLIO(folio_zonenum(folio) != zone, folio); + prefetchw_next_lru_folio(folio, head, flags); + pos = pos->next; new_gen = __folio_inc_gen(folio, old_gen, &gen_increased); if (gen_increased) { delta += nr_pages; -- 2.34.1