From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 91EA0403EBA for ; Thu, 2 Jul 2026 12:23:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782995031; cv=none; b=sza4HBpFTL1RVLqK0GNeMgkCR6vURHX+QR83s1pZd0yhp9s1JHzfYM6xMmAeKVCxJRDOD7+zbIshKtfXBXwWcW5Vx12dGhMaLRbW0Vojiag+qXD4vbA+o23fZrQqJ3UwGRSZN7u2I0b2ssxnbpKEnDsch7r5vj6DQYk5NDaWGSs= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782995031; c=relaxed/simple; bh=qGMx9eunjKj1+HroyMYGm+yuHaEyBdNfYh03Smp56UI=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=phjHlyTdb7vtIy3EWPZuOF8H5mWxLIaKylP80pwozy3W4KgIRtyxNHXr/09UOMeIyl42TWfHlwvr8UDIlfjIatHP3wv+wxoovxPMOxbIFp0Y6sJn7XdtlalaD02RlULHFdEE0ErB7PvFcBxyqPJ6tiMDtncLr2Fv4xs9iZIFkis= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=GKGhU4no; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="GKGhU4no" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E3B6F1F00AC4; Thu, 2 Jul 2026 12:23:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1782995030; bh=FU6Q1fEXtxxFWF1vTBxkrkETlNKEIy1bqSJpKDM+1rI=; h=From:To:Cc:Subject:Date; b=GKGhU4noxDfkoAKHiEPSl/Q7q989zThZnfJpP3ywsZE6DZtejVy/kZo8CpTk5dFqP jBJdKDcvgdKcHWyUwa4jlIE/vMugNv1iIsoaWuXgljp1dIlbyGzQ1Cp9rRI2fID7nc uHAAd2EWV3Hzh8vMbBWyu6VgkL02ZeJ5VxXDjs1sswv2SEEL6DWcGUtTcKRFVNAmsy JnjgPA5ZqwxCthon6M9uTgk0tP85lJ6j8bTAvcjSHTzGYoKjurExGJw3tRwr8DdyZ9 PQ3yMjteVAmCfoStzFIFYCm+twbFs35jMj2oaf0inay7+68lYC6SM9xo3Qukqwt2VL o+MgAiIe7LDeA== From: "Barry Song (Xiaomi)" To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, "Barry Song (Xiaomi)" , David Hildenbrand , Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Lance Yang Subject: [RFC PATCH] mm: allow smaller large folios to use lru_cache Date: Thu, 2 Jul 2026 20:23:42 +0800 Message-Id: <20260702122342.26721-1-baohua@kernel.org> X-Mailer: git-send-email 2.39.3 (Apple Git-146) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit For a system which primarily uses smaller orders for large folios, it could be beneficial to enable lru_cache to avoid lock contention. For large folios with higher orders, the number of folios involved would be small, so the contention is less significant. The following small program runs on a system with 16 KiB mTHP enabled. #include #include #include #define NUM_THREADS 20 #define MEM_SIZE (16 * 1024 * 1024) #define LOOP_COUNT 1000 void* thread_worker(void* arg) { void *addr = mmap(NULL, MEM_SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); for (int i = 0; i < LOOP_COUNT; i++) { memset(addr, 0x55, MEM_SIZE); madvise(addr, MEM_SIZE, MADV_DONTNEED); } munmap(addr, MEM_SIZE); pthread_exit(NULL); } int main() { pthread_t threads[NUM_THREADS]; for (long t = 0; t < NUM_THREADS; t++) { pthread_create(&threads[t], NULL, thread_worker, (void*)t); } for (int t = 0; t < NUM_THREADS; t++) { pthread_join(threads[t], NULL); } return 0; } Before patch: root@barry-desktop:/home/barry# time ./a.out real 0m20.233s user 0m21.739s sys 6m21.143s Perf lock report: Name acquired contended avg wait total wait max wait min wait 21158662 21158662 29.20 us 10.30 m 67.83 us 884 ns 13547 13547 14.68 us 198.93 ms 135.39 us 982 ns rcu_state 41 41 1.83 us 75.21 us 3.46 us 1.27 us rcu_state 34 34 1.83 us 62.09 us 2.60 us 1.16 us 5 5 21.31 us 106.57 us 25.77 us 18.17 us 5 5 3.01 us 15.07 us 5.19 us 1.53 us 3 3 6.42 us 19.26 us 9.63 us 4.22 us 3 3 4.20 us 12.61 us 7.45 us 2.10 us 2 2 5.03 us 10.06 us 7.88 us 2.18 us 2 2 2.79 us 5.59 us 3.34 us 2.25 us 1 1 4.70 us 4.70 us 4.70 us 4.70 us 1 1 9.77 us 9.77 us 9.77 us 9.77 us After patch: root@barry-desktop:/home/barry# time ./a.out real 0m17.796s user 0m24.962s sys 5m28.774s Perf lock report: Name acquired contended avg wait total wait max wait min wait 1407013 1407013 25.44 us 35.80 s 127.83 us 972 ns 845113 845113 47.96 us 40.54 s 152.86 us 1.21 us rcu_state 1920 1920 5.87 us 11.27 ms 25.02 us 1.76 us rcu_state 1889 1889 5.80 us 10.96 ms 30.35 us 1.61 us 140 140 3.97 us 555.52 us 12.51 us 1.20 us 10 10 61.34 us 613.42 us 96.90 us 44.48 us 8 8 24.03 us 192.26 us 37.64 us 6.95 us 8 8 12.32 us 98.56 us 25.55 us 4.16 us Both system time and lock contention are noticeably reduced. Cc: David Hildenbrand Cc: Lorenzo Stoakes Cc: Zi Yan Cc: Baolin Wang Cc: Liam R. Howlett Cc: Nico Pache Cc: Ryan Roberts Cc: Dev Jain Cc: Lance Yang Signed-off-by: Barry Song (Xiaomi) --- include/linux/swap.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index 8f0f68e245ba..fce1d84aa2c8 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -322,9 +322,9 @@ static inline bool folio_may_be_lru_cached(struct folio *folio) /* * Holding PMD-sized folios in per-CPU LRU cache unbalances accounting. * Holding small numbers of low-order mTHP folios in per-CPU LRU cache - * will be sensible, but nobody has implemented and tested that yet. + * will be sensible. */ - return !folio_test_large(folio); + return folio_order(folio) < PAGE_ALLOC_COSTLY_ORDER; } extern atomic_t lru_disable_count; -- 2.39.3 (Apple Git-146)