From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 135E91F12F4 for ; Wed, 17 Sep 2025 01:06:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758071188; cv=none; b=k/iebRdqWn7aL2XE+SfgRgT8dXkSTq9SSjK13hsHD4aX0S+mZ8oP+0MZIEhhuLHpy1bswTFLU6Zym6LJwunoW7UO1FH6WaGk1lZPAgeXaC84ppxOA6GI7knsac1zHx38rPkIK6Il8mt1vtTsOLKK9CX7E18hkfcuZmi0RsjU4nA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758071188; c=relaxed/simple; bh=w6NK4wmu0lmQ463xtOQfxAba8zhYvoWvPcHgXNOJNtk=; h=Date:To:From:Subject:Message-Id; b=Wx675l3c5e7dCgrrPmj7xMT7MG357YIZMARot/aLcaNUH5cmXvwkc5/DMukYIEfgYSwOd8VdxtN8zgYkd+omD7TycL7aLNpmqGI2goSYCpNTI3GdCzDyVjA+KNILlMbmSI2qcFZI9G/VvVhvkH80QGX2SXSHfmYNMfgTgy/Pug8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b=skQXtVV1; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b="skQXtVV1" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DF18FC4CEF9; Wed, 17 Sep 2025 01:06:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1758071187; bh=w6NK4wmu0lmQ463xtOQfxAba8zhYvoWvPcHgXNOJNtk=; h=Date:To:From:Subject:From; b=skQXtVV1mFe0sElOx9MkdUyw5cQVIbt0bplxZ7jeLQYS57lJgPHipr/alrBQhIRMG J6W20CaRbu9O1Upr+2upZXG72XRO+LT8YbVQU6XATvqgDD30h1t0iTeRXWWZspuA6C /V2zE8H4L0zsqAecYgffWcpW0QNecaUvEJU81l+0= Date: Tue, 16 Sep 2025 18:06:27 -0700 To: mm-commits@vger.kernel.org,ziy@nvidia.com,yosryahmed@google.com,ying.huang@linux.alibaba.com,willy@infradead.org,shikemeng@huaweicloud.com,oliver.sang@intel.com,nphamcs@gmail.com,lorenzo.stoakes@oracle.com,hughd@google.com,hannes@cmpxchg.org,david@redhat.com,chrisl@kernel.org,bhe@redhat.com,baolin.wang@linux.alibaba.com,baohua@kernel.org,kasong@tencent.com,akpm@linux-foundation.org From: Andrew Morton Subject: + mm-swap-use-a-single-page-for-swap-table-when-the-size-fits.patch added to mm-unstable branch Message-Id: <20250917010627.DF18FC4CEF9@smtp.kernel.org> Precedence: bulk X-Mailing-List: mm-commits@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: The patch titled Subject: mm, swap: use a single page for swap table when the size fits has been added to the -mm mm-unstable branch. Its filename is mm-swap-use-a-single-page-for-swap-table-when-the-size-fits.patch This patch will shortly appear at https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/mm-swap-use-a-single-page-for-swap-table-when-the-size-fits.patch This patch will later appear in the mm-unstable branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's *** Remember to use Documentation/process/submit-checklist.rst when testing your code *** The -mm tree is included into linux-next via the mm-everything branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm and is updated there every 2-3 working days ------------------------------------------------------ From: Kairui Song Subject: mm, swap: use a single page for swap table when the size fits Date: Wed, 17 Sep 2025 00:01:00 +0800 We have a cluster size of 512 slots. Each slot consumes 8 bytes in swap table so the swap table size of each cluster is exactly one page (4K). If that condition is true, allocate one page direct and disable the slab cache to reduce the memory usage of swap table and avoid fragmentation. Link: https://lkml.kernel.org/r/20250916160100.31545-16-ryncsn@gmail.com Co-developed-by: Chris Li Signed-off-by: Chris Li Signed-off-by: Kairui Song Acked-by: Chris Li Suggested-by: Chris Li Cc: Baolin Wang Cc: Baoquan He Cc: Barry Song Cc: David Hildenbrand Cc: "Huang, Ying" Cc: Hugh Dickins Cc: Johannes Weiner Cc: Kemeng Shi Cc: kernel test robot Cc: Lorenzo Stoakes Cc: Matthew Wilcox (Oracle) Cc: Nhat Pham Cc: Yosry Ahmed Cc: Zi Yan Signed-off-by: Andrew Morton --- mm/swap_table.h | 2 + mm/swapfile.c | 51 ++++++++++++++++++++++++++++++++++++---------- 2 files changed, 43 insertions(+), 10 deletions(-) --- a/mm/swapfile.c~mm-swap-use-a-single-page-for-swap-table-when-the-size-fits +++ a/mm/swapfile.c @@ -434,6 +434,38 @@ static inline unsigned int cluster_offse return cluster_index(si, ci) * SWAPFILE_CLUSTER; } +static struct swap_table *swap_table_alloc(gfp_t gfp) +{ + struct folio *folio; + + if (!SWP_TABLE_USE_PAGE) + return kmem_cache_zalloc(swap_table_cachep, gfp); + + folio = folio_alloc(gfp | __GFP_ZERO, 0); + if (folio) + return folio_address(folio); + return NULL; +} + +static void swap_table_free_folio_rcu_cb(struct rcu_head *head) +{ + struct folio *folio; + + folio = page_folio(container_of(head, struct page, rcu_head)); + folio_put(folio); +} + +static void swap_table_free(struct swap_table *table) +{ + if (!SWP_TABLE_USE_PAGE) { + kmem_cache_free(swap_table_cachep, table); + return; + } + + call_rcu(&(folio_page(virt_to_folio(table), 0)->rcu_head), + swap_table_free_folio_rcu_cb); +} + static void swap_cluster_free_table(struct swap_cluster_info *ci) { unsigned int ci_off; @@ -447,7 +479,7 @@ static void swap_cluster_free_table(stru table = (void *)rcu_dereference_protected(ci->table, true); rcu_assign_pointer(ci->table, NULL); - kmem_cache_free(swap_table_cachep, table); + swap_table_free(table); } /* @@ -470,8 +502,7 @@ swap_cluster_alloc_table(struct swap_inf /* The cluster must be free and was just isolated from the free list. */ VM_WARN_ON_ONCE(ci->flags || !cluster_is_empty(ci)); - table = kmem_cache_zalloc(swap_table_cachep, - __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN); + table = swap_table_alloc(__GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN); if (table) { rcu_assign_pointer(ci->table, table); return ci; @@ -487,8 +518,7 @@ swap_cluster_alloc_table(struct swap_inf spin_unlock(&si->global_cluster_lock); local_unlock(&percpu_swap_cluster.lock); - table = kmem_cache_zalloc(swap_table_cachep, - __GFP_HIGH | __GFP_NOMEMALLOC | GFP_KERNEL); + table = swap_table_alloc(__GFP_HIGH | __GFP_NOMEMALLOC | GFP_KERNEL); /* * Back to atomic context. We might have migrated to a new CPU with a @@ -506,7 +536,7 @@ swap_cluster_alloc_table(struct swap_inf /* Nothing except this helper should touch a dangling empty cluster. */ if (WARN_ON_ONCE(cluster_table_is_alloced(ci))) { if (table) - kmem_cache_free(swap_table_cachep, table); + swap_table_free(table); return ci; } @@ -734,7 +764,7 @@ static int inc_cluster_info_page(struct ci = cluster_info + idx; if (!ci->table) { - table = kmem_cache_zalloc(swap_table_cachep, GFP_KERNEL); + table = swap_table_alloc(GFP_KERNEL); if (!table) return -ENOMEM; rcu_assign_pointer(ci->table, table); @@ -4072,9 +4102,10 @@ static int __init swapfile_init(void) * only, and all swap cache readers (swap_cache_*) verifies * the content before use. So it's safe to use RCU slab here. */ - swap_table_cachep = kmem_cache_create("swap_table", - sizeof(struct swap_table), - 0, SLAB_PANIC | SLAB_TYPESAFE_BY_RCU, NULL); + if (!SWP_TABLE_USE_PAGE) + swap_table_cachep = kmem_cache_create("swap_table", + sizeof(struct swap_table), + 0, SLAB_PANIC | SLAB_TYPESAFE_BY_RCU, NULL); #ifdef CONFIG_MIGRATION if (swapfile_maximum_size >= (1UL << SWP_MIG_TOTAL_BITS)) --- a/mm/swap_table.h~mm-swap-use-a-single-page-for-swap-table-when-the-size-fits +++ a/mm/swap_table.h @@ -11,6 +11,8 @@ struct swap_table { atomic_long_t entries[SWAPFILE_CLUSTER]; }; +#define SWP_TABLE_USE_PAGE (sizeof(struct swap_table) == PAGE_SIZE) + /* * A swap table entry represents the status of a swap slot on a swap * (physical or virtual) device. The swap table in each cluster is a _ Patches currently in -mm which might be from kasong@tencent.com are mm-swap-use-unified-helper-for-swap-cache-look-up.patch mm-swap-fix-swap-cache-index-error-when-retrying-reclaim.patch mm-swap-check-page-poison-flag-after-locking-it.patch mm-swap-always-lock-and-check-the-swap-cache-folio-before-use.patch mm-swap-rename-and-move-some-swap-cluster-definition-and-helpers.patch mm-swap-tidy-up-swap-device-and-cluster-info-helpers.patch mm-swap-cleanup-swap-cache-api-and-add-kerneldoc.patch mm-shmem-swap-remove-redundant-error-handling-for-replacing-folio.patch mm-swap-wrap-swap-cache-replacement-with-a-helper.patch mm-swap-use-the-swap-table-for-the-swap-cache-and-switch-api.patch mm-swap-mark-swap-address-space-ro-and-add-context-debug-check.patch mm-swap-remove-contention-workaround-for-swap-cache.patch mm-swap-implement-dynamic-allocation-of-swap-table.patch mm-swap-use-a-single-page-for-swap-table-when-the-size-fits.patch