From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9015438F95A for ; Mon, 18 May 2026 21:52:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779141146; cv=none; b=ibSYWj0V+v8qt2dR3QhM65e3wUcuGxkO2Flzwg5SleLwyPwal1KbfmCIg07ROABjnNQD3zpAJG40WPWzG4oZqWDsld7qA6Nypar0JBBf+eV2WkUeiO2Mp0qTgsiqO4sjRBP2Hsj+Giixca0LFkprDpkzYM07iI/LpCtBWNL8/W8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779141146; c=relaxed/simple; bh=0fjqTD4hlAF7oC31NtSFIY8Y/uon3Q6WUGKcoe9dLaA=; h=Date:To:From:Subject:Message-Id; b=DEE2l4EkDvMUPW3cdpgLAqmvq8j01ijxGe28OjlDUr0zmnfKyWiCRnpFQcc8n6fn7eUU9W7OJgJFq+Poc8YClxsVG7sRg2ck0rX/7iZGgNIkHIeoDp5k76KTAwSGC/Xjap9qA6h8I+uTat55FGGEQBBzQwzjcWBB0uU3r8zDjE8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b=2sFecITG; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b="2sFecITG" Received: by smtp.kernel.org (Postfix) with ESMTPSA id F1160C2BCB7; Mon, 18 May 2026 21:52:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1779141146; bh=0fjqTD4hlAF7oC31NtSFIY8Y/uon3Q6WUGKcoe9dLaA=; h=Date:To:From:Subject:From; b=2sFecITGPdjDyeB64ly680Bh70sDgezOrPKsjkz+gMG3PcRtz/1rqtuEaSJnKJKOH Ko5FMaJQDGGOz0KsqEJlEQOFRYoMY+nCNFr75u2eaFKBrYCAOSx7Kd+VmFHROzmOEa Ge0rv4WShbryPVHX+lZzkCPMZLvwtl8X3RzRbbVo= Date: Mon, 18 May 2026 14:52:25 -0700 To: mm-commits@vger.kernel.org,kasong@tencent.com,akpm@linux-foundation.org From: Andrew Morton Subject: + mm-swap-delay-and-unify-memcg-lookup-and-charging-for-swapin.patch added to mm-new branch Message-Id: <20260518215225.F1160C2BCB7@smtp.kernel.org> Precedence: bulk X-Mailing-List: mm-commits@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: The patch titled Subject: mm, swap: delay and unify memcg lookup and charging for swapin has been added to the -mm mm-new branch. Its filename is mm-swap-delay-and-unify-memcg-lookup-and-charging-for-swapin.patch This patch will shortly appear at https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/mm-swap-delay-and-unify-memcg-lookup-and-charging-for-swapin.patch This patch will later appear in the mm-new branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm Note, mm-new is a provisional staging ground for work-in-progress patches, and acceptance into mm-new is a notification for others take notice and to finish up reviews. Please do not hesitate to respond to review feedback and post updated versions to replace or incrementally fixup patches in mm-new. The mm-new branch of mm.git is not included in linux-next If a few days of testing in mm-new is successful, the patch will me moved into mm.git's mm-unstable branch, which is included in linux-next Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's *** Remember to use Documentation/process/submit-checklist.rst when testing your code *** The -mm tree is included into linux-next via various branches at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm and is updated there most days ------------------------------------------------------ From: Kairui Song Subject: mm, swap: delay and unify memcg lookup and charging for swapin Date: Sun, 17 May 2026 23:39:47 +0800 Instead of checking the cgroup private ID during page table walk in swap_pte_batch(), move the memcg lookup into __swap_cache_add_check() under the cluster lock. The first pre-alloc check is speculative and skips the memcg check since the post-alloc stable check ensures all slots covered by the folio belong to the same memcg. It is very rare for contiguous and aligned entries across a contiguous region of a page table of the same process or shmem mapping to belong to different memcgs. This also prepares for recording the memcg info in the cluster's table. Also make the order check and fallback more compact. There should be no user-observable behavior change. Link: https://lore.kernel.org/20260517-swap-table-p4-v5-8-88ae43e064c7@tencent.com Signed-off-by: Kairui Song Acked-by: Chris Li Cc: Baolin Wang Cc: Baoquan He Cc: Barry Song Cc: Chengming Zhou Cc: David Hildenbrand Cc: Hugh Dickins Cc: Johannes Weiner Cc: Kemeng Shi Cc: Lorenzo Stoakes Cc: Muchun Song Cc: Nhat Pham Cc: Roman Gushchin Cc: Shakeel Butt Cc: Youngjun Park Cc: Zi Yan Signed-off-by: Andrew Morton --- include/linux/memcontrol.h | 6 +++--- mm/internal.h | 10 +--------- mm/memcontrol.c | 10 ++++------ mm/swap_state.c | 28 +++++++++++++++++++--------- 4 files changed, 27 insertions(+), 27 deletions(-) --- a/include/linux/memcontrol.h~mm-swap-delay-and-unify-memcg-lookup-and-charging-for-swapin +++ a/include/linux/memcontrol.h @@ -646,8 +646,8 @@ static inline int mem_cgroup_charge(stru int mem_cgroup_charge_hugetlb(struct folio* folio, gfp_t gfp); -int mem_cgroup_swapin_charge_folio(struct folio *folio, struct mm_struct *mm, - gfp_t gfp, swp_entry_t entry); +int mem_cgroup_swapin_charge_folio(struct folio *folio, unsigned short id, + struct mm_struct *mm, gfp_t gfp); void __mem_cgroup_uncharge(struct folio *folio); @@ -1137,7 +1137,7 @@ static inline int mem_cgroup_charge_huge } static inline int mem_cgroup_swapin_charge_folio(struct folio *folio, - struct mm_struct *mm, gfp_t gfp, swp_entry_t entry) + unsigned short id, struct mm_struct *mm, gfp_t gfp) { return 0; } --- a/mm/internal.h~mm-swap-delay-and-unify-memcg-lookup-and-charging-for-swapin +++ a/mm/internal.h @@ -451,24 +451,16 @@ static inline int swap_pte_batch(pte_t * { pte_t expected_pte = pte_next_swp_offset(pte); const pte_t *end_ptep = start_ptep + max_nr; - const softleaf_t entry = softleaf_from_pte(pte); pte_t *ptep = start_ptep + 1; - unsigned short cgroup_id; VM_WARN_ON(max_nr < 1); - VM_WARN_ON(!softleaf_is_swap(entry)); + VM_WARN_ON(!softleaf_is_swap(softleaf_from_pte(pte))); - cgroup_id = lookup_swap_cgroup_id(entry); while (ptep < end_ptep) { - softleaf_t entry; - pte = ptep_get(ptep); if (!pte_same(pte, expected_pte)) break; - entry = softleaf_from_pte(pte); - if (lookup_swap_cgroup_id(entry) != cgroup_id) - break; expected_pte = pte_next_swp_offset(expected_pte); ptep++; } --- a/mm/memcontrol.c~mm-swap-delay-and-unify-memcg-lookup-and-charging-for-swapin +++ a/mm/memcontrol.c @@ -5076,27 +5076,25 @@ out: /** * mem_cgroup_swapin_charge_folio - Charge a newly allocated folio for swapin. - * @folio: folio to charge. + * @folio: the folio to charge + * @id: memory cgroup id * @mm: mm context of the victim * @gfp: reclaim mode - * @entry: swap entry for which the folio is allocated * * This function charges a folio allocated for swapin. Please call this before * adding the folio to the swapcache. * * Returns 0 on success. Otherwise, an error code is returned. */ -int mem_cgroup_swapin_charge_folio(struct folio *folio, struct mm_struct *mm, - gfp_t gfp, swp_entry_t entry) +int mem_cgroup_swapin_charge_folio(struct folio *folio, unsigned short id, + struct mm_struct *mm, gfp_t gfp) { struct mem_cgroup *memcg; - unsigned short id; int ret; if (mem_cgroup_disabled()) return 0; - id = lookup_swap_cgroup_id(entry); rcu_read_lock(); memcg = mem_cgroup_from_private_id(id); if (!memcg || !css_tryget_online(&memcg->css)) --- a/mm/swap_state.c~mm-swap-delay-and-unify-memcg-lookup-and-charging-for-swapin +++ a/mm/swap_state.c @@ -142,17 +142,21 @@ void *swap_cache_get_shadow(swp_entry_t * @ci: The locked swap cluster * @targ_entry: The target swap entry to check, will be rounded down by @nr * @nr: Number of slots to check, must be a power of 2 - * @shadowp: Returns the shadow value if one exists in the range. + * @shadowp: Returns the shadow value if one exists in the range + * @memcg_id: Returns the memory cgroup id, NULL to ignore cgroup check * * Check if all slots covered by given range have a swap count >= 1. - * Retrieves the shadow if there is one. + * Retrieves the shadow if there is one. If @memcg_id is not NULL, also + * checks if all slots belong to the same cgroup and return the cgroup + * private id. * * Context: Caller must lock the cluster. * Return: 0 if success, error code if failed. */ static int __swap_cache_add_check(struct swap_cluster_info *ci, swp_entry_t targ_entry, - unsigned long nr, void **shadowp) + unsigned long nr, void **shadowp, + unsigned short *memcg_id) { unsigned int ci_off, ci_end; unsigned long old_tb; @@ -172,19 +176,24 @@ static int __swap_cache_add_check(struct return -EEXIST; if (!__swp_tb_get_count(old_tb)) return -ENOENT; - if (swp_tb_is_shadow(old_tb) && shadowp) + if (shadowp && swp_tb_is_shadow(old_tb)) *shadowp = swp_tb_to_shadow(old_tb); + if (memcg_id) + *memcg_id = lookup_swap_cgroup_id(targ_entry); if (nr == 1) return 0; + targ_entry.val = round_down(targ_entry.val, nr); ci_off = round_down(ci_off, nr); ci_end = ci_off + nr; do { old_tb = __swap_table_get(ci, ci_off); if (unlikely(swp_tb_is_folio(old_tb) || - !__swp_tb_get_count(old_tb))) + !__swp_tb_get_count(old_tb) || + (memcg_id && *memcg_id != lookup_swap_cgroup_id(targ_entry)))) return -EBUSY; + targ_entry.val++; } while (++ci_off < ci_end); return 0; @@ -400,6 +409,7 @@ static struct folio *__swap_cache_alloc( swp_entry_t entry; struct folio *folio; void *shadow = NULL; + unsigned short memcg_id; unsigned long address, nr_pages = 1UL << order; struct vm_area_struct *vma = vmf ? vmf->vma : NULL; @@ -408,7 +418,7 @@ static struct folio *__swap_cache_alloc( /* Check if the slot and range are available, skip allocation if not */ spin_lock(&ci->lock); - err = __swap_cache_add_check(ci, targ_entry, nr_pages, NULL); + err = __swap_cache_add_check(ci, targ_entry, nr_pages, NULL, NULL); spin_unlock(&ci->lock); if (unlikely(err)) return ERR_PTR(err); @@ -431,7 +441,7 @@ static struct folio *__swap_cache_alloc( /* Double check the range is still not in conflict */ spin_lock(&ci->lock); - err = __swap_cache_add_check(ci, targ_entry, nr_pages, &shadow); + err = __swap_cache_add_check(ci, targ_entry, nr_pages, &shadow, &memcg_id); if (unlikely(err)) { spin_unlock(&ci->lock); folio_put(folio); @@ -443,8 +453,8 @@ static struct folio *__swap_cache_alloc( __swap_cache_do_add_folio(ci, folio, entry); spin_unlock(&ci->lock); - if (mem_cgroup_swapin_charge_folio(folio, vmf ? vmf->vma->vm_mm : NULL, - gfp, entry)) { + if (mem_cgroup_swapin_charge_folio(folio, memcg_id, + vmf ? vmf->vma->vm_mm : NULL, gfp)) { spin_lock(&ci->lock); __swap_cache_do_del_folio(ci, folio, entry, shadow); spin_unlock(&ci->lock); _ Patches currently in -mm which might be from kasong@tencent.com are mm-mglru-consolidate-common-code-for-retrieving-evictable-size.patch mm-mglru-rename-variables-related-to-aging-and-rotation.patch mm-mglru-relocate-the-lru-scan-batch-limit-to-callers.patch mm-mglru-restructure-the-reclaim-loop.patch mm-mglru-scan-and-count-the-exact-number-of-folios.patch mm-mglru-use-a-smaller-batch-for-reclaim.patch mm-mglru-dont-abort-scan-immediately-right-after-aging.patch mm-mglru-remove-redundant-swap-constrained-check-upon-isolation.patch mm-mglru-use-the-common-routine-for-dirty-writeback-reactivation.patch mm-mglru-simplify-and-improve-dirty-writeback-handling.patch mm-mglru-remove-no-longer-used-reclaim-argument-for-folio-protection.patch mm-vmscan-remove-sc-file_taken.patch mm-vmscan-remove-sc-unqueued_dirty.patch mm-vmscan-unify-writeback-reclaim-statistic-and-throttling.patch mm-swap-avoid-leaving-unused-extend-table-after-alloc-race.patch mm-swap-simplify-swap-cache-allocation-helper.patch mm-swap-move-common-swap-cache-operations-into-standalone-helpers.patch mm-huge_memory-move-thp-gfp-limit-helper-into-header.patch mm-swap-add-support-for-stable-large-allocation-in-swap-cache-directly.patch mm-swap-unify-large-folio-allocation.patch mm-memcg-swap-tidy-up-cgroup-v1-memsw-swap-helpers.patch mm-swap-support-flexible-batch-freeing-of-slots-in-different-memcgs.patch mm-swap-delay-and-unify-memcg-lookup-and-charging-for-swapin.patch mm-swap-consolidate-cluster-allocation-helpers.patch mm-memcg-swap-store-cgroup-id-in-cluster-table-directly.patch mm-memcg-remove-no-longer-used-swap-cgroup-array.patch mm-swap-merge-zeromap-into-swap-table.patch