From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1344A53E24 for ; Tue, 18 Jun 2024 20:32:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718742748; cv=none; b=BLoJqpk3AUtgFX9X9S5Ul9Jobg+Jt+EdTJ2IwSVTVk2t0ViyOoFAa6b60yhrh/hzS4Swpl5jwGSXGFRufrUv4xslI5kMHqOpspKSFYQ66IAa/SbATWSxLYxSxvU6QuPHviD8UT/iuMm0/Zo7fDJzuFX39IoGnIKC5/OWsvayKUE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718742748; c=relaxed/simple; bh=mRGQBTHwdHHLQ9PXm8mRQGvMiGzTYOxyIeJiaX95WPY=; h=Date:To:From:Subject:Message-Id; b=lpPh1kLNmA6Va5dUyKa0rMjBDoIyf1Hfl9X9dq8NqN+Afw6jYXHpg2PzznCZFge/bPL0CyjzedanH48mtHRggiBdRjQBWCkIYBtBdbDO4CiDgGm5FLEriJSXmoAzjXaOQq9UpXercAthThXrbY/VMBIPns64yuTpgcneBKRAO0M= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b=NRosxKJY; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b="NRosxKJY" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7FA18C4AF1C; Tue, 18 Jun 2024 20:32:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1718742747; bh=mRGQBTHwdHHLQ9PXm8mRQGvMiGzTYOxyIeJiaX95WPY=; h=Date:To:From:Subject:From; b=NRosxKJYZFPMxYkSGMT44ViqnJK7w5ajlb7Fe6/mrZMn8Zl3OLE5e3zcjlQYo0D3j fU/Tj3t7aypsW4yQ31Vkp41+wXOFB/qDerWgeUAO+pE1Z61HvuFsdXEyCi35jrbyd1 IwrMo9OIAaOQO5vDw4T5fKv4vCg8diiOseqH+Gwk= Date: Tue, 18 Jun 2024 13:32:27 -0700 To: mm-commits@vger.kernel.org,ziy@nvidia.com,ying.huang@intel.com,willy@infradead.org,wangkefeng.wang@huawei.com,shy828301@gmail.com,ryan.roberts@arm.com,p.raghav@samsung.com,ioworker0@gmail.com,hughd@google.com,david@redhat.com,da.gomez@samsung.com,chrisl@kernel.org,21cnbao@gmail.com,baolin.wang@linux.alibaba.com,akpm@linux-foundation.org From: Andrew Morton Subject: + mm-shmem-support-large-folio-allocation-for-shmem_replace_folio.patch added to mm-unstable branch Message-Id: <20240618203227.7FA18C4AF1C@smtp.kernel.org> Precedence: bulk X-Mailing-List: mm-commits@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: The patch titled Subject: mm: shmem: support large folio allocation for shmem_replace_folio() has been added to the -mm mm-unstable branch. Its filename is mm-shmem-support-large-folio-allocation-for-shmem_replace_folio.patch This patch will shortly appear at https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/mm-shmem-support-large-folio-allocation-for-shmem_replace_folio.patch This patch will later appear in the mm-unstable branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's *** Remember to use Documentation/process/submit-checklist.rst when testing your code *** The -mm tree is included into linux-next via the mm-everything branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm and is updated there every 2-3 working days ------------------------------------------------------ From: Baolin Wang Subject: mm: shmem: support large folio allocation for shmem_replace_folio() Date: Tue, 18 Jun 2024 14:54:19 +0800 To support large folio swapin for shmem in the following patches, add large folio allocation for the new replacement folio in shmem_replace_folio(). Moreover large folios occupy N consecutive entries in the swap cache instead of using multi-index entries like the page cache, therefore we should replace each consecutive entries in the swap cache instead of using the shmem_replace_entry(). As well as updating statistics and folio reference count using the number of pages in the folio. Link: https://lkml.kernel.org/r/75d1c1f2fbc88dce476a53fa90f085b3bebb83c5.1718690645.git.baolin.wang@linux.alibaba.com Signed-off-by: Baolin Wang Cc: Barry Song <21cnbao@gmail.com> Cc: Chris Li Cc: Daniel Gomez Cc: David Hildenbrand Cc: "Huang, Ying" Cc: Hugh Dickins Cc: Kefeng Wang Cc: Lance Yang Cc: Matthew Wilcox (Oracle) Cc: Pankaj Raghav Cc: Ryan Roberts Cc: Yang Shi Cc: Zi Yan Signed-off-by: Andrew Morton --- mm/shmem.c | 53 ++++++++++++++++++++++++++++----------------------- 1 file changed, 30 insertions(+), 23 deletions(-) --- a/mm/shmem.c~mm-shmem-support-large-folio-allocation-for-shmem_replace_folio +++ a/mm/shmem.c @@ -1878,28 +1878,24 @@ static bool shmem_should_replace_folio(s static int shmem_replace_folio(struct folio **foliop, gfp_t gfp, struct shmem_inode_info *info, pgoff_t index) { - struct folio *old, *new; - struct address_space *swap_mapping; - swp_entry_t entry; - pgoff_t swap_index; - int error; - - old = *foliop; - entry = old->swap; - swap_index = swap_cache_index(entry); - swap_mapping = swap_address_space(entry); + struct folio *new, *old = *foliop; + swp_entry_t entry = old->swap; + struct address_space *swap_mapping = swap_address_space(entry); + pgoff_t swap_index = swap_cache_index(entry); + XA_STATE(xas, &swap_mapping->i_pages, swap_index); + int nr_pages = folio_nr_pages(old); + int error = 0, i; /* * We have arrived here because our zones are constrained, so don't * limit chance of success by further cpuset and node constraints. */ gfp &= ~GFP_CONSTRAINT_MASK; - VM_BUG_ON_FOLIO(folio_test_large(old), old); - new = shmem_alloc_folio(gfp, 0, info, index); + new = shmem_alloc_folio(gfp, folio_order(old), info, index); if (!new) return -ENOMEM; - folio_get(new); + folio_ref_add(new, nr_pages); folio_copy(new, old); flush_dcache_folio(new); @@ -1909,18 +1905,24 @@ static int shmem_replace_folio(struct fo new->swap = entry; folio_set_swapcache(new); - /* - * Our caller will very soon move newpage out of swapcache, but it's - * a nice clean interface for us to replace oldpage by newpage there. - */ + /* Swap cache still stores N entries instead of a high-order entry */ xa_lock_irq(&swap_mapping->i_pages); - error = shmem_replace_entry(swap_mapping, swap_index, old, new); + for (i = 0; i < nr_pages; i++) { + void *item = xas_store(&xas, new); + + if (item != old) { + error = -ENOENT; + break; + } + + xas_next(&xas); + } if (!error) { mem_cgroup_replace_folio(old, new); - __lruvec_stat_mod_folio(new, NR_FILE_PAGES, 1); - __lruvec_stat_mod_folio(new, NR_SHMEM, 1); - __lruvec_stat_mod_folio(old, NR_FILE_PAGES, -1); - __lruvec_stat_mod_folio(old, NR_SHMEM, -1); + __lruvec_stat_mod_folio(new, NR_FILE_PAGES, nr_pages); + __lruvec_stat_mod_folio(new, NR_SHMEM, nr_pages); + __lruvec_stat_mod_folio(old, NR_FILE_PAGES, -nr_pages); + __lruvec_stat_mod_folio(old, NR_SHMEM, -nr_pages); } xa_unlock_irq(&swap_mapping->i_pages); @@ -1940,7 +1942,12 @@ static int shmem_replace_folio(struct fo old->private = NULL; folio_unlock(old); - folio_put_refs(old, 2); + /* + * The old folio are removed from swap cache, drop the 'nr_pages' + * reference, as well as one temporary reference getting from swap + * cache. + */ + folio_put_refs(old, nr_pages + 1); return error; } _ Patches currently in -mm which might be from baolin.wang@linux.alibaba.com are mm-memory-extend-finish_fault-to-support-large-folio.patch mm-memory-extend-finish_fault-to-support-large-folio-fix.patch mm-memory-extend-finish_fault-to-support-large-folio-fix-fix.patch mm-shmem-add-thp-validation-for-pmd-mapped-thp-related-statistics.patch mm-shmem-add-multi-size-thp-sysfs-interface-for-anonymous-shmem.patch mm-shmem-add-multi-size-thp-sysfs-interface-for-anonymous-shmem-fix.patch mm-shmem-add-multi-size-thp-sysfs-interface-for-anonymous-shmem-fix-3.patch mm-shmem-add-mthp-support-for-anonymous-shmem.patch mm-shmem-add-mthp-size-alignment-in-shmem_get_unmapped_area.patch mm-shmem-add-mthp-counters-for-anonymous-shmem.patch mm-shmem-add-mthp-counters-for-anonymous-shmem-fix.patch mm-memcontrol-add-vm_bug_on_folio-to-catch-lru-folio-in-mem_cgroup_migrate.patch mm-vmscan-add-validation-before-spliting-shmem-large-folio.patch mm-swap-extend-swap_shmem_alloc-to-support-batch-swap_map_shmem-flag-setting.patch mm-shmem-extend-shmem_partial_swap_usage-to-support-large-folio-swap.patch mm-filemap-use-xa_get_order-to-get-the-swap-entry-order.patch mm-shmem-use-swap_free_nr-to-free-shmem-swap-entries.patch mm-shmem-support-large-folio-allocation-for-shmem_replace_folio.patch mm-shmem-drop-folio-reference-count-using-nr_pages-in-shmem_delete_from_page_cache.patch mm-shmem-support-large-folio-swap-out.patch