* + mm-huge_memory-add-two-new-not-yet-used-functions-for-folio_split-fix.patch added to mm-unstable branch
@ 2025-02-15 2:54 Andrew Morton
0 siblings, 0 replies; 2+ messages in thread
From: Andrew Morton @ 2025-02-15 2:54 UTC (permalink / raw)
To: mm-commits, yuzhao, yang, willy, wangkefeng.wang, ryan.roberts,
linmiaohe, kirill.shutemov, jhubbard, hughd, david, baolin.wang,
ziy, akpm
The patch titled
Subject: mm/huge_memory: do not drop the original folio during truncate
has been added to the -mm mm-unstable branch. Its filename is
mm-huge_memory-add-two-new-not-yet-used-functions-for-folio_split-fix.patch
This patch will shortly appear at
https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/mm-huge_memory-add-two-new-not-yet-used-functions-for-folio_split-fix.patch
This patch will later appear in the mm-unstable branch at
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
Before you just go and hit "reply", please:
a) Consider who else should be cc'ed
b) Prefer to cc a suitable mailing list as well
c) Ideally: find the original patch on the mailing list and do a
reply-to-all to that, adding suitable additional cc's
*** Remember to use Documentation/process/submit-checklist.rst when testing your code ***
The -mm tree is included into linux-next via the mm-everything
branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
and is updated there every 2-3 working days
------------------------------------------------------
From: Zi Yan <ziy@nvidia.com>
Subject: mm/huge_memory: do not drop the original folio during truncate
Date: Fri, 14 Feb 2025 16:18:24 -0500
The caller expects to handle the original folio itself.
Also make __split_unmapped_folio() never fail, per discussion with David
Hildenbrand.
Link: https://lkml.kernel.org/r/E2B8D781-01EE-4832-8587-4C5FF57F91B3@nvidia.com
Link: https://lore.kernel.org/all/67af65cb.050a0220.21dd3.004a.GAE@google.com/
Link: https://lore.kernel.org/linux-mm/db77d017-4a1e-4a47-9064-e335cb0313af@redhat.com/
Signed-off-by: Zi Yan <ziy@nvidia.com>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: John Hubbard <jhubbard@nvidia.com>
Cc: Kefeng Wang <wangkefeng.wang@huawei.com>
Cc: Kirill A. Shuemov <kirill.shutemov@linux.intel.com>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Miaohe Lin <linmiaohe@huawei.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Yang Shi <yang@os.amperecomputing.com>
Cc: Yu Zhao <yuzhao@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
mm/huge_memory.c | 22 ++++++----------------
1 file changed, 6 insertions(+), 16 deletions(-)
--- a/mm/huge_memory.c~mm-huge_memory-add-two-new-not-yet-used-functions-for-folio_split-fix
+++ a/mm/huge_memory.c
@@ -3386,16 +3386,12 @@ bool can_split_folio(struct folio *folio
* It splits @folio into @new_order folios and copies the @folio metadata to
* all the resulting folios.
*/
-static int __split_folio_to_order(struct folio *folio, int new_order)
+static void __split_folio_to_order(struct folio *folio, int new_order)
{
- int curr_order = folio_order(folio);
long nr_pages = folio_nr_pages(folio);
long new_nr_pages = 1 << new_order;
long index;
- if (curr_order <= new_order)
- return -EINVAL;
-
/*
* Skip the first new_nr_pages, since the new folio from them have all
* the flags from the original folio.
@@ -3490,8 +3486,6 @@ static int __split_folio_to_order(struct
if (!new_order)
ClearPageCompound(&folio->page);
-
- return 0;
}
/*
@@ -3585,7 +3579,6 @@ static int __split_unmapped_folio(struct
int old_order = folio_order(folio);
struct folio *release;
struct folio *end_folio = folio_next(folio);
- int status;
/* order-1 anonymous folio is not supported */
if (folio_test_anon(folio) && split_order == 1)
@@ -3618,12 +3611,7 @@ static int __split_unmapped_folio(struct
split_page_owner(&folio->page, old_order, split_order);
pgalloc_tag_split(folio, old_order, split_order);
- status = __split_folio_to_order(folio, split_order);
-
- if (status < 0) {
- stop_split = true;
- ret = -EINVAL;
- }
+ __split_folio_to_order(folio, split_order);
after_split:
/*
@@ -3661,8 +3649,10 @@ after_split:
folio_test_swapcache(origin_folio)) ?
folio_nr_pages(release) : 0));
- if (release != origin_folio)
- lru_add_page_tail(origin_folio, &release->page,
+ if (release == origin_folio)
+ continue;
+
+ lru_add_page_tail(origin_folio, &release->page,
lruvec, list);
/* Some pages can be beyond EOF: drop them from page cache */
_
Patches currently in -mm which might be from ziy@nvidia.com are
selftests-mm-make-file-backed-thp-split-work-by-writing-pmd-size-data.patch
mm-huge_memory-allow-split-shmem-large-folio-to-any-lower-order.patch
selftests-mm-test-splitting-file-backed-thp-to-any-lower-order.patch
xarray-add-xas_try_split-to-split-a-multi-index-entry.patch
xarray-add-xas_try_split-to-split-a-multi-index-entry-fix.patch
mm-huge_memory-add-two-new-not-yet-used-functions-for-folio_split.patch
mm-huge_memory-add-two-new-not-yet-used-functions-for-folio_split-fix.patch
mm-huge_memory-move-folio-split-common-code-to-__folio_split.patch
mm-huge_memory-add-buddy-allocator-like-non-uniform-folio_split.patch
mm-huge_memory-remove-the-old-unused-__split_huge_page.patch
mm-huge_memory-add-folio_split-to-debugfs-testing-interface.patch
mm-truncate-use-buddy-allocator-like-folio-split-for-truncate-operation.patch
selftests-mm-add-tests-for-folio_split-buddy-allocator-like-split.patch
mm-filemap-use-xas_try_split-in-__filemap_add_folio.patch
mm-shmem-use-xas_try_split-in-shmem_split_large_entry.patch
^ permalink raw reply [flat|nested] 2+ messages in thread* + mm-huge_memory-add-two-new-not-yet-used-functions-for-folio_split-fix.patch added to mm-unstable branch
@ 2025-03-10 23:29 Andrew Morton
0 siblings, 0 replies; 2+ messages in thread
From: Andrew Morton @ 2025-03-10 23:29 UTC (permalink / raw)
To: mm-commits, yuzhao, yang, willy, wangkefeng.wang, ryan.roberts,
linmiaohe, kirill.shutemov, kasong, jhubbard, hughd, david,
baolin.wang, ziy, akpm
The patch titled
Subject: mm/huge_memory: unfreeze head folio after page cache entries are updated
has been added to the -mm mm-unstable branch. Its filename is
mm-huge_memory-add-two-new-not-yet-used-functions-for-folio_split-fix.patch
This patch will shortly appear at
https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/mm-huge_memory-add-two-new-not-yet-used-functions-for-folio_split-fix.patch
This patch will later appear in the mm-unstable branch at
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
Before you just go and hit "reply", please:
a) Consider who else should be cc'ed
b) Prefer to cc a suitable mailing list as well
c) Ideally: find the original patch on the mailing list and do a
reply-to-all to that, adding suitable additional cc's
*** Remember to use Documentation/process/submit-checklist.rst when testing your code ***
The -mm tree is included into linux-next via the mm-everything
branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
and is updated there every 2-3 working days
------------------------------------------------------
From: Zi Yan <ziy@nvidia.com>
Subject: mm/huge_memory: unfreeze head folio after page cache entries are updated
Date: Mon, 10 Mar 2025 11:59:42 -0400
Otherwise others can grab the head folio and see stale page cache entries.
Data corruption can happen because of that.
Drop large EOF tail folios with the right number of refs to prevent memory
leak.
Also include Matthew's suggestion on __split_folio_to_order()[1]
[1] https://lore.kernel.org/all/Z88ar5YS99HsIRYo@casper.infradead.org/
Link: https://lkml.kernel.org/r/0F15DA7F-1977-412F-9A3E-F06B515D4BD2@nvidia.com
Signed-off-by: Zi Yan <ziy@nvidia.com>
Reported-by: Hugh Dickins <hughd@google.com>
Closes: https://lore.kernel.org/all/fcbadb7f-dd3e-21df-f9a7-2853b53183c4@google.com/
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: John Hubbard <jhubbard@nvidia.com>
Cc: Kairui Song <kasong@tencent.com>
Cc: Kefeng Wang <wangkefeng.wang@huawei.com>
Cc: Kirill A. Shuemov <kirill.shutemov@linux.intel.com>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Miaohe Lin <linmiaohe@huawei.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Yang Shi <yang@os.amperecomputing.com>
Cc: Yu Zhao <yuzhao@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
mm/huge_memory.c | 52 +++++++++++++++++++++++++--------------------
1 file changed, 29 insertions(+), 23 deletions(-)
--- a/mm/huge_memory.c~mm-huge_memory-add-two-new-not-yet-used-functions-for-folio_split-fix
+++ a/mm/huge_memory.c
@@ -3525,15 +3525,14 @@ static void __split_folio_to_order(struc
{
long new_nr_pages = 1 << new_order;
long nr_pages = 1 << old_order;
- long index;
+ long i;
/*
* Skip the first new_nr_pages, since the new folio from them have all
* the flags from the original folio.
*/
- for (index = new_nr_pages; index < nr_pages; index += new_nr_pages) {
- struct page *head = &folio->page;
- struct page *new_head = head + index;
+ for (i = new_nr_pages; i < nr_pages; i += new_nr_pages) {
+ struct page *new_head = &folio->page + i;
/*
* Careful: new_folio is not a "real" folio before we cleared PageTail.
@@ -3541,7 +3540,7 @@ static void __split_folio_to_order(struc
*/
struct folio *new_folio = (struct folio *)new_head;
- VM_BUG_ON_PAGE(atomic_read(&new_head->_mapcount) != -1, new_head);
+ VM_BUG_ON_PAGE(atomic_read(&new_folio->_mapcount) != -1, new_head);
/*
* Clone page flags before unfreezing refcount.
@@ -3556,8 +3555,8 @@ static void __split_folio_to_order(struc
* unreferenced sub-pages of an anonymous THP: we can simply drop
* PG_anon_exclusive (-> PG_mappedtodisk) for these here.
*/
- new_head->flags &= ~PAGE_FLAGS_CHECK_AT_PREP;
- new_head->flags |= (head->flags &
+ new_folio->flags &= ~PAGE_FLAGS_CHECK_AT_PREP;
+ new_folio->flags |= (folio->flags &
((1L << PG_referenced) |
(1L << PG_swapbacked) |
(1L << PG_swapcache) |
@@ -3576,23 +3575,20 @@ static void __split_folio_to_order(struc
(1L << PG_dirty) |
LRU_GEN_MASK | LRU_REFS_MASK));
- /* ->mapping in first and second tail page is replaced by other uses */
- VM_BUG_ON_PAGE(new_nr_pages > 2 && new_head->mapping != TAIL_MAPPING,
- new_head);
- new_head->mapping = head->mapping;
- new_head->index = head->index + index;
+ new_folio->mapping = folio->mapping;
+ new_folio->index = folio->index + i;
/*
* page->private should not be set in tail pages. Fix up and warn once
* if private is unexpectedly set.
*/
- if (unlikely(new_head->private)) {
+ if (unlikely(new_folio->private)) {
VM_WARN_ON_ONCE_PAGE(true, new_head);
- new_head->private = 0;
+ new_folio->private = 0;
}
if (folio_test_swapcache(folio))
- new_folio->swap.val = folio->swap.val + index;
+ new_folio->swap.val = folio->swap.val + i;
/* Page flags must be visible before we make the page non-compound. */
smp_wmb();
@@ -3788,17 +3784,18 @@ after_split:
}
/*
- * Unfreeze refcount first. Additional reference from
- * page cache.
+ * origin_folio should be kept frozon until page cache
+ * entries are updated with all the other after-split
+ * folios to prevent others seeing stale page cache
+ * entries.
*/
- folio_ref_unfreeze(release,
- 1 + ((!folio_test_anon(origin_folio) ||
- folio_test_swapcache(origin_folio)) ?
- folio_nr_pages(release) : 0));
-
if (release == origin_folio)
continue;
+ folio_ref_unfreeze(release, 1 +
+ ((mapping || swap_cache) ?
+ folio_nr_pages(release) : 0));
+
lru_add_page_tail(origin_folio, &release->page,
lruvec, list);
@@ -3810,7 +3807,7 @@ after_split:
folio_account_cleaned(release,
inode_to_wb(mapping->host));
__filemap_remove_folio(release, NULL);
- folio_put(release);
+ folio_put_refs(release, folio_nr_pages(release));
} else if (mapping) {
__xa_store(&mapping->i_pages,
release->index, release, 0);
@@ -3822,6 +3819,15 @@ after_split:
}
}
+ /*
+ * Unfreeze origin_folio only after all page cache entries, which used
+ * to point to it, have been updated with new folios. Otherwise,
+ * a parallel folio_try_get() can grab origin_folio and its caller can
+ * see stale page cache entries.
+ */
+ folio_ref_unfreeze(origin_folio, 1 +
+ ((mapping || swap_cache) ? folio_nr_pages(origin_folio) : 0));
+
unlock_page_lruvec(lruvec);
if (swap_cache)
_
Patches currently in -mm which might be from ziy@nvidia.com are
mm-migrate-fix-shmem-xarray-update-during-migration.patch
selftests-mm-make-file-backed-thp-split-work-by-writing-pmd-size-data.patch
mm-huge_memory-allow-split-shmem-large-folio-to-any-lower-order.patch
selftests-mm-test-splitting-file-backed-thp-to-any-lower-order.patch
xarray-add-xas_try_split-to-split-a-multi-index-entry.patch
mm-huge_memory-add-two-new-not-yet-used-functions-for-folio_split.patch
mm-huge_memory-add-two-new-not-yet-used-functions-for-folio_split-fix.patch
mm-huge_memory-move-folio-split-common-code-to-__folio_split.patch
mm-huge_memory-add-buddy-allocator-like-non-uniform-folio_split.patch
mm-huge_memory-remove-the-old-unused-__split_huge_page.patch
mm-huge_memory-add-folio_split-to-debugfs-testing-interface.patch
mm-truncate-use-folio_split-in-truncate-operation.patch
selftests-mm-add-tests-for-folio_split-buddy-allocator-like-split.patch
mm-filemap-use-xas_try_split-in-__filemap_add_folio.patch
mm-shmem-use-xas_try_split-in-shmem_split_large_entry.patch
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2025-03-10 23:29 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-02-15 2:54 + mm-huge_memory-add-two-new-not-yet-used-functions-for-folio_split-fix.patch added to mm-unstable branch Andrew Morton
-- strict thread matches above, loose matches on Subject: below --
2025-03-10 23:29 Andrew Morton
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.