[PATCH v2 0/5] Keep tail page private zero at free and folio split time

[PATCH v2 0/5] Keep tail page private zero at free and folio split time

[Zi Yan]

Hi all,

This patchset makes sure tail_page->private is zero before compound or
high-order pages are returned to the allocator. It also checks tail pages
that become new folio heads during large folio split, before their private
fields are used by new folios.

It is based on mm-new.

Note on ZONE_DEVICE and DAX page/folio
===
ZONE_DEVICE and DAX use prep_compound_tail() to reinitialize folios, so
tail_page->private was reset before this patchset. There was a concern that
after this patchset stale ->private can appear after ZONE_DEVICE/DAX folio
initialization. My reasoning is that no code sets ZONE_DEVICE/DAX
page->private, so their page->private stays zero all the time.
ZONE_DEVICE_PRIVATE page migration only supports anonymous memory without
swapcache, so after the migration ->private remains zero.

But let me know if my reasoning is wrong. It can be fixed by adding
->private zeroing code in ZONE_DEVICE/DAX folio initialization code.

Motivation
===

page->private is zeroed at page free time since commit ac1ea219590c0
("mm/page_alloc: clear page->private in free_pages_prepare()"), since we
concluded that it might be too much to ask every page user to free a page
with ->private zeroed. The holder of the last page reference might not know
whether ->private needs to be cleared.

For compound and high-order pages, tail_page->private can also leak to
later users if it is left uncleared. The page allocation path does not zero
every tail_page->private field, so they can be seen by new users and cause
unexpected issues[1].

Check tail_page->private at page free time, and check tail pages that
become new folio heads during large folio split. With those checks in
place, prep_compound_tail() no longer needs to clear tail_page->private
when preparing compound page metadata.

Overview
===

1. Patch 1 clears all pages ->private before percpu-km frees them.
2. Patch 2 removes setting page->private in compaction code when a free
   page is taken out of the buddy allocator. cc->freepages is indexed by
   page order, so storing the free page order in page->private is
   redundant. In alloc_contig_frozen_range_noprof(),
   isolate_freepages_range() is used to grab free pages from buddy
   allocator and it leaves the aforementioned page->private set until
   either split_free_frozen_pages() or prep_new_page() is called. That
   stale value without resetting triggers the tail_page->private nonzero
   check once set_page_private(0) is removed from prep_compound_tail().

3. Patch 3 adds back the page->private check for tail pages promoted to new
   folio heads in __split_folio_to_order().
4. Patch 4 adds a tail_page->private check in the page free path.
5. Patch 5 removes tail_page->private zeroing from prep_compound_tail().

Link: https://lore.kernel.org/all/20260206174017.128673-1-mikhail.v.gavrilov@gmail.com/ [1]

Signed-off-by: Zi Yan <ziy@nvidia.com>
---
Changes in v2:
1. added reset page->private when percpu-km frees pages
2. replaced subpage with tail page/tail_page in all patches
3. moved implementation details from cc->freepages patch message to cover
   letter, since it is too much for a patch description.
4. used VM_WARN_ON_ONCE_PAGE() in __split_folio_to_order() patch without
   fixup. The expectation is to catch any violation during development
   phase.
5. guarded tail_page->private check behind is_check_pages_enabled().
6. replaced tail_page->private reset code with VM_WARN_ON_ONCE() instead of
   deletion in prep_compound_tail
7. the pre-existing issue in alloc_contig_frozen_range_noprof() is under
   discussion and might not be worth fixing.
   - Link: https://lore.kernel.org/all/d44ae8a5-ec70-456b-92a0-ce7ccabf6917@kernel.org/
- Link to v1: https://lore.kernel.org/r/20260628-keep-subpage-private-zero-at-free-v1-0-f4ce3930d10f@nvidia.com

---
Zi Yan (5):
      mm/percpu-km: clear page->private before free them
      mm/compaction: stop recording free page order in page->private
      mm/huge_memory: add page->private check back in __split_folio_to_order()
      mm/page_alloc: make sure tail_page->private is zero at page free time
      mm/page_alloc: remove set_page_private() in prep_compound_tail()

 mm/compaction.c  |  3 ---
 mm/huge_memory.c |  7 +++++++
 mm/internal.h    |  2 +-
 mm/page_alloc.c  | 13 ++++++++++---
 mm/percpu-km.c   |  9 ++++++++-
 5 files changed, 26 insertions(+), 8 deletions(-)
---
base-commit: e031e55776cf9193b4720a253e92539ca536d224
change-id: 20260603-keep-subpage-private-zero-at-free-a1e1435025dc

Best regards,
-- 
Yan, Zi

[PATCH v2 1/5] mm/percpu-km: clear page->private before free them

[Zi Yan]

page->private is cleared in free page path. In a subsequent commit,
tail_page->private will be checked and ensured to be zero. Clearing
percpu-km allocated pages' ->private to prevent triggering warnings later.

Signed-off-by: Zi Yan <ziy@nvidia.com>
---
 mm/percpu-km.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/mm/percpu-km.c b/mm/percpu-km.c
index 4efa74a495cb6..7ffe84adadb9d 100644
--- a/mm/percpu-km.c
+++ b/mm/percpu-km.c
@@ -94,8 +94,15 @@ static void pcpu_destroy_chunk(struct pcpu_chunk *chunk)
 	pcpu_stats_chunk_dealloc();
 	trace_percpu_destroy_chunk(chunk->base_addr);
 
-	if (chunk->data)
+	if (chunk->data) {
+		struct page *pages = (struct page *)chunk->data;
+		int i;
+
+		/* clear chunk info from each page before free them */
+		for (i = 0; i < nr_pages; i++)
+			pcpu_set_page_chunk(pages + i, NULL);
 		__free_pages(chunk->data, order_base_2(nr_pages));
+	}
 	pcpu_free_chunk(chunk);
 }
 

-- 
2.53.0

[PATCH v2 2/5] mm/compaction: stop recording free page order in page->private

[Zi Yan]

Commit 733aea0b3a7bb ("mm/compaction: add support for >0 order folio
memory compaction.") stores isolated free pages in an array indexed by free
page orders, it is no longer needed to store the order in each page's
->private field. And there is no code using the stored order. Stop doing
that.

It also prepares for an upcoming change that ensures subpage->private is
zero at page free time and the removal of set_page_private(0) from
prep_compound_tail().

Reviewed-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Signed-off-by: Zi Yan <ziy@nvidia.com>
---
 mm/compaction.c | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/mm/compaction.c b/mm/compaction.c
index f08765ade014c..9f4204853afd6 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -644,7 +644,6 @@ static unsigned long isolate_freepages_block(struct compact_control *cc,
 		isolated = __isolate_free_page(page, order);
 		if (!isolated)
 			break;
-		set_page_private(page, order);
 
 		nr_scanned += isolated - 1;
 		total_isolated += isolated;
@@ -1617,7 +1616,6 @@ static void fast_isolate_freepages(struct compact_control *cc)
 		/* Isolate the page if available */
 		if (page) {
 			if (__isolate_free_page(page, order)) {
-				set_page_private(page, order);
 				nr_isolated = 1 << order;
 				nr_scanned += nr_isolated - 1;
 				total_isolated += nr_isolated;
@@ -1846,7 +1844,6 @@ static struct folio *compaction_alloc_noprof(struct folio *src, unsigned long da
 		size >>= 1;
 
 		list_add(&freepage[size].lru, &cc->freepages[start_order]);
-		set_page_private(&freepage[size], start_order);
 	}
 	dst = (struct folio *)freepage;
 

-- 
2.53.0

[PATCH v2 3/5] mm/huge_memory: add page->private check back in __split_folio_to_order()

[Zi Yan]

page->private should not be set in tail pages. Commit 4265d67e405a
("mm/migrate_device: add THP splitting during migration") removed it
without a proper reason[1]. Add it back.

Link: https://lore.kernel.org/all/13f3fcda-7328-4aa5-afc6-75a294a82b2a@nvidia.com/ [1]
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Signed-off-by: Zi Yan <ziy@nvidia.com>
---
 mm/huge_memory.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index bdd8635922f96..06d0671cff899 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -3591,6 +3591,13 @@ static void __split_folio_to_order(struct folio *folio, int old_order,
 		new_folio->mapping = folio->mapping;
 		new_folio->index = folio->index + i;
 
+		/*
+		 * page->private should not be set in tail pages. Warn once
+		 * if private is unexpectedly set. Do it before swap.val assignment
+		 * since private overlaps with swap.val.
+		 */
+		VM_WARN_ON_ONCE_PAGE(new_folio->private, new_head);
+
 		if (folio_test_swapcache(folio))
 			new_folio->swap.val = folio->swap.val + i;
 

-- 
2.53.0

[PATCH v2 4/5] mm/page_alloc: make sure tail_page->private is zero at page free time

[Zi Yan]

Any code using tail_page->private of a folio, a compound page or a
high-order page is supposed to reset it after use, otherwise ->private data
can leak to new page user and cause unexpected issues. Add a bad_page()
check at page free path for it.

Signed-off-by: Zi Yan <ziy@nvidia.com>
---
 mm/page_alloc.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 762d9b6bc792f..723a52dfc80f6 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1376,15 +1376,22 @@ static __always_inline bool __free_pages_prepare(struct page *page,
 #endif
 		}
 		for (i = 1; i < (1 << order); i++) {
+			struct page *tail_page = page + i;
+
 			if (compound)
-				bad += free_tail_page_prepare(page, page + i);
+				bad += free_tail_page_prepare(page, tail_page);
 			if (is_check_pages_enabled()) {
-				if (free_page_is_bad(page + i)) {
+				if (free_page_is_bad(tail_page)) {
 					bad++;
 					continue;
 				}
 			}
-			(page + i)->flags.f &= ~PAGE_FLAGS_CHECK_AT_PREP;
+			tail_page->flags.f &= ~PAGE_FLAGS_CHECK_AT_PREP;
+			if (is_check_pages_enabled() && tail_page->private) {
+				bad_page(tail_page, "nonzero private");
+				bad++;
+				continue;
+			}
 		}
 	}
 	if (folio_test_anon(folio)) {

-- 
2.53.0

[PATCH v2 5/5] mm/page_alloc: remove set_page_private() in prep_compound_tail()

[Zi Yan]

Tail pages are expected to have (and optionally be checked) zeroed
->private when they are freed. It stays true during subsequent
reallocation, so replace the tail_page->private initialization with a
VM_WARN_ON_ONCE() in compound page preparation.

Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Signed-off-by: Zi Yan <ziy@nvidia.com>
---
 mm/internal.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/internal.h b/mm/internal.h
index fa4fb69444ecd..fbd9fb84341bc 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -902,7 +902,7 @@ static inline void prep_compound_tail(struct page *tail,
 {
 	tail->mapping = TAIL_MAPPING;
 	set_compound_head(tail, head, order);
-	set_page_private(tail, 0);
+	VM_WARN_ON_ONCE(tail->private);
 }
 
 static inline void init_compound_tail(struct page *tail,

-- 
2.53.0

Re: [PATCH v2 5/5] mm/page_alloc: remove set_page_private() in prep_compound_tail()

[Lance Yang]

On 2026/7/3 21:47, Zi Yan wrote:
> Tail pages are expected to have (and optionally be checked) zeroed
> ->private when they are freed. It stays true during subsequent
> reallocation, so replace the tail_page->private initialization with a
> VM_WARN_ON_ONCE() in compound page preparation.
> 
> Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
> Signed-off-by: Zi Yan <ziy@nvidia.com>
> ---

LGTM. Feel free to add:

Reviewed-by: Lance Yang <lance.yang@linux.dev>

>   mm/internal.h | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/mm/internal.h b/mm/internal.h
> index fa4fb69444ecd..fbd9fb84341bc 100644
> --- a/mm/internal.h
> +++ b/mm/internal.h
> @@ -902,7 +902,7 @@ static inline void prep_compound_tail(struct page *tail,
>   {
>   	tail->mapping = TAIL_MAPPING;
>   	set_compound_head(tail, head, order);
> -	set_page_private(tail, 0);
> +	VM_WARN_ON_ONCE(tail->private);
>   }
>   
>   static inline void init_compound_tail(struct page *tail,
> 

Re: [PATCH v2 0/5] Keep tail page private zero at free and folio split time

[Zi Yan]

On Fri Jul 3, 2026 at 9:47 AM EDT, Zi Yan wrote:
> Hi all,
>
> This patchset makes sure tail_page->private is zero before compound or
> high-order pages are returned to the allocator. It also checks tail pages
> that become new folio heads during large folio split, before their private
> fields are used by new folios.
>
> It is based on mm-new.
>
> Note on ZONE_DEVICE and DAX page/folio
> ===
> ZONE_DEVICE and DAX use prep_compound_tail() to reinitialize folios, so
> tail_page->private was reset before this patchset. There was a concern that
> after this patchset stale ->private can appear after ZONE_DEVICE/DAX folio
> initialization. My reasoning is that no code sets ZONE_DEVICE/DAX
> page->private, so their page->private stays zero all the time.
> ZONE_DEVICE_PRIVATE page migration only supports anonymous memory without
> swapcache, so after the migration ->private remains zero.
>
> But let me know if my reasoning is wrong. It can be fixed by adding
> ->private zeroing code in ZONE_DEVICE/DAX folio initialization code.
>
> Motivation
> ===
>
> page->private is zeroed at page free time since commit ac1ea219590c0
> ("mm/page_alloc: clear page->private in free_pages_prepare()"), since we
> concluded that it might be too much to ask every page user to free a page
> with ->private zeroed. The holder of the last page reference might not know
> whether ->private needs to be cleared.
>
> For compound and high-order pages, tail_page->private can also leak to
> later users if it is left uncleared. The page allocation path does not zero
> every tail_page->private field, so they can be seen by new users and cause
> unexpected issues[1].
>
> Check tail_page->private at page free time, and check tail pages that
> become new folio heads during large folio split. With those checks in
> place, prep_compound_tail() no longer needs to clear tail_page->private
> when preparing compound page metadata.
>
> Overview
> ===
>
> 1. Patch 1 clears all pages ->private before percpu-km frees them.
> 2. Patch 2 removes setting page->private in compaction code when a free
>    page is taken out of the buddy allocator. cc->freepages is indexed by
>    page order, so storing the free page order in page->private is
>    redundant. In alloc_contig_frozen_range_noprof(),
>    isolate_freepages_range() is used to grab free pages from buddy
>    allocator and it leaves the aforementioned page->private set until
>    either split_free_frozen_pages() or prep_new_page() is called. That
>    stale value without resetting triggers the tail_page->private nonzero
>    check once set_page_private(0) is removed from prep_compound_tail().
>
> 3. Patch 3 adds back the page->private check for tail pages promoted to new
>    folio heads in __split_folio_to_order().
> 4. Patch 4 adds a tail_page->private check in the page free path.
> 5. Patch 5 removes tail_page->private zeroing from prep_compound_tail().
>
> Link: https://lore.kernel.org/all/20260206174017.128673-1-mikhail.v.gavrilov@gmail.com/ [1]
>
> Signed-off-by: Zi Yan <ziy@nvidia.com>
> ---
> Changes in v2:
> 1. added reset page->private when percpu-km frees pages
> 2. replaced subpage with tail page/tail_page in all patches
> 3. moved implementation details from cc->freepages patch message to cover
>    letter, since it is too much for a patch description.
> 4. used VM_WARN_ON_ONCE_PAGE() in __split_folio_to_order() patch without
>    fixup. The expectation is to catch any violation during development
>    phase.
> 5. guarded tail_page->private check behind is_check_pages_enabled().
> 6. replaced tail_page->private reset code with VM_WARN_ON_ONCE() instead of
>    deletion in prep_compound_tail
> 7. the pre-existing issue in alloc_contig_frozen_range_noprof() is under
>    discussion and might not be worth fixing.
>    - Link: https://lore.kernel.org/all/d44ae8a5-ec70-456b-92a0-ce7ccabf6917@kernel.org/
> - Link to v1: https://lore.kernel.org/r/20260628-keep-subpage-private-zero-at-free-v1-0-f4ce3930d10f@nvidia.com
>
> ---
> Zi Yan (5):
>       mm/percpu-km: clear page->private before free them
>       mm/compaction: stop recording free page order in page->private
>       mm/huge_memory: add page->private check back in __split_folio_to_order()
>       mm/page_alloc: make sure tail_page->private is zero at page free time
>       mm/page_alloc: remove set_page_private() in prep_compound_tail()
>
>  mm/compaction.c  |  3 ---
>  mm/huge_memory.c |  7 +++++++
>  mm/internal.h    |  2 +-
>  mm/page_alloc.c  | 13 ++++++++++---
>  mm/percpu-km.c   |  9 ++++++++-
>  5 files changed, 26 insertions(+), 8 deletions(-)
> ---
> base-commit: e031e55776cf9193b4720a253e92539ca536d224
> change-id: 20260603-keep-subpage-private-zero-at-free-a1e1435025dc
>
> Best regards,

Answers to Sashiko's reviews:
https://sashiko.dev/#/patchset/20260703-keep-subpage-private-zero-at-free-v2-0-2970fe777dd6%40nvidia.com

Q1: To Patch 1, this isn't a bug introduced by this patch, but does
pcpu_create_chunk() overflow chunk->populated on SMP configs?

Answer: I am not familiar with the code, but based on my understanding
and the chat with codex, a patch like below could fix the issue. I will
wait for the feedback from percpu-km people about it.


diff --git a/mm/percpu-km.c b/mm/percpu-km.c
--- a/mm/percpu-km.c
+++ b/mm/percpu-km.c
@@ -74,8 +74,13 @@ static struct pcpu_chunk *pcpu_create_chunk(gfp_t gfp)
 	chunk->data = pages;
 	chunk->base_addr = page_address(pages);
 
+	/*
+	 * nr_pages covers the physical backing for all units.  The populated
+	 * bitmap and pcpu_nr_populated accounting are per-unit, so only mark
+	 * the logical chunk page range populated.
+	 */
 	spin_lock_irqsave(&pcpu_lock, flags);
-	pcpu_chunk_populated(chunk, 0, nr_pages);
+	pcpu_chunk_populated(chunk, 0, chunk->nr_pages);
 	spin_unlock_irqrestore(&pcpu_lock, flags);
 
 	pcpu_stats_chunk_alloc();


Q2: To Patch 5, does replacing the explicit zeroing with a warning leave
the private field uninitialized on production kernels?

Answer: there are a lot of ifs in the question. It starts from one could
allocate a non-compound high-order page and free it without clearing
tail_page->private. This assumption is wrong, since Patch 4 will catch
such code. So there is no issue.


-- 
Best Regards,
Yan, Zi