[RFC PATCH v2 0/2] kho: support preserving unsplit high-order pages

[RFC PATCH v2 0/2] kho: support preserving unsplit high-order pages

[Pranjal Shrivastava]

Introduction
============
This series is required for the ongoing effort to preserve DMA allocations
across KHO [1]. It addresses a fundamental mismatch between the current KHO
restoration logic and the physical reality of high-order buddy allocations.

The Problem
===========
The current KHO restore implementation treats all multi-page blocks as
split pages during restoration. Specifically, kho_restore_pages()
initializes every 4KB sub-page with a refcount of 1.

However, many kernel subsystems, most notably the DMA allocator (via
dma_alloc_coherent), frequently return high-order non-compound pages.
In this unsplit state, only the head page carries a refcount of 1, while
all tail pages have a refcount of 0.

Consequently, when these contiguous but unsplit blocks are restored by KHO in
the new kernel, the forced reference count of 1 on tail pages causes some 
trouble with the buddy allocator. Downstream of the eventual free path,
__free_pages_prepare() [2] ends up calling page_expected_state() [3]
when is_check_pages_enabled() returns true (triggered when CONFIG_DEBUG_VM
is enabled or debug_pagealloc=on).

This detects the unexpected non-zero reference counts on tail pages [4] and
incorrectly taints the kernel while leaking the physical pages in question.

Proposed Solution
=================
Following feedback on the v1 RFC, this series moves away from auto type 
detection and instead introduces explicit preserve / restore APIs for 
high-order unsplit pages.

Callers are now explicitly specify if a block is unsplit & preserve it by using
kho_preserve_page() and kho_restore_page(). These functions apply a refcount
of 1 to the head page while leaving tail pages at 0.

The existing APIs (kho_preserve_pages / kho_restore_pages) remain as is 
for ranges of independent 4KB pages, continuing to use the split refcount.

The internal initialization logic is refactored to provide a helper:
kho_init_unsplit_pages(), which is shared between folios and unsplit page 
restore APIs.

[v2]
 - Dropped automatic type detection via higher bits in Radix key.
 - Introduced explicit kho_preserve_page and kho_restore_page helpers.
 - Refactored internal init logic to share code between folios & unsplit pages.

[v1] https://lore.kernel.org/all/20260703020832.1731864-1-praan@google.com/

Thanks,
Praan

[1] https://lore.kernel.org/all/20260708234854.4044652-1-skhawaja@google.com/
[2] https://elixir.bootlin.com/linux/v7.1.1/source/mm/page_alloc.c#L1370
[3] https://elixir.bootlin.com/linux/v7.1.1/source/mm/page_alloc.c#L1027
[4] https://elixir.bootlin.com/linux/v7.1.1/source/mm/page_alloc.c#L1034


Pranjal Shrivastava (2):
  kho: Introduce a helper to init unsplit pages
  kho: Introduce preserve/restore APIs for unsplit pages

 include/linux/kexec_handover.h     |  10 +++
 kernel/liveupdate/kexec_handover.c | 115 +++++++++++++++++++++++++----
 2 files changed, 111 insertions(+), 14 deletions(-)


base-commit: 0f26556c5eeea62cc934fa8938b148aa5844a6b6
-- 
2.55.0.141.g00534a21ce-goog

[RFC PATCH v2 1/2] kho: Introduce a helper to init unsplit pages

[Pranjal Shrivastava]

The current KHO restoration logic assumes all multi-page blocks are
split into independent 4KB pages. Break out a helper to prepare for
supporting high-order non-compound pages.

Extract kho_init_unsplit_pages() to handle the refcount pattern
where only the head page is refcounted. Use the helper for folio
restoration that requires a similar refcount logic.

Signed-off-by: Pranjal Shrivastava <praan@google.com>
---
 kernel/liveupdate/kexec_handover.c | 29 +++++++++++++++++++----------
 1 file changed, 19 insertions(+), 10 deletions(-)

diff --git a/kernel/liveupdate/kexec_handover.c b/kernel/liveupdate/kexec_handover.c
index 4834a809985a..07f6f453cd69 100644
--- a/kernel/liveupdate/kexec_handover.c
+++ b/kernel/liveupdate/kexec_handover.c
@@ -357,6 +357,24 @@ int kho_radix_walk_tree(struct kho_radix_tree *tree,
 }
 EXPORT_SYMBOL_GPL(kho_radix_walk_tree);
 
+/* For physically contiguous pages. */
+static void kho_init_unsplit_pages(struct page *page, unsigned int order)
+{
+	unsigned long nr_pages = (1UL << order);
+
+	/* Head page gets refcount of 1. */
+	set_page_count(page, 1);
+	/* Clear head page's codetag to avoid accounting mismatch. */
+	clear_page_tag_ref(page);
+
+	/* For high-order blocks, tail pages get a page count of zero. */
+	for (unsigned long i = 1; i < nr_pages; i++) {
+		set_page_count(page + i, 0);
+		/* Clear each page's codetag to avoid accounting mismatch. */
+		clear_page_tag_ref(page + i);
+	}
+}
+
 /* For physically contiguous 0-order pages. */
 static void kho_init_pages(struct page *page, unsigned long nr_pages)
 {
@@ -369,16 +387,7 @@ static void kho_init_pages(struct page *page, unsigned long nr_pages)
 
 static void kho_init_folio(struct page *page, unsigned int order)
 {
-	unsigned long nr_pages = (1 << order);
-
-	/* Head page gets refcount of 1. */
-	set_page_count(page, 1);
-	/* Clear head page's codetag to avoid accounting mismatch. */
-	clear_page_tag_ref(page);
-
-	/* For higher order folios, tail pages get a page count of zero. */
-	for (unsigned long i = 1; i < nr_pages; i++)
-		set_page_count(page + i, 0);
+	kho_init_unsplit_pages(page, order);
 
 	if (order > 0)
 		prep_compound_page(page, order);
-- 
2.55.0.141.g00534a21ce-goog

Re: [RFC PATCH v2 1/2] kho: Introduce a helper to init unsplit pages

[Samiullah Khawaja]

On Mon, Jul 13, 2026 at 08:49:34PM +0000, Pranjal Shrivastava wrote:
>The current KHO restoration logic assumes all multi-page blocks are
>split into independent 4KB pages. Break out a helper to prepare for
>supporting high-order non-compound pages.
>
>Extract kho_init_unsplit_pages() to handle the refcount pattern
>where only the head page is refcounted. Use the helper for folio
>restoration that requires a similar refcount logic.
>
>Signed-off-by: Pranjal Shrivastava <praan@google.com>
>---
> kernel/liveupdate/kexec_handover.c | 29 +++++++++++++++++++----------
> 1 file changed, 19 insertions(+), 10 deletions(-)
>

Reviewed-by: Samiullah Khawaja <skhawaja@google.com>

Re: [RFC PATCH v2 1/2] kho: Introduce a helper to init unsplit pages

[Mike Rapoport]

> The current KHO restoration logic assumes all multi-page blocks are
> split into independent 4KB pages. Break out a helper to prepare for
> supporting high-order non-compound pages.
> 
> Extract kho_init_unsplit_pages() to handle the refcount pattern
> where only the head page is refcounted. Use the helper for folio
> restoration that requires a similar refcount logic.
> 
> Signed-off-by: Pranjal Shrivastava <praan@google.com>
>
> diff --git a/kernel/liveupdate/kexec_handover.c b/kernel/liveupdate/kexec_handover.c
> index 4834a809985ab..07f6f453cd69d 100644
> --- a/kernel/liveupdate/kexec_handover.c
> +++ b/kernel/liveupdate/kexec_handover.c
> @@ -357,6 +357,24 @@ int kho_radix_walk_tree(struct kho_radix_tree *tree,
>  }
>  EXPORT_SYMBOL_GPL(kho_radix_walk_tree);
>  
> +/* For physically contiguous pages. */
> +static void kho_init_unsplit_pages(struct page *page, unsigned int order)
> +{

Can't say I like "unsplit". How about kho_init_high_order_page?

And whatever name we pick here, it's surely must end with "page" rather
than "pages".

-- 
Sincerely yours,
Mike.

Re: [RFC PATCH v2 1/2] kho: Introduce a helper to init unsplit pages

[Pranjal Shrivastava]

On Mon, Jul 20, 2026 at 07:57:15PM +0300, Mike Rapoport wrote:
> > The current KHO restoration logic assumes all multi-page blocks are
> > split into independent 4KB pages. Break out a helper to prepare for
> > supporting high-order non-compound pages.
> > 
> > Extract kho_init_unsplit_pages() to handle the refcount pattern
> > where only the head page is refcounted. Use the helper for folio
> > restoration that requires a similar refcount logic.
> > 
> > Signed-off-by: Pranjal Shrivastava <praan@google.com>
> >
> > diff --git a/kernel/liveupdate/kexec_handover.c b/kernel/liveupdate/kexec_handover.c
> > index 4834a809985ab..07f6f453cd69d 100644
> > --- a/kernel/liveupdate/kexec_handover.c
> > +++ b/kernel/liveupdate/kexec_handover.c
> > @@ -357,6 +357,24 @@ int kho_radix_walk_tree(struct kho_radix_tree *tree,
> >  }
> >  EXPORT_SYMBOL_GPL(kho_radix_walk_tree);
> >  
> > +/* For physically contiguous pages. */
> > +static void kho_init_unsplit_pages(struct page *page, unsigned int order)
> > +{
> 
> Can't say I like "unsplit". How about kho_init_high_order_page?
> 
> And whatever name we pick here, it's surely must end with "page" rather
> than "pages".
> 

Ack. I'll rename this to kho_init_high_order_page().

Thanks,
Praan

[RFC PATCH v2 2/2] kho: Introduce preserve/restore APIs for unsplit pages

[Pranjal Shrivastava]

The current KHO page preservation APIs (e.g. kho_preserve_pages) assume
that multi-page blocks are split into independent 4KB pages during
restoration. This is incompatible with high-order non-compound pages,
such as DMA buffers, which must be restored with tail pages having a
zero reference count.

Introduce explicit preserve and restore APIs for unsplit pages,
which preserve and restore a high-order page block as a single unit,
applying a refcount of 1 to the head page while leaving tail pages at 0.
Rename the existing internal helper to __kho_restore_page().

Signed-off-by: Pranjal Shrivastava <praan@google.com>
---
 include/linux/kexec_handover.h     | 10 ++++
 kernel/liveupdate/kexec_handover.c | 86 ++++++++++++++++++++++++++++--
 2 files changed, 92 insertions(+), 4 deletions(-)

diff --git a/include/linux/kexec_handover.h b/include/linux/kexec_handover.h
index 8968c56d2d73..6e2c75b16209 100644
--- a/include/linux/kexec_handover.h
+++ b/include/linux/kexec_handover.h
@@ -22,6 +22,8 @@ bool is_kho_boot(void);
 
 int kho_preserve_folio(struct folio *folio);
 void kho_unpreserve_folio(struct folio *folio);
+int kho_preserve_page(struct page *page, unsigned int order);
+void kho_unpreserve_page(struct page *page, unsigned int order);
 int kho_preserve_pages(struct page *page, unsigned long nr_pages);
 void kho_unpreserve_pages(struct page *page, unsigned long nr_pages);
 int kho_preserve_vmalloc(void *ptr, struct kho_vmalloc *preservation);
@@ -30,6 +32,7 @@ void *kho_alloc_preserve(size_t size);
 void kho_unpreserve_free(void *mem);
 void kho_restore_free(void *mem);
 struct folio *kho_restore_folio(phys_addr_t phys);
+struct page *kho_restore_page(phys_addr_t phys);
 struct page *kho_restore_pages(phys_addr_t phys, unsigned long nr_pages);
 void *kho_restore_vmalloc(const struct kho_vmalloc *preservation);
 int kho_add_subtree(const char *name, void *blob, size_t size);
@@ -65,6 +68,13 @@ static inline int kho_preserve_pages(struct page *page, unsigned int nr_pages)
 
 static inline void kho_unpreserve_pages(struct page *page, unsigned int nr_pages) { }
 
+static inline int kho_preserve_page(struct page *page, unsigned int order)
+{
+	return -EOPNOTSUPP;
+}
+
+static inline void kho_unpreserve_page(struct page *page, unsigned int order) { }
+
 static inline int kho_preserve_vmalloc(void *ptr,
 				       struct kho_vmalloc *preservation)
 {
diff --git a/kernel/liveupdate/kexec_handover.c b/kernel/liveupdate/kexec_handover.c
index 07f6f453cd69..43ae3e2de5da 100644
--- a/kernel/liveupdate/kexec_handover.c
+++ b/kernel/liveupdate/kexec_handover.c
@@ -393,7 +393,7 @@ static void kho_init_folio(struct page *page, unsigned int order)
 		prep_compound_page(page, order);
 }
 
-static struct page *kho_restore_page(phys_addr_t phys, bool is_folio)
+static struct page *__kho_restore_page(phys_addr_t phys, bool is_folio)
 {
 	struct page *page = pfn_to_online_page(PHYS_PFN(phys));
 	unsigned long nr_pages;
@@ -432,12 +432,44 @@ static struct page *kho_restore_page(phys_addr_t phys, bool is_folio)
  */
 struct folio *kho_restore_folio(phys_addr_t phys)
 {
-	struct page *page = kho_restore_page(phys, true);
+	struct page *page = __kho_restore_page(phys, true);
 
 	return page ? page_folio(page) : NULL;
 }
 EXPORT_SYMBOL_GPL(kho_restore_folio);
 
+/**
+ * kho_restore_page - restore a higher-order unsplit page block.
+ * @phys: physical address of the first page.
+ *
+ * Restore a higher-order unsplit page block that was preserved with
+ * kho_preserve_page().
+ *
+ * Return: the head page on success, NULL on failure.
+ */
+struct page *kho_restore_page(phys_addr_t phys)
+{
+	struct page *page = pfn_to_online_page(PHYS_PFN(phys));
+	unsigned long nr_pages;
+	union kho_page_info info;
+
+	if (!page)
+		return NULL;
+
+	info.page_private = page->private;
+	if (WARN_ON_ONCE(info.magic != KHO_PAGE_MAGIC))
+		return NULL;
+
+	nr_pages = (1UL << info.order);
+	page->private = 0;
+
+	kho_init_unsplit_pages(page, info.order);
+	adjust_managed_page_count(page, nr_pages);
+
+	return page;
+}
+EXPORT_SYMBOL_GPL(kho_restore_page);
+
 /**
  * kho_restore_pages - restore list of contiguous order 0 pages.
  * @phys: physical address of the first page.
@@ -457,7 +489,7 @@ struct page *kho_restore_pages(phys_addr_t phys, unsigned long nr_pages)
 	while (pfn < end_pfn) {
 		const unsigned int order =
 			min(count_trailing_zeros(pfn), ilog2(end_pfn - pfn));
-		struct page *page = kho_restore_page(PFN_PHYS(pfn), false);
+		struct page *page = __kho_restore_page(PFN_PHYS(pfn), false);
 
 		if (!page)
 			return NULL;
@@ -890,6 +922,51 @@ void kho_unpreserve_folio(struct folio *folio)
 }
 EXPORT_SYMBOL_GPL(kho_unpreserve_folio);
 
+/**
+ * kho_preserve_page - preserve a higher-order "unsplit" page block.
+ * @page: head page of the block.
+ * @order: order of the allocation.
+ *
+ * Instructs KHO to preserve a higher-order contiguous allocation (like a DMA
+ * buffer) as a single unit. It must be restored using kho_restore_page() to
+ * ensure the tail pages are correctly initialized with a zero refcount.
+ *
+ * If a driver needs to split a block that has been preserved with this
+ * function, it must first unpreserve the block using kho_unpreserve_page(),
+ * perform the split, and then re-preserve the individual pages using
+ * kho_preserve_pages().
+ *
+ * Return: 0 on success, error code on failure
+ */
+int kho_preserve_page(struct page *page, unsigned int order)
+{
+	struct kho_radix_tree *tree = &kho_out.radix_tree;
+	const unsigned long pfn = page_to_pfn(page);
+
+	if (WARN_ON(kho_scratch_overlap(pfn << PAGE_SHIFT, PAGE_SIZE << order)))
+		return -EINVAL;
+
+	return kho_radix_add_page(tree, pfn, order);
+}
+EXPORT_SYMBOL_GPL(kho_preserve_page);
+
+/**
+ * kho_unpreserve_page - unpreserve a higher-order "unsplit" page block.
+ * @page: head page of the block.
+ * @order: order of the allocation.
+ *
+ * Instructs KHO to unpreserve a high-order block that was preserved by
+ * kho_preserve_page() before.
+ */
+void kho_unpreserve_page(struct page *page, unsigned int order)
+{
+	struct kho_radix_tree *tree = &kho_out.radix_tree;
+	const unsigned long pfn = page_to_pfn(page);
+
+	kho_radix_del_page(tree, pfn, order);
+}
+EXPORT_SYMBOL_GPL(kho_unpreserve_page);
+
 static unsigned int __kho_preserve_pages_order(unsigned long start_pfn,
 					       unsigned long end_pfn)
 {
@@ -927,7 +1004,8 @@ static void __kho_unpreserve(struct kho_radix_tree *tree,
  * @nr_pages: number of pages.
  *
  * Preserve a contiguous list of order 0 pages. Must be restored using
- * kho_restore_pages() to ensure the pages are restored properly as order 0.
+ * kho_restore_pages() to ensure the pages are restored properly as order 0
+ * with each page having a reference count of 1 (split).
  *
  * Return: 0 on success, error code on failure
  */
-- 
2.55.0.141.g00534a21ce-goog

Re: [RFC PATCH v2 2/2] kho: Introduce preserve/restore APIs for unsplit pages

[Mike Rapoport]

> The current KHO page preservation APIs (e.g. kho_preserve_pages) assume
> that multi-page blocks are split into independent 4KB pages during
> restoration. This is incompatible with high-order non-compound pages,
> such as DMA buffers, which must be restored with tail pages having a
> zero reference count.
> 
> Introduce explicit preserve and restore APIs for unsplit pages,

                                 high-order non-compound ^

> which preserve and restore a high-order page block as a single unit,
> applying a refcount of 1 to the head page while leaving tail pages at 0.
> Rename the existing internal helper to __kho_restore_page().
> 
> Signed-off-by: Pranjal Shrivastava <praan@google.com>
>
> diff --git a/include/linux/kexec_handover.h b/include/linux/kexec_handover.h
> index 8968c56d2d73e..6e2c75b16209c 100644
> --- a/include/linux/kexec_handover.h
> +++ b/include/linux/kexec_handover.h
> @@ -22,6 +22,8 @@ bool is_kho_boot(void);
>  
>  int kho_preserve_folio(struct folio *folio);
>  void kho_unpreserve_folio(struct folio *folio);
> +int kho_preserve_page(struct page *page, unsigned int order);
> +void kho_unpreserve_page(struct page *page, unsigned int order);
>  int kho_preserve_pages(struct page *page, unsigned long nr_pages);
>  void kho_unpreserve_pages(struct page *page, unsigned long nr_pages);
>  int kho_preserve_vmalloc(void *ptr, struct kho_vmalloc *preservation);
> @@ -30,6 +32,7 @@ void *kho_alloc_preserve(size_t size);
>  void kho_unpreserve_free(void *mem);
>  void kho_restore_free(void *mem);
>  struct folio *kho_restore_folio(phys_addr_t phys);
> +struct page *kho_restore_page(phys_addr_t phys);
>  struct page *kho_restore_pages(phys_addr_t phys, unsigned long nr_pages);
>  void *kho_restore_vmalloc(const struct kho_vmalloc *preservation);
>  int kho_add_subtree(const char *name, void *blob, size_t size);
> @@ -65,6 +68,13 @@ static inline int kho_preserve_pages(struct page *page, unsigned int nr_pages)
>  
>  static inline void kho_unpreserve_pages(struct page *page, unsigned int nr_pages) { }
>  
> +static inline int kho_preserve_page(struct page *page, unsigned int order)
> +{
> +	return -EOPNOTSUPP;
> +}
> +
> +static inline void kho_unpreserve_page(struct page *page, unsigned int order) { }
> +
>  static inline int kho_preserve_vmalloc(void *ptr,
>  				       struct kho_vmalloc *preservation)
>  {
> diff --git a/kernel/liveupdate/kexec_handover.c b/kernel/liveupdate/kexec_handover.c
> index 07f6f453cd69d..43ae3e2de5da2 100644
> --- a/kernel/liveupdate/kexec_handover.c
> +++ b/kernel/liveupdate/kexec_handover.c
> @@ -393,7 +393,7 @@ static void kho_init_folio(struct page *page, unsigned int order)
>  		prep_compound_page(page, order);
>  }
>  
> -static struct page *kho_restore_page(phys_addr_t phys, bool is_folio)
> +static struct page *__kho_restore_page(phys_addr_t phys, bool is_folio)
>  {
>  	struct page *page = pfn_to_online_page(PHYS_PFN(phys));
>  	unsigned long nr_pages;
> @@ -432,12 +432,44 @@ static struct page *kho_restore_page(phys_addr_t phys, bool is_folio)
>   */
>  struct folio *kho_restore_folio(phys_addr_t phys)
>  {
> -	struct page *page = kho_restore_page(phys, true);
> +	struct page *page = __kho_restore_page(phys, true);
>  
>  	return page ? page_folio(page) : NULL;
>  }
>  EXPORT_SYMBOL_GPL(kho_restore_folio);
>  
> +/**
> + * kho_restore_page - restore a higher-order unsplit page block.
> + * @phys: physical address of the first page.
> + *
> + * Restore a higher-order unsplit page block that was preserved with
> + * kho_preserve_page().
> + *
> + * Return: the head page on success, NULL on failure.
> + */
> +struct page *kho_restore_page(phys_addr_t phys)
> +{
> +	struct page *page = pfn_to_online_page(PHYS_PFN(phys));
> +	unsigned long nr_pages;
> +	union kho_page_info info;
> +
> +	if (!page)
> +		return NULL;
> +
> +	info.page_private = page->private;
> +	if (WARN_ON_ONCE(info.magic != KHO_PAGE_MAGIC))
> +		return NULL;
> +
> +	nr_pages = (1UL << info.order);
> +	page->private = 0;
> +
> +	kho_init_unsplit_pages(page, info.order);
> +	adjust_managed_page_count(page, nr_pages);
> +
> +	return page;

Hmm, it copies a lot from __kho_init_page().
We should keep the common part of order-0 page/folio/high-order page
initialization in a single place.

-- 
Sincerely yours,
Mike.

Re: [RFC PATCH v2 2/2] kho: Introduce preserve/restore APIs for unsplit pages

[Pranjal Shrivastava]

On Mon, Jul 20, 2026 at 07:57:15PM +0300, Mike Rapoport wrote:
> > The current KHO page preservation APIs (e.g. kho_preserve_pages) assume
> > that multi-page blocks are split into independent 4KB pages during
> > restoration. This is incompatible with high-order non-compound pages,
> > such as DMA buffers, which must be restored with tail pages having a
> > zero reference count.
> > 
> > Introduce explicit preserve and restore APIs for unsplit pages,
> 
>                                  high-order non-compound ^
> 

Ack, will update the commit msg.

> > which preserve and restore a high-order page block as a single unit,
> > applying a refcount of 1 to the head page while leaving tail pages at 0.
> > Rename the existing internal helper to __kho_restore_page().
> > 
> > Signed-off-by: Pranjal Shrivastava <praan@google.com>
> >
> > diff --git a/include/linux/kexec_handover.h b/include/linux/kexec_handover.h
> > index 8968c56d2d73e..6e2c75b16209c 100644
> > --- a/include/linux/kexec_handover.h
> > +++ b/include/linux/kexec_handover.h
> > @@ -22,6 +22,8 @@ bool is_kho_boot(void);
> >  

[...]

> > +struct page *kho_restore_page(phys_addr_t phys)
> > +{
> > +	struct page *page = pfn_to_online_page(PHYS_PFN(phys));
> > +	unsigned long nr_pages;
> > +	union kho_page_info info;
> > +
> > +	if (!page)
> > +		return NULL;
> > +
> > +	info.page_private = page->private;
> > +	if (WARN_ON_ONCE(info.magic != KHO_PAGE_MAGIC))
> > +		return NULL;
> > +
> > +	nr_pages = (1UL << info.order);
> > +	page->private = 0;
> > +
> > +	kho_init_unsplit_pages(page, info.order);
> > +	adjust_managed_page_count(page, nr_pages);
> > +
> > +	return page;
> 
> Hmm, it copies a lot from __kho_init_page().
> We should keep the common part of order-0 page/folio/high-order page
> initialization in a single place.
> 

Alright, I'll break out a helper.

Thanks,
Praan