[PATCH v2 0/2] xen/mm: Ensure page offlining only creates properly aligned buddies

[PATCH v2 0/2] xen/mm: Ensure page offlining only creates properly aligned buddies

[Bernhard Kaindl]

This series fixes a bug in reserve_offlined_page() where growing
buddies around offlined pages may create misaligned buddies and
return them to the free lists.

For example, pages may be offlined following an MCE caused by faulty RAM.
Once a misaligned buddy has been placed on a free list, a particular
sequence of allocations and frees may cause the same page to be allocated
more than once, eventually triggering a Xen BUG() in alloc_heap_pages().

As requested for backporting the fix, the first patch adds the alignment
check while the second patch now adds the regression test for this issue.

As minimal example, consider an order-2 buddy (4 pages) with this layout:

   +---------------+-----------------+-----------------+----------------+
   | head page     | tail page 1     | tail page 2     | tail page 3    |
   +---------------+-----------------+-----------------+----------------+

reserve_offline_page() then merges unaligned tail pages:

   +---------------+-----------------+-----------------+----------------+
   | offlined page |     head page with a tail page    | single page    |
   +---------------+-----------------+-----------------+----------------+

When a single page is allocated from this buddy, MFN 7 is allocated:

        MFN 4             MFN 5             MFN 6             MFN 7
  +---------------+-----------------+-----------------+----------------+
  | offlined page |    head page        tail page     | allocated page |
  +---------------+-----------------+-----------------+----------------+

If MFN 7 is freed, the predecessor merge in free_heap_pages() kicks in,
merging MFN 7 with its naturally aligned predecessor page at MFN 6:

        MFN 4             MFN 5             MFN 6            MFN 7
  +---------------+-----------------+-----------------+
  | offlined page |    head page         tail page    |
  +---------------+-----------------+-----------------+----------------+
                                    |    head page        tail page    |
                                    +-----------------+----------------+

The next allocations would allocate MFN 7 again and MFN 6 as well:

        MFN 4             MFN 5             MFN 6            MFN 7
  +---------------+-----------------+-----------------+
  | offlined page |    head page         tail page    | <- BUG() on alloc
  +---------------+-----------------+-----------------+----------------+
                                    |   in-use page   |   in-use page  |
                                    +-----------------+----------------+

When the next page from this buddy is allocated, get_free_page() returns
the buddy head MFN 5. An order-0 allocation splits page 6, or an order-1
allocation allocates the whole unaligned buddy. alloc_heap_page()
catches the attempt to allocate MFN 6 for a second time as a bug:

   pg[0] MFN 842adc c=0x4000000000000000 o=0 v=0 t=0
   Xen BUG at common/page_alloc.c:1324

You can pull this series with the regression test environment to run it:
https://lists.xen.org/archives/html/xen-devel/2026-05/msg01163.html
git pull git@gitlab.com:bernhardkaindl/xen.git offline-unaligned-buddies-v2
make -C tools/tests/native TARGETS=offline-unaligned test

Fixes: e4865c2315 ('Page offline support in Xen side')
Signed-off-by: Bernhard Kaindl <bernhard.kaindl@citrix.com>

Bernhard Kaindl (2):
  xen/page_alloc: verify buddy alignment in reserve_offlined_page()
  tools/tests: Regression test checking unaligned pages after offlining

 tools/tests/native/offline-unaligned.c | 60 ++++++++++++++++++++++++++
 xen/common/page_alloc.c                |  5 +++
 2 files changed, 65 insertions(+)
 create mode 100644 tools/tests/native/offline-unaligned.c

-- 
2.39.5

[PATCH v2 1/2] xen/page_alloc: verify buddy alignment in reserve_offlined_page()

[Bernhard Kaindl]

reserve_offlined_page() fails to verify alignment when growing
buddies around offlined pages. Consequently, misaligned buddies
may be constructed from non-offlined page ranges and returned to
the free lists.

After a particular sequence of allocations and frees, pages
from such a misaligned buddy may be allocated more than once,
eventually triggering a Xen BUG() in alloc_heap_pages().

Fixes: e4865c2315 ('Page offline support in Xen side')
Signed-off-by: Bernhard Kaindl <bernhard.kaindl@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
v2:
- Updated the title for clarity.
- Bugfix isolated from the test case for backporting.
- Removed excess parentheses from the alignment check if() expression.
- Simplified the alignment check to use '& (1UL << cur_order)'. Because
  the covering buddy head is size-aligned, cur_head is also aligned to
  cur_order, making this reduction safe (verified against extended tests).
- Updated the inline code comment to accurately state that only the upper
  half of the next_order range is checked for offlined pages.
---
 xen/common/page_alloc.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 2c4ff2c34c70..2767376a710b 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -1202,6 +1202,11 @@ static int reserve_offlined_page(struct page_info *head)
             if ( (cur_head + (1 << next_order)) >= (head + ( 1 << head_order)) )
                 goto merge;
 
+            /* Do not grow to next_order if cur_head is not aligned to it. */
+            if ( mfn_x(page_to_mfn(cur_head)) & (1UL << cur_order) )
+                goto merge;
+
+            /* Check for offlined pages in upper half of next_order range. */
             for ( i = (1 << cur_order), pg = cur_head + (1 << cur_order );
                   i < (1 << next_order);
                   i++, pg++ )
-- 
2.39.5

Re: [PATCH v2 1/2] xen/page_alloc: verify buddy alignment in reserve_offlined_page()

[Jan Beulich]

On 03.06.2026 16:17, Bernhard Kaindl wrote:
> reserve_offlined_page() fails to verify alignment when growing
> buddies around offlined pages. Consequently, misaligned buddies
> may be constructed from non-offlined page ranges and returned to
> the free lists.
> 
> After a particular sequence of allocations and frees, pages
> from such a misaligned buddy may be allocated more than once,
> eventually triggering a Xen BUG() in alloc_heap_pages().
> 
> Fixes: e4865c2315 ('Page offline support in Xen side')
> Signed-off-by: Bernhard Kaindl <bernhard.kaindl@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Oleksii, thoughts towards 4.22?

Jan

> ---
> v2:
> - Updated the title for clarity.
> - Bugfix isolated from the test case for backporting.
> - Removed excess parentheses from the alignment check if() expression.
> - Simplified the alignment check to use '& (1UL << cur_order)'. Because
>   the covering buddy head is size-aligned, cur_head is also aligned to
>   cur_order, making this reduction safe (verified against extended tests).
> - Updated the inline code comment to accurately state that only the upper
>   half of the next_order range is checked for offlined pages.
> ---
>  xen/common/page_alloc.c | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
> index 2c4ff2c34c70..2767376a710b 100644
> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -1202,6 +1202,11 @@ static int reserve_offlined_page(struct page_info *head)
>              if ( (cur_head + (1 << next_order)) >= (head + ( 1 << head_order)) )
>                  goto merge;
>  
> +            /* Do not grow to next_order if cur_head is not aligned to it. */
> +            if ( mfn_x(page_to_mfn(cur_head)) & (1UL << cur_order) )
> +                goto merge;
> +
> +            /* Check for offlined pages in upper half of next_order range. */
>              for ( i = (1 << cur_order), pg = cur_head + (1 << cur_order );
>                    i < (1 << next_order);
>                    i++, pg++ )

Re: [PATCH v2 1/2] xen/page_alloc: verify buddy alignment in reserve_offlined_page()

[Oleksii Kurochko]

On 6/3/26 4:30 PM, Jan Beulich wrote:
> On 03.06.2026 16:17, Bernhard Kaindl wrote:
>> reserve_offlined_page() fails to verify alignment when growing
>> buddies around offlined pages. Consequently, misaligned buddies
>> may be constructed from non-offlined page ranges and returned to
>> the free lists.
>>
>> After a particular sequence of allocations and frees, pages
>> from such a misaligned buddy may be allocated more than once,
>> eventually triggering a Xen BUG() in alloc_heap_pages().
>>
>> Fixes: e4865c2315 ('Page offline support in Xen side')
>> Signed-off-by: Bernhard Kaindl <bernhard.kaindl@citrix.com>
>> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> 
> Oleksii, thoughts towards 4.22?

I've waited for v2 of this patch series to R-Ack, I see patches 
separately but they aren't grouped into one patch series for some reason.

Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

~ Oleksii

> 
> Jan
> 
>> ---
>> v2:
>> - Updated the title for clarity.
>> - Bugfix isolated from the test case for backporting.
>> - Removed excess parentheses from the alignment check if() expression.
>> - Simplified the alignment check to use '& (1UL << cur_order)'. Because
>>    the covering buddy head is size-aligned, cur_head is also aligned to
>>    cur_order, making this reduction safe (verified against extended tests).
>> - Updated the inline code comment to accurately state that only the upper
>>    half of the next_order range is checked for offlined pages.
>> ---
>>   xen/common/page_alloc.c | 5 +++++
>>   1 file changed, 5 insertions(+)
>>
>> diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
>> index 2c4ff2c34c70..2767376a710b 100644
>> --- a/xen/common/page_alloc.c
>> +++ b/xen/common/page_alloc.c
>> @@ -1202,6 +1202,11 @@ static int reserve_offlined_page(struct page_info *head)
>>               if ( (cur_head + (1 << next_order)) >= (head + ( 1 << head_order)) )
>>                   goto merge;
>>   
>> +            /* Do not grow to next_order if cur_head is not aligned to it. */
>> +            if ( mfn_x(page_to_mfn(cur_head)) & (1UL << cur_order) )
>> +                goto merge;
>> +
>> +            /* Check for offlined pages in upper half of next_order range. */
>>               for ( i = (1 << cur_order), pg = cur_head + (1 << cur_order );
>>                     i < (1 << next_order);
>>                     i++, pg++ )
> 

[PATCH v2 2/2] tools/tests: Regression test checking unaligned pages after offlining

[Bernhard Kaindl]

Add a regression test to check that offlining pages does not return
a misaligned buddy like to the free lists after reseving the first
offlined sub-page from a buddy and returning the healthy  to the free list:

   +---------------+-----------------+-----------------+----------------+
   | offlined page |     head page with a tail page    | single page    |
   +---------------+-----------------+-----------------+----------------+

After a seres of allocations and frees, such an unaligned buddy would
trigger a Xen BUG if this state is reached when the tail is allocated:

  +---------------+-----------------+-----------------+
  | offlined page |    head page         tail page    |
  +---------------+-----------------+-----------------+----------------+
                                    |   in-use page   |   in-use page  |
                                    +-----------------+----------------+

Signed-off-by: Bernhard Kaindl <bernhard.kaindl@citrix.com>
---
 tools/tests/native/offline-unaligned.c | 60 ++++++++++++++++++++++++++
 1 file changed, 60 insertions(+)
 create mode 100644 tools/tests/native/offline-unaligned.c

diff --git a/tools/tests/native/offline-unaligned.c b/tools/tests/native/offline-unaligned.c
new file mode 100644
index 000000000000..198bf952ba7d
--- /dev/null
+++ b/tools/tests/native/offline-unaligned.c
@@ -0,0 +1,60 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Regression test offlining to not return misaligned buddies to free lists.
+ * Copyright (C) 2026 Cloud Software Group
+ */
+#include "harness/native.h"
+
+/*
+ * Verify that offlining the head page of an order-2 buddy does not return
+ * a misaligned buddy to the free list, and the pages returned to the free
+ * list are usable for allocations.
+ */
+static void test_unaligned_buddy_merge(int start_mfn)
+{
+    struct page_info *pg = frame_table + start_mfn;
+    uint32_t status = 0;
+
+    /*
+     * Prepare a valid order-2 buddy (4 pages) with this layout:
+     * +---------------+-----------------+-----------------+----------------+
+     * | head page     | tail page 1     | tail page 2     | tail page 3    |
+     * +---------------+-----------------+-----------------+----------------+
+     */
+    test_page_list_add_buddy(pg, order2);
+
+    /* Act */
+    offline_page(page_to_mfn(pg), 0, &status);
+    CHECK(status & PG_OFFLINE_OFFLINED, "Page should be offlined");
+
+    /*
+     * The expected free list state after offlining the buddy head is:
+     * +---------------+---------------+----------------+---------------+
+     * | offlined page |  single page  |    head page with a tail page  |
+     * +---------------+---------------+----------------+---------------+
+     */
+    CHECK(page_aligned(pg + 1), "The buddy #%lu is not aligned to order-%d",
+          mfn_x(page_to_mfn(pg + 1)), PFN_ORDER(pg + 1));
+
+    /* Allocate and free a page to trigger buddy merging on free. */
+    free_domheap_pages(alloc_domheap_pages(dom1, order0, 0), order0);
+    CHECK((pg = alloc_domheap_pages(dom1, order1, 0)), "Alloc the order-1 pg");
+
+    /* Inspect the predecessor (pg is the tail of the unaligned buddy) */
+    CHECK(page_aligned(pg - 1), "The buddy #%lu is not aligned to order-%d!",
+          mfn_x(page_to_mfn(pg - 1)), PFN_ORDER(pg - 1));
+
+    /* Test allocating the remaining page */
+    alloc_domheap_pages(dom1, order0, 0);
+}
+
+int main(int argc, char *argv[])
+{
+    if ( !parse_args(argc, argv, "Test offlining to return aligned buddies") )
+        return EXIT_FAILURE;
+
+    init_page_alloc_tests();
+    RUN_TESTCASE("TUBM", test_unaligned_buddy_merge, 4);
+
+    return test_complete();
+}
-- 
2.39.5