Re: Cannot boot PVH dom0 with big initrd

["Marek Marczykowski-Górecki" <marmarek@invisiblethingslab.com>]

[-- Attachment #1: Type: text/plain, Size: 6310 bytes --]

On Fri, Feb 13, 2026 at 04:56:39PM +0100, Roger Pau Monné wrote:
> On Fri, Feb 13, 2026 at 09:56:42AM +0100, Jan Beulich wrote:
> > On 13.02.2026 05:02, Marek Marczykowski-Górecki wrote:
> > > Hi,
> > > 
> > > After fixing the xhci crash, I hit another issue - booting with 236MB
> > > initrd doesn't work, I get:
> > > 
> > >     (XEN) [    3.151856] *** Building a PVH Dom0 ***
> > >     ...
> > >     (XEN) [    3.593940] Unable to allocate memory with order 0!
> > >     (XEN) [    3.597110] Failed to setup Dom0 physical memory map
> > >     (XEN) [    3.599884] 
> > >     (XEN) [    3.602482] ****************************************
> > >     (XEN) [    3.605272] Panic on CPU 0:
> > >     (XEN) [    3.607928] Could not construct d0
> > >     (XEN) [    3.610692] ****************************************
> > >     (XEN) [    3.613463] 
> > >     (XEN) [    3.616035] Reboot in five seconds...
> > >     (XEN) [    8.626565] Resetting with ACPI MEMORY or I/O RESET_REG.
> > > 
> > > Full console log: https://gist.github.com/marmarek/c9dbc87bf07b76f2899781755762f565
> > > 
> > > If I skip initrd, then it boots just fine (but dom0 is not happy about
> > > that). 164MB initrd failed too, but 13MB started ok.
> > > Just in case, I tried skipping XHCI console, but it didn't change
> > > anything.
> > > 
> > > Host has 16GB of memory, and there is no dom0_mem= parameter. Xen is
> > > started from GRUB, using MB2+EFI.
> > 
> > Hmm, yes, there's an ordering issue: Of course we free initrd space (as used
> > for passing from the boot loader to Xen) only after copying to the designated
> > guest area. Yet dom0_compute_nr_pages(), intentionally, includes the space in
> > its calculation (adding initial_images_nrpages()'s return value). PV Dom0
> > isn't affected because to load huge initrd there, the kernel has to request
> > the initrd to not be mapped into the initial allocation.
> 
> Right, on PV dom0 we do not copy the image to a new set of pages, we
> simply assign the pages where the initrd resides to the domain.  We
> can't populate those pages in the p2m as-is, otherwise we would
> shatter super pages.
> 
> I think the fix below should do it, it's likely the best we can do.
> Can you please give it a try Marek?

With this, it's different:

    (XEN) [    4.510345] Dom0 memory allocation stats:
    (XEN) [    4.513110] order  0 allocations: 1
    (XEN) [    4.515847] order  1 allocations: 1
    (XEN) [    4.518593] order  2 allocations: 2
    (XEN) [    4.521329] order  3 allocations: 2
    (XEN) [    4.524076] order  4 allocations: 1
    (XEN) [    4.526799] order  5 allocations: 1
    (XEN) [    4.529518] order  6 allocations: 1
    (XEN) [    4.532118] order  7 allocations: 2
    (XEN) [    4.534807] order  8 allocations: 2
    (XEN) [    4.537483] order  9 allocations: 1
    (XEN) [    4.540052] order 10 allocations: 2
    (XEN) [    4.542618] order 11 allocations: 2
    (XEN) [    4.545276] order 12 allocations: 1
    (XEN) [    4.547949] order 13 allocations: 1
    (XEN) [    4.550501] order 14 allocations: 1
    (XEN) [    4.553147] order 15 allocations: 1
    (XEN) [    5.393487] ELF: phdr: paddr=0x200000 memsz=0x1ff3928
    (XEN) [    5.396196] ELF: phdr: paddr=0x2200000 memsz=0x1c00000
    (XEN) [    5.398891] ELF: memory: 0x200000 -> 0x3e00000
    (XEN) [    5.401592] ELF: note: PHYS32_RELOC align: 0x200000 min: 0x200000 max: 0x3fffffff
    (XEN) [    5.404632] ELF: note: PHYS32_ENTRY = 0x16a2ca0
    (XEN) [    5.407695] ELF: note: GUEST_OS = "linux"
    (XEN) [    5.410748] ELF: note: GUEST_VERSION = "2.6"
    (XEN) [    5.413810] ELF: note: XEN_VERSION = "xen-3.0"
    (XEN) [    5.416891] ELF: note: VIRT_BASE = 0xffffffff80000000
    (XEN) [    5.419976] ELF: note: INIT_P2M = 0x8000000000
    (XEN) [    5.423062] ELF: note: ENTRY = 0xffffffff82d3c160
    (XEN) [    5.426155] ELF: note: FEATURES = "!writable_page_tables"
    (XEN) [    5.429260] ELF: note: PAE_MODE = "yes"
    (XEN) [    5.432343] ELF: note: L1_MFN_VALID
    (XEN) [    5.434952] ELF: note: MOD_START_PFN = 0x1
    (XEN) [    5.437975] ELF: note: PADDR_OFFSET = 0
    (XEN) [    5.440928] ELF: note: SUPPORTED_FEATURES = 0x8801
    (XEN) [    5.443871] ELF: note: LOADER = "generic"
    (XEN) [    5.446785] ELF: note: SUSPEND_CANCEL = 0x1
    (XEN) [    5.449696] ELF: Found PVH image
    (XEN) [    5.452574] ELF: addresses:
    (XEN) [    5.455002]     virt_base        = 0x0
    (XEN) [    5.457569]     elf_paddr_offset = 0x0
    (XEN) [    5.460102]     virt_offset      = 0x0
    (XEN) [    5.462569]     virt_kstart      = 0x200000
    (XEN) [    5.465049]     virt_kend        = 0x3e00000
    (XEN) [    5.467515]     virt_entry       = 0x16a2ca0
    (XEN) [    5.469983]     p2m_base         = 0x8000000000
    (XEN) [    5.472465] ELF: phdr 0 at 0x200000 -> 0x21f3928
    (XEN) [    5.480548] ELF: phdr 1 at 0x2200000 -> 0x3e00000
    (XEN) [    5.487915] Unable to find a memory region to load initrd and metadata
    (XEN) [    5.490466] Failed to load Dom0 kernel
    (XEN) [    5.492954] 
    (XEN) [    5.495266] ****************************************
    (XEN) [    5.497755] Panic on CPU 0:
    (XEN) [    5.500065] Could not construct d0
    (XEN) [    5.502470] ****************************************
    (XEN) [    5.504897] 
    (XEN) [    5.507134] Reboot in five seconds...
    (XEN) [   10.517191] Resetting with ACPI MEMORY or I/O RESET_REG.

FWIW memory map is in the full console dump linked earlier.

I'll test the other patch next.

> ---
> diff --git a/xen/arch/x86/dom0_build.c b/xen/arch/x86/dom0_build.c
> index 0b467fd4a4fc..8e3cb5d0db76 100644
> --- a/xen/arch/x86/dom0_build.c
> +++ b/xen/arch/x86/dom0_build.c
> @@ -343,7 +343,7 @@ unsigned long __init dom0_compute_nr_pages(
>  
>      for_each_node_mask ( node, dom0_nodes )
>          avail += avail_domheap_pages_region(node, 0, 0) +
> -                 initial_images_nrpages(node);
> +                 is_pv_domain(d) ? initial_images_nrpages(node) : 0;
>  
>      /* Reserve memory for further dom0 vcpu-struct allocations... */
>      avail -= (d->max_vcpus - 1UL)
> 

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]