Re: Linux dom0 hangs on boot with Xen 4.22 and Heads firmware

["Marek Marczykowski-Górecki" <marmarek@invisiblethingslab.com>]

[-- Attachment #1: Type: text/plain, Size: 7906 bytes --]

On Wed, Jun 24, 2026 at 09:43:22AM +0200, Jan Beulich wrote:
> On 24.06.2026 02:31, Marek Marczykowski-Górecki wrote:
> > On Mon, Jun 22, 2026 at 10:53:01AM +0200, Jan Beulich wrote:
> >> On 19.06.2026 16:38, Marek Marczykowski-Górecki wrote:
> >>> After updating Xen to 4.22-rc2 on a system with Heads firmware, dom0
> >>> doesn't start anymore. It worked fine with Xen 4.19.
> >>> The last messages on console are:
> >>>
> >>>     [    1.495140] installing Xen timer for CPU 2
> >>>     [    1.496149] installing Xen timer for CPU 4
> >>>     [    1.496587] installing Xen timer for CPU 5
> >>>     [    1.496809] installing Xen timer for CPU 7
> >>>     [    0.008235] [Firmware Bug]: CPU   2: APIC ID mismatch. CPUID: 0x0002 APIC: 0x0028
> >>>     [    0.008235] [Firmware Bug]: CPU   2: APIC ID mismatch. Firmware: 0x0011 APIC: 0x0028
> >>>     [    1.497055] cpu 2 spinlock event irq 200
> >>>     [    0.008235] [Firmware Bug]: CPU   4: APIC ID mismatch. CPUID: 0x0004 APIC: 0x0000
> >>>     [    0.008235] [Firmware Bug]: CPU   4: APIC ID mismatch. Firmware: 0x0019 APIC: 0x0000
> >>>     [    1.497074] cpu 4 spinlock event irq 201
> >>>     [    0.008235] [Firmware Bug]: CPU   5: APIC ID mismatch. CPUID: 0x0005 APIC: 0x0002
> >>>     [    0.008235] [Firmware Bug]: CPU   5: APIC ID mismatch. Firmware: 0x0021 APIC: 0x0002
> >>>     [    1.497074] cpu 5 spinlock event irq 202
> >>>     [    0.008235] [Firmware Bug]: CPU   7: APIC I
> >>>
> >>> Full console log (containing both successful boot of Xen 4.19, and then
> >>> reboot into 4.22):
> >>> https://openqa.qubes-os.org/tests/184780/logfile?filename=serial0.txt
> >>
> >> The 4.19 log also has an anomaly around this point in time. Can you try
> >> again with sync_console added to both the 4.19 and the 4.22 attempt?
> > 
> > Yes, sync_console helped quite a bit, now I get full dom0 panic message:
> > 
> >     [   10.334800] vesafb: cannot reserve video memory at 0x0
> >     [   10.340009] vesafb: mode is 0x0x0, linelength=0, pages=0
> >     [   10.345515] Oops: divide error: 0000 [#1] SMP NOPTI
> >     [   10.346503] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted 6.17.9-1.qubes.fc41.x86_64 #1 PREEMPT(full) 
> >     [   10.346503] Hardware name: Notebook V54x_6x_TU/V54x_6x_TU, BIOS Dasharo (coreboot+heads) v0.9.0 01/01/1970
> >     [   10.346503] RIP: e030:vesafb_probe.cold+0xd4/0x5fb
> >     [   10.346503] Code: 08 75 1f 83 3d a8 8c 1d 02 00 75 16 48 c7 c7 90 4a fd 81 e8 8a ef f9 ff c7 05 d4 54 09 02 05 00 00 00 8b 05 c2 54 09 02 31 d2 <f7> 35 d2 54 09 02 8b 15 fc 54 09 02 48 89 c1 48 c1 e1 20 48 09 ca
> >     [   10.346503] RSP: e02b:ffffc9004001fbb8 EFLAGS: 00010246
> >     [   10.346503] RAX: 0000000000000000 RBX: ffff888101d86f28 RCX: ffffffff823666e8
> >     [   10.346503] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000003
> >     [   10.346503] RBP: ffff88810197d400 R08: 0000000000000000 R09: 6c656e696c202c30
> >     [   10.346503] R10: 0000000000000030 R11: 203a626661736576 R12: 0000000000000000
> >     [   10.346503] R13: 0000000000000000 R14: 0000000000000000 R15: ffff888106351000
> >     [   10.346503] FS:  0000000000000000(0000) GS:ffff888cef7c1000(0000) knlGS:0000000000000000
> >     [   10.346503] CS:  e030 DS: 0000 ES: 0000 CR0: 0000000080050033
> >     [   10.346503] CR2: ffffc900064ff000 CR3: 000000000222c000 CR4: 0000000000050660
> >     [   10.346503] Call Trace:
> >     [   10.346503]  <TASK>
> >     [   10.346503]  ? __pfx___device_attach_driver+0x10/0x10
> >     [   10.346503]  platform_probe+0x39/0x70
> >     [   10.346503]  really_probe+0xdb/0x340
> >     [   10.346503]  ? pm_runtime_barrier+0x54/0x90
> >     [   10.346503]  __driver_probe_device+0x78/0x110
> >     [   10.346503]  driver_probe_device+0x1f/0xa0
> >     [   10.346503]  __device_attach_driver+0x89/0x110
> >     [   10.346503]  bus_for_each_drv+0x94/0xf0
> >     [   10.346503]  __device_attach+0xaf/0x1b0
> >     [   10.346503]  bus_probe_device+0x8d/0xa0
> >     [   10.346503]  device_add+0x508/0x710
> >     [   10.346503]  platform_device_add+0xed/0x250
> >     [   10.346503]  sysfb_init+0x283/0x320
> >     [   10.346503]  ? __pfx_sysfb_init+0x10/0x10
> >     [   10.346503]  do_one_initcall+0x57/0x310
> >     [   10.346503]  do_initcalls+0x1ef/0x240
> >     [   10.346503]  kernel_init_freeable+0x187/0x210
> >     [   10.346503]  ? __pfx_kernel_init+0x10/0x10
> >     [   10.346503]  kernel_init+0x1a/0x140
> >     [   10.346503]  ret_from_fork+0xf2/0x110
> >     [   10.346503]  ? __pfx_kernel_init+0x10/0x10
> >     [   10.346503]  ret_from_fork_asm+0x1a/0x30
> >     [   10.346503]  </TASK>
> >     [   10.346503] Modules linked in:
> >     [   10.559786] ---[ end trace 0000000000000000 ]---
> >     [   10.564581] RIP: e030:vesafb_probe.cold+0xd4/0x5fb
> >     [   10.569546] Code: 08 75 1f 83 3d a8 8c 1d 02 00 75 16 48 c7 c7 90 4a fd 81 e8 8a ef f9 ff c7 05 d4 54 09 02 05 00 00 00 8b 05 c2 54 09 02 31 d2 <f7> 35 d2 54 09 02 8b 15 fc 54 09 02 48 89 c1 48 c1 e1 20 48 09 ca
> >     [   10.588833] RSP: e02b:ffffc9004001fbb8 EFLAGS: 00010246
> >     [   10.594255] RAX: 0000000000000000 RBX: ffff888101d86f28 RCX: ffffffff823666e8
> >     [   10.601622] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000003
> >     [   10.609000] RBP: ffff88810197d400 R08: 0000000000000000 R09: 6c656e696c202c30
> >     [   10.616378] R10: 0000000000000030 R11: 203a626661736576 R12: 0000000000000000
> >     [   10.623755] R13: 0000000000000000 R14: 0000000000000000 R15: ffff888106351000
> >     [   10.631136] FS:  0000000000000000(0000) GS:ffff888cef7c1000(0000) knlGS:0000000000000000
> >     [   10.639483] CS:  e030 DS: 0000 ES: 0000 CR0: 0000000080050033
> >     [   10.645434] CR2: ffffc900064ff000 CR3: 000000000222c000 CR4: 0000000000050660
> >     [   10.652817] Kernel panic - not syncing: Fatal exception
> >     [   10.653803] Kernel Offset: disabled
> >     (XEN) Hardware Dom0 crashed: 'noreboot' set - not rebooting.
> > 
> >>
> >>> It doesn't reach loading graphics driver in dom0, so I don't have
> >>> anything interesting on VGA (the last output is about the kexec call
> >>> done by Heads). But at least I have a serial console.
> >>
> >> Yet interestingly Xen's "  VGA is ... mode ..." lines differ as well.
> > 
> > This might be relevant given the above.
> 
> Indeed. It looks like d5a73cdc6b90 ("x86/boot: Use boot_vid_info variable
> directly from C code") is at fault, breaking this piece of pre-existing
> code at the bottom of mbi2_reloc():
> 
> #ifdef CONFIG_VIDEO
>     if ( video )
>         video->orig_video_isVGA = 0x23;
> #endif
> 
> Does the patch below help?

Yes, this helps, thanks!

> Jan
> 
> x86/boot: don't blindly mark VGA in graphics mode on MB2 path
> 
> Setting ->orig_video_isVGA to the specific marker should be done only when
> the VBE tag is present and the FRAMEBUFFER is either absent or indicates
> RGB type. Since the "video" variable now starts out non-NULL, this
> property was broken when in particular neither of the tags are present. To
> move back to at least close to original behavior, add a 2nd check to said
> conditional.
> 
> Fixes: d5a73cdc6b90 ("x86/boot: Use boot_vid_info variable directly from C code")
> Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Tested-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

> --- a/xen/arch/x86/boot/reloc.c
> +++ b/xen/arch/x86/boot/reloc.c
> @@ -339,7 +339,7 @@ static multiboot_info_t *mbi2_reloc(uint
>   end:
>  
>  #ifdef CONFIG_VIDEO
> -    if ( video )
> +    if ( video && video->lfb_size )
>          video->orig_video_isVGA = 0x23;
>  #endif
>  
> 

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]