Re: Regression between Xen 4.6.0 and 4.7.0, Direct kernel boot on a qemu-xen and seabios HVM guest doesn't work anymore.

[Wei Liu <wei.liu2@citrix.com>]

On Mon, Sep 05, 2016 at 11:20:30AM +0200, linux@eikelenboom.it wrote:
> On 2016-08-25 23:18, linux@eikelenboom.it wrote:
> >On 2016-08-25 22:34, Doug Goldstein wrote:
> >>On 8/25/16 4:21 PM, linux@eikelenboom.it wrote:
> >>>Today i tried to switch some of my HVM guests (qemu-xen) from booting
> >>>of
> >>>a kernel *inside* the guest, to a dom0 supplied kernel, which is
> >>>described as "Direct Kernel Boot" here:
> >>>https://xenbits.xen.org/docs/unstable/man/xl.cfg.5.html :
> >>>
> >>>    Direct Kernel Boot
> >>>
> >>>    Direct kernel boot allows booting directly from a kernel and
> >>>initrd
> >>>stored in the host physical
> >>>    machine OS, allowing command line arguments to be passed directly.
> >>>PV guest direct kernel boot
> >>>    is supported. HVM guest direct kernel boot is supported with
> >>>limitation (it's supported when
> >>>    using qemu-xen and default BIOS 'seabios'; not supported in case
> >>>of
> >>>stubdom-dm and old rombios.)
> >>>
> >>>    kernel="PATHNAME"    Load the specified file as the kernel image.
> >>>    ramdisk="PATHNAME"   Load the specified file as the ramdisk.
> >>>
> >>>But qemu fails to start, output appended below.
> >>>
> >>>I tested with:
> >>>- current Xen-unstable, which fails.
> >>>- xen-stable-4.7.0 release, which fails.
> >>>- xen-stable-4.6.0 release, works fine.
> >>
> >>Can you include the logs from xl dmesg around that time frame as well?
> >
> >Ah i thought there wasn't any, but didn't check thoroughly or wasn't there
> >since the release builds are non-debug by default.
> >
> >However, back on xen-unstable:
> >(XEN) [2016-08-25 21:09:15.172] HVM19 save: CPU
> >(XEN) [2016-08-25 21:09:15.172] HVM19 save: PIC
> >(XEN) [2016-08-25 21:09:15.172] HVM19 save: IOAPIC
> >(XEN) [2016-08-25 21:09:15.172] HVM19 save: LAPIC
> >(XEN) [2016-08-25 21:09:15.172] HVM19 save: LAPIC_REGS
> >(XEN) [2016-08-25 21:09:15.172] HVM19 save: PCI_IRQ
> >(XEN) [2016-08-25 21:09:15.172] HVM19 save: ISA_IRQ
> >(XEN) [2016-08-25 21:09:15.172] HVM19 save: PCI_LINK
> >(XEN) [2016-08-25 21:09:15.172] HVM19 save: PIT
> >(XEN) [2016-08-25 21:09:15.172] HVM19 save: RTC
> >(XEN) [2016-08-25 21:09:15.172] HVM19 save: HPET
> >(XEN) [2016-08-25 21:09:15.172] HVM19 save: PMTIMER
> >(XEN) [2016-08-25 21:09:15.172] HVM19 save: MTRR
> >(XEN) [2016-08-25 21:09:15.172] HVM19 save: VIRIDIAN_DOMAIN
> >(XEN) [2016-08-25 21:09:15.172] HVM19 save: CPU_XSAVE
> >(XEN) [2016-08-25 21:09:15.172] HVM19 save: VIRIDIAN_VCPU
> >(XEN) [2016-08-25 21:09:15.172] HVM19 save: VMCE_VCPU
> >(XEN) [2016-08-25 21:09:15.172] HVM19 save: TSC_ADJUST
> >(XEN) [2016-08-25 21:09:15.172] HVM19 restore: CPU 0
> >(XEN) [2016-08-25 21:09:16.126] d0v1 Over-allocation for domain 19:
> >262401 > 262400
> >(XEN) [2016-08-25 21:09:16.126] memory.c:213:d0v1 Could not allocate
> >order=0 extent: id=19 memflags=0 (192 of 512)
> >
> >Hmm some off by one issue ?
> >
> >
> >>Just wondering how much RAM you're domain is defined with as well?
> >
> >1024 Mb, there is more than enough unallocated memory for xen to start
> >the guest (and dom0 is fixed with dom0_mem=1536M,max:1536M and
> >ballooning is off)
> 
> 
> Hmm it seems my thread was kind of hijacked and i was dropped from the CC.
> 

Oops, I thought you were CC'ed. Sorry.

> I had some time and bisected the issue and it resulted in:
> 
> 5a3ce8f85e7e7bdd339d259daa19f6bc5cb4735f is the first bad commit
> commit 5a3ce8f85e7e7bdd339d259daa19f6bc5cb4735f
> Author: Jan Beulich <jbeulich@suse.com>
> Date:   Wed Oct 21 10:56:31 2015 +0200
> 
>     x86/shadow: drop stray name tags from sh_{guest_get,map}_eff_l1e()
> 
>     They (as a now being removed comment validly says) depend only on Xen's
>     number of page table levels, and hence their tags didn't serve any
>     useful purpose (there could only ever be one instance in a single
>     binary, even back in the x86-32 days).
> 
>     Further conditionalize the inclusion of PV-specific hook pointers, at
>     once making sure that PV guests can't ever get other than 4-level mode
>     enabled for them.
> 
>     For consistency reasons shadow_{write,cmpxchg}_guest_entry() also get
>     moved next to the other PV-only actors, allowing them to become static
>     just like the $subject ones do.
> 
>     Signed-off-by: Jan Beulich <jbeulich@suse.com>
>     Acked-by: Tim Deegan <tim@xen.org>
> 
> :040000 040000 0c2e3475f81547f934a5960d9f1ac4849707d4ed
> f17f5ff17ca50d6ab908afe9a2d8555d954d3d0a M  xen
> 

Unfortunately I can't see immediately why this would affect QEMU direct
boot. It also suggests that it only affects shadow code -- what kind of
hardware are you using?

Wei.

> 
> --
> Sander
> 
> 
> >
> >--
> >Sander

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel