Re: debian stretch dom0 + xen 4.9 fails to boot

[Boris Ostrovsky <boris.ostrovsky@oracle.com>]

On 06/06/2017 12:28 PM, Paul Durrant wrote:
>> -----Original Message-----
>> From: Xen-devel [mailto:xen-devel-bounces@lists.xen.org] On Behalf Of
>> Paul Durrant
>> Sent: 06 June 2017 16:52
>> To: 'Jan Beulich' <JBeulich@suse.com>
>> Cc: xen-devel (xen-devel@lists.xenproject.org) <xen-
>> devel@lists.xenproject.org>
>> Subject: Re: [Xen-devel] debian stretch dom0 + xen 4.9 fails to boot
>>
>>> -----Original Message-----
>>> From: Jan Beulich [mailto:JBeulich@suse.com]
>>> Sent: 06 June 2017 16:11
>>> To: Paul Durrant <Paul.Durrant@citrix.com>
>>> Cc: xen-devel (xen-devel@lists.xenproject.org) <xen-
>>> devel@lists.xenproject.org>
>>> Subject: Re: [Xen-devel] debian stretch dom0 + xen 4.9 fails to boot
>>>
>>>>>> On 06.06.17 at 16:32, <Paul.Durrant@citrix.com> wrote:
>>>> I've been having fun setting up a new test rig...
>>>>
>>>> I have a skull canyon NUC and I put debian stretch (rc4) on it (so that's a
>>>> 4.9 kernel) and then tried building and installing the latest Xen staging-4.9
>>>> code. The system failed to boot... basically it got stuck before even
>>>> managing to get sufficiently into Xen to spit out anything on the console.
>>>> Xen 4.8 OTOH booted just fine so I started bisecting and after 14
>> iterations
>>>> I got down to the following commit is being the problem:
>>>>
>>>> commit c0655e492e6b33e26ec9cd33f59725d0db89cdd0
>>>> Author: Juergen Gross <jgross@suse.com>
>>>> Date:   Fri Mar 24 14:18:54 2017 +0100
>>>>
>>>>     x86: split boot trampoline into permanent and temporary part
>>>>
>>>>     The hypervisor needs a trampoline in low memory for early boot and
>>>>     later for bringing up cpus and during wakeup from suspend. Today this
>>>>     trampoline is kept completely even if most of it isn't needed later.
>>>>
>>>>     Split the trampoline into a permanent part and a temporary part
>> needed
>>>>     at early boot only. Introduce a new entry at the boundary.
>>>>
>>>>     Reduce the stack for wakeup code in order for the permanent
>>>>     trampoline to fit in a single page. 4k of stack seems excessive, about
>>>>     3k should be more than enough.
>>>>
>>>>     Add an ASSERT() to the linker script to ensure the wakeup stack is
>>>>     always at least 3k.
>>>>
>>>>     Signed-off-by: Juergen Gross <jgross@suse.com>
>>>>     Reviewed-by: Jan Beulich <jbeulich@suse.com>
>>>>
>>>> To verify this I checked out master, reverted that commit, and tried again.
>>>> The NUC still booted fine.
>>> Well, interesting, but I don't think it is very realistic to expect any
>>> fix with just the information you supply. There must be something
>>> rather special about that system, and likely it would help if we
>>> knew what that is. E.g. an unusual E820 map. Worse would be if
>>> they used memory outside of properly marked E820 regions in a
>>> way colliding with what we do.
>>>
>>> Otherwise I'm afraid we need to hope for you to debug the issue.
>>>
>> Yes, I was posting this more a heads-up for the moment, so that 4.9 does not
>> go out with this regression.
>>
>> I will try to figure out what is going on... My initial thoughts on looking at what
>> the patch does are that it may be something to do with the fact I am using a
>> vga console rather than a serial one. I need to try another 4.9 on another
>> system (gigabyte brix) to see if the problem manifests there too. I'll also have
>> to play with the BIOS settings on the skull canyon.
>>
> The problem definitely doesn't manifest on the brix, so the next theory is that it is something to do with the BIOS of the skull canyon.
>


FWIW, one of machines in our test farm choked on this very patch. I
don't remember details now but essentially it turned out that syslinux
(we are pxe-booting) could not handle changes in ELF sections layout
(the way syslinux calculated how to load the binary into memory resulted
in overlap of some sort).

I hacked it (mboot.c32 specifically) to work around this but never came
up with a proper solution.

-boris

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel