Re: HVM domains crash after upgrade from XEN 4.5.1 to 4.5.2

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Mon, Nov 16, 2015 at 01:39:08PM -0600, Doug Goldstein wrote:
> On 11/16/15 1:25 PM, Konrad Rzeszutek Wilk wrote:
> > On Mon, Nov 16, 2015 at 08:16:33PM +0100, Atom2 wrote:
> >>
> >>
> >> Am 16.11.15 um 16:31 schrieb Konrad Rzeszutek Wilk:
> >>>>>> Your analysis was absolutely spot on. After re-thinking this for a
> >>>>>> moment, I thought going down that route first would make a lot of sense
> >>>>>> as PV guests still do work and one of the differences to HVM domUs is
> >>>>>> that the former do _not_ require SeaBIOS. Looking at my log files of
> >>>>>> installed packages confirmed an upgrade from SeaBIOS 1.7.5 to 1.8.2 in
> >>>>>> the relevant timeframe which obviously had not made it to the hvmloader
> >>>>>> of xen-4.5.1 as I did not re-compile xen after the upgrade of SeaBIOS.
> >>>>>>
> >>>>>> So I re-compiled xen-4.5.1 (obviously now using the installed SeaBIOS
> >>>>>> 1.8.2) and the same error as with xen-4.5.2 popped up - and that seemed
> >>>>>> to strongly indicate that there indeed might be an issue with SeaBIOS as
> >>>>>> this probably was the only variable that had changed from the original
> >>>>>> install of xen-4.5.1.
> >>> I recall seeing this way back in Fedora 20 days. I narrowed it down the
> >>> SeaBIOS version that was a standalone package to not have CONFIG_XEN.
> >>>
> >>> Having that fixed in the SeaBIOS package fixed it.
> >> Hi Konrad, Doug, Andrew (specifically added to this part of the thread)!
> >> Konrad, you might have found an interesting point. I did have a look at the
> >> ebuild for the failing version and in there I found the following comment:
> >> ====== comment from ebuild =======
> >>     # Upstream hasn't released a new binary.  We snipe ours from Fedora for
> >> now.
> >>     # http://code.coreboot.org/p/seabios/downloads/get/bios.bin-${PV}.gz
> >> ====== end comment from ebuild =======
> >> which might in fact underline that there might be an issue similar to what
> >> you described above.
> >>
> >> What is also pretty interesting is the fact that the old (working) SeaBIOS
> >> version 1.7.5 installed as "bios.bin" under /usr/share/seabios is actually
> >> 262.144 bytes in size whereas the new (invalid) SeaBIOS 1.8.2 installed in
> >> the same location is only half as big: 131.072 bytes.
> >>
> >> I checked at the download site and the 1.8.2 binary version is indeed not
> >> available from http://code.coreboot.org/p/seabios/downloads/. But both the
> >> binary versions for 1.7.5 and 1.8.0 are available and both are acutually
> >> 262.144 bytes in size, so I'd be very surprised if the 1.8.2 version is
> >> really only half that size. By the way, the old working version (according
> >> to the ebuild) was directly downloaded from the above url and also shows an
> >> identical SHA1 digest to that version available for download there.
> > 
> > <blinks>I thought Gentoo was all about rebuilding from source and not
> > taking binary blobs.
> 
> So since SeaBIOS and friends (the blobs) are so sensitive to compilers
> and environments and to avoid as much problems for people as possible. I
> setup the ebuilds in Gentoo to grab the binary blobs by default and if
> the user disabled the binary option it would build from source. The idea
> was that Fedora doesn't ship any blobs that can't be rebuilt so I would
> follow so the same approach and even use their built blobs. But it
> appears to be that there are definitely differences between what QEMU
> needs/uses and what upstream ships.
> 
> Its pretty common for saying Gentoo is about building from source but
> really its about user choice.

Aaah! Thank you for educating me!
> 
> That said I haven't been maintaining these for some time now and I've
> looked at the state of the way that SeaBIOS and friends are built and I
> believe there's an issue and I intend on remedying things soon to avoid
> issues like this.

Woot!
> 
> >>
> >> To me this looks as if something is really wrong here. If anybody of you has
> >> access to a 1.8.2 version, could you please confirm whether there's really
> >> that big a size difference between the 1.7.5 and the 1.8.2 versions? Or is
> >> that difference probably attributable to the missing CONFIG_XEN option?
> > 
> > It may be other options too - like CONFIG_XHCI, or a huge amount of other
> > ones.
> 
> Yes. There's definitely differences.
> 
> >>
> >> Andrew: I havent't gotten around to run the debug version of the hypervisor
> >> again, but if the current suspicion turns out to be true, there's probably
> >> not much value in that anyways. Would you agree?
> > 
> > I am not Andrew and can't really speak for him, but I am going to take a 
> > leap here and say he will agree with you.
> > 
> 
> 
> -- 
> Doug Goldstein
>