[Qemu-devel] 128Kb bios size and older machines?

[Qemu-devel] 128Kb bios size and older machines?

[Michael Tokarev]

Since the "small" (128Kb) seabios is now built without
support of xen, does that mean that xen will be unable
to use qemu with -M 1.7 and before?  Does it _ever_ use
-M option like this?

I'm asking because I don't really understand how this works.
 We updated seabios in debian to 1.7.4 (with qemu 1.7), it
grew past 128Kb, and now I'm trying to understand what
exactly is broken and how to fix it.  I should either
build it w/o xen support for it to fit in 128Kb again,
or keep it at larger size and live with (migration) breakage
caused by different bios size.

BTW, recompiling seabios-1.7.4 without xchi and a few other
recently added stuff (pvscsi boot et al) with gcc-4.7.2 on
debian does not produce 128Kb binary, it is larger than 128k,
so I can't produce 128kb bios on debian anymore.. ;)

Thanks,

/mjt

Re: [Qemu-devel] 128Kb bios size and older machines?

[Gerd Hoffmann]

On Do, 2014-02-06 at 12:28 +0400, Michael Tokarev wrote:
> Since the "small" (128Kb) seabios is now built without
> support of xen, does that mean that xen will be unable
> to use qemu with -M 1.7 and before?  Does it _ever_ use
> -M option like this?

As far I know xen has its own seabios builds anyway as they
have to add hvmloader to the mix.  Thats why it has been
turned off in the upstream blob builds, nobody will use these
with xen anyway.

Not sure how all that is organized in the debian xen/seabios packages.

> I'm asking because I don't really understand how this works.
>  We updated seabios in debian to 1.7.4 (with qemu 1.7), it
> grew past 128Kb, and now I'm trying to understand what
> exactly is broken and how to fix it.  I should either
> build it w/o xen support for it to fit in 128Kb again,
> or keep it at larger size and live with (migration) breakage
> caused by different bios size.

Turn off xhci should be enough to make it fit into 128k again (used to
be the case for me last time I tried).

> BTW, recompiling seabios-1.7.4 without xchi and a few other
> recently added stuff (pvscsi boot et al) with gcc-4.7.2 on
> debian does not produce 128Kb binary, it is larger than 128k,
> so I can't produce 128kb bios on debian anymore.. ;)

Hmm.  I had the same problem with gcc 4.4 on rhel6, but gcc 4.7+ works
for me.  No idea, sorry.

cheers,
  Gerd

Re: [Qemu-devel] 128Kb bios size and older machines?

[Michael Tokarev]

06.02.2014 12:50, Gerd Hoffmann wrote:
> On Do, 2014-02-06 at 12:28 +0400, Michael Tokarev wrote:
>> Since the "small" (128Kb) seabios is now built without
>> support of xen, does that mean that xen will be unable
>> to use qemu with -M 1.7 and before?  Does it _ever_ use
>> -M option like this?
> 
> As far I know xen has its own seabios builds anyway as they
> have to add hvmloader to the mix.  Thats why it has been
> turned off in the upstream blob builds, nobody will use these
> with xen anyway.

Thank you Gerd for the answer.  Ian, can you give some light
here, what is still missing in seabios for xen?  Because,
well, ..

> Not sure how all that is organized in the debian xen/seabios packages.

.. debian uses upstream seabios directly with xen now, not
the one which comes from xen repository.

(And yes I know about
  http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=5f2875739beef3a75c7a7e8579b6cbcb464e61b3
which talks about >128kb bios size.  This is actually exactly
the same issue which I'm trying to address now, from a few PoVs:
qemu now builds/uses stripped-down bios for older (pre-2.0) machine
types, and that stripped-down version does not include Xen bits
in order to fit in 128Kb, because with xen it doesn't fit anymore.
Maybe we should start building special xen variant of seabios --
stripped down but WITH xen.  Oh well.)

Thanks,

/mjt

>> I'm asking because I don't really understand how this works.
>>  We updated seabios in debian to 1.7.4 (with qemu 1.7), it
>> grew past 128Kb, and now I'm trying to understand what
>> exactly is broken and how to fix it.  I should either
>> build it w/o xen support for it to fit in 128Kb again,
>> or keep it at larger size and live with (migration) breakage
>> caused by different bios size.
> 
> Turn off xhci should be enough to make it fit into 128k again (used to
> be the case for me last time I tried).
> 
>> BTW, recompiling seabios-1.7.4 without xchi and a few other
>> recently added stuff (pvscsi boot et al) with gcc-4.7.2 on
>> debian does not produce 128Kb binary, it is larger than 128k,
>> so I can't produce 128kb bios on debian anymore.. ;)
> 
> Hmm.  I had the same problem with gcc 4.4 on rhel6, but gcc 4.7+ works
> for me.  No idea, sorry.
> 
> cheers,
>   Gerd
> 
> 

Re: [Qemu-devel] 128Kb bios size and older machines?

[Ian Campbell]

On Thu, 2014-02-06 at 17:08 +0400, Michael Tokarev wrote:
> 06.02.2014 12:50, Gerd Hoffmann wrote:
> > On Do, 2014-02-06 at 12:28 +0400, Michael Tokarev wrote:
> >> Since the "small" (128Kb) seabios is now built without
> >> support of xen, does that mean that xen will be unable
> >> to use qemu with -M 1.7 and before?  Does it _ever_ use
> >> -M option like this?
> > 
> > As far I know xen has its own seabios builds anyway as they
> > have to add hvmloader to the mix.  Thats why it has been
> > turned off in the upstream blob builds, nobody will use these
> > with xen anyway.
> 
> Thank you Gerd for the answer.  Ian, can you give some light
> here, what is still missing in seabios for xen?  Because,
> well, ..

Nothing is missing in seabios.

The above patch is a fix to Xen not SeaBIOS, which lets it work
correctly regardless of the size of the seabios binary.

There is no need to do anything other than apply that fix to the Xen
packages (perhaps by upgrading to 4.3.1) AFAIK.

> > Not sure how all that is organized in the debian xen/seabios packages.
> 
> .. debian uses upstream seabios directly with xen now, not
> the one which comes from xen repository.
> 
> (And yes I know about
>   http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=5f2875739beef3a75c7a7e8579b6cbcb464e61b3
> which talks about >128kb bios size.  This is actually exactly
> the same issue which I'm trying to address now, from a few PoVs:
> qemu now builds/uses stripped-down bios for older (pre-2.0) machine
> types, and that stripped-down version does not include Xen bits
> in order to fit in 128Kb, because with xen it doesn't fit anymore.
> Maybe we should start building special xen variant of seabios --
> stripped down but WITH xen.  Oh well.)

Why is this stripped down SeaBIOS needed?

Is there some reason the Xen build can't pickup the non-stripped down
(post-2.0) SeaBIOS?

Ian.

Re: [Qemu-devel] 128Kb bios size and older machines?

[Michael Tokarev]

06.02.2014 17:15, Ian Campbell wrote:
> On Thu, 2014-02-06 at 17:08 +0400, Michael Tokarev wrote:
>> 06.02.2014 12:50, Gerd Hoffmann wrote:
[]
>>> As far I know xen has its own seabios builds anyway as they
>>> have to add hvmloader to the mix.  Thats why it has been
>>> turned off in the upstream blob builds, nobody will use these
>>> with xen anyway.
>>
>> Thank you Gerd for the answer.  Ian, can you give some light
>> here, what is still missing in seabios for xen?  Because,
>> well, ..
> 
> Nothing is missing in seabios.
> 
> The above patch is a fix to Xen not SeaBIOS, which lets it work
> correctly regardless of the size of the seabios binary.
> 
> There is no need to do anything other than apply that fix to the Xen
> packages (perhaps by upgrading to 4.3.1) AFAIK.

Okay.  That is good to know, because it started to become
quite confusing :)

Yes I understand the xen hvmloader change is necessary for
xen to support larger bios sizes, and new full-blown bios
is larger than 128Kb.

[]
>> (And yes I know about
>>   http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=5f2875739beef3a75c7a7e8579b6cbcb464e61b3
>> which talks about >128kb bios size.  This is actually exactly
>> the same issue which I'm trying to address now, from a few PoVs:
>> qemu now builds/uses stripped-down bios for older (pre-2.0) machine
>> types, and that stripped-down version does not include Xen bits
>> in order to fit in 128Kb, because with xen it doesn't fit anymore.
>> Maybe we should start building special xen variant of seabios --
>> stripped down but WITH xen.  Oh well.)
> 
> Why is this stripped down SeaBIOS needed?

Because it fits into 128Kb, while complete build doesn't.
And when the bios size grows (provided that software -
xen and qemu - actually support this), it breaks guest
migration between old and new systems.

> Is there some reason the Xen build can't pickup the non-stripped down
> (post-2.0) SeaBIOS?

The question is exactly about old systems.  Qemu has an option,
-M foo.  Right now, if foo is less than 2.0 (the upcoming release),
it picks up the old small and now stripped-down verion of seabios.
For new versions - like -M pc-2.0, or by default - it uses the
new larger bios with all features.

But I've no idea if xen actually uses or cares about -M to start
with.  Qemu supports it for 2 things: migration, and keeping, say,
windows guests activated across version changes (with variable
success on both fronts :)

Thanks,

/mjt

Re: [Qemu-devel] 128Kb bios size and older machines?

[Ian Campbell]

On Thu, 2014-02-06 at 17:30 +0400, Michael Tokarev wrote:
> 06.02.2014 17:15, Ian Campbell wrote:
> > On Thu, 2014-02-06 at 17:08 +0400, Michael Tokarev wrote:
> >> 06.02.2014 12:50, Gerd Hoffmann wrote:
> []
> >>> As far I know xen has its own seabios builds anyway as they
> >>> have to add hvmloader to the mix.  Thats why it has been
> >>> turned off in the upstream blob builds, nobody will use these
> >>> with xen anyway.
> >>
> >> Thank you Gerd for the answer.  Ian, can you give some light
> >> here, what is still missing in seabios for xen?  Because,
> >> well, ..
> > 
> > Nothing is missing in seabios.
> > 
> > The above patch is a fix to Xen not SeaBIOS, which lets it work
> > correctly regardless of the size of the seabios binary.
> > 
> > There is no need to do anything other than apply that fix to the Xen
> > packages (perhaps by upgrading to 4.3.1) AFAIK.
> 
> Okay.  That is good to know, because it started to become
> quite confusing :)
> 
> Yes I understand the xen hvmloader change is necessary for
> xen to support larger bios sizes, and new full-blown bios
> is larger than 128Kb.
> 
> []
> >> (And yes I know about
> >>   http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=5f2875739beef3a75c7a7e8579b6cbcb464e61b3
> >> which talks about >128kb bios size.  This is actually exactly
> >> the same issue which I'm trying to address now, from a few PoVs:
> >> qemu now builds/uses stripped-down bios for older (pre-2.0) machine
> >> types, and that stripped-down version does not include Xen bits
> >> in order to fit in 128Kb, because with xen it doesn't fit anymore.
> >> Maybe we should start building special xen variant of seabios --
> >> stripped down but WITH xen.  Oh well.)
> > 
> > Why is this stripped down SeaBIOS needed?
> 
> Because it fits into 128Kb, while complete build doesn't.
> And when the bios size grows (provided that software -
> xen and qemu - actually support this), it breaks guest
> migration between old and new systems.
> 
> > Is there some reason the Xen build can't pickup the non-stripped down
> > (post-2.0) SeaBIOS?
> 
> The question is exactly about old systems.  Qemu has an option,
> -M foo.  Right now, if foo is less than 2.0 (the upcoming release),
> it picks up the old small and now stripped-down verion of seabios.
> For new versions - like -M pc-2.0, or by default - it uses the
> new larger bios with all features.
> 
> But I've no idea if xen actually uses or cares about -M to start
> with.  Qemu supports it for 2 things: migration, and keeping, say,
> windows guests activated across version changes (with variable
> success on both fronts :)

Xen only supports migration from version N to N+1, and since this issue
is fixed it should be fine.

I'm not sure why the BIOS size is relevant to migration anyway, Xen
doesn't load a new BIOS on the target host, the BIOS image is part of
the migrated RAM.

Ian.

Re: [Qemu-devel] 128Kb bios size and older machines?

[Gerd Hoffmann]

  Hi,

> I'm not sure why the BIOS size is relevant to migration anyway, Xen
> doesn't load a new BIOS on the target host, the BIOS image is part of
> the migrated RAM.

The memory layout is different with a larger bios as qemu makes the
virtual flash as big as the bios image is.  The different memory layout
in turn makes live migration fail.

cheers,
  Gerd

Re: [Qemu-devel] 128Kb bios size and older machines?

[Ian Campbell]

On Thu, 2014-02-06 at 15:31 +0100, Gerd Hoffmann wrote:
>   Hi,
> 
> > I'm not sure why the BIOS size is relevant to migration anyway, Xen
> > doesn't load a new BIOS on the target host, the BIOS image is part of
> > the migrated RAM.
> 
> The memory layout is different with a larger bios as qemu makes the
> virtual flash as big as the bios image is.  The different memory layout
> in turn makes live migration fail.

OK. Xen doesn't emulate flash so it doesn't have this issue.

Ian.

Re: [Qemu-devel] 128Kb bios size and older machines?

[Paolo Bonzini]

Il 06/02/2014 15:48, Ian Campbell ha scritto:
>>> > > I'm not sure why the BIOS size is relevant to migration anyway, Xen
>>> > > doesn't load a new BIOS on the target host, the BIOS image is part of
>>> > > the migrated RAM.
>> >
>> > The memory layout is different with a larger bios as qemu makes the
>> > virtual flash as big as the bios image is.  The different memory layout
>> > in turn makes live migration fail.
> OK. Xen doesn't emulate flash so it doesn't have this issue.

It's problematic even without flash.  At least in Xen, all ROM memory 
blocks are migrated, not just the main RAM.  Migration of RAM/ROM block 
data is independent from where the RAM/ROM is mapped in guest physical 
address space.

In the guest, a 128k BIOS is at fffe0000-ffffffff.

A 256k BIOS is at fffc0000-ffffffff.

If the destination prepares the memory map for a 256k BIOS, and the 
source sends the data for a 128k BIOS, the 128k BIOS is erroneously 
placed at ffffc0000-fffdffff.  There is no reset vector fffffff0.  The 
guest works fine until you reboot it, then it breaks.

Paolo

Re: [Qemu-devel] 128Kb bios size and older machines?

[Paolo Bonzini]

Il 06/02/2014 14:15, Ian Campbell ha scritto:
> Nothing is missing in seabios.
>
> The above patch is a fix to Xen not SeaBIOS, which lets it work
> correctly regardless of the size of the seabios binary.
>
> There is no need to do anything other than apply that fix to the Xen
> packages (perhaps by upgrading to 4.3.1) AFAIK.

Yes, but do you use the Debian SeaBIOS build?  Wasn't SeaBIOS a blob 
inside the Xen hvmloader binary?

Paolo

Re: [Qemu-devel] 128Kb bios size and older machines?

[Michael Tokarev]

07.02.2014 03:56, Paolo Bonzini wrote:
> Il 06/02/2014 14:15, Ian Campbell ha scritto:
>> Nothing is missing in seabios.
>>
>> The above patch is a fix to Xen not SeaBIOS, which lets it work
>> correctly regardless of the size of the seabios binary.
>>
>> There is no need to do anything other than apply that fix to the Xen
>> packages (perhaps by upgrading to 4.3.1) AFAIK.
> 
> Yes, but do you use the Debian SeaBIOS build?  Wasn't SeaBIOS a blob inside the Xen hvmloader binary?

In the past debian had 3 copies of qemu too -- regular qemu package,
qemu-kvm and xen-qemu.  With all the blobs and whatnot.  Now,
*finally*, we moved to just one set.  Including seabios build.

That was a miracle, when finally it weren't necessary anymore
to carry all those copies and debug each one independently,
finding and fixing the same bugs... ;)

But for now I sorted out the seabios issue in debian, by finding
the right combination of compiler and seabios config options to
produce xen-enabled but small seabios binary.  This is very
fragile still (we use 99.7% of 128Kb), but it works for now.
It'd be sad if we'll have to build another variant of seabios,
but it isn't that difficult now anyway.  Maybe it wont really
be necessary in a near future when xen in debian will be able
to use larger bios (and we'll just switch it to bios-256k.bin,
building 128kb bios without xen).

Thanks,

/mjt