upstream PowerPC qemu breakage

upstream PowerPC qemu breakage

[Hollis Blanchard]

Hi Avi, we're having a problem with the qemu merge you just did in
kvm-userspace.

Upstream qemu recently added the TCG code generator to phase out dyngen.
When he did that, Fabrice explicitly broke the build every non-x86
architecture, and since you've now pulled that breakage into KVM, we're
stuck in an awkward situation.

In the short term we'll have to fork a working userspace, since we're in
the middle of some other stuff (such as real guest IO, which I think is
pretty important :) .

Long term, one option is to try to define a new qemu target that
completely bypasses the code generation parts of qemu. Anthony did that
for x86 once, but there are at least a couple sticking points; not sure
how long it will take. This is probably the best long-term way to avoid
this situation in the future.

Another long-term option is to fix TCG for PowerPC upstream, and I'm
afraid that isn't feasible.

I guess merging with qemu while it's in a period of massive change
wasn't the most opportune moment. Were there some device model changes
you were eager to pick up?

-- 
Hollis Blanchard
IBM Linux Technology Center


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: upstream PowerPC qemu breakage

[Anthony Liguori]

Hollis Blanchard wrote:
> Hi Avi, we're having a problem with the qemu merge you just did in
> kvm-userspace.
>
> Upstream qemu recently added the TCG code generator to phase out dyngen.
> When he did that, Fabrice explicitly broke the build every non-x86
> architecture, and since you've now pulled that breakage into KVM, we're
> stuck in an awkward situation.
>   

This sucks.  This seems like a big step backwards for QEMU since it 
doesn't appear that there is an obvious way to fix things for non-x86 hosts.

> In the short term we'll have to fork a working userspace, since we're in
> the middle of some other stuff (such as real guest IO, which I think is
> pretty important :) .
>
> Long term, one option is to try to define a new qemu target that
> completely bypasses the code generation parts of qemu. Anthony did that
> for x86 once, but there are at least a couple sticking points; not sure
> how long it will take. This is probably the best long-term way to avoid
> this situation in the future.
>   

This is very easy to do and is probably the best long term and short 
term solution.  If you introduce a new target type (ppcemb-kvm) and drop 
the TCG/dyngen bits from the build for it, then you should be okay.  It 
will require a small stub file but there's not more than a dozen or so 
functions required for that.

I think this would be generally useful for other architectures too (like 
ia64, s390, and even x86).  At least ia64 and s390 aren't going to have 
functioning translation bits so having a -kvm target really makes a lot 
more sense than faking out a -softmmu target.

Regards,

Anthony Liguori

> Another long-term option is to fix TCG for PowerPC upstream, and I'm
> afraid that isn't feasible.
>
> I guess merging with qemu while it's in a period of massive change
> wasn't the most opportune moment. Were there some device model changes
> you were eager to pick up?
>
>   


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: upstream PowerPC qemu breakage

[Avi Kivity]

Anthony Liguori wrote:
>
>> In the short term we'll have to fork a working userspace, since we're in
>> the middle of some other stuff (such as real guest IO, which I think is
>> pretty important :) .
>>
>> Long term, one option is to try to define a new qemu target that
>> completely bypasses the code generation parts of qemu. Anthony did that
>> for x86 once, but there are at least a couple sticking points; not sure
>> how long it will take. This is probably the best long-term way to avoid
>> this situation in the future.
>>   
>
> This is very easy to do and is probably the best long term and short 
> term solution.  If you introduce a new target type (ppcemb-kvm) and 
> drop the TCG/dyngen bits from the build for it, then you should be 
> okay.  It will require a small stub file but there's not more than a 
> dozen or so functions required for that.
>
> I think this would be generally useful for other architectures too 
> (like ia64, s390, and even x86).  At least ia64 and s390 aren't going 
> to have functioning translation bits so having a -kvm target really 
> makes a lot more sense than faking out a -softmmu target.
>

I don't think this is the right fix.  A machine type (if I understand it 
correctly) refers to a combination of a target processor, 
chipset/busses, and devices, not to how they are implemented in qemu.

I believe a better fix is to introduce an orthogonal 
--without-cpu-emulation.

-- 
Any sufficiently difficult bug is indistinguishable from a feature.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: upstream PowerPC qemu breakage

[Avi Kivity]

Hollis Blanchard wrote:
> Hi Avi, we're having a problem with the qemu merge you just did in
> kvm-userspace.
>
> Upstream qemu recently added the TCG code generator to phase out dyngen.
> When he did that, Fabrice explicitly broke the build every non-x86
> architecture, and since you've now pulled that breakage into KVM, we're
> stuck in an awkward situation.
>
> In the short term we'll have to fork a working userspace, since we're in
> the middle of some other stuff (such as real guest IO, which I think is
> pretty important :) .
>   

I meant to drop Xiantao and you a note about this, but qemu merges tend 
to erase short-term memory.  I figured that since tcg is not used when 
using kvm, you could just stub it out.  The downside is that -no-kvm 
breaks, but we can live with that.


> Long term, one option is to try to define a new qemu target that
> completely bypasses the code generation parts of qemu. Anthony did that
> for x86 once, but there are at least a couple sticking points; not sure
> how long it will take. This is probably the best long-term way to avoid
> this situation in the future.
>   

It kills -no-kvm, which is a powerful debugging aid.

> Another long-term option is to fix TCG for PowerPC upstream, and I'm
> afraid that isn't feasible.
>   

I saw some talk that dyngen and tcg can coexist; but apparently that's 
not the case.  Hopefully qemu upstream will unbreak the damage.

> I guess merging with qemu while it's in a period of massive change
> wasn't the most opportune moment. Were there some device model changes
> you were eager to pick up?
>   

The e1000 patch for one; also doing regular small merges is much easier 
than irregular large ones.

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: upstream PowerPC qemu breakage

[Hollis Blanchard]

On Tue, 2008-02-12 at 12:45 +0200, Avi Kivity wrote:
> Hollis Blanchard wrote:
> > Long term, one option is to try to define a new qemu target that
> > completely bypasses the code generation parts of qemu. Anthony did that
> > for x86 once, but there are at least a couple sticking points; not sure
> > how long it will take. This is probably the best long-term way to avoid
> > this situation in the future.
> 
> It kills -no-kvm, which is a powerful debugging aid.

Build failures kill a lot more functionality than -no-kvm.

Beyond the immediate issue, there is also the question of carrying the
memory footprint for a bunch of functionality that we aren't using. I
guess it could increase exposure security issues too. Generally, I don't
see that it makes sense to build a bunch of code we don't use,
especially if your only merge criterion is "x86 works"...

(By the way, upstream qemu doesn't even support 440 or IA64 instruction
emulation right now, so -no-kvm is worthless to us anyways.)

> > Another long-term option is to fix TCG for PowerPC upstream, and I'm
> > afraid that isn't feasible.
> 
> I saw some talk that dyngen and tcg can coexist; but apparently that's 
> not the case.

I have no reason to believe that's not true, in theory. In practice,
we're broken right now...

> Hopefully qemu upstream will unbreak the damage.

What do you suggest, waiting until they fix it?

-- 
Hollis Blanchard
IBM Linux Technology Center


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: upstream PowerPC qemu breakage

[Avi Kivity]

Hollis Blanchard wrote:
> On Tue, 2008-02-12 at 12:45 +0200, Avi Kivity wrote:
>   
>> Hollis Blanchard wrote:
>>     
>>> Long term, one option is to try to define a new qemu target that
>>> completely bypasses the code generation parts of qemu. Anthony did that
>>> for x86 once, but there are at least a couple sticking points; not sure
>>> how long it will take. This is probably the best long-term way to avoid
>>> this situation in the future.
>>>       
>> It kills -no-kvm, which is a powerful debugging aid.
>>     
>
> Build failures kill a lot more functionality than -no-kvm.
>
>   

I am not advocating that as a useful feature.

> Beyond the immediate issue, there is also the question of carrying the
> memory footprint for a bunch of functionality that we aren't using. I
> guess it could increase exposure security issues too. Generally, I don't
> see that it makes sense to build a bunch of code we don't use,
>   

I think the fix for that is a compile time option to disable emulation.  
Just like you can disable kvm and kqemu support at compile time.

> especially if your only merge criterion is "x86 works"...
>   

I'll try to set up F8 ppc on a qemu instance, in order to reduce 
breakage in the future.  As it is,  many libkvm and qemu-kvm.c changes 
will break the build.

It'll need to be built against your kernel tree; please provide a URL.

>   
>> Hopefully qemu upstream will unbreak the damage.
>>     
>
> What do you suggest, waiting until they fix it?
>
>   

No.  Stubbing out tcg-target.h as a band-aid, and 
--without-cpu-emulation ./configure switch as a long term fix (which you 
want anyway).

-- 
Any sufficiently difficult bug is indistinguishable from a feature.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [kvm-ppc-devel] upstream PowerPC qemu breakage

[Hollis Blanchard]

On Wed, 2008-02-13 at 08:58 +0200, Avi Kivity wrote:
> It'll need to be built against your kernel tree; please provide a URL.

curl http://penguinppc.org/~hollisb/kvm/kvm-powerpc.mbox | git-am

-- 
Hollis Blanchard
IBM Linux Technology Center


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [kvm-ppc-devel] upstream PowerPC qemu breakage

[Avi Kivity]

Hollis Blanchard wrote:
> On Wed, 2008-02-13 at 08:58 +0200, Avi Kivity wrote:
>   
>> It'll need to be built against your kernel tree; please provide a URL.
>>     
>
> curl http://penguinppc.org/~hollisb/kvm/kvm-powerpc.mbox | git-am
>
>   

Unfortunately I wasn't able to get an F8 ppc rescue cd ISO to boot with 
qemu 0.9.0.  Can you point me to a working combination?

-- 
Any sufficiently difficult bug is indistinguishable from a feature.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [kvm-ppc-devel] upstream PowerPC qemu breakage

[Hollis Blanchard]

On Sat, 2008-02-16 at 10:47 +0200, Avi Kivity wrote:
> Hollis Blanchard wrote:
> > On Wed, 2008-02-13 at 08:58 +0200, Avi Kivity wrote:
> >   
> >> It'll need to be built against your kernel tree; please provide a URL.
> >>     
> >
> > curl http://penguinppc.org/~hollisb/kvm/kvm-powerpc.mbox | git-am  
> 
> Unfortunately I wasn't able to get an F8 ppc rescue cd ISO to boot with 
> qemu 0.9.0.  Can you point me to a working combination?

It's difficult to get anything booting with upstream PowerPC qemu,
mostly because of the unmaintained firmware they use (called Open
Hackware).

That said, upstream qemu does not support 440 cores, which is what our
KVM work is targeting. Also, Fedora 8 doesn't either, though it may be
possible to get F8 working by providing your own kernel and some small
configuration tweaks (inittab, securetty).

There are other distributions that will work with little to no tweaking
for 440, but until we can get IO (other than console) working we have
been using very simple root filesystems.

However, none of these will be useful to you in KVM unless you have a
440 host to run them on.

Originally you said you just need a kernel tree to build against. Can
you elaborate on what you're trying to do now?

-- 
Hollis Blanchard
IBM Linux Technology Center


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [kvm-ppc-devel] upstream PowerPC qemu breakage

[Avi Kivity]

Hollis Blanchard wrote:
>> Unfortunately I wasn't able to get an F8 ppc rescue cd ISO to boot with 
>> qemu 0.9.0.  Can you point me to a working combination?
>>     
>
> It's difficult to get anything booting with upstream PowerPC qemu,
> mostly because of the unmaintained firmware they use (called Open
> Hackware).
>
> That said, upstream qemu does not support 440 cores, which is what our
> KVM work is targeting. Also, Fedora 8 doesn't either, though it may be
> possible to get F8 working by providing your own kernel and some small
> configuration tweaks (inittab, securetty).
>
> There are other distributions that will work with little to no tweaking
> for 440, but until we can get IO (other than console) working we have
> been using very simple root filesystems.
>
> However, none of these will be useful to you in KVM unless you have a
> 440 host to run them on.
>
> Originally you said you just need a kernel tree to build against. Can
> you elaborate on what you're trying to do now?
>   

I'd like to have a full setup so I can do run testing as well.

I was thinking of running a ppc distro on qemu, and building and running 
kvm-ppc on that to test.  Even if  I don't run it, it's a much better 
build-time environment.  Of course running the unit tests and an image 
or two will be even better.

So, at best, I'd like an emulation environment that allows me to build 
and run; at worst, build only.

-- 
Any sufficiently difficult bug is indistinguishable from a feature.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [kvm-ppc-devel] upstream PowerPC qemu breakage

[Hollis Blanchard]

On Mon, 2008-02-18 at 22:22 +0200, Avi Kivity wrote:
> Hollis Blanchard wrote:
> >> Unfortunately I wasn't able to get an F8 ppc rescue cd ISO to boot with 
> >> qemu 0.9.0.  Can you point me to a working combination?
> >>     
> >
> > It's difficult to get anything booting with upstream PowerPC qemu,
> > mostly because of the unmaintained firmware they use (called Open
> > Hackware).
> >
> > That said, upstream qemu does not support 440 cores, which is what our
> > KVM work is targeting. Also, Fedora 8 doesn't either, though it may be
> > possible to get F8 working by providing your own kernel and some small
> > configuration tweaks (inittab, securetty).
> >
> > There are other distributions that will work with little to no tweaking
> > for 440, but until we can get IO (other than console) working we have
> > been using very simple root filesystems.
> >
> > However, none of these will be useful to you in KVM unless you have a
> > 440 host to run them on.
> >
> > Originally you said you just need a kernel tree to build against. Can
> > you elaborate on what you're trying to do now?
> >   
> 
> I'd like to have a full setup so I can do run testing as well.

I'd love to help make this happen.

> I was thinking of running a ppc distro on qemu, and building and running 
> kvm-ppc on that to test.

This isn't going to work in the near future, because KVM today requires
a 440 host, and qemu today doesn't emulate 440 instructions.

> Even if  I don't run it, it's a much better 
> build-time environment.  Of course running the unit tests and an image 
> or two will be even better.
> 
> So, at best, I'd like an emulation environment that allows me to build 
> and run; at worst, build only.

Building would be better done with a cross-compiler than inside qemu.

-- 
Hollis Blanchard
IBM Linux Technology Center


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/