[Qemu-devel] Target vs architecture for QEMU binary

[Qemu-devel] Target vs architecture for QEMU binary

[Andrea Bolognani]

Hi,

at the moment, libvirt is using some ad-hoc logic to allow
i686 guests to run on qemu-system-x86_64 (by using the CPU
model qemu32); in all other cases, it's assumed that a $arch
guest needs qemu-system-$arch to run.

This is causing a problem right now with ppc64le guests
because, even though qemu-system-ppc64 is perfectly capable
of running them, libvirt will refuse to.

We want to change the logic so that it reflects the actual
capabilities of the QEMU binary, but AFAICT there isn't eg.
a QMP command we can use to query the binary for the list
of architectures it implements.

Am I missing something? Is such an interface available?

Failing that, we'll have to map QEMU targets with implemented
guest architectures inside libvirt, in which case it would be
great if you could point me towards either some up-to-date
documentation or a reliable way to extract the information
myself.

Thank you for your help.

-- 
Andrea Bolognani
Software Engineer - Virtualization Team

Re: [Qemu-devel] Target vs architecture for QEMU binary

[Daniel P. Berrange]

On Tue, Sep 08, 2015 at 03:27:38PM +0200, Andrea Bolognani wrote:
> Hi,
> 
> at the moment, libvirt is using some ad-hoc logic to allow
> i686 guests to run on qemu-system-x86_64 (by using the CPU
> model qemu32); in all other cases, it's assumed that a $arch
> guest needs qemu-system-$arch to run.
> 
> This is causing a problem right now with ppc64le guests
> because, even though qemu-system-ppc64 is perfectly capable
> of running them, libvirt will refuse to.

Is there a bug report somewhere for that, because libvirt
already has code in virQEMUCapsFindBinaryForArch() which
forces it to look at qemu-system-ppc64 when asked to use
ppc64le, so I'd expect it to already work.

> We want to change the logic so that it reflects the actual
> capabilities of the QEMU binary, but AFAICT there isn't eg.
> a QMP command we can use to query the binary for the list
> of architectures it implements.
> 
> Am I missing something? Is such an interface available?

We have a bit of a chicken and egg problem, because to query
QEMU for capabilities, you already have to know what system
emulator binary is required for the architeture you want to
run.

> Failing that, we'll have to map QEMU targets with implemented
> guest architectures inside libvirt, in which case it would be
> great if you could point me towards either some up-to-date
> documentation or a reliable way to extract the information
> myself.

The various rules in virQEMUCapsFindBinaryForArch() already
try todo a suitable mapping

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] Target vs architecture for QEMU binary

[Andrea Bolognani]

On Tue, 2015-09-08 at 15:37 +0100, Daniel P. Berrange wrote:
> > at the moment, libvirt is using some ad-hoc logic to allow
> > i686 guests to run on qemu-system-x86_64 (by using the CPU
> > model qemu32); in all other cases, it's assumed that a $arch
> > guest needs qemu-system-$arch to run.
> > 
> > This is causing a problem right now with ppc64le guests
> > because, even though qemu-system-ppc64 is perfectly capable
> > of running them, libvirt will refuse to.
> 
> Is there a bug report somewhere for that, because libvirt
> already has code in virQEMUCapsFindBinaryForArch() which
> forces it to look at qemu-system-ppc64 when asked to use
> ppc64le, so I'd expect it to already work.

You're right, starting a guest actually works.

The bug report[1] (which was filed by none other than myself :)
is still related to figuring out stuff starting from the guest
architecture, but it's apparently going through a different
code path where the adjustment you're referring to is not
applied[2].

> > We want to change the logic so that it reflects the actual
> > capabilities of the QEMU binary, but AFAICT there isn't eg.
> > a QMP command we can use to query the binary for the list
> > of architectures it implements.
> > 
> > Am I missing something? Is such an interface available?
> 
> We have a bit of a chicken and egg problem, because to query
> QEMU for capabilities, you already have to know what system
> emulator binary is required for the architeture you want to
> run.

Or we could just query everything that looks like a QEMU
binary and then lookup the correct one for the guest based
on the query results, couldn't we? Again, assuming such
interface even exists.

> > Failing that, we'll have to map QEMU targets with implemented
> > guest architectures inside libvirt, in which case it would be
> > great if you could point me towards either some up-to-date
> > documentation or a reliable way to extract the information
> > myself.
> 
> The various rules in virQEMUCapsFindBinaryForArch() already
> try todo a suitable mapping

I'm not sure they're covering all possible combinations,
though. Which is why it would be really nice to be able to
ask this stuff to QEMU itself.

Cheers.


[1] https://bugzilla.redhat.com/show_bug.cgi?id=1260753
[2] If I'm not mistaken, qemuCaps->arch is set in
    virQEMUCapsInitArchQMPBasic() and none of the logic
    you're talking about seems to be used there.
-- 
Andrea Bolognani
Software Engineer - Virtualization Team

Re: [Qemu-devel] Target vs architecture for QEMU binary

[Daniel P. Berrange]

On Tue, Sep 08, 2015 at 05:34:42PM +0200, Andrea Bolognani wrote:
> On Tue, 2015-09-08 at 15:37 +0100, Daniel P. Berrange wrote:
> > > at the moment, libvirt is using some ad-hoc logic to allow
> > > i686 guests to run on qemu-system-x86_64 (by using the CPU
> > > model qemu32); in all other cases, it's assumed that a $arch
> > > guest needs qemu-system-$arch to run.
> > > 
> > > This is causing a problem right now with ppc64le guests
> > > because, even though qemu-system-ppc64 is perfectly capable
> > > of running them, libvirt will refuse to.
> > 
> > Is there a bug report somewhere for that, because libvirt
> > already has code in virQEMUCapsFindBinaryForArch() which
> > forces it to look at qemu-system-ppc64 when asked to use
> > ppc64le, so I'd expect it to already work.
> 
> You're right, starting a guest actually works.
> 
> The bug report[1] (which was filed by none other than myself :)
> is still related to figuring out stuff starting from the guest
> architecture, but it's apparently going through a different
> code path where the adjustment you're referring to is not
> applied[2].

Ok, thanks I see the codepath now.

> > > We want to change the logic so that it reflects the actual
> > > capabilities of the QEMU binary, but AFAICT there isn't eg.
> > > a QMP command we can use to query the binary for the list
> > > of architectures it implements.
> > > 
> > > Am I missing something? Is such an interface available?
> > 
> > We have a bit of a chicken and egg problem, because to query
> > QEMU for capabilities, you already have to know what system
> > emulator binary is required for the architeture you want to
> > run.
> 
> Or we could just query everything that looks like a QEMU
> binary and then lookup the correct one for the guest based
> on the query results, couldn't we? Again, assuming such
> interface even exists.

I'd prefer libvirt to not have a trawl through every QEMU
binary to do this really.

> > > Failing that, we'll have to map QEMU targets with implemented
> > > guest architectures inside libvirt, in which case it would be
> > > great if you could point me towards either some up-to-date
> > > documentation or a reliable way to extract the information
> > > myself.
> > 
> > The various rules in virQEMUCapsFindBinaryForArch() already
> > try todo a suitable mapping
> 
> I'm not sure they're covering all possible combinations,
> though. Which is why it would be really nice to be able to
> ask this stuff to QEMU itself.

So, I think what we need do is to just refactor the
virQEMUCapsFindBinaryForArch(), to pull out the
architecture canonocalization out into a separate
method eg virArch virQEMUCapsCanonicalSystemArch(virArch)
and then just call it from both places

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] Target vs architecture for QEMU binary

[Andrea Bolognani]

On Tue, 2015-09-08 at 16:47 +0100, Daniel P. Berrange wrote:
> > Or we could just query everything that looks like a QEMU
> > binary and then lookup the correct one for the guest based
> > on the query results, couldn't we? Again, assuming such
> > interface even exists.
> 
> I'd prefer libvirt to not have a trawl through every QEMU
> binary to do this really.

AFAIK we're already querying every binary for other stuff
we're interested in, so adding one more query shouldn't
change anything. Or am I missing something?

> > I'm not sure they're covering all possible combinations,
> > though. Which is why it would be really nice to be able to
> > ask this stuff to QEMU itself.
> 
> So, I think what we need do is to just refactor the
> virQEMUCapsFindBinaryForArch(), to pull out the
> architecture canonocalization out into a separate
> method eg virArch virQEMUCapsCanonicalSystemArch(virArch)
> and then just call it from both places

Sounds reasonable. Are we sure we have a complete
understanding of the relationship between targets and
architectures, though? For example, I don't see anything
about s390, and the ARM stuff doesn't look like it covers
everything. I just want to make sure we're not doing
anything wrong or missing any possible combination.

Cheers.

-- 
Andrea Bolognani
Software Engineer - Virtualization Team

Re: [Qemu-devel] Target vs architecture for QEMU binary

[Daniel P. Berrange]

On Wed, Sep 09, 2015 at 10:17:34AM +0200, Andrea Bolognani wrote:
> On Tue, 2015-09-08 at 16:47 +0100, Daniel P. Berrange wrote:
> > > Or we could just query everything that looks like a QEMU
> > > binary and then lookup the correct one for the guest based
> > > on the query results, couldn't we? Again, assuming such
> > > interface even exists.
> > 
> > I'd prefer libvirt to not have a trawl through every QEMU
> > binary to do this really.
> 
> AFAIK we're already querying every binary for other stuff
> we're interested in, so adding one more query shouldn't
> change anything. Or am I missing something?

In all the other cases we already know which binary we need
to query. We happen to cache the results of querying binaries
we find in $PATH, but none of our code has a fixed assumption
that we have caps available for every single binary. I don't
want such an assumption to get baked into libvirt code,
because it will limit our flexibility to change the way we
probe / cache capabilities data later, if we have a requirement
to always query every binary upfront.

> > > I'm not sure they're covering all possible combinations,
> > > though. Which is why it would be really nice to be able to
> > > ask this stuff to QEMU itself.
> > 
> > So, I think what we need do is to just refactor the
> > virQEMUCapsFindBinaryForArch(), to pull out the
> > architecture canonocalization out into a separate
> > method eg virArch virQEMUCapsCanonicalSystemArch(virArch)
> > and then just call it from both places
> 
> Sounds reasonable. Are we sure we have a complete
> understanding of the relationship between targets and
> architectures, though? For example, I don't see anything
> about s390, and the ARM stuff doesn't look like it covers
> everything. I just want to make sure we're not doing
> anything wrong or missing any possible combination.

Yeah, we've just added exceptions on an as-nedeed basis, so we could
well need more special cases added


Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|