Integrate kqemu emulation into xen hypervisor possible?

Integrate kqemu emulation into xen hypervisor possible?

[Ross S. W. Walker]

I was wondering if now that the kqemu source has been GPL'd if it was
possible if it's functionality could be incorporated into the Xen
hypervisor to provide full virtualization on hardware that doesn't
support it?

Call it FVM instead of HVM.

-Ross

______________________________________________________________________
This e-mail, and any attachments thereto, is intended only for use by
the addressee(s) named herein and may contain legally privileged
and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination,
distribution or copying of this e-mail, and any attachments thereto,
is strictly prohibited. If you have received this e-mail in error,
please immediately notify the sender and permanently delete the
original and any copy or printout thereof.

Re: Integrate kqemu emulation into xen hypervisor possible?

[Samuel Thibault]

Ross S. W. Walker, le Tue 01 Apr 2008 11:19:35 -0400, a écrit :
> I was wondering if now that the kqemu source has been GPL'd if it was
> possible if it's functionality could be incorporated into the Xen
> hypervisor to provide full virtualization on hardware that doesn't
> support it?

What would be the benefit, compared to just running qemu in user space?

Samuel

RE: Integrate kqemu emulation into xen hypervisor possible?

[Ross S. W. Walker]

Samuel Thibault wrote:
> Ross S. W. Walker, le Tue 01 Apr 2008 11:19:35 -0400, a écrit :
> > I was wondering if now that the kqemu source has been GPL'd if it was
> > possible if it's functionality could be incorporated into the Xen
> > hypervisor to provide full virtualization on hardware that doesn't
> > support it?
> 
> What would be the benefit, compared to just running qemu in 
> user space?

A higher performance of the VM as opposed to running it strictly
in full emulation mode in user space.

Here is the technical page of the project:

http://fabrice.bellard.free.fr/qemu/kqemu-tech.html

Maybe it just can't be done, I don't know the internal architecture
of the hypervisor to make that call.

At a minimum how about putting back in the full emulation features
of qemu so one can run and manage a fully emulated VM in Xen
without having to make any changes to the system. Then maybe
some brilliant mind can figure a way to make it run better.


-Ross

______________________________________________________________________
This e-mail, and any attachments thereto, is intended only for use by
the addressee(s) named herein and may contain legally privileged
and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination,
distribution or copying of this e-mail, and any attachments thereto,
is strictly prohibited. If you have received this e-mail in error,
please immediately notify the sender and permanently delete the
original and any copy or printout thereof.

Re: Integrate kqemu emulation into xen hypervisor possible?

[Daniel P. Berrange]

On Tue, Apr 01, 2008 at 11:19:35AM -0400, Ross S. W. Walker wrote:
> 
> I was wondering if now that the kqemu source has been GPL'd if it was
> possible if it's functionality could be incorporated into the Xen
> hypervisor to provide full virtualization on hardware that doesn't
> support it?

Nice april fools joke.

Dan.
-- 
|: Red Hat, Engineering, Boston   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: Integrate kqemu emulation into xen hypervisor possible?

[Samuel Thibault]

Ross S. W. Walker, le Tue 01 Apr 2008 11:41:24 -0400, a écrit :
> Samuel Thibault wrote:
> > Ross S. W. Walker, le Tue 01 Apr 2008 11:19:35 -0400, a écrit :
> > > I was wondering if now that the kqemu source has been GPL'd if it was
> > > possible if it's functionality could be incorporated into the Xen
> > > hypervisor to provide full virtualization on hardware that doesn't
> > > support it?
> > 
> > What would be the benefit, compared to just running qemu in 
> > user space?

Oooh, oops, you are talking about *k*qemu, not qemu, sorry, ok, I see.
Well, a first step would be to make kqemu work with Xen's dom0 Linux.
Then, adding the feature in the Hypervisor may make sense indeed, but
I'm not sure you'd get much bigger performance than with just the dom0
support.

Samuel

RE: Integrate kqemu emulation into xen hypervisor possible?

[Ross S. W. Walker]

Daniel P. Berrange wrote:
> On Tue, Apr 01, 2008 at 11:19:35AM -0400, Ross S. W. Walker wrote:
> > 
> > I was wondering if now that the kqemu source has been GPL'd if it was
> > possible if it's functionality could be incorporated into the Xen
> > hypervisor to provide full virtualization on hardware that doesn't
> > support it?
> 
> Nice april fools joke.

Lol, no I wasn't joking, maybe naive, but not joking.

No I was trying to see if there was any interest if adding support
for full virtualization in Xen to allow non-xenified domains to
run on systems that have no hardware virtualization support. Maybe
slower then hardware virtualization, but at least run.

-Ross

______________________________________________________________________
This e-mail, and any attachments thereto, is intended only for use by
the addressee(s) named herein and may contain legally privileged
and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination,
distribution or copying of this e-mail, and any attachments thereto,
is strictly prohibited. If you have received this e-mail in error,
please immediately notify the sender and permanently delete the
original and any copy or printout thereof.

RE: Integrate kqemu emulation into xen hypervisor possible?

[Dan Magenheimer]

> Ross S. W. Walker, le Tue 01 Apr 2008 11:41:24 -0400, a écrit :
> > Samuel Thibault wrote:
> > > Ross S. W. Walker, le Tue 01 Apr 2008 11:19:35 -0400, a écrit :
> > > > I was wondering if now that the kqemu source has been 
> GPL'd if it was
> > > > possible if it's functionality could be incorporated 
> into the Xen
> > > > hypervisor to provide full virtualization on hardware 
> that doesn't
> > > > support it?
> > >
> > > What would be the benefit, compared to just running qemu in
> > > user space?
> 
> Oooh, oops, you are talking about *k*qemu, not qemu, sorry, ok, I see.
> Well, a first step would be to make kqemu work with Xen's dom0 Linux.
> Then, adding the feature in the Hypervisor may make sense indeed, but
> I'm not sure you'd get much bigger performance than with just the dom0
> support.

If I'm not misunderstanding, an interesting side effect of this
would be recursive virtual machines, e.g. Xen could run Xen running
an hvm.

Who cares about such a thing in the real world, you ask?
Probably not particularly useful in production, but it
would be useful in education and debugging.

On the other hand, kqemu isn't really needed for that, just full
emulation of the VT instruction set, correct?

What kqemu-in-xen *could* be used for would be running x86 VMs on
a non-x86 architecture! (Insert shameless plug for googling
MagiXen here :-)

Dan

RE: Integrate kqemu emulation into xen hypervisor possible?

[Ross S. W. Walker]

Dan Magenheimer wrote:
> > Ross S. W. Walker, le Tue 01 Apr 2008 11:41:24 -0400, a écrit :
> > > Samuel Thibault wrote:
> > > > Ross S. W. Walker, le Tue 01 Apr 2008 11:19:35 -0400, a écrit :
> > > > > I was wondering if now that the kqemu source has been GPL'd if it was
> > > > > possible if it's functionality could be incorporated into the Xen
> > > > > hypervisor to provide full virtualization on hardware that doesn't
> > > > > support it?
> > > >
> > > > What would be the benefit, compared to just running qemu in
> > > > user space?
> > 
> > Oooh, oops, you are talking about *k*qemu, not qemu, sorry, ok, I see.
> > Well, a first step would be to make kqemu work with Xen's dom0 Linux.
> > Then, adding the feature in the Hypervisor may make sense indeed, but
> > I'm not sure you'd get much bigger performance than with just the dom0
> > support.
> 
> If I'm not misunderstanding, an interesting side effect of this
> would be recursive virtual machines, e.g. Xen could run Xen running
> an hvm.

You can do that today by running a qemu user-space daemon in a
PVM (or HVM). Not very fast, but for debugging that's probably
a good thing.

> Who cares about such a thing in the real world, you ask?
> Probably not particularly useful in production, but it
> would be useful in education and debugging.

Or in Philosophy class on the discussion of existence or a
demonstration of deus ex machina.

> On the other hand, kqemu isn't really needed for that, just full
> emulation of the VT instruction set, correct?

Yup, not really needed and can be done now with just the
user-space qemu.

> What kqemu-in-xen *could* be used for would be running x86 VMs on
> a non-x86 architecture! (Insert shameless plug for googling
> MagiXen here :-)

Well in theory, if the Xen development team added back the
parts of qemu that provided translative emulation and an
option to use those parts, then yes, one in theory would
be able to emulate different architectures as well.

I'm mainly interested in full virtualization without hardware
support though...

-Ross

______________________________________________________________________
This e-mail, and any attachments thereto, is intended only for use by
the addressee(s) named herein and may contain legally privileged
and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination,
distribution or copying of this e-mail, and any attachments thereto,
is strictly prohibited. If you have received this e-mail in error,
please immediately notify the sender and permanently delete the
original and any copy or printout thereof.

Re: Integrate kqemu emulation into xen hypervisor possible?

[Daniel P. Berrange]

On Tue, Apr 01, 2008 at 10:50:36AM -0600, Dan Magenheimer wrote:
> > Ross S. W. Walker, le Tue 01 Apr 2008 11:41:24 -0400, a écrit :
> > > Samuel Thibault wrote:
> > > > Ross S. W. Walker, le Tue 01 Apr 2008 11:19:35 -0400, a écrit :
> > > > > I was wondering if now that the kqemu source has been 
> > GPL'd if it was
> > > > > possible if it's functionality could be incorporated 
> > into the Xen
> > > > > hypervisor to provide full virtualization on hardware 
> > that doesn't
> > > > > support it?
> > > >
> > > > What would be the benefit, compared to just running qemu in
> > > > user space?
> > 
> > Oooh, oops, you are talking about *k*qemu, not qemu, sorry, ok, I see.
> > Well, a first step would be to make kqemu work with Xen's dom0 Linux.
> > Then, adding the feature in the Hypervisor may make sense indeed, but
> > I'm not sure you'd get much bigger performance than with just the dom0
> > support.
> 
> If I'm not misunderstanding, an interesting side effect of this
> would be recursive virtual machines, e.g. Xen could run Xen running
> an hvm.
> 
> Who cares about such a thing in the real world, you ask?
> Probably not particularly useful in production, but it
> would be useful in education and debugging.
> 
> On the other hand, kqemu isn't really needed for that, just full
> emulation of the VT instruction set, correct?

Indeed that is correct - atlhough I've not tried it out, upstream QEMU does
already emulate the SVM instructions which should allow Xen / KVM to do
fullyvirt.

> What kqemu-in-xen *could* be used for would be running x86 VMs on
> a non-x86 architecture! (Insert shameless plug for googling
> MagiXen here :-)

No it can't - AFAIK kqemu is for matched host-guest arch only. Only plain
emulated QEMU can do mixed host-guest archiecture, and it cannot take
advantage of kqemu when doing this emulation.

Dan.
-- 
|: Red Hat, Engineering, Boston   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

RE: Integrate kqemu emulation into xen hypervisor possible?

[Ross S. W. Walker]

Samuel Thibault wrote:
> Ross S. W. Walker, le Tue 01 Apr 2008 11:41:24 -0400, a écrit :
> > Samuel Thibault wrote:
> > > Ross S. W. Walker, le Tue 01 Apr 2008 11:19:35 -0400, a écrit :
> > > > I was wondering if now that the kqemu source has been GPL'd if it was
> > > > possible if it's functionality could be incorporated into the Xen
> > > > hypervisor to provide full virtualization on hardware that doesn't
> > > > support it?
> > > 
> > > What would be the benefit, compared to just running qemu in 
> > > user space?
> 
> Oooh, oops, you are talking about *k*qemu, not qemu, sorry, ok, I see.
> Well, a first step would be to make kqemu work with Xen's dom0 Linux.
> Then, adding the feature in the Hypervisor may make sense indeed, but
> I'm not sure you'd get much bigger performance than with just the dom0
> support.

Yes, you are probably right here. The hypervisor should probably just
"allow" the existence of the kqemu kernel module in dom0 by recognizing
it and allowing it to do it's thing without either hypervisor stepping
on each other, and no, integrating it in the xen kernel would not
provide any more performance then dom0 I believe, it merely means that
the kernel can do full virtualization without the addition of another
software project.

I have always wondered why the xen developers decided to keep the qemu
name for the domain device provider after they have completely gutted
and pratically rewritten the original code?

How about calling it just 'dm' for domain manager?

-Ross


______________________________________________________________________
This e-mail, and any attachments thereto, is intended only for use by
the addressee(s) named herein and may contain legally privileged
and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination,
distribution or copying of this e-mail, and any attachments thereto,
is strictly prohibited. If you have received this e-mail in error,
please immediately notify the sender and permanently delete the
original and any copy or printout thereof.

ioemu/qemu

[Samuel Thibault]

Ross S. W. Walker, le Tue 01 Apr 2008 14:41:17 -0400, a écrit :
> I have always wondered why the xen developers decided to keep the qemu
> name for the domain device provider

The directory is actually called ioemu (but indeed the program is called
qemu-dm).  The fact that all functions are still named qemu_* is just to
make upstream tracking easier.

> after they have completely gutted
> and pratically rewritten the original code?

No. The device emulation is as close to upstream as possible (or at
least, it should be :) )

Samuel

Re: Integrate kqemu emulation into xen hypervisor possible?

[Daniel P. Berrange]

On Tue, Apr 01, 2008 at 02:41:17PM -0400, Ross S. W. Walker wrote:
> I have always wondered why the xen developers decided to keep the qemu
> name for the domain device provider after they have completely gutted
> and pratically rewritten the original code?

Xen has not re-written the original code. QEMU provides both a CPU
emulator and a plethora of devices. Xen merely uses the device emulation
and not the CPU emulation. There's a few patches to integrate with Xen
but it is not a re-write.

> How about calling it just 'dm' for domain manager?

No, renaming functional components for mere cosmetic reasons, will cause real 
world breakage for existing deployments. This is not a net win.

Dan.
-- 
|: Red Hat, Engineering, Boston   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

RE: Integrate kqemu emulation into xen hypervisor possible?

[Ross S. W. Walker]

Daniel P. Berrange wrote:
> On Tue, Apr 01, 2008 at 02:41:17PM -0400, Ross S. W. Walker wrote:
> > I have always wondered why the xen developers decided to keep the qemu
> > name for the domain device provider after they have completely gutted
> > and pratically rewritten the original code?
> 
> Xen has not re-written the original code. QEMU provides both a CPU
> emulator and a plethora of devices. Xen merely uses the device emulation
> and not the CPU emulation. There's a few patches to integrate with Xen
> but it is not a re-write.

Ok, I understand, merely removed the CPU emulation. Does that mean it
tracks very closely with upstream's version?

> > How about calling it just 'dm' for domain manager?
> 
> No, renaming functional components for mere cosmetic reasons, will cause real 
> world breakage for existing deployments. This is not a net win.

Well I was thinking of the situation where one might be running qemu
instances on dom0 and when trying to work out which instance belonged
to Xen and which to QEmu it would be difficult.

'killall qemu' would be very bad to find in a shutdown script.

-Ross

______________________________________________________________________
This e-mail, and any attachments thereto, is intended only for use by
the addressee(s) named herein and may contain legally privileged
and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination,
distribution or copying of this e-mail, and any attachments thereto,
is strictly prohibited. If you have received this e-mail in error,
please immediately notify the sender and permanently delete the
original and any copy or printout thereof.

Re: Integrate kqemu emulation into xen hypervisor possible?

[Samuel Thibault]

Ross S. W. Walker, le Tue 01 Apr 2008 15:04:58 -0400, a écrit :
> Ok, I understand, merely removed the CPU emulation. Does that mean it
> tracks very closely with upstream's version?

Not so closely, but efforts are being done into that direction :)

> Well I was thinking of the situation where one might be running qemu
> instances on dom0 and when trying to work out which instance belonged
> to Xen and which to QEmu it would be difficult.
> 
> 'killall qemu' would be very bad to find in a shutdown script.

Qemu's qemu is called qemu, Xen's qemu-dm is called qemu-dm.  killall
should work fine.

Samuel

RE: Integrate kqemu emulation into xen hypervisor possible?

[Ross S. W. Walker]

Samuel Thibault wrote:
> Ross S. W. Walker, le Tue 01 Apr 2008 15:04:58 -0400, a écrit :
> > Ok, I understand, merely removed the CPU emulation. Does that mean it
> > tracks very closely with upstream's version?
> 
> Not so closely, but efforts are being done into that direction :)
> 
> > Well I was thinking of the situation where one might be running qemu
> > instances on dom0 and when trying to work out which instance belonged
> > to Xen and which to QEmu it would be difficult.
> > 
> > 'killall qemu' would be very bad to find in a shutdown script.
> 
> Qemu's qemu is called qemu, Xen's qemu-dm is called qemu-dm.  killall
> should work fine.

Doh! I even said 'qemu-dm' in my earlier post. Time for some more
caffeine.

-Ross

______________________________________________________________________
This e-mail, and any attachments thereto, is intended only for use by
the addressee(s) named herein and may contain legally privileged
and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination,
distribution or copying of this e-mail, and any attachments thereto,
is strictly prohibited. If you have received this e-mail in error,
please immediately notify the sender and permanently delete the
original and any copy or printout thereof.

Re: Integrate kqemu emulation into xen hypervisor possible?

[Tristan Gingold]

On Tue, Apr 01, 2008 at 10:50:36AM -0600, Dan Magenheimer wrote:
> What kqemu-in-xen *could* be used for would be running x86 VMs on
> a non-x86 architecture! (Insert shameless plug for googling
> MagiXen here :-)

Are MagiXen sources available ?

Tristan.

Re: Integrate kqemu emulation into xen hypervisor possible?

[Mark Williamson]

> > > What would be the benefit, compared to just running qemu in
> > > user space?
>
> Oooh, oops, you are talking about *k*qemu, not qemu, sorry, ok, I see.
> Well, a first step would be to make kqemu work with Xen's dom0 Linux.
> Then, adding the feature in the Hypervisor may make sense indeed, but
> I'm not sure you'd get much bigger performance than with just the dom0
> support.

IIRC, kqemu has a load of x86-specific code setting up shadow pagetables, etc.  
My expectation would be that this would want a fair bit of fixing up in order 
to work properly under Xen (if it can work at all under the constraints of 
PV...?).

In a way, it seems a shame not to leverage the shadow PT support that's in the 
hypervisor already.  It's probably less work to integrate kqemu into XenLinux 
though, and it has fewer security implications (I don't think kqemu is 
recommended for secure sandboxing, so you probably wouldn't want it in the 
hypervisor!).  It'd be nice if it were possible to use it in domUs too...

Cheers,
Mark

-- 
Push Me Pull You - Distributed SCM tool (http://www.cl.cam.ac.uk/~maw48/pmpu/)

RE: Integrate kqemu emulation into xen hypervisor possible?

[Dan Magenheimer]

> Are MagiXen sources available ?

I doubt it.  They had not been released to open source by HP when
I left HP, and probably have not been since.

Alex, any chance they might be released?

Dan

RE: Integrate kqemu emulation into xen hypervisor possible?

[Ross S. W. Walker]

Mark Williamson wrote:
> > > > What would be the benefit, compared to just running qemu in
> > > > user space?
> >
> > Oooh, oops, you are talking about *k*qemu, not qemu, sorry, ok, I see.
> > Well, a first step would be to make kqemu work with Xen's dom0 Linux.
> > Then, adding the feature in the Hypervisor may make sense indeed, but
> > I'm not sure you'd get much bigger performance than with just the dom0
> > support.
> 
> IIRC, kqemu has a load of x86-specific code setting up shadow pagetables, etc.  
> My expectation would be that this would want a fair bit of fixing up in order 
> to work properly under Xen (if it can work at all under the constraints of 
> PV...?).
> 
> In a way, it seems a shame not to leverage the shadow PT support that's in the 
> hypervisor already.  It's probably less work to integrate kqemu into XenLinux 
> though, and it has fewer security implications (I don't think kqemu is 
> recommended for secure sandboxing, so you probably wouldn't want it in the 
> hypervisor!).  It'd be nice if it were possible to use it in domUs too...

Cursory glance at the code tells me that it's current implementation
will not play well at all between dom0 and the hypervisor.

It may just be that the technology behind the functional idea of kqemu
would need to be re-worked to expressly support Xen.

A project that will in the end mean that kqemu-xen will end up being
a separately maintained branch of kqemu, since as it will be Xen
specific it will be less likely to be adopted by Qemu, as it is a
separate kernel module it will be less likely to be adopted by Xen,
and since it isn't KVM related it is less likely to be merged into
the kernel. :-(

So whoever follows this route should be prepared to be the maintainer
of a separate project for a long time.

Of course I could be wrong, and if it works well, either Qemu or Xen
may adopt it...

All I know is I personnally don't have the time to learn the memory
management aspects necessary to make it a reality.

-Ross

______________________________________________________________________
This e-mail, and any attachments thereto, is intended only for use by
the addressee(s) named herein and may contain legally privileged
and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination,
distribution or copying of this e-mail, and any attachments thereto,
is strictly prohibited. If you have received this e-mail in error,
please immediately notify the sender and permanently delete the
original and any copy or printout thereof.

RE: Integrate kqemu emulation into xen hypervisor possible?

[Alex Williamson]

On Wed, 2008-04-02 at 07:11 -0600, Dan Magenheimer wrote:
> > Are MagiXen sources available ?
> 
> I doubt it.  They had not been released to open source by HP when
> I left HP, and probably have not been since.
> 
> Alex, any chance they might be released?

   They have not been, and I don't think we have a plan to, but if
someone was really interested in making use of it and getting it into an
upstream-able state, I could try to make it available.  It's currently
suffering from major bit-rot and casual internal attempts to make it run
again have not been successful.

	Alex

-- 
Alex Williamson                             HP Open Source & Linux Org.