Xen inside Xen with VMX?

Xen inside Xen with VMX?

[Morten Hansen]

[-- Attachment #1.1: Type: text/plain, Size: 943 bytes --]


Hello,

out of curiosity, I was wondering if it is possible to support recursive emulation in Xen, i.e. if it is possible to have Xen as a guest in Xen (possibly with other guests inside those Xen's)? I'm particularly curious if this is possible when running Xen using Intel's VT extensions (or AMD's Pacifica), i.e. do these enhancements provide the hypervisor with enough control that it can virtualize the virtualization enhancements themselves (without a prohibitive loss in performance)?
If it's not possible I guess VT etc. doesn't really offer true emulation since the guest would be able to detect the emulation by noticing that it wasn't itself able to become a hypervisor.

Thank you in advance!

Best regards,

MH

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE!
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

[-- Attachment #1.2: Type: text/html, Size: 1101 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Xen inside Xen with VMX?

[pradeep singh rautela]

Hi
On 31/01/2008, Morten Hansen <paradigm__82@hotmail.com> wrote:
>
>  Hello,
>
> out of curiosity, I was wondering if it is possible to support recursive
> emulation in Xen, i.e. if it is possible to have Xen as a guest in Xen
> (possibly with other guests inside those Xen's)? I'm particularly curious if
> this is possible when running Xen using Intel's VT extensions (or AMD's
> Pacifica), i.e. do these enhancements provide the hypervisor with enough
> control that it can virtualize the virtualization enhancements themselves
> (without a prohibitive loss in performance)?

AFAIK I guess xen can still run inside a HVM guest. yes/no?
And therefore atleast can run PV guests over it, though i doubt HVM
guests will work too.

Thanks,
        --Pradeep
> If it's not possible I guess VT etc. doesn't really offer true emulation
> since the guest would be able to detect the emulation by noticing that it
> wasn't itself able to become a hypervisor.
>
> Thank you in advance!
>
> Best regards,
>
> MH
>
> ________________________________
> Express yourself instantly with MSN Messenger! MSN Messenger
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>


-- 
Pradeep Singh Rautela
http://eagain.wordpress.com
http://emptydomain.googlepages.com

Re: Xen inside Xen with VMX?

[Ben Guthro]

[-- Attachment #1.1: Type: text/plain, Size: 1278 bytes --]

Current VT implementations do not have this ability, AFAIK.

Morten Hansen wrote:
> Hello,
>
> out of curiosity, I was wondering if it is possible to support 
> recursive emulation in Xen, i.e. if it is possible to have Xen as a 
> guest in Xen (possibly with other guests inside those Xen's)? I'm 
> particularly curious if this is possible when running Xen using 
> Intel's VT extensions (or AMD's Pacifica), i.e. do these enhancements 
> provide the hypervisor with enough control that it can virtualize the 
> virtualization enhancements themselves (without a prohibitive loss in 
> performance)?
> If it's not possible I guess VT etc. doesn't really offer true 
> emulation since the guest would be able to detect the emulation by 
> noticing that it wasn't itself able to become a hypervisor.
>
> Thank you in advance!
>
> Best regards,
>
> MH
>
> ------------------------------------------------------------------------
> Express yourself instantly with MSN Messenger! MSN Messenger 
> <http://clk.atdmt.com/AVE/go/onm00200471ave/direct/01/>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>   


[-- Attachment #1.2: Type: text/html, Size: 1831 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Xen inside Xen with VMX?

[Daniel Stodden]

On Thu, 2008-01-31 at 07:07 -0500, Ben Guthro wrote:
> Current VT implementations do not have this ability, AFAIK.

Indeed. There's only one root mode, and the processor has no concept of
recursively stacking roots and accompanying protection levels on top of
each other.

Nonetheless, root mode could be emulated, i.e. via shadow VMCBs,
optional shadow NPTs and emulation of the respective instruction subset.

Would suffer from the same (most probably solvable) problems regarding
privilege compression and the like, but probably an interesting
excercise.

Maybe one should put it on some (nonexisting) list for interesting
[academic] projects.

regards,
daniel

-- 
Daniel Stodden
LRR     -      Lehrstuhl für Rechnertechnik und Rechnerorganisation
Institut für Informatik der TU München             D-85748 Garching
http://www.lrr.in.tum.de/~stodden         mailto:stodden@cs.tum.edu
PGP Fingerprint: F5A4 1575 4C56 E26A 0B33  3D80 457E 82AE B0D8 735B

RE: Xen inside Xen with VMX?

[Morten Hansen]

[-- Attachment #1.1: Type: text/plain, Size: 2381 bytes --]


Hello,
 
thanks to everyone who replied! What I was thinking of was something along the lines of Daniel's suggestion below, i.e. if the VMX architecture makes it possible to setup enough proper traps etc. so that you can actually "cheaply" emulate all these VMX registers/features and thus have multiple, nested HVM's. I don't know if it's useful but it is at least interesting to think about how it would work and how big/small a performance cost you could get away with if you had, say, 10 nested HVM's with Linux or something running in the innermost ;) I guess the emulation cost will be bigger for the "non-primary" HVM's (the primary being the one running closest to the hardware). But I guess that even stuff like VT-d directed I/O could be recursively emulated and yet still have "true" direct I/O for the innermost operating system.
 
Cheers,
 
MH > From: stodden@cs.tum.edu> To: bguthro@virtualiron.com> Date: Thu, 31 Jan 2008 14:33:46 +0100> Subject: Re: [Xen-devel] Xen inside Xen with VMX?> CC: xen-devel@lists.xensource.com; paradigm__82@hotmail.com> > > On Thu, 2008-01-31 at 07:07 -0500, Ben Guthro wrote:> > Current VT implementations do not have this ability, AFAIK.> > Indeed. There's only one root mode, and the processor has no concept of> recursively stacking roots and accompanying protection levels on top of> each other.> > Nonetheless, root mode could be emulated, i.e. via shadow VMCBs,> optional shadow NPTs and emulation of the respective instruction subset.> > Would suffer from the same (most probably solvable) problems regarding> privilege compression and the like, but probably an interesting> excercise.> > Maybe one should put it on some (nonexisting) list for interesting> [academic] projects.> > regards,> daniel> > -- > Daniel Stodden> LRR - Lehrstuhl für Rechnertechnik und Rechnerorganisation> Institut für Informatik der TU München D-85748 Garching> http://www.lrr.in.tum.de/~stodden mailto:stodden@cs.tum.edu> PGP Fingerprint: F5A4 1575 4C56 E26A 0B33 3D80 457E 82AE B0D8 735B> > > > _______________________________________________> Xen-devel mailing list> Xen-devel@lists.xensource.com> http://lists.xensource.com/xen-devel
_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE!
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

[-- Attachment #1.2: Type: text/html, Size: 2873 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Xen inside Xen with VMX?

[Mark Williamson]

> thanks to everyone who replied! What I was thinking of was something along
> the lines of Daniel's suggestion below, i.e. if the VMX architecture makes
> it possible to setup enough proper traps etc. so that you can actually
> "cheaply" emulate all these VMX registers/features and thus have multiple,
> nested HVM's.

I wasn't able to find if it's possible to trap VMX instructions that are 
executed in noon-root mode but it's my assumption that Intel did the sensible 
thing and made that work :-)  They definitely cause a trap in AMD's SVM.

> I don't know if it's useful but it is at least interesting to 
> think about how it would work and how big/small a performance cost you
> could get away with if you had, say, 10 nested HVM's with Linux or
> something running in the innermost ;) I guess the emulation cost will be
> bigger for the "non-primary" HVM's (the primary being the one running
> closest to the hardware). But I guess that even stuff like VT-d directed
> I/O could be recursively emulated and yet still have "true" direct I/O for
> the innermost operating system.

Yeah, that sounds plausible.  Similarly, with nested pagetables / hardware 
assisted paging, you'd save overhead relative to nesting n shadow pagetables.

It'd be interesting whether by emulating these virtualisation-friendly 
features it might be possible to do nested full virtualisation without 
completely destroying performance...  I guess it'd entail a fair bit of scary 
coding to make it work but it would be really cool.

It could certainly be useful for testing stuff if nothing else.  I already do 
development and testing by running PV Linux on Xen, all wrapped up in a Xen 
HVM domain on my test box.  It works quite well and the performance is usable 
for testing / debugging.

I think Qemu can now emulate VMX / SVM instructions when it's run as an 
emulator (not as a virtualiser), so anyone who really needs virtual machines 
with HVM support right now can get it that way - if they don't mind taking a 
massive performance hit.

Cheers,
Mark

> MH > From: stodden@cs.tum.edu> To: bguthro@virtualiron.com> Date: Thu, 31
> Jan 2008 14:33:46 +0100> Subject: Re: [Xen-devel] Xen inside Xen with VMX?>
> CC: xen-devel@lists.xensource.com; paradigm__82@hotmail.com> > > On Thu,
> 2008-01-31 at 07:07 -0500, Ben Guthro wrote:> > Current VT implementations
> do not have this ability, AFAIK.> > Indeed. There's only one root mode, and
> the processor has no concept of> recursively stacking roots and
> accompanying protection levels on top of> each other.> > Nonetheless, root
> mode could be emulated, i.e. via shadow VMCBs,> optional shadow NPTs and
> emulation of the respective instruction subset.> > Would suffer from the
> same (most probably solvable) problems regarding> privilege compression and
> the like, but probably an interesting> excercise.> > Maybe one should put
> it on some (nonexisting) list for interesting> [academic] projects.> >
> regards,> daniel> > -- > Daniel Stodden> LRR - Lehrstuhl für Rechnertechnik
> und Rechnerorganisation> Institut für Informatik der TU München D-85748
> Garching> http://www.lrr.in.tum.de/~stodden mailto:stodden@cs.tum.edu> PGP
> Fingerprint: F5A4 1575 4C56 E26A 0B33 3D80 457E 82AE B0D8 735B> > > >
> _______________________________________________> Xen-devel mailing list>
> Xen-devel@lists.xensource.com> http://lists.xensource.com/xen-devel
> _________________________________________________________________
> Express yourself instantly with MSN Messenger! Download today it's FREE!
> http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/



-- 
Push Me Pull You - Distributed SCM tool (http://www.cl.cam.ac.uk/~maw48/pmpu/)

RE: Xen inside Xen with VMX?

[Daniel Stodden]

On Thu, 2008-01-31 at 22:14 +0100, Morten Hansen wrote:

>  But I guess that even stuff like VT-d directed I/O could be
> recursively emulated and yet still have "true" direct I/O for the
> innermost operating system.

AMD's IOMMU docs (used to?) have some useful comments regarding
emulation. Different from CPU virtualization, emulation I/O memory
management is much more interesting in practice, since different from
hvm it's not a VMM-only feature. Most operating systems would use it,
e.g. for remapping 32bit DMA-capable devices to transfers into 64bit
memory space.

Hence, emulation of the machine interface for full virtualization is
indeed quite useful, even without recursion. And yes, it would remain
effective and efficient. It's only control operations which are to be
emulated in shadow structures, running is _not_ subject to emulation.

regards,
daniel

-- 
Daniel Stodden
LRR     -      Lehrstuhl für Rechnertechnik und Rechnerorganisation
Institut für Informatik der TU München             D-85748 Garching
http://www.lrr.in.tum.de/~stodden         mailto:stodden@cs.tum.edu
PGP Fingerprint: F5A4 1575 4C56 E26A 0B33  3D80 457E 82AE B0D8 735B

Re: Xen inside Xen with VMX?

[Mark Williamson]

You can run Xen within an HVM domain but you won't be able to run HVM domains 
inside that.  The HVM environment does not provide emulated VMX / SVM support 
at the moment.

> If it's not possible I guess
> VT etc. doesn't really offer true emulation since the guest would be able
> to detect the emulation by noticing that it wasn't itself able to become a
> hypervisor.

Well, there are plenty of existing machines out there that don't support HVM - 
and even one ones that theoretically do it may be disabled in the BIOS.  So 
you can't tell just from that aspect of the CPU behaviour that you're running 
in a VM.

There are more obvious Xen/Qemu-specific interfaces that can tell the guest 
straightforwardly that it's in a VM, though.

Cheers,
Mark

-- 
Push Me Pull You - Distributed SCM tool (http://www.cl.cam.ac.uk/~maw48/pmpu/)