Re: nested virtualization test report for Xen 4.3-RC1

Re: nested virtualization test report for Xen 4.3-RC1

[Ren, Yongjie]

> -----Original Message-----
> From: Pasi Kärkkäinen [mailto:pasik@iki.fi]
> Sent: Thursday, June 27, 2013 8:47 PM
> To: George Dunlap
> Cc: Ren, Yongjie; Xu, Dongxiao; xen-devel; Zhang, Xiantao
> Subject: Re: [Xen-devel] nested virtualizaiton test report for Xen 4.3-RC1
> 
> On Thu, Jun 27, 2013 at 12:37:56PM +0100, George Dunlap wrote:
> > On Fri, May 10, 2013 at 12:07 PM, Ren, Yongjie <yongjie.ren@intel.com>
> wrote:
> > > Hi All,
> > > This the a nested virtualization test report for Xen 4.3-RC1 on Intel
> hardware. We use Linux 3.9.1 as Dom0.
> > > a. Virtual EPT and VMCS shadowing features can work fine.
> > > b. Xen, KVM and VMware can basically work on top of L0 Xen.
> > > c. 32bit/64bit Linux and Windows are covered as L2 guests.
> >
> > Sorry I just saw this -- thanks for the nice enumeration.
> >
> > Two questions.  First, I don't see the Win7 "XP compatibility mode" on
> > this list -- that would be L0 Xen, L1 Win7, L2 XP.  This seems like
> > probably the most likely actual real-world use of nested virt.  Is
> > that on your radar at all?
> >

So far, I didn't test this case.
That's a good use case. I'll add this case in my next report about nested virt.

> > Secondly, what do you think is the primary use case for Xen-on-Xen (or
> > KVM-on-Xen, &c)?  Who would want to use it and why?
> >
I'm not sure which type of nested virt is more important. But I heard some use cases as following.

1. public cloud (e.g. Amazon EC2: if they can provide virtual VMX to the VMs, that can be a value-added service.)

2. QA: Nested virt can make QAs easy to set up the testing env (especially: virt/cloud areas).
In a Ovirt Workshop, I heard some guys have been testing Ovirt in KVM nested virtualization. 
Some OpenStack guys also told me that they want to test OpenStack in a VM not on physical host.

3. competition counter: In some BBS, I ever heard someone said "VMware have nested virt cap; why Xen doesn't support?"


Best Regards,
     Yongjie (Jay)

> 
> I can think of at least two use-cases:
> 
> - Xen-on-Xen might be good for testing/debugging the hypervisor.. Much
> easier to crash and debug the virtual Xen rather than physical machine.
> - Xen-on-Xen makes it possible to create easy-and-fast-to-clone
> lab/poc/test environments with "full" functionality thanks to virtual vmx
> and ept..
> 
> 
> -- Pasi
> 
> 
> > Thanks,
> >  -George
> >
> > >
> > > There are three basic entities in Xen nested virtualization.
> > >         L0: Xen (64bit Xen and 64bit Dom0), which is at the bottom
> of the nested stack.
> > >         L1: Xen or KVM or VMware or VirtualBox  (all in 64bit
> mode)
> > >         L2: Linux or Windows guest, which is at the top of the nested
> stack.
> > > (when saying 'KVM on Xen', I mean L0 hypervisor is Xen and L1
> hypervisor is KVM.)
> > >
> > > Workable cases: (Pass)
> > > 1. virtual EPT and VMCS shadowing feature enabled
> > > 2. 64bit Linux/Windows as L2 guest for "Xen on Xen"
> > > 3. 64bit Linux guest as L2 guest for "KVM on Xen"
> > > 4. L1 KVM and L1 Xen simultaneously running on a L0 Xen
> > > 5. L2 guest Save/Restore and local migration for "KVM on Xen"
> > > 6. AVX and XSAVE in L2 guest for "KVM on Xen"
> > > 7. some workloads (e.g. LTP, Kernel-build, UnixBench) can work fine in
> 64bit L2 Linux guest
> > > 8. 64bit Linux L2 guest can boot up for "VMware on Xen"
> > > 9. 32bit L2 guest (Linux/Windows) booting on "Xen on Xen"  (not use
> EPT in L1)
> > > 10. 32bit/64bit Windows and 32bit Linux L2 guest booting on "VMware
> on Xen"  (not use EPT in L1)
> > > N.B. Only if you don't use EPT feature in L1 hypervisor, case #9 and #10
> can work fine.
> > >
> > > Non-workable cases: (Fail)
> > > 1. 32bit/64bit Windows L2 guest booting on "KVM on Xen"
> > > 2. L2 guest Save/Restore and local migration for "Xen on Xen"
> > > 3. Migration "from L0 to L1" for "Xen on Xen"
> > > 4. Migration "from L1 to L0" for "Xen on Xen"
> > > 5. Migration a L1 Xen/KVM guest with a L2 running in that L1
> > > 6. L2 guest booting on "VirtualBox on Xen"
> > >
> > >
> > > Best Regards,
> > >      Yongjie (Jay)
> > >
> > >
> > >
> > > _______________________________________________
> > > Xen-devel mailing list
> > > Xen-devel@lists.xen.org
> > > http://lists.xen.org/xen-devel
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xen.org
> > http://lists.xen.org/xen-devel

Re: nested virtualization test report for Xen 4.3-RC1

[konrad wilk]

On 6/27/2013 9:27 AM, Ren, Yongjie wrote:
>> -----Original Message-----
>> From: Pasi Kärkkäinen [mailto:pasik@iki.fi]
>> Sent: Thursday, June 27, 2013 8:47 PM
>> To: George Dunlap
>> Cc: Ren, Yongjie; Xu, Dongxiao; xen-devel; Zhang, Xiantao
>> Subject: Re: [Xen-devel] nested virtualizaiton test report for Xen 4.3-RC1
>>
>> On Thu, Jun 27, 2013 at 12:37:56PM +0100, George Dunlap wrote:
>>> On Fri, May 10, 2013 at 12:07 PM, Ren, Yongjie <yongjie.ren@intel.com>
>> wrote:
>>>> Hi All,
>>>> This the a nested virtualization test report for Xen 4.3-RC1 on Intel
>> hardware. We use Linux 3.9.1 as Dom0.
>>>> a. Virtual EPT and VMCS shadowing features can work fine.
>>>> b. Xen, KVM and VMware can basically work on top of L0 Xen.
>>>> c. 32bit/64bit Linux and Windows are covered as L2 guests.
>>> Sorry I just saw this -- thanks for the nice enumeration.
>>>
>>> Two questions.  First, I don't see the Win7 "XP compatibility mode" on
>>> this list -- that would be L0 Xen, L1 Win7, L2 XP.  This seems like
>>> probably the most likely actual real-world use of nested virt.  Is
>>> that on your radar at all?
>>>
> So far, I didn't test this case.
> That's a good use case. I'll add this case in my next report about nested virt.
>
>>> Secondly, what do you think is the primary use case for Xen-on-Xen (or
>>> KVM-on-Xen, &c)?  Who would want to use it and why?
>>>
> I'm not sure which type of nested virt is more important. But I heard some use cases as following.
>
> 1. public cloud (e.g. Amazon EC2: if they can provide virtual VMX to the VMs, that can be a value-added service.)
>
> 2. QA: Nested virt can make QAs easy to set up the testing env (especially: virt/cloud areas).
> In a Ovirt Workshop, I heard some guys have been testing Ovirt in KVM nested virtualization.
> Some OpenStack guys also told me that they want to test OpenStack in a VM not on physical host.
And also development. I do use that for some of the low-level assembler 
coding - as I can trigger QEMU to dump the CPU state when printing out 
to the debug port. Quite useful when I was poking in suspend/resume logic.

I was quite happy to see Xen 4.3 + Fedora 18 allow me to nicely boot 
under Xen an Xen HVM OS. Didn't try yet to run an HVM guest within the 
HVM context.