Re: Re: [Xen-users] Nested Virtualisation question

[Keir Fraser <keir.xen@gmail.com>]

On 07/04/2011 06:36, "Jeroen Groenewegen van der Weyden"
<groen692@grosc.com> wrote:

> I couldn't help noticing some patches were added in the
> /staging/xen-unstable branch. So I gave it a go.
> 
> I know it is not called the unstable branch for nothing but, but I want
> to let you know that it the entire system crashes when any domain is
> created:
> xl create someDomain -> results in immediate system crash

Probably the same as this, happening often in the automated tests:

Apr  7 03:09:18.695893 (XEN) hvm_vcpu_initialise: nestedhvm_vcpu_initialise
returned -95
Apr  7 03:09:18.703835 (XEN) Assertion 'd->arch.paging.hap.total_pages == 0'
failed at hap.c:699
[...]
Apr  7 03:09:18.915908 (XEN) Xen call trace:
Apr  7 03:09:18.915950 (XEN)    [<ffff82c48020429f>]
hap_teardown+0x1a8/0x229
Apr  7 03:09:18.923861 (XEN)    [<ffff82c4801ccb7c>]
paging_teardown+0x25/0x135
Apr  7 03:09:18.923922 (XEN)    [<ffff82c4801584d1>]
domain_relinquish_resources+0x6b/0x23c
Apr  7 03:09:18.935903 (XEN)    [<ffff82c480105287>] domain_kill+0x95/0xe9
Apr  7 03:09:18.943841 (XEN)    [<ffff82c48010324c>] do_domctl+0xd8e/0x1a02
Apr  7 03:09:18.943900 (XEN)    [<ffff82c480212f28>]
syscall_enter+0xc8/0x122
Apr  7 03:09:18.955837 (XEN)
Apr  7 03:09:18.955875 (XEN)
Apr  7 03:09:18.955909 (XEN) ****************************************
Apr  7 03:09:18.955961 (XEN) Panic on CPU 7:
Apr  7 03:09:18.956003 (XEN) Assertion 'd->arch.paging.hap.total_pages == 0'
failed at hap.c:699
Apr  7 03:09:18.963863 (XEN) ****************************************


> System:
> - domO system openSUSE 11.4 x86_64. kernel 2.6.37.1-1.2-xen
> - xen build from staging/xen-unstable (c/s 23172)
> - proc intel q9450
> 
> 
> +++++++++++++console output
> hydra:/etc/xen/vm # xl create groscWindowsXpWorkstation
> Parsing config file groscWindowsXpWorkstation
> xc: info: VIRTUAL MEMORY ARRANGEMENT:
>    Loader:        0000000000100000->00000000001795b0
>    TOTAL:         0000000000000000->0000000017800000
>    ENTRY ADDRESS: 0000000000100000
> xc: info: PHYSICAL MEMORY ALLOCATION:
>    4KB PAGES: 0x0000000000000200
>    2MB PAGES: 0x00000000000000bb
>    1GB PAGES: 0x0000000000000000
> libxl: error: libxl_device.c:475:libxl__wait_for_device_model Device
> Model not ready
> xl: fatal error: libxl_create.c:532, rc=-1:
> libxl__confirm_device_model_startup
> 
> +++++++++++++
> 
> Op 4-4-2011 11:51, Tim Deegan schreef:
>> At 07:47 +0100 on 04 Apr (1301903251), Jeroen Groenewegen van der Weyden
>> wrote:
>>> Hello,
>>> 
>>> I have A question about nested virtualisation, the xen user list
>>> suggested to post the question here.
>>> I need nested virtualisation to run an application on out en
>>> environment, which uses kvm it-self.
>>> 
>>> I suppose I have to apply these patches:
>>> -
>>> http://xen.markmail.org/search/?q=nested%20vmx#query:nested%20vmx+page:1+mid
>>> :z33lihxnbdjghua5+state:results
>>> -  or http://xenbits.xen.org/ext/cegger/nestedhvm/
>>> 
>>> Is it correct these patches are now in the xen-unstable branch?
>> Not just yet.  There were two problems outstanding with patch #12 of the
>> latest version (a TLB flush race and a subsequent deadlock in the fix),
>> which are both fixed.  As soon as Christoph posts his fixes, the patch
>> series will be applied.
>> 
>> Cheers,
>> 
>> Tim.
>> 
>>> Anything else to get me on my way.
>>> 
>>> 
>>> My goal is
>>> 
>>> DOM0
>>>    openSUSE 11.4 with xen 4.X
>>> 
>>> DOMU1
>>>    SLES 11 SP1
>>>     kvm guest HVM domu1a (propetary OS)
>>>     kvm guest HVM domu1b (sles 10 sp3)
>>>     kvm guest HVM domu1c (sles 10 sp3)
>>>     kvm guest HVM domu1b (sles 10 sp3)
>>> 
>>> Op 3-4-2011 23:29, Todd Deshane schreef:
>>>> On Sun, Apr 3, 2011 at 8:32 PM, Jeroen Groenewegen van der Weyden
>>>> <groen692@grosc.com>   wrote:
>>>>> I now am in the position to start testing your suggestion.
>>>>> 
>>>>> An additional question, is it correct the patches you mention here are
>>>>> part
>>>>> of the xen4.2-unstable branch?
>>>>> are these patches for vt-x or for svm only? the developer seems to be from
>>>>> amd. My machines are running on intel.
>>>>> 
>>>> The latest patch series is here:
>>>> http://xen.1045712.n5.nabble.com/PATCH-00-12-Nested-Virtualization-Overview
>>>> -td3415672.html
>>>> 
>>>> Not sure if it has been accepted yet or not. You can email the
>>>> developer and/or devel list for confirmation.
>>>> 
>>>> Thanks,
>>>> Todd
>>>> 
>>>>> mfg,
>>>>> Jeroen
>>>>> 
>>>>> Op 15-2-2011 16:21, Todd Deshane schreef:
>>>>>> On Tue, Feb 15, 2011 at 6:47 AM, Jeroen Groenewegen van der Weyden
>>>>>> <groen692@grosc.com>     wrote:
>>>>>>> The why is a good question,
>>>>>>> We use xen as out main virtualisation platform. But now there are
>>>>>>> applications that use virtualisation it self. In my case I have an
>>>>>>> application that uses kvm. So to put it on our xen server it needs to be
>>>>>>> nested. I guess in the future more of these situation will appear.
>>>>>>> 
>>>>>>> I read somethings about nested virtualisation and I wonder If somebody
>>>>>>> has
>>>>>>> any experience with it.
>>>>>>> 
>>>>>> To run KVM inside of a guest you need virtualization support within
>>>>>> the guest (running nested xen doesn't require the double nested
>>>>>> (unless you want to do HVM within the nested of course))
>>>>>> 
>>>>>> Patches can be found here:
>>>>>> 
>>>>>> http://xen.markmail.org/search/?q=nested%20vmx#query:nested%20vmx+page:1+
>>>>>> mid:z33lihxnbdjghua5+state:results
>>>>>> 
>>>>>> I believe KVM should also have support for nested vmx and nested svm.
>>>>>> 
>>>>>> The use cases are fairly slim, so finding users/testers is often
>>>>>> difficult.
>>>>>> 
>>>>>> Please report any bugs you find to the appropriate devel lists and cc
>>>>>> the appropriate nested virtualization maintainers .
>>>>>> 
>>>>>> Hope that helps.
>>>>>> 
>>>>>> Thanks,
>>>>>> Todd
>>>>>> 
>>>>>> _______________________________________________
>>>>>> Xen-users mailing list
>>>>>> Xen-users@lists.xensource.com
>>>>>> http://lists.xensource.com/xen-users
>>>>>> 
>>>> 
>>> 
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xensource.com
>>> http://lists.xensource.com/xen-devel
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel