Windows-10 virtualization and nested virtualization

Windows-10 virtualization and nested virtualization

[Jeff Forbes]

We have a Windows-10 application which uses a virtual as part of the
process. I am trying to get the application to work in a virtual
environment, which of course will require nested 	virtualization.

Using a libvirt xml description for a RHEL virtual where nested 
	virtualization works, I installed windows-10, Hyper-v and the
application software under the latest updates of Fedora 22.
Windows-10 works great; however, VMBus device required for Hyper-V to
work would not start and was flagged in the Device Manager. Trying to
start a virtual resulted in the error that a component required by
Hyper-V failed to start.

I then ported the VM to VMWare Workstation 12 and in this environment t
he VMBus device starts and nested virtuals start as expected. These
results indicated that the VM was working correctly and the problem was
likely with the QEMU-KVM hypervisor.

Reading recent messages on the KVM and QEMU mail lists indicated that
there were many KVM updates made to the 4.6.0-rc1 kernel, so I
download, compiled and installed this kernel. I also cloned the latest
version of qemu. When I run my Windows-10 VM in this environment with
the latest updates. the VMBus device starts and there are not warnings
in the Device Manager; however, the nested VMs do not start and the
same error about a component required by Hyper-V failed to start.

Do I need to turn something on to get this to work or is the Hyper-V
support still under development?

Please advise.


Best,

Jeff

Re: Windows-10 virtualization and nested virtualization

[Bandan Das]

Jeff Forbes <jeff.forbes@mail.nacon.com> writes:

> We have a Windows-10 application which uses a virtual as part of the
> process. I am trying to get the application to work in a virtual
> environment, which of course will require nested 	virtualization.
>
> Using a libvirt xml description for a RHEL virtual where nested 
> 	virtualization works, I installed windows-10, Hyper-v and the
> application software under the latest updates of Fedora 22.
> Windows-10 works great; however, VMBus device required for Hyper-V to
> work would not start and was flagged in the Device Manager. Trying to
> start a virtual resulted in the error that a component required by
> Hyper-V failed to start.
>
> I then ported the VM to VMWare Workstation 12 and in this environment t
> he VMBus device starts and nested virtuals start as expected. These
> results indicated that the VM was working correctly and the problem was
> likely with the QEMU-KVM hypervisor.
>
> Reading recent messages on the KVM and QEMU mail lists indicated that
> there were many KVM updates made to the 4.6.0-rc1 kernel, so I
> download, compiled and installed this kernel. I also cloned the latest
> version of qemu. When I run my Windows-10 VM in this environment with
> the latest updates. the VMBus device starts and there are not warnings
> in the Device Manager; however, the nested VMs do not start and the
> same error about a component required by Hyper-V failed to start.
>
> Do I need to turn something on to get this to work or is the Hyper-V
> support still under development?

I am not sure if anyone has been successful with running Hyper-V nested.
The first obstacle I remember is Hyper-V refusing to run when it finds
the "hypervisor" flag in guest cpu. If you are past that step,  maybe
the vmbus dependency is something else.

> Please advise.
>
>
> Best,
>
> Jeff--
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Windows-10 virtualization and nested virtualization

[Jeff Forbes]

[-- Attachment #1: Type: text/plain, Size: 2419 bytes --]

When I use "-cpu host,+VMX,-hypervisor", Hyper-V gives this error when trying to start a VM:
Virtual Machine could not be started because the hypervisor is not running.

So with the latest kernel and qemu-kvm, the hypervisor flag is needed.


On Mon, 2016-04-04 at 17:35 -0400, Bandan Das wrote:

Jeff Forbes <jeff.forbes@mail.nacon.com<mailto:jeff.forbes@mail.nacon.com>> writes:



We have a Windows-10 application which uses a virtual as part of the
process. I am trying to get the application to work in a virtual
environment, which of course will require nested        virtualization.

Using a libvirt xml description for a RHEL virtual where nested
        virtualization works, I installed windows-10, Hyper-v and the
application software under the latest updates of Fedora 22.
Windows-10 works great; however, VMBus device required for Hyper-V to
work would not start and was flagged in the Device Manager. Trying to
start a virtual resulted in the error that a component required by
Hyper-V failed to start.

I then ported the VM to VMWare Workstation 12 and in this environment t
he VMBus device starts and nested virtuals start as expected. These
results indicated that the VM was working correctly and the problem was
likely with the QEMU-KVM hypervisor.

Reading recent messages on the KVM and QEMU mail lists indicated that
there were many KVM updates made to the 4.6.0-rc1 kernel, so I
download, compiled and installed this kernel. I also cloned the latest
version of qemu. When I run my Windows-10 VM in this environment with
the latest updates. the VMBus device starts and there are not warnings
in the Device Manager; however, the nested VMs do not start and the
same error about a component required by Hyper-V failed to start.

Do I need to turn something on to get this to work or is the Hyper-V
support still under development?



I am not sure if anyone has been successful with running Hyper-V nested.
The first obstacle I remember is Hyper-V refusing to run when it finds
the "hypervisor" flag in guest cpu. If you are past that step,  maybe
the vmbus dependency is something else.



Please advise.


Best,

Jeff--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@vger.kernel.org<mailto:majordomo@vger.kernel.org>
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[-- Attachment #2: Type: text/html, Size: 2863 bytes --]

Re: Windows-10 virtualization and nested virtualization

[Paolo Bonzini]

On 06/04/2016 17:27, Jeff Forbes wrote:
> When I use "-cpu host,+VMX,-hypervisor", Hyper-V gives this error
> when trying to start a VM: Virtual Machine could not be started
> because the hypervisor is not running.
> 
> So with the latest kernel and qemu-kvm, the hypervisor flag is
> needed.

That is not what the error means.  The error means that Hyper-V has
failed to start.

Paolo

> 
> On Mon, 2016-04-04 at 17:35 -0400, Bandan Das wrote:
>> Jeff Forbes <jeff.forbes-o7saAZo6sAzSUeElwK9/Pw@public.gmane.org <mailto:jeff.forbes-o7saAZo6sAzSUeElwK9/Pw@public.gmane.org>> writes:
>>
>>> We have a Windows-10 application which uses a virtual as part of the
>>> process. I am trying to get the application to work in a virtual
>>> environment, which of course will require nested virtualization.
>>> Using a libvirt xml description for a RHEL virtual where nested
>>> virtualization works, I installed windows-10, Hyper-v and the
>>> application software under the latest updates of Fedora 22.
>>> Windows-10 works great; however, VMBus device required for Hyper-V to
>>> work would not start and was flagged in the Device Manager. Trying to
>>> start a virtual resulted in the error that a component required by
>>> Hyper-V failed to start. I then ported the VM to VMWare Workstation
>>> 12 and in this environment t he VMBus device starts and nested
>>> virtuals start as expected. These results indicated that the VM was
>>> working correctly and the problem was likely with the QEMU-KVM
>>> hypervisor. Reading recent messages on the KVM and QEMU mail lists
>>> indicated that there were many KVM updates made to the 4.6.0-rc1
>>> kernel, so I download, compiled and installed this kernel. I also
>>> cloned the latest version of qemu. When I run my Windows-10 VM in
>>> this environment with the latest updates. the VMBus device starts and
>>> there are not warnings in the Device Manager; however, the nested VMs
>>> do not start and the same error about a component required by Hyper-V
>>> failed to start. Do I need to turn something on to get this to work
>>> or is the Hyper-V support still under development? 
>>
>>
>> I am not sure if anyone has been successful with running Hyper-V nested.
>> The first obstacle I remember is Hyper-V refusing to run when it finds
>> the "hypervisor" flag in guest cpu. If you are past that step,  maybe
>> the vmbus dependency is something else.
>>
>>> Please advise. Best, Jeff-- To unsubscribe from this list: send the
>>> line "unsubscribe kvm" in the body of a message to
>>> majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org <mailto:majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org> More
>>> majordomo info at http://vger.kernel.org/majordomo-info.html 

Re: Windows-10 virtualization and nested virtualization

[Jeff Forbes]

I was responding to Bandan Das' comment about Hyper-v NOT running when
the hypervisor flag is present. What I observed was just the opposite. 
With "-cpu host,+vmx" Hyper-V will try to start the VM and then report
that a component required by Hyper-V failed to start. The end result is
the same; however, with the kernel and qemu-kvm in Fedora-22 the VMBus
device does not start. Now with the the latest kernel and qemu-kvm,
VMBus starts, but the same error is given. Progress from all of the
hard work of the KVM developers. I am afraid that helping in this area
is beyond my current abilities, so I must defer to you and your colleag
ues.

Paolo,
Since you are the most likely person to know, is Hyper-V nested
virtualization likely to work with KVM in the near term? 

Jeff


On Wed, 2016-04-06 at 18:11 +0200, Paolo Bonzini wrote:
> 
> On 06/04/2016 17:27, Jeff Forbes wrote:
> > When I use "-cpu host,+VMX,-hypervisor", Hyper-V gives this error
> > when trying to start a VM: Virtual Machine could not be started
> > because the hypervisor is not running.
> > 
> > So with the latest kernel and qemu-kvm, the hypervisor flag is
> > needed.
> 
> That is not what the error means.  The error means that Hyper-V has
> failed to start.
> 
> Paolo
> 
> > 
> > On Mon, 2016-04-04 at 17:35 -0400, Bandan Das wrote:
> > > Jeff Forbes <jeff.forbes@mail.nacon.com <mailto:
> > > jeff.forbes@mail.nacon.com>> writes:
> > > 
> > > > We have a Windows-10 application which uses a virtual as part
> > > > of the
> > > > process. I am trying to get the application to work in a
> > > > virtual
> > > > environment, which of course will require nested
> > > > virtualization.
> > > > Using a libvirt xml description for a RHEL virtual where nested
> > > > virtualization works, I installed windows-10, Hyper-v and the
> > > > application software under the latest updates of Fedora 22.
> > > > Windows-10 works great; however, VMBus device required for
> > > > Hyper-V to
> > > > work would not start and was flagged in the Device Manager.
> > > > Trying to
> > > > start a virtual resulted in the error that a component required
> > > > by
> > > > Hyper-V failed to start. I then ported the VM to VMWare
> > > > Workstation
> > > > 12 and in this environment t he VMBus device starts and nested
> > > > virtuals start as expected. These results indicated that the VM
> > > > was
> > > > working correctly and the problem was likely with the QEMU-KVM
> > > > hypervisor. Reading recent messages on the KVM and QEMU mail
> > > > lists
> > > > indicated that there were many KVM updates made to the 4.6.0
> > > > -rc1
> > > > kernel, so I download, compiled and installed this kernel. I
> > > > also
> > > > cloned the latest version of qemu. When I run my Windows-10 VM
> > > > in
> > > > this environment with the latest updates. the VMBus device
> > > > starts and
> > > > there are not warnings in the Device Manager; however, the
> > > > nested VMs
> > > > do not start and the same error about a component required by
> > > > Hyper-V
> > > > failed to start. Do I need to turn something on to get this to
> > > > work
> > > > or is the Hyper-V support still under development? 
> > > 
> > > 
> > > I am not sure if anyone has been successful with running Hyper-V
> > > nested.
> > > The first obstacle I remember is Hyper-V refusing to run when it
> > > finds
> > > the "hypervisor" flag in guest cpu. If you are past that step, 
> > >  maybe
> > > the vmbus dependency is something else.
> > > 
> > > > Please advise. Best, Jeff-- To unsubscribe from this list: send
> > > > the
> > > > line "unsubscribe kvm" in the body of a message to
> > > > majordomo@vger.kernel.org <mailto:majordomo@vger.kernel.org>
> > > > More
> > > > majordomo info at http://vger.kernel.org/majordomo-info.html 

Re: Windows-10 virtualization and nested virtualization

[Paolo Bonzini]

On 06/04/2016 19:45, Jeff Forbes wrote:
> I was responding to Bandan Das' comment about Hyper-v NOT running when
> the hypervisor flag is present. What I observed was just the opposite. 
> With "-cpu host,+vmx" Hyper-V will try to start the VM and then report
> that a component required by Hyper-V failed to start.

Remember that Hyper-V's primary partition is _also_ running under the
nested hypervisor and thus is expecting to see the hypervisor bit in
CPUID.  So the error messages can be confusing.

For example "a component required by Hyper-V failed to start" could mean
"the primary partition found the CPUID bit to be set, as expected, but
didn't find the right hypervisor interface".

> Since you are the most likely person to know, is Hyper-V nested
> virtualization likely to work with KVM in the near term? 

It needs manpower.  If somebody wants the feature enough that they will
spend time looking at why it doesn't work, then that will help.

Paolo