[PATCH 0/27] nVMX: Nested VMX, v6

["Nadav Har'El" <nyh@il.ibm.com>]

Hi,

About three months have passed since my previous (v5) nested VMX patch set,
and it can no longer be applied cleanly to the current KVM trunk.

This version of the patches can be applied to the current trunk, and addresses
dozens of concerns that have been raised by Avi Kivity, Marcelo Tosatti, Gleb
Natapov, and Eddie Dong while reviewing v5.
There are still several outstanding issues (e.g., that IDT handling code that
we've been discussing) that are not addressed in this version, but rest assured
that I have not forgotten them - I simply want a newer version, and one that
works with a current KVM, to be available to potential reviewers or testers.

About nested VMX:
-----------------

The following 27 patches implement nested VMX support. This feature enables a
guest to use the VMX APIs in order to run its own nested guests. In other
words, it allows running hypervisors (that use VMX) under KVM.
Multiple guest hypervisors can be run concurrently, and each of those can
in turn host multiple guests.

The theory behind this work, our implementation, and its performance
characteristics were presented this month in OSDI (the USENIX Symposium on
Operating Systems Design and Implementation). Our paper was titled
"The Turtles Project: Design and Implementation of Nested Virtualization",
and was awarded "Jay Lepreau Best Paper". The paper is available online, at:

	http://www.usenix.org/events/osdi10/tech/full_papers/Ben-Yehuda.pdf

This patch set does not include all the features described in the paper.
In particular, this patch set is missing nested EPT (shadow page tables are
used) and missing some features required to run VMWare Server as a guest.
These missing features will be sent as follow-on patchs.

Running nested VMX:
------------------

The current patches have a number of requirements, which will be relaxed in
follow-on patches:

1. This version was only tested with KVM (64-bit) as a guest hypervisor, and
   Linux as a nested guest.

2. SMP is supported in the code, but is unfortunately buggy in this version
   and often leads to hangs. Use the "nosmp" option in the L0 (topmost)
   kernel to avoid this bug (and to reduce your performance ;-))..

3. No modifications are required to user space (qemu). However, qemu does not
   currently list "VMX" as a CPU feature in its emulated CPUs (even when they
   are named after CPUs that do normally have VMX). Therefore, the "-cpu host"
   option should be given to qemu, to tell it to support CPU features which
   exist in the host - and in particular VMX.
   This requirement can be made unnecessary by a trivial patch to qemu (which
   I will submit in the future).

4. The nested VMX feature is currently disabled by default. It must be
   explicitly enabled with the "nested=1" option to the kvm-intel module.

5. Nested EPT and VPID are not properly supported in this version. You must
   give the "ept=0 vpid=0" module options to kvm-intel to turn both features
   off.


Patch statistics:
-----------------

 Documentation/kvm/nested-vmx.txt |  237 ++
 arch/x86/include/asm/kvm_host.h  |    2 
 arch/x86/include/asm/vmx.h       |   31 
 arch/x86/kvm/svm.c               |    6 
 arch/x86/kvm/vmx.c               | 2396 ++++++++++++++++++++++++++++-
 arch/x86/kvm/x86.c               |   16 
 arch/x86/kvm/x86.h               |    6 
 7 files changed, 2657 insertions(+), 37 deletions(-)

--
Nadav Har'El
IBM Haifa Research Lab