Re: KVM call minutes for Sept 21

["Nadav Har'El" <nyh@math.technion.ac.il>]

On Sun, Sep 26, 2010, Avi Kivity wrote about "Re: KVM call minutes for Sept 21":
> Don't worry, I want to merge nvmx as soon as possible (but not sooner).

Thanks, I'm happy to hear that.

> >So I don't think there has been any lack of reviews. I don't think that
> >getting more reviews is the most important task ahead of us.
> 
> The code is so incredibly complex that each review round raises new 
> issues, simply because they were hidden by other issues previously, or 
> because the reviewer's understanding only reached the point where they 
> can notice it recently.  Usually after a few rounds the review 
> converges.
> This hasn't yet happened with nvmx, because it is so complicated.

I completely agree. If you look at the sheer size of the VMX spec, it is
simply unavoidable that nested VMX, which is basically a VMX implementation,
will be complex. So if you want the nested VMX feature in KVM (and I
understand that you do), then you can't avoid this added complexity...

What is now my task is to do my best to make sure that while nested VMX
is complex, it shouldn't be more complex than it needs to be. But it
can't be less complex than it needs to be ;-)

> Don't worry, nvmx will not get rejected on those grounds.  However, the 
> lack of use cases is worrying.  I haven't seen the use cases you list 
> above used with nsvm, and no bug reports from users either.

I don't know why nested x86 virtualization hasn't caught on. I'll be honest
and admit that - yes - it's possibile that it's simply not useful to actual
users.

But there are other possibilities - which I'd like to think are the right ones.
Maybe it's a chicken-and-egg problem: Perhaps nobody knows how to use nested
virtualization because the most common hypervisor (vmware) doesn't support it,
and common clouds (e.g., Amazon EC2) don't support these kind of use cases.
Maybe the number of people who use KVM *and* AMD *and* need nested
virtualization is not big enough to have any sort of critical mass for building
use cases for this feature. And maybe nested virtualization is one of those
features that is useless to the majority of users, but to the few who need it,
it is very important.

I've been told that IBM System Z has supported (hardware-assisted) nested
virtualization right from the start (in the 70s), and there people actually
use nested virtualization all the time, typically of depth 2. I understand
(but admit that I don't have any first-hand knowledge of this) that nested
virtualization is mostly useful there for organization needs - one very
strong machine is divided into many virtual machines, and often a need arises
for a second level of organization. Maybe someone on this list has actual
experience with these systems and can relate war-stories about them?

> No need.  But, as we haven't converged yet, we may see new review items.

Of course, I'm used to those by now :-)

> What's absolutely critical is having code that is understood by more 
> people than just you.  Part of the process of increasing knowledge of 
> the code is reading it and raising issues... nothing much we can do to 
> change it.

I agree. It's definitely a really bad thing when only one person knows any
part of the code, and I'm happy that you've dived so deeply into the code
(and even though you feel like you don't understand it - I feel that you
do :-)). There are more people who understand much of this code even better
than I do (the people who wrote it originally), and of course anybody can
learn this code just like I did this year.

> I'm worried about maintaining core vmx after nvmx is merged, not nvmx 
> itself.  There are simply many more things to consider when making a change.

Right, but how can we avoid this issue, assuming that you do want nvmx in?
May I ask how this effected nested SVM?

> Note that we can work on a test suite in parallel with the kvm code.  
> Look at kvm-unit-tests.git x86/svm.c.

I will.

-- 
Nadav Har'El                        |      Sunday, Sep 26 2010, 18 Tishri 5771
nyh@math.technion.ac.il             |-----------------------------------------
Phone +972-523-790466, ICQ 13349191 |I'm a peripheral visionary: I see into
http://nadav.harel.org.il           |the future, but mostly off to the sides.