From mboxrd@z Thu Jan  1 00:00:00 1970
From: Joerg Roedel <joro@8bytes.org>
Subject: Re: Nested SVM and migration
Date: Sat, 20 Feb 2010 21:18:23 +0100
Message-ID: <20100220201822.GG20833@8bytes.org>
References: <4B80347E.7000003@redhat.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Cc: Joerg Roedel <joerg.roedel@amd.com>, Avi Kivity <avi@redhat.com>,
	kvm <kvm@vger.kernel.org>
To: Zachary Amsden <zamsden@redhat.com>
Return-path: <kvm-owner@vger.kernel.org>
Received: from 8bytes.org ([88.198.83.132]:35966 "EHLO 8bytes.org"
	rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
	id S1753189Ab0BTUSY (ORCPT <rfc822;kvm@vger.kernel.org>);
	Sat, 20 Feb 2010 15:18:24 -0500
Content-Disposition: inline
In-Reply-To: <4B80347E.7000003@redhat.com>
Sender: kvm-owner@vger.kernel.org
List-ID: <kvm.vger.kernel.org>

On Sat, Feb 20, 2010 at 09:14:06AM -1000, Zachary Amsden wrote:
> Perhaps I am misunderstanding, but I don't see how nested SVM instances  
> can be properly migrated.  How does one extract and rebuild the nested  
> hsave control block?

Migrating guests which run in nested mode could not be migrated in a
save way currently but there are plans to fix that :-)
The first step is to save the l1 cpu state in the guest supplied hsave
area. But that is not sufficient because this does not work for all l1
state.

> If it isn't done already, one possible way to add it as an extension  
> might be to represent the data as additional MSRs which are saved and  
> restored with migration.

This sounds complicated.

> Actually, looking deeper, there doesn't even appear to be any way to  
> export the nested CPU data at all, meaning basic features like  
> suspending and restoring the VM are not possible.  Is there any plan to  
> make it work in the near future?  I'm not complaining; if my  
> understanding is correct, this actually makes my current task easier.

I think we should introduce a flag to indicate userspace if a vcpu is in
a state that could be migrated in a save way together with a way for
userspace to request that the vcpu enters a migratable state. In the
kernel we could do something like that:

nested_svm_vmrun(...)
{
	/* ... */
	kvm_migration_disable(vcpu);
	/* ... */
}

nested_svm_vmexit(...)
{
	/* ... */
	kvm_migration_enable(vcpu);
	/* ... */
}

and somewhere in the vcpu_run loop:

if (vcpu->arch.migration_win_req)
	nested_svm_vmexit(INTR);

This might be helpful in other situations too. Thoughts?

	Joerg