Re: [PATCH 5/5] KVM: MMU: fast invalid all mmio sptes

[Gleb Natapov <gleb@redhat.com>]

On Tue, Mar 19, 2013 at 11:15:36AM +0800, Xiao Guangrong wrote:
> On 03/19/2013 06:16 AM, Eric Northup wrote:
> > On Fri, Mar 15, 2013 at 8:29 AM, Xiao Guangrong
> > <xiaoguangrong@linux.vnet.ibm.com> wrote:
> >> This patch tries to introduce a very simple and scale way to invalid all
> >> mmio sptes - it need not walk any shadow pages and hold mmu-lock
> >>
> >> KVM maintains a global mmio invalid generation-number which is stored in
> >> kvm->arch.mmio_invalid_gen and every mmio spte stores the current global
> >> generation-number into his available bits when it is created
> >>
> >> When KVM need zap all mmio sptes, it just simply increase the global
> >> generation-number. When guests do mmio access, KVM intercepts a MMIO #PF
> >> then it walks the shadow page table and get the mmio spte. If the
> >> generation-number on the spte does not equal the global generation-number,
> >> it will go to the normal #PF handler to update the mmio spte
> >>
> >> Since 19 bits are used to store generation-number on mmio spte, the
> >> generation-number can be round after 33554432 times. It is large enough
> >> for nearly all most cases, but making the code be more strong, we zap all
> >> shadow pages when the number is round
> >>
> >> Signed-off-by: Xiao Guangrong <xiaoguangrong@linux.vnet.ibm.com>
> >> ---
> >>  arch/x86/include/asm/kvm_host.h |    2 +
> >>  arch/x86/kvm/mmu.c              |   61 +++++++++++++++++++++++++++++++++------
> >>  arch/x86/kvm/mmutrace.h         |   17 +++++++++++
> >>  arch/x86/kvm/paging_tmpl.h      |    7 +++-
> >>  arch/x86/kvm/vmx.c              |    4 ++
> >>  arch/x86/kvm/x86.c              |    6 +--
> >>  6 files changed, 82 insertions(+), 15 deletions(-)
> >>
> >> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> >> index ef7f4a5..572398e 100644
> >> --- a/arch/x86/include/asm/kvm_host.h
> >> +++ b/arch/x86/include/asm/kvm_host.h
> >> @@ -529,6 +529,7 @@ struct kvm_arch {
> >>         unsigned int n_requested_mmu_pages;
> >>         unsigned int n_max_mmu_pages;
> >>         unsigned int indirect_shadow_pages;
> >> +       unsigned int mmio_invalid_gen;
> > 
> > Could this get initialized to something close to the wrap-around
> > value, so that the wrap-around case gets more real-world coverage?
> 
> I am afraid we can not. We cache the current mmio_invalid_gen into mmio spte when
> it is created no matter what the initiation value is.
> 
> If you have a better way, please show me. ;)
> 
The idea is to initialize mmio_invalid_gen to value close to MAX_GEN in
order to exercise 

+       if (unlikely(kvm->arch.mmio_invalid_gen++ == MAX_GEN)) {
+               kvm->arch.mmio_invalid_gen = 0;
+               return kvm_mmu_zap_all(kvm);
+       }

path more often.

--
			Gleb.