Re: [PATCH 5/5] KVM: MMU: fast invalid all mmio sptes

[Xiao Guangrong <xiaoguangrong@linux.vnet.ibm.com>]

On 03/19/2013 03:36 PM, Gleb Natapov wrote:
> On Tue, Mar 19, 2013 at 11:15:36AM +0800, Xiao Guangrong wrote:
>> On 03/19/2013 06:16 AM, Eric Northup wrote:
>>> On Fri, Mar 15, 2013 at 8:29 AM, Xiao Guangrong
>>> <xiaoguangrong@linux.vnet.ibm.com> wrote:
>>>> This patch tries to introduce a very simple and scale way to invalid all
>>>> mmio sptes - it need not walk any shadow pages and hold mmu-lock
>>>>
>>>> KVM maintains a global mmio invalid generation-number which is stored in
>>>> kvm->arch.mmio_invalid_gen and every mmio spte stores the current global
>>>> generation-number into his available bits when it is created
>>>>
>>>> When KVM need zap all mmio sptes, it just simply increase the global
>>>> generation-number. When guests do mmio access, KVM intercepts a MMIO #PF
>>>> then it walks the shadow page table and get the mmio spte. If the
>>>> generation-number on the spte does not equal the global generation-number,
>>>> it will go to the normal #PF handler to update the mmio spte
>>>>
>>>> Since 19 bits are used to store generation-number on mmio spte, the
>>>> generation-number can be round after 33554432 times. It is large enough
>>>> for nearly all most cases, but making the code be more strong, we zap all
>>>> shadow pages when the number is round
>>>>
>>>> Signed-off-by: Xiao Guangrong <xiaoguangrong@linux.vnet.ibm.com>
>>>> ---
>>>>  arch/x86/include/asm/kvm_host.h |    2 +
>>>>  arch/x86/kvm/mmu.c              |   61 +++++++++++++++++++++++++++++++++------
>>>>  arch/x86/kvm/mmutrace.h         |   17 +++++++++++
>>>>  arch/x86/kvm/paging_tmpl.h      |    7 +++-
>>>>  arch/x86/kvm/vmx.c              |    4 ++
>>>>  arch/x86/kvm/x86.c              |    6 +--
>>>>  6 files changed, 82 insertions(+), 15 deletions(-)
>>>>
>>>> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
>>>> index ef7f4a5..572398e 100644
>>>> --- a/arch/x86/include/asm/kvm_host.h
>>>> +++ b/arch/x86/include/asm/kvm_host.h
>>>> @@ -529,6 +529,7 @@ struct kvm_arch {
>>>>         unsigned int n_requested_mmu_pages;
>>>>         unsigned int n_max_mmu_pages;
>>>>         unsigned int indirect_shadow_pages;
>>>> +       unsigned int mmio_invalid_gen;
>>>
>>> Could this get initialized to something close to the wrap-around
>>> value, so that the wrap-around case gets more real-world coverage?
>>
>> I am afraid we can not. We cache the current mmio_invalid_gen into mmio spte when
>> it is created no matter what the initiation value is.
>>
>> If you have a better way, please show me. ;)
>>
> The idea is to initialize mmio_invalid_gen to value close to MAX_GEN in
> order to exercise 

Oh, got it, i understood it as "initialize mmio_invalid properly to _avoid_ more
wrap-around"... Sorry for my careless! :(

The idea can check the hardly-run code at runtime, i am fine with this.

Thank you, Eric and Gleb!