Re: [PATCH 09/13] KVM: MMU: get expected spte out of mmu-lock

Xiao Guangrong
2012-04-05T18:25:41Z

On 04/01/2012 11:53 PM, Avi Kivity wrote:

> On 03/29/2012 11:25 AM, Xiao Guangrong wrote:
>> It depends on PTE_LIST_WRITE_PROTECT bit in rmap which let us quickly know
>> whether the page is writable out of mmu-lock
>>
>> Signed-off-by: Xiao Guangrong <xiaoguangrong@linux.vnet.ibm.com>
>> ---
>> arch/x86/kvm/mmu.c | 17 +++++++++++++----
>> arch/x86/kvm/paging_tmpl.h | 2 +-
>> 2 files changed, 14 insertions(+), 5 deletions(-)
>>
>> diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
>> index 3887a07..c029185 100644
>> --- a/arch/x86/kvm/mmu.c
>> +++ b/arch/x86/kvm/mmu.c
>> @@ -1148,6 +1148,12 @@ static int rmap_write_protect(struct kvm *kvm, u64 gfn)
>>
>> *rmapp |= PTE_LIST_WRITE_PROTECT;
>>
>> + /*
>> + * Setting PTE_LIST_WRITE_PROTECT bit before doing page
>> + * write-protect.
>> + */
>> + smp_mb();
>> +
>
> wmb only needed.
>

We should ensure setting this bit before reading spte, it cooperates with
fast page fault path to avoid this case:

On fast page fault path: On rmap_write_protect path:
read spte: old_spte = *spte
(reading spte is reordered to the front of
setting PTE_LIST_WRITE_PROTECT bit)
set spte.identification
smp_mb
if (!rmap.PTE_LIST_WRITE_PROTECT)
set rmap.PTE_LIST_WRITE_PROTECT
cmpxchg(sptep, spte, spte | WRITABLE)
see old_spte.identification is not set,
so it does not write-protect this page
OOPS!!!

> Would it be better to store this bit in all the sptes instead? We're
> touching them in any case. More work to clear them, but
> un-write-protecting a page is beneficial anyway as it can save a fault.
>

There are two reasons:
- if we set this bit in rmap, we can do the quickly check to see the page is
writble before doing shadow page walking.

- since a full barrier is needed, we should use smp_mb for every spte like this:

while ((spte = rmap_next(rmapp, spte))) {
read spte
smp_mb
write-protect spte
}

smp_mb is called in the loop, i think it is not good, yes?

If you just want to save the fault, we can let all spte to be writeable in
mmu_need_write_protect, but we should cache gpte access bits into spte firstly.
It should be another patchset i think. :)