Re: [PATCH v2 8/10] KVM: MMU: prefetch ptes when intercepted guest #PF

Marcelo Tosatti
2010-06-29T12:00:37Z

On Tue, Jun 29, 2010 at 04:07:40PM +0800, Xiao Guangrong wrote:
>
>
> Marcelo Tosatti wrote:
>
> >> +
> >> + if (sp->role.level > PT_PAGE_TABLE_LEVEL)
> >> + return;
> >> +
> >> + if (sp->role.direct)
> >> + return direct_pte_prefetch(vcpu, sptep);
> >
> > Can never happen.
> >
>
> Marcelo,
>
> Thanks for your comment. You mean that we can't meet sp->role.direct here?
> could you please tell me why? During my test, it can be triggered.

Ah, for 1->1 emulation, right.

> >> @@ -322,6 +395,7 @@ static u64 *FNAME(fetch)(struct kvm_vcpu *vcpu, gva_t addr,
> >> user_fault, write_fault,
> >> dirty, ptwrite, level,
> >> gw->gfn, pfn, false, true);
> >> + FNAME(pte_prefetch)(vcpu, sptep);
> >> break;
> >> }
> >
> >
> > I'm afraid this can introduce regressions since it increases mmu_lock
> > contention. Can you get some numbers with 4-vcpu or 8-vcpu guest and
> > many threads benchmarks, such as kernbench and apachebench? (on
> > non-EPT).
> >
>
> The pte prefetch is the fast path, it only occupies little time, for the worst
> case, only need read 128 byte form the guest pte, and if it prefetched success,
> the #PF cause by later access will avoid, then we avoid to exit form the guest,
> and walk guest pte, walk shadow pages, flush local tlb... a lots of work can be
> reduced.
>
> Before i post this patchset firstly, i do the performance test by using unixbench,
> it improved ~3.6% under EPT disable case.
> (it's in the first version's chagelog)
>
> Today, i do the kernbench test with 4 vcpu and 1G memory, the result shows it
> improved ~1.6% :-)

OK, nice.

> > Also prefetch should be disabled for EPT, due to lack of accessed bit.
> >
>
> But we call mmu_set_spte() with speculative == false, it not touch the accessed bit.

There is no accessed bit on EPT. So the aging code (kvm_age_rmapp)
considers any present translation as accessed. There is no way to
distinguish between actually accessed translations and prefetched (but
unused) ones.