Re: [PATCH] KVM: MMU: Add wrapper to check whether MMU is in direct mode

[Yu Zhang <yu.c.zhang@linux.intel.com>]

On Fri, Jan 20, 2023 at 03:38:24PM +0800, Yu Zhang wrote:
> On Fri, Jan 20, 2023 at 01:18:45AM +0000, Sean Christopherson wrote:
> > +David and Ben
> > 
> > On Tue, Dec 06, 2022, Yu Zhang wrote:
> > > Simplify the code by introducing a wrapper, mmu_is_direct(),
> > > instead of using vcpu->arch.mmu->root_role.direct everywhere.
> > > 
> > > Meanwhile, use temporary variable 'direct', in routines such
> > > as kvm_mmu_load()/kvm_mmu_page_fault() etc. instead of checking
> > > vcpu->arch.mmu->root_role.direct repeatedly.
> 
> Thanks Sean. I already forgot the existence of this patch. :)
> > 
> > I've looked at this patch at least four times and still can't decide whether or
> > not I like the helper.  On one had, it's shorter and easier to read.  On the other
> > hand, I don't love that mmu_is_nested() looks at a completely different MMU, which
> > is weird if not confusing.
> 
> Do you mean mmu_is_direct()? Why it's about a different MMU? 
> 
> > 
> > Anyone else have an opinion?
> > 
> > > No functional change intended.
> > > 
> > > Signed-off-by: Yu Zhang <yu.c.zhang@linux.intel.com>
> > > ---
> > >  arch/x86/kvm/mmu/mmu.c | 26 +++++++++++++-------------
> > >  arch/x86/kvm/x86.c     |  9 +++++----
> > >  arch/x86/kvm/x86.h     |  5 +++++
> > >  3 files changed, 23 insertions(+), 17 deletions(-)
> > > 
> > > diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
> > > index 4736d7849c60..d2d0fabdb702 100644
> > > --- a/arch/x86/kvm/mmu/mmu.c
> > > +++ b/arch/x86/kvm/mmu/mmu.c
> > > @@ -2280,7 +2280,7 @@ static void shadow_walk_init_using_root(struct kvm_shadow_walk_iterator *iterato
> > >  
> > >  	if (iterator->level >= PT64_ROOT_4LEVEL &&
> > >  	    vcpu->arch.mmu->cpu_role.base.level < PT64_ROOT_4LEVEL &&
> > > -	    !vcpu->arch.mmu->root_role.direct)
> > > +	    !mmu_is_direct(vcpu))
> > >  		iterator->level = PT32E_ROOT_LEVEL;
> > >  
> > >  	if (iterator->level == PT32E_ROOT_LEVEL) {
> > > @@ -2677,7 +2677,7 @@ static int kvm_mmu_unprotect_page_virt(struct kvm_vcpu *vcpu, gva_t gva)
> > >  	gpa_t gpa;
> > >  	int r;
> > >  
> > > -	if (vcpu->arch.mmu->root_role.direct)
> > > +	if (mmu_is_direct(vcpu))
> > >  		return 0;
> > >  
> > >  	gpa = kvm_mmu_gva_to_gpa_read(vcpu, gva, NULL);
> > > @@ -3918,7 +3918,7 @@ void kvm_mmu_sync_roots(struct kvm_vcpu *vcpu)
> > >  	int i;
> > >  	struct kvm_mmu_page *sp;
> > >  
> > > -	if (vcpu->arch.mmu->root_role.direct)
> > > +	if (mmu_is_direct(vcpu))
> > >  		return;
> > >  
> > >  	if (!VALID_PAGE(vcpu->arch.mmu->root.hpa))
> > > @@ -4147,7 +4147,7 @@ static bool kvm_arch_setup_async_pf(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,
> > >  
> > >  	arch.token = alloc_apf_token(vcpu);
> > >  	arch.gfn = gfn;
> > > -	arch.direct_map = vcpu->arch.mmu->root_role.direct;
> > > +	arch.direct_map = mmu_is_direct(vcpu);
> > >  	arch.cr3 = vcpu->arch.mmu->get_guest_pgd(vcpu);
> > >  
> > >  	return kvm_setup_async_pf(vcpu, cr2_or_gpa,
> > > @@ -4157,17 +4157,16 @@ static bool kvm_arch_setup_async_pf(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,
> > >  void kvm_arch_async_page_ready(struct kvm_vcpu *vcpu, struct kvm_async_pf *work)
> > >  {
> > >  	int r;
> > > +	bool direct = mmu_is_direct(vcpu);
> > 
> > I would prefer to not add local bools and instead due a 1:1 replacement.  "direct"
> > loses too much context (direct what?), and performance wise I doubt it will
> > influence the compiler.
> 
> If we want to use a new temp value, how about "mmu_direct_mode"?
> 
> But I am also open to use mmu_is_direct(). Because I just realized the benifit
> is too marginal: the second read of vcpu->arch.mmu->root_role.direct should be
> a cache hit, so the gain of adding a local variable is to only reduce one L1
> cache read.
Sorry, should be one TLB and one cache access(I guess VIPT will also bring some
parallelism).

B.R.
Yu