Re: [PATCH v10 035/108] KVM: x86/mmu: Track shadow MMIO value on a per-VM basis

[Sean Christopherson <seanjc@google.com>]

On Fri, Nov 25, 2022, Yan Zhao wrote:
> On Fri, Nov 25, 2022 at 09:07:09AM +0800, Huang, Kai wrote:
> > On Fri, 2022-11-25 at 08:37 +0800, Yan Zhao wrote:
> > > On Fri, Nov 25, 2022 at 08:45:01AM +0800, Huang, Kai wrote:
> > > > On Fri, 2022-11-25 at 08:12 +0800, Yan Zhao wrote:
> > > > > On Fri, Nov 25, 2022 at 08:13:48AM +0800, Huang, Kai wrote:
> > > > > > On Tue, 2022-11-22 at 10:10 +0800, Yan Zhao wrote:
> > > > > > > Also make enable_mmio_caching to be a per-VM value?
> > > > > > > As if the shadow_mmio_value is 0, mmio_caching needs to be disabled.
> > > > > > 
> > > > > > If I recall correctly, Sean said we can disable TDX guests if mmio_caching is
> > > > > > disabled (we also will need to change to allow enable_mmio_caching to still be
> > > > > > true when mmio_value is 0).
> > > > > > 
> > > > > > SEV_ES has similar logic:
> > > > > > 
> > > > > > void __init sev_hardware_setup(void)
> > > > > > {
> > > > > > 
> > > > > > 	...
> > > > > > 
> > > > > >         /*
> > > > > >          * SEV-ES requires MMIO caching as KVM doesn't have access to the guest
> > > > > >          * instruction stream, i.e. can't emulate in response to a #NPF and
> > > > > >          * instead relies on #NPF(RSVD) being reflected into the guest as #VC
> > > > > >          * (the guest can then do a #VMGEXIT to request MMIO emulation).
> > > > > >          */
> > > > > >         if (!enable_mmio_caching)
> > > > > >                 goto out;
> > > > > > 
> > > > > 
> > > > > Would enabling mmio caching in per-VM basis be better?
> > > > > 
> > > > 
> > > > We need Paolo/Sean to decide.
> > > > 
> > > > The thing is TDX guests always require mmio_caching being enabled.  For VMX
> > > > guests, normally we will always enable mmio_caching too.  So I think per-VM
> > > > basis mmio_caching is not that useful.
> > > With per-VM basis enabling, I think we can get rid of the kvm_gfn_shared_mask(kvm)
> > > in below code and also in handle_abnormal_pfn()
> > > 
> > > static inline bool is_mmio_spte(struct kvm *kvm, u64 spte)
> > > {
> > >         return (spte & shadow_mmio_mask) == kvm->arch.shadow_mmio_value &&
> > >                likely(enable_mmio_caching || kvm_gfn_shared_mask(kvm));
> > > }
> > > 
> > 
> > It needs to go anyway regardless per-VM mmio_caching or not, as explained we
> > need to change to allow enable_mmio_caching to be true even mmio_value is 0.

Yes, the kvm_gfn_shared_mask() in is_mmio_spte() should not exist.

> Or it's better to check enable_mmio_caching is true in
> kvm_mmu_set_mmio_spte_value() as below. 
> 
> void kvm_mmu_set_mmio_spte_value(struct kvm *kvm, u64 mmio_value)
> {
>         WARN_ON(!enable_mmio_caching);
>         kvm->arch.shadow_mmio_value = mmio_value;

Eh, if you're going to bother with that assert, capture the more subtle aspects
as well, e.g. that EPT is enabled and that shadow_mmio_mask has been set to the
expected value.  I would also ditch the helper and just open code this in TDX
code, I doubt there's a use case outside of TDX+VMX coexistence.