Re: [RFC PATCH 01/13] KVM: nSVM: Track the ASID per-VMCB

[Sean Christopherson <seanjc@google.com>]

On Mon, Mar 03, 2025, Jim Mattson wrote:
> On Fri, Feb 28, 2025 at 4:03 PM Sean Christopherson <seanjc@google.com> wrote:
> >
> > +Jim, for his input on VPIDs.
> >
> > On Wed, Feb 05, 2025, Yosry Ahmed wrote:
> > > The ASID is currently tracked per-vCPU, because the same ASID is used by
> > > L1 and L2. That ASID is flushed on every transition between L1 and L2.
> > >
> > > Track the ASID separately for each VMCB (similar to the
> > > asid_generation), giving L2 a separate ASID. This is in preparation for
> > > doing fine-grained TLB flushes on nested transitions instead of
> > > unconditional full flushes.
> >
> > After having some time to think about this, rather than track ASIDs per VMCB, I
> > think we should converge on a single approach for nVMX (VPID) and nSVM (ASID).
> >
> > Per **VM**, one VPID/ASID for L1, and one VPID/ASID for L2.
> 
> When using EPT on VMX, there is probably no advantage to using one
> VPID per VM. The physical ASID is determined by <EPTRTA, VPID, PCID>.
> Two different VMs are not going to share an EPTRTA, so they already
> have different ASIDs, even if they have the same VPID.

For posterity, which the SDM says this:

  Linear mappings may be created. They are derived from the paging structures
  referenced (directly or indirectly) by the current value of CR3 and are associated
  with the current VPID and the current PCID.

it explicitly disallows creating or using linear mappings when EPT is enabled:

  No linear mappings are created while EPT is in use.

  no linear mappings are used while EPT is in use.

I think it's still worth assigning a unique VPID though, e.g. it would provide
some amount of defense in depth.  I.e. two different VMs *shouldn't* share an
EPTRTA :-)

> > For SVM, the dynamic ASID crud is a holdover from KVM's support for CPUs that
> > don't support FLUSHBYASID, i.e. needed to purge the entire TLB in order to flush
> > guest mappings.  FLUSHBYASID was added in 2010, and AFAIK has been supported by
> > all AMD CPUs since.
> 
> > KVM already mostly keeps the same ASID, except for when a vCPU is migrated, in
> > which case KVM assigns a new ASID.  I suspect that following VMX's lead and
> > simply doing a TLB flush in this situation would be an improvement for modern
> > CPUs, as it would flush the entries that need to be flushed, and not pollute the
> > TLBs with stale, unused entries.
> >
> > Using a static per-VM ASID would also allow using broadcast invalidations[*],
> > would simplify the SVM code base, and I think/hope would allow us to move much
> > of the TLB flushing logic, e.g. for task migration, to common code.
> >
> > For VPIDs, maybe it's because it's Friday afternoon, but for the life of me I
> > can't think of any reason why KVM needs to assign VPIDs per vCPU.  Especially
> > since KVM is ridiculously conservative and flushes _all_ EPT/VPID contexts when
> > running a different vCPU on a pCPU (which I suspect we can trim down?).
> >
> > Am I forgetting something?
> 
> TDX? IIRC, TDX requires a unique VPID for each vCPU in a VM.

Ha!  Nope, the TDX module actually does what I'm suggesting, and uses a per-VM
VPID.  So if I'm forgetting some TLB edge case, TDX is already hosed.

FWIW, the hypervisor, i.e. KVM, has no control over the VPID used by the TDX
module.  Intel incorporated SEAM mode into the ASID tag to prevent TLB collisions
between the hypervisor and the TDX module, and that also conveniently provides
separation between VPIDs for non-TDX VMs and TDX VMs (and now I'm curious if TDX
enabling does the "right" thing and skips VPID allocation).

FWIW, TDX's scheme would match what I'm proposing almost exactly.  TDX "composes"
the VPID using the HKID (guaranteed unique per VM) and then a "VM identifier",
which at a glance differentiates L1 from L2.

> > [*] https://lore.kernel.org/all/Z8HdBg3wj8M7a4ts@google.com