Kernel KVM virtualization development
 help / color / mirror / Atom feed
From: Yosry Ahmed <yosry@kernel.org>
To: Sean Christopherson <seanjc@google.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>,
	Jim Mattson <jmattson@google.com>,
	 Maxim Levitsky <mlevitsk@redhat.com>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	 Tom Lendacky <thomas.lendacky@amd.com>,
	kvm@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v1 12/28] KVM: nSVM: Add a placeholder ASID for L2
Date: Wed, 5 Aug 2026 15:25:19 +0000	[thread overview]
Message-ID: <anNSYpoXYXoGBewo@google.com> (raw)
In-Reply-To: <anKCocdyhY6PFT3l@google.com>

On Wed, Aug 05, 2026 at 12:26:30AM +0000, Yosry Ahmed wrote:
> On Fri, Jul 31, 2026 at 11:34:34PM -0700, Yosry Ahmed wrote:
> > On Mon, Jul 27, 2026 at 5:36 PM Yosry Ahmed <yosry@kernel.org> wrote:
> > >
> > > In preparation for introducing a separate ASID for L2, introduce a
> > > 'placeholder' ASID that is still the same as L1's ASID. This will
> > > facilitate future changes that need to distinguish L1 and L2's ASIDs,
> > > before actually using a different ASID for L1 and L2.
> > >
> > > No functional change intended.
> > >
> > > Signed-off-by: Yosry Ahmed <yosry@kernel.org>
> > > ---
> > >  arch/x86/kvm/svm/nested.c | 5 +++--
> > >  arch/x86/kvm/svm/svm.h    | 2 ++
> > >  2 files changed, 5 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
> > > index ed4af5a08f794..eb60d6b959d86 100644
> > > --- a/arch/x86/kvm/svm/nested.c
> > > +++ b/arch/x86/kvm/svm/nested.c
> > > @@ -698,7 +698,6 @@ static void nested_svm_transition_tlb_flush(struct kvm_vcpu *vcpu)
> > >          *  - Honor L1's request to flush an ASID on nested VMRUN
> > >          *  - Sync nested NPT MMU on VMRUN that flushes L2's ASID[*]
> > >          *  - Don't crush a pending TLB flush in vmcb02 on nested VMRUN
> > > -        *  - Flush L1's ASID on KVM_REQ_TLB_FLUSH_GUEST
> > >          *
> > >          * [*] Unlike nested EPT, SVM's ASID management can invalidate nested
> > >          *     NPT guest-physical mappings on VMRUN.
> > > @@ -928,7 +927,7 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm)
> > >         else
> > >                 vmcb02->control.bus_lock_counter = 0;
> > >
> > > -       vmcb02->control.asid = vmcb01->control.asid;
> > > +       vmcb02->control.asid = svm->nested.asid02;
> > >
> > >         /* Overwritten later if necessary.  */
> > >         vmcb_clr_flush_asid(vmcb02);
> > > @@ -1498,6 +1497,8 @@ int svm_allocate_nested(struct vcpu_svm *svm)
> > >         if (!svm->nested.msrpm)
> > >                 goto err_free_vmcb02;
> > >
> > > +       svm->nested.asid02 = svm->asid;
> > > +
> > 
> > From internal Sashiko:
> > ---
> > Does caching the ASID here risk leaving nested.asid02 stale during an
> > intra-host SEV VM migration?
> > 
> > If userspace restores EFER.SVME on a destination vCPU before completing
> > the migration, svm_allocate_nested() is called and nested.asid02 is
> > initialized with a normal non-SEV TLB tag.
> > 
> > Later, when KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM completes the migration,
> > sev_migrate_from() updates the primary ASID but misses this new
> > nested.asid02 field:
> > 
> > arch/x86/kvm/svm/sev.c:sev_migrate_from() {
> >         ...
> >         dst_svm->asid = dst->asid;
> >         sev_init_vmcb(dst_svm, false);
> >         ...
> > }
> > 
> > When L1 subsequently executes VMRUN to enter L2,
> > nested_vmcb02_prepare_control() will assign this stale non-SEV
> > nested.asid02 to vmcb02->control.asid.
> > 
> > Does this mismatch cause L2 to run with an invalid non-SEV ASID while the
> > VMCB's SEV bit is set, resulting in an unconditional #VMEXIT_INVALID
> > hardware fault?
> > ---
> > 
> > I am starting to hate SEV..
> > 
> > This would also be correct if the vCPUs are created on the destination
> > VM before it's marked as an SEV VM. This is a recurring theme at this
> > point.
> 
> So for this problem and the other two SEV migration problems reported in
> patch #10, I am thinking we end up with something like this over both
> patches (untested) -- any feedback would be appreciated as I have ~0
> knowledge of SEV and ~0 testing capabilities:
> 
> diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
> index 087d18a5eda69..09a0572996352 100644
> --- a/arch/x86/kvm/svm/sev.c
> +++ b/arch/x86/kvm/svm/sev.c
> @@ -2019,6 +2019,29 @@ static void sev_unlock_two_vms(struct kvm *dst_kvm, struct kvm *src_kvm)
>         atomic_set_release(&src_sev->migration_in_progress, 0);
>  }
> 
> +static void sev_vcpu_migrate_asid(struct vcpu_svm *dst_svm, struct vcpu_svm *src_svm,
> +                                 unsigned int asid)
> +{
> +       /*
> +        * Free the (potentially non-SEV) ASID on the destination vCPU before
> +        * setting the new SEV ASID. Clear the ASID on the source vCPU to avoid
> +        * mistakenly attempting to free an SEV ASID after the vCPU is no longer
> +        * an SEV vCPU.
> +        */
> +       free_asid(dst_svm->asid);

Actually this doesn't solve the problem of leaking the non-SEV ASID,
because is_sev_guest() will return true for the destination vCPU at this
point AFAICT.

It's probably best to have free_asid() not key off is_sev_guest() at
all, and explicitly check if the ASID is reserved. Something like:

static inline void free_asid(kvm_tlb_tag_t asid)
{
	if (asid >= svm_nr_reserved_asids && asid != fallback_asid)
                kvm_free_tlb_tag(asid);
}

, or even better, we can have fallback_asid be a reserved ASID as well
(if all ASIDs are not allocated to SEV), then we can just check reserved
ASIDs and we don't have to worry about freeing the fallback_asid.

> +       dst_svm->asid = asid;
> +       src_svm->asid = 0;

..and the above would also remove te need to clear src_svm->asid here.

> +
> +       /*
> +        * If nested is already initialized on the destination vCPU, update the
> +        * nested ASID as well to match the new SEV ASID.
> +        */
> +       if (dst_svm->nested.vmcb02.ptr) {
> +               dst_svm->nested.asid02 = asid;
> +               dst_svm->nested.vmcb02.ptr->control.asid = asid;
> +       }
> +}
> +
>  static void sev_migrate_from(struct kvm *dst_kvm, struct kvm *src_kvm)
>  {
>         struct kvm_sev_info *dst = to_kvm_sev_info(dst_kvm);
> @@ -2071,12 +2094,6 @@ static void sev_migrate_from(struct kvm *dst_kvm, struct kvm *src_kvm)
>         kvm_for_each_vcpu(i, dst_vcpu, dst_kvm) {
>                 dst_svm = to_svm(dst_vcpu);
> 
> -               dst_svm->asid = dst->asid;
> -               sev_init_vmcb(dst_svm, false);
> -
> -               if (!dst->es_active)
> -                       continue;
> -
>                 /*
>                  * Note, the source is not required to have the same number of
>                  * vCPUs as the destination when migrating a vanilla SEV VM.
> @@ -2084,6 +2101,12 @@ static void sev_migrate_from(struct kvm *dst_kvm, struct kvm *src_kvm)
>                 src_vcpu = kvm_get_vcpu(src_kvm, i);
>                 src_svm = to_svm(src_vcpu);
> 
> +               sev_vcpu_migrate_asid(dst_svm, src_svm, dst->asid);
> +               sev_init_vmcb(dst_svm, false);
> +
> +               if (!dst->es_active)
> +                       continue;
> +
>                 /*
>                  * Transfer VMSA and GHCB state to the destination.  Nullify and
>                  * clear source fields as appropriate, the state now belongs to
> 
> 

  reply	other threads:[~2026-08-05 15:25 UTC|newest]

Thread overview: 45+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-07-28  0:35 [PATCH v1 00/28] KVM: nSVM: Optimize nSVM TLB flushes Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 01/28] KVM: nSVM: Flush the TLB after forcefully leaving nested Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 02/28] KVM: SVM: Document number of ASIDs CPUID setting Yosry Ahmed
2026-08-01  6:10   ` Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 03/28] KVM: VMX: Generalize VPID allocation to be vendor-neutral Yosry Ahmed
2026-08-01  6:13   ` Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 04/28] KVM: x86/mmu: Support specifying reserved TLB tags Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 05/28] KVM: SVM: Add helpers to set/clear ASID flush in VMCB Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 06/28] KVM: SVM: Fallback to flush everything if FLUSHBYASID is not available Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 07/28] KVM: SVM: Duplicate pre-run ASID check for SEV and non-SEV guests Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 08/28] KVM: SEV: Do ASID initialization at VMCB initialization Yosry Ahmed
2026-08-01  6:15   ` Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 09/28] KVM: SEV: Expose sev_get_asid() outside of sev.c Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 10/28] KVM: SVM: Use a static ASID per vCPU Yosry Ahmed
2026-08-01  6:18   ` Yosry Ahmed
2026-08-01  6:25   ` Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 11/28] KVM: SVM: Only flush the fallback ASID when used by a different vCPU Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 12/28] KVM: nSVM: Add a placeholder ASID for L2 Yosry Ahmed
2026-08-01  6:34   ` Yosry Ahmed
2026-08-05  0:26     ` Yosry Ahmed
2026-08-05 15:25       ` Yosry Ahmed [this message]
2026-07-28  0:35 ` [PATCH v1 13/28] KVM: x86: hyper-v: Rename kvm_hv_vcpu_purge_flush_tlb() Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 14/28] KVM: x86: hyper-v: Allow puring all TLB flush FIFOs Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 15/28] KVM: nSVM: Drop svm->nested.initialized Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 16/28] KVM: nSVM: Flush both L1 and L2 ASIDs on KVM_REQ_TLB_FLUSH Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 17/28] KVM: nSVM: Always switch VMCB before leaving guest mode Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 18/28] KVM: nSVM: Split nested_svm_transition_tlb_flush() into entry/exit fns Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 19/28] KVM: nSVM: Service local TLB flushes before nested transitions Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 20/28] KVM: nSVM: Handle nested TLB flush requests through TLB_CONTROL Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 21/28] KVM: nSVM: Flush the TLB if L1 changes L2's ASID in vmcb12 Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 22/28] KVM: nSVM: Do not reset TLB_CONTROL in vmcb02 on nested VM-Enter Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 23/28] KVM: x86/mmu: Rename __kvm_mmu_invalidate_addr() to kvm_mmu_sync_addr() Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 24/28] KVM: x86/mmu: Refactor kvm_mmu_invlpg() to allow skipping the GVA flush Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 25/28] KVM: nSVM: Flush L2's ASID when emulating INVLPGA Yosry Ahmed
2026-08-01  6:41   ` Yosry Ahmed
2026-08-03 17:21     ` Yosry Ahmed
2026-08-03 22:27       ` Sean Christopherson
2026-07-28  0:35 ` [PATCH v1 26/28] KVM: nSVM: Flush the ASID on nested transitions if shared by L1 and L2 Yosry Ahmed
2026-08-01  6:36   ` Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 27/28] KVM: nSVM: Use different ASIDs for " Yosry Ahmed
2026-07-28  0:35 ` [PATCH v1 28/28] KVM: selftests: Add a test for nested TLB flushes Yosry Ahmed
2026-07-28  1:05 ` [PATCH v1 00/28] KVM: nSVM: Optimize nSVM " Yosry Ahmed
2026-07-28  1:18   ` Sean Christopherson
2026-07-30 21:52     ` Yosry Ahmed
2026-08-01  6:39     ` Yosry Ahmed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=anNSYpoXYXoGBewo@google.com \
    --to=yosry@kernel.org \
    --cc=jmattson@google.com \
    --cc=kvm@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mlevitsk@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=seanjc@google.com \
    --cc=thomas.lendacky@amd.com \
    --cc=vkuznets@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox