[PATCH 1/3] KVM: nVMX: Always flush vpid02 on first use

[PATCH 1/3] KVM: nVMX: Always flush vpid02 on first use

[Yosry Ahmed]

Make sure vpid02 is always flushed on first use by setting last_vpid=0
when allocating vpid02.  nested_vmx_transition_tlb_flush() will always
detect a VPID change on first VM-Enter after VMXON, because VPID=0 in
vmcb12 is not allowed if L1 enables VPID.

This avoids using stale TLB entries from a previous lifetime of the
VPID, that might have been associated with a different vCPU (or a
completely different VM).

Note that last_vpid is already being initialized as 0 when the vCPU is
created, but it is not reset when vpid02 is freed on VMXOFF. Hence, the
problem can only occur if L1 does VMXOFF -> VMXON, runs an L2, and KVM
happens to reuse a VPID that has TLB entries on the physical CPU.

Cc: stable@vger.kernel.org
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
---
 arch/x86/kvm/vmx/nested.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index b2c851cc7d5c8..a49115d9a5a54 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -1290,6 +1290,9 @@ static void nested_vmx_transition_tlb_flush(struct kvm_vcpu *vcpu,
 	 * is the VPID incorporated into the MMU context.  I.e. KVM must assume
 	 * that the new vpid12 has never been used and thus represents a new
 	 * guest ASID that cannot have entries in the TLB.
+	 *
+	 * Note, last_vpid is initialized as 0, so the first nested VM-Enter
+	 * after VMXON will always flush the TLB to avoid using stale entries.
 	 */
 	if (is_vmenter && vmcs12->virtual_processor_id != vmx->nested.last_vpid) {
 		vmx->nested.last_vpid = vmcs12->virtual_processor_id;
@@ -5447,6 +5450,13 @@ static int enter_vmx_operation(struct kvm_vcpu *vcpu)
 
 	vmx->nested.vpid02 = allocate_vpid();
 
+	/*
+	 * Clear last_vpid to ensure that the VPID is flushed on the first
+	 * nested VM-Enter. Otherwise, stale TLB entries from a previous life of
+	 * the VPID (e.g. different vCPU or even different VM) could be used.
+	 */
+	vmx->nested.last_vpid = 0;
+
 	vmx->nested.vmcs02_initialized = false;
 	vmx->nested.vmxon = true;
 
-- 
2.54.0.1136.gdb2ca164c4-goog

Re: [PATCH 1/3] KVM: nVMX: Always flush vpid02 on first use

[Jim Mattson]

On Tue, Jun 16, 2026 at 2:47 PM Yosry Ahmed <yosry@kernel.org> wrote:
>
> Make sure vpid02 is always flushed on first use by setting last_vpid=0
> when allocating vpid02.  nested_vmx_transition_tlb_flush() will always
> detect a VPID change on first VM-Enter after VMXON, because VPID=0 in
> vmcb12 is not allowed if L1 enables VPID.
>
> This avoids using stale TLB entries from a previous lifetime of the
> VPID, that might have been associated with a different vCPU (or a
> completely different VM).
>
> Note that last_vpid is already being initialized as 0 when the vCPU is
> created, but it is not reset when vpid02 is freed on VMXOFF. Hence, the
> problem can only occur if L1 does VMXOFF -> VMXON, runs an L2, and KVM
> happens to reuse a VPID that has TLB entries on the physical CPU.
>
> Cc: stable@vger.kernel.org
> Signed-off-by: Yosry Ahmed <yosry@kernel.org>
> ---
Reviewed-by: Jim Mattson <jmattson@google.com>

Re: [PATCH 1/3] KVM: nVMX: Always flush vpid02 on first use

[Huang, Kai]

On Tue, 2026-06-16 at 21:46 +0000, Yosry Ahmed wrote:
> Make sure vpid02 is always flushed on first use by setting last_vpid=0
> when allocating vpid02.  nested_vmx_transition_tlb_flush() will always
> detect a VPID change on first VM-Enter after VMXON, because VPID=0 in
> vmcb12 is not allowed if L1 enables VPID.

vmcs12 :-)

> 
> This avoids using stale TLB entries from a previous lifetime of the
> VPID, that might have been associated with a different vCPU (or a
> completely different VM).
> 
> Note that last_vpid is already being initialized as 0 when the vCPU is
> created, but it is not reset when vpid02 is freed on VMXOFF. Hence, the
> problem can only occur if L1 does VMXOFF -> VMXON, runs an L2, and KVM
> happens to reuse a VPID that has TLB entries on the physical CPU.

Not sure whether it's better to set it to 0 in free_nested(), which also resets
some other nested fields to clean slate AFAICT?

> 
> Cc: stable@vger.kernel.org
> Signed-off-by: Yosry Ahmed <yosry@kernel.org>

Anyway:

Reviewed-by: Kai Huang <kai.huang@intel.com>

Re: [PATCH 1/3] KVM: nVMX: Always flush vpid02 on first use

[Sean Christopherson]

On Wed, Jun 17, 2026, Kai Huang wrote:
> On Tue, 2026-06-16 at 21:46 +0000, Yosry Ahmed wrote:
> > Make sure vpid02 is always flushed on first use by setting last_vpid=0
> > when allocating vpid02.  nested_vmx_transition_tlb_flush() will always
> > detect a VPID change on first VM-Enter after VMXON, because VPID=0 in
> > vmcb12 is not allowed if L1 enables VPID.
> 
> vmcs12 :-)
> 
> > 
> > This avoids using stale TLB entries from a previous lifetime of the
> > VPID, that might have been associated with a different vCPU (or a
> > completely different VM).
> > 
> > Note that last_vpid is already being initialized as 0 when the vCPU is
> > created, but it is not reset when vpid02 is freed on VMXOFF. Hence, the
> > problem can only occur if L1 does VMXOFF -> VMXON, runs an L2, and KVM
> > happens to reuse a VPID that has TLB entries on the physical CPU.
> 
> Not sure whether it's better to set it to 0 in free_nested(), which also resets
> some other nested fields to clean slate AFAICT?

It needs to be set on first use, for the same reason that kvm_mmu_load() flushes
the root:

	/*
	 * Flush any TLB entries for the new root, the provenance of the root
	 * is unknown.  Even if KVM ensures there are no stale TLB entries
	 * for a freed root, in theory another hypervisor could have left
	 * stale entries.  Flushing on alloc also allows KVM to skip the TLB
	 * flush when freeing a root (see kvm_tdp_mmu_put_root()).
	 */
	kvm_x86_call(flush_tlb_current)(vcpu);

Re: [PATCH 1/3] KVM: nVMX: Always flush vpid02 on first use

[Huang, Kai]

On Wed, 2026-06-17 at 06:03 -0700, Sean Christopherson wrote:
> On Wed, Jun 17, 2026, Kai Huang wrote:
> > On Tue, 2026-06-16 at 21:46 +0000, Yosry Ahmed wrote:
> > > Make sure vpid02 is always flushed on first use by setting last_vpid=0
> > > when allocating vpid02.  nested_vmx_transition_tlb_flush() will always
> > > detect a VPID change on first VM-Enter after VMXON, because VPID=0 in
> > > vmcb12 is not allowed if L1 enables VPID.
> > 
> > vmcs12 :-)
> > 
> > > 
> > > This avoids using stale TLB entries from a previous lifetime of the
> > > VPID, that might have been associated with a different vCPU (or a
> > > completely different VM).
> > > 
> > > Note that last_vpid is already being initialized as 0 when the vCPU is
> > > created, but it is not reset when vpid02 is freed on VMXOFF. Hence, the
> > > problem can only occur if L1 does VMXOFF -> VMXON, runs an L2, and KVM
> > > happens to reuse a VPID that has TLB entries on the physical CPU.
> > 
> > Not sure whether it's better to set it to 0 in free_nested(), which also resets
> > some other nested fields to clean slate AFAICT?
> 
> It needs to be set on first use, for the same reason that kvm_mmu_load() flushes
> the root:
> 
> 	/*
> 	 * Flush any TLB entries for the new root, the provenance of the root
> 	 * is unknown.  Even if KVM ensures there are no stale TLB entries
> 	 * for a freed root, in theory another hypervisor could have left
> 	 * stale entries.  Flushing on alloc also allows KVM to skip the TLB
> 	 * flush when freeing a root (see kvm_tdp_mmu_put_root()).
> 	 */
> 	kvm_x86_call(flush_tlb_current)(vcpu);

I think you mean the "actual flush" needs to be done on the first use.  But
setting last_vpid to 0 is a setting which is to make sure the actual flush will
always be done on the first use, i.e., the actual flush will always be done on
the first use.  For this purpose seems to me there's no difference between
setting last_vpid to 0 in enter_vmx_operation() and free_nested(), but maybe I
am missing something.

But I guess doing it in enter_vmx_operation() matches the logic of "doing actual
flush on first use" more :-)

Re: [PATCH 1/3] KVM: nVMX: Always flush vpid02 on first use

[Yosry Ahmed]

On Wed, Jun 17, 2026 at 3:03 PM Huang, Kai <kai.huang@intel.com> wrote:
>
> On Wed, 2026-06-17 at 06:03 -0700, Sean Christopherson wrote:
> > On Wed, Jun 17, 2026, Kai Huang wrote:
> > > On Tue, 2026-06-16 at 21:46 +0000, Yosry Ahmed wrote:
> > > > Make sure vpid02 is always flushed on first use by setting last_vpid=0
> > > > when allocating vpid02.  nested_vmx_transition_tlb_flush() will always
> > > > detect a VPID change on first VM-Enter after VMXON, because VPID=0 in
> > > > vmcb12 is not allowed if L1 enables VPID.
> > >
> > > vmcs12 :-)
> > >
> > > >
> > > > This avoids using stale TLB entries from a previous lifetime of the
> > > > VPID, that might have been associated with a different vCPU (or a
> > > > completely different VM).
> > > >
> > > > Note that last_vpid is already being initialized as 0 when the vCPU is
> > > > created, but it is not reset when vpid02 is freed on VMXOFF. Hence, the
> > > > problem can only occur if L1 does VMXOFF -> VMXON, runs an L2, and KVM
> > > > happens to reuse a VPID that has TLB entries on the physical CPU.
> > >
> > > Not sure whether it's better to set it to 0 in free_nested(), which also resets
> > > some other nested fields to clean slate AFAICT?
> >
> > It needs to be set on first use, for the same reason that kvm_mmu_load() flushes
> > the root:
> >
> >       /*
> >        * Flush any TLB entries for the new root, the provenance of the root
> >        * is unknown.  Even if KVM ensures there are no stale TLB entries
> >        * for a freed root, in theory another hypervisor could have left
> >        * stale entries.  Flushing on alloc also allows KVM to skip the TLB
> >        * flush when freeing a root (see kvm_tdp_mmu_put_root()).
> >        */
> >       kvm_x86_call(flush_tlb_current)(vcpu);
>
> I think you mean the "actual flush" needs to be done on the first use.  But
> setting last_vpid to 0 is a setting which is to make sure the actual flush will
> always be done on the first use, i.e., the actual flush will always be done on
> the first use.  For this purpose seems to me there's no difference between
> setting last_vpid to 0 in enter_vmx_operation() and free_nested(), but maybe I
> am missing something.
>
> But I guess doing it in enter_vmx_operation() matches the logic of "doing actual
> flush on first use" more :-)

Yup. I thought about putting it free_nested() as it looks like
cleanup, but semantically it makes more sense to put it in
enter_vmx_operation().

Re: [PATCH 1/3] KVM: nVMX: Always flush vpid02 on first use

[Huang, Kai]

On Wed, 2026-06-17 at 15:04 -0700, Yosry Ahmed wrote:
> On Wed, Jun 17, 2026 at 3:03 PM Huang, Kai <kai.huang@intel.com> wrote:
> > 
> > On Wed, 2026-06-17 at 06:03 -0700, Sean Christopherson wrote:
> > > On Wed, Jun 17, 2026, Kai Huang wrote:
> > > > On Tue, 2026-06-16 at 21:46 +0000, Yosry Ahmed wrote:
> > > > > Make sure vpid02 is always flushed on first use by setting last_vpid=0
> > > > > when allocating vpid02.  nested_vmx_transition_tlb_flush() will always
> > > > > detect a VPID change on first VM-Enter after VMXON, because VPID=0 in
> > > > > vmcb12 is not allowed if L1 enables VPID.
> > > > 
> > > > vmcs12 :-)
> > > > 
> > > > > 
> > > > > This avoids using stale TLB entries from a previous lifetime of the
> > > > > VPID, that might have been associated with a different vCPU (or a
> > > > > completely different VM).
> > > > > 
> > > > > Note that last_vpid is already being initialized as 0 when the vCPU is
> > > > > created, but it is not reset when vpid02 is freed on VMXOFF. Hence, the
> > > > > problem can only occur if L1 does VMXOFF -> VMXON, runs an L2, and KVM
> > > > > happens to reuse a VPID that has TLB entries on the physical CPU.
> > > > 
> > > > Not sure whether it's better to set it to 0 in free_nested(), which also resets
> > > > some other nested fields to clean slate AFAICT?
> > > 
> > > It needs to be set on first use, for the same reason that kvm_mmu_load() flushes
> > > the root:
> > > 
> > >       /*
> > >        * Flush any TLB entries for the new root, the provenance of the root
> > >        * is unknown.  Even if KVM ensures there are no stale TLB entries
> > >        * for a freed root, in theory another hypervisor could have left
> > >        * stale entries.  Flushing on alloc also allows KVM to skip the TLB
> > >        * flush when freeing a root (see kvm_tdp_mmu_put_root()).
> > >        */
> > >       kvm_x86_call(flush_tlb_current)(vcpu);
> > 
> > I think you mean the "actual flush" needs to be done on the first use.  But
> > setting last_vpid to 0 is a setting which is to make sure the actual flush will
> > always be done on the first use, i.e., the actual flush will always be done on
> > the first use.  For this purpose seems to me there's no difference between
> > setting last_vpid to 0 in enter_vmx_operation() and free_nested(), but maybe I
> > am missing something.
> > 
> > But I guess doing it in enter_vmx_operation() matches the logic of "doing actual
> > flush on first use" more :-)
> 
> Yup. I thought about putting it free_nested() as it looks like
> cleanup, but semantically it makes more sense to put it in
> enter_vmx_operation().

Sounds good to me.  :-)

Re: [PATCH 1/3] KVM: nVMX: Always flush vpid02 on first use

[Yosry Ahmed]

> > > I think you mean the "actual flush" needs to be done on the first use.  But
> > > setting last_vpid to 0 is a setting which is to make sure the actual flush will
> > > always be done on the first use, i.e., the actual flush will always be done on
> > > the first use.  For this purpose seems to me there's no difference between
> > > setting last_vpid to 0 in enter_vmx_operation() and free_nested(), but maybe I
> > > am missing something.
> > >
> > > But I guess doing it in enter_vmx_operation() matches the logic of "doing actual
> > > flush on first use" more :-)
> >
> > Yup. I thought about putting it free_nested() as it looks like
> > cleanup, but semantically it makes more sense to put it in
> > enter_vmx_operation().
>
> Sounds good to me.  :-)

Thanks for taking a look and reviewing!

Re: [PATCH 1/3] KVM: nVMX: Always flush vpid02 on first use

[Sean Christopherson]

On Wed, Jun 17, 2026, Yosry Ahmed wrote:
> On Wed, Jun 17, 2026 at 3:03 PM Huang, Kai <kai.huang@intel.com> wrote:
> >
> > On Wed, 2026-06-17 at 06:03 -0700, Sean Christopherson wrote:
> > > On Wed, Jun 17, 2026, Kai Huang wrote:
> > > > On Tue, 2026-06-16 at 21:46 +0000, Yosry Ahmed wrote:
> > > > > Make sure vpid02 is always flushed on first use by setting last_vpid=0
> > > > > when allocating vpid02.  nested_vmx_transition_tlb_flush() will always
> > > > > detect a VPID change on first VM-Enter after VMXON, because VPID=0 in
> > > > > vmcb12 is not allowed if L1 enables VPID.
> > > >
> > > > vmcs12 :-)
> > > >
> > > > >
> > > > > This avoids using stale TLB entries from a previous lifetime of the
> > > > > VPID, that might have been associated with a different vCPU (or a
> > > > > completely different VM).
> > > > >
> > > > > Note that last_vpid is already being initialized as 0 when the vCPU is
> > > > > created, but it is not reset when vpid02 is freed on VMXOFF. Hence, the
> > > > > problem can only occur if L1 does VMXOFF -> VMXON, runs an L2, and KVM
> > > > > happens to reuse a VPID that has TLB entries on the physical CPU.
> > > >
> > > > Not sure whether it's better to set it to 0 in free_nested(), which also resets
> > > > some other nested fields to clean slate AFAICT?
> > >
> > > It needs to be set on first use, for the same reason that kvm_mmu_load() flushes
> > > the root:
> > >
> > >       /*
> > >        * Flush any TLB entries for the new root, the provenance of the root
> > >        * is unknown.  Even if KVM ensures there are no stale TLB entries
> > >        * for a freed root, in theory another hypervisor could have left
> > >        * stale entries.  Flushing on alloc also allows KVM to skip the TLB
> > >        * flush when freeing a root (see kvm_tdp_mmu_put_root()).
> > >        */
> > >       kvm_x86_call(flush_tlb_current)(vcpu);
> >
> > I think you mean the "actual flush" needs to be done on the first use.  But
> > setting last_vpid to 0 is a setting which is to make sure the actual flush will
> > always be done on the first use, i.e., the actual flush will always be done on
> > the first use.  For this purpose seems to me there's no difference between
> > setting last_vpid to 0 in enter_vmx_operation() and free_nested(), but maybe I
> > am missing something.

You're not missing anything.  It's just that putting it in free_nested() subtly
relies on zero-allocating vmx->nested, so that the *very* first use also flushes
vpid02.  Relying on zero-allocating is generally a-ok, but in this case it would
require documenting the same base logic in multiple places.

> > But I guess doing it in enter_vmx_operation() matches the logic of "doing actual
> > flush on first use" more :-)
> 
> Yup. I thought about putting it free_nested() as it looks like
> cleanup, but semantically it makes more sense to put it in
> enter_vmx_operation().

Re: [PATCH 1/3] KVM: nVMX: Always flush vpid02 on first use

[Huang, Kai]

On Wed, 2026-06-17 at 15:26 -0700, Sean Christopherson wrote:
> > > I think you mean the "actual flush" needs to be done on the first use.  But
> > > setting last_vpid to 0 is a setting which is to make sure the actual flush will
> > > always be done on the first use, i.e., the actual flush will always be done on
> > > the first use.  For this purpose seems to me there's no difference between
> > > setting last_vpid to 0 in enter_vmx_operation() and free_nested(), but maybe I
> > > am missing something.
> 
> You're not missing anything.  It's just that putting it in free_nested() subtly
> relies on zero-allocating vmx->nested, so that the *very* first use also flushes
> vpid02.  Relying on zero-allocating is generally a-ok, but in this case it would
> require documenting the same base logic in multiple places.

Right. Agreed.

[PATCH 2/3] KVM: nVMX: Decouple INVVPID operand checks from flushing of vpid02

[Yosry Ahmed]

From: Sean Christopherson <seanjc@google.com>

Separate the INVVPID operand checks from the actual flushing of vpid02 so
the flushing can be adjusted to do the right thing when vmcs12 was last
loaded on a different pCPU, without having to duplicate the logic across
multiple case-statements.

Opportunistically let the VM-Fail paths poke out past 80 chars.

No functional change intended.

Cc: stable@vger.kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
---
 arch/x86/kvm/vmx/nested.c | 43 ++++++++++++---------------------------
 1 file changed, 13 insertions(+), 30 deletions(-)

diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index a49115d9a5a54..596dec7ba2b78 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -6084,7 +6084,6 @@ static int handle_invvpid(struct kvm_vcpu *vcpu)
 		u64 vpid;
 		u64 gla;
 	} operand;
-	u16 vpid02;
 	int r, gpr_index;
 
 	if (!(vmx->nested.msrs.secondary_ctls_high &
@@ -6119,8 +6118,15 @@ static int handle_invvpid(struct kvm_vcpu *vcpu)
 		return kvm_handle_memory_failure(vcpu, r, &e);
 
 	if (operand.vpid >> 16)
-		return nested_vmx_fail(vcpu,
-			VMXERR_INVALID_OPERAND_TO_INVEPT_INVVPID);
+		return nested_vmx_fail(vcpu, VMXERR_INVALID_OPERAND_TO_INVEPT_INVVPID);
+
+	if (type != VMX_VPID_EXTENT_ALL_CONTEXT && !operand.vpid)
+		return nested_vmx_fail(vcpu, VMXERR_INVALID_OPERAND_TO_INVEPT_INVVPID);
+
+	/* LAM doesn't apply to addresses that are inputs to TLB invalidation. */
+	if (type == VMX_VPID_EXTENT_INDIVIDUAL_ADDR &&
+	    is_noncanonical_invlpg_address(operand.gla, vcpu))
+		return nested_vmx_fail(vcpu, VMXERR_INVALID_OPERAND_TO_INVEPT_INVVPID);
 
 	/*
 	 * Always flush the effective vpid02, i.e. never flush the current VPID
@@ -6128,33 +6134,10 @@ static int handle_invvpid(struct kvm_vcpu *vcpu)
 	 * VMCS, and so whether or not the current vmcs12 has VPID enabled is
 	 * irrelevant (and there may not be a loaded vmcs12).
 	 */
-	vpid02 = nested_get_vpid02(vcpu);
-	switch (type) {
-	case VMX_VPID_EXTENT_INDIVIDUAL_ADDR:
-		/*
-		 * LAM doesn't apply to addresses that are inputs to TLB
-		 * invalidation.
-		 */
-		if (!operand.vpid ||
-		    is_noncanonical_invlpg_address(operand.gla, vcpu))
-			return nested_vmx_fail(vcpu,
-				VMXERR_INVALID_OPERAND_TO_INVEPT_INVVPID);
-		vpid_sync_vcpu_addr(vpid02, operand.gla);
-		break;
-	case VMX_VPID_EXTENT_SINGLE_CONTEXT:
-	case VMX_VPID_EXTENT_SINGLE_NON_GLOBAL:
-		if (!operand.vpid)
-			return nested_vmx_fail(vcpu,
-				VMXERR_INVALID_OPERAND_TO_INVEPT_INVVPID);
-		vpid_sync_context(vpid02);
-		break;
-	case VMX_VPID_EXTENT_ALL_CONTEXT:
-		vpid_sync_context(vpid02);
-		break;
-	default:
-		WARN_ON_ONCE(1);
-		return kvm_skip_emulated_instruction(vcpu);
-	}
+	if (type == VMX_VPID_EXTENT_INDIVIDUAL_ADDR)
+		vpid_sync_vcpu_addr(nested_get_vpid02(vcpu), operand.gla);
+	else
+		vpid_sync_context(nested_get_vpid02(vcpu));
 
 	/*
 	 * Sync the shadow page tables if EPT is disabled, L1 is invalidating
-- 
2.54.0.1136.gdb2ca164c4-goog

Re: [PATCH 2/3] KVM: nVMX: Decouple INVVPID operand checks from flushing of vpid02

[Huang, Kai]

On Tue, 2026-06-16 at 21:46 +0000, Yosry Ahmed wrote:
> From: Sean Christopherson <seanjc@google.com>
> 
> Separate the INVVPID operand checks from the actual flushing of vpid02 so
> the flushing can be adjusted to do the right thing when vmcs12 was last

Nit: vmcs12 or vmcs02?

> loaded on a different pCPU, without having to duplicate the logic across
> multiple case-statements.
> 
> Opportunistically let the VM-Fail paths poke out past 80 chars.
> 
> No functional change intended.
> 
> Cc: stable@vger.kernel.org
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> Signed-off-by: Yosry Ahmed <yosry@kernel.org>

Reviewed-by: Kai Huang <kai.huang@intel.com>

Re: [PATCH 2/3] KVM: nVMX: Decouple INVVPID operand checks from flushing of vpid02

[Yosry Ahmed]

On Thu, Jun 18, 2026 at 3:57 AM Huang, Kai <kai.huang@intel.com> wrote:
>
> On Tue, 2026-06-16 at 21:46 +0000, Yosry Ahmed wrote:
> > From: Sean Christopherson <seanjc@google.com>
> >
> > Separate the INVVPID operand checks from the actual flushing of vpid02 so
> > the flushing can be adjusted to do the right thing when vmcs12 was last
>
> Nit: vmcs12 or vmcs02?

I think vmcs02 is more accurate here, good eye.

>
> > loaded on a different pCPU, without having to duplicate the logic across
> > multiple case-statements.
> >
> > Opportunistically let the VM-Fail paths poke out past 80 chars.
> >
> > No functional change intended.
> >
> > Cc: stable@vger.kernel.org
> > Signed-off-by: Sean Christopherson <seanjc@google.com>
> > Signed-off-by: Yosry Ahmed <yosry@kernel.org>
>
> Reviewed-by: Kai Huang <kai.huang@intel.com>

Thank you!

[PATCH 3/3] KVM: nVM: Ensure INVVPID is emulated on the correct physical CPU

[Yosry Ahmed]

When emulating INVVPID, KVM executes INVVPID on the physical CPU using
vpid02 (instead of the L1 assigned VPID), after doing some validations
on the operands. However, it is possible that the physical CPU KVM
executes INVVPID on is different from the CPU L2 is running on.

For example, in the following scenario:
- L2 runs on CPU #1 and exits to L1 (vmx->nested.vmcs02.cpu=1)
- L1 migrates to CPU #2 and executes INVVPID
- KVM executes INVVPID on CPU #2
- L1 migrates back to CPU #1 and runs L2 (vmx->nested.vmcs02.cpu=1)

The TLB entries on CPU #1 are never invalidated, because INVVPID was
executed on CPU #2, and vmcs02 never ran on a different pCPU (i.e.
vmx_vcpu_load_vmcs() will *not* request KVM_REQ_TLB_FLUSH).

Ensure that INVVPID is being executed on the same pCPU that L2 last ran
on, and if not, fallback to clearing last_vpid=0 to trigger a full VPID
flush on the next nested VM-Enter (as KVM will detect L1 using a
different VPID for L2). If L2 ends up running on a different pCPU, KVM
will flush the TLB anyway through vmx_vcpu_load_vmcs().

Cc: stable@vger.kernel.org
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
---
 arch/x86/kvm/vmx/nested.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index 596dec7ba2b78..2d1cd2c2a46d7 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -6085,6 +6085,7 @@ static int handle_invvpid(struct kvm_vcpu *vcpu)
 		u64 gla;
 	} operand;
 	int r, gpr_index;
+	int cpu;
 
 	if (!(vmx->nested.msrs.secondary_ctls_high &
 	      SECONDARY_EXEC_ENABLE_VPID) ||
@@ -6133,11 +6134,19 @@ static int handle_invvpid(struct kvm_vcpu *vcpu)
 	 * and never explicitly flush vpid01.  INVVPID targets a VPID, not a
 	 * VMCS, and so whether or not the current vmcs12 has VPID enabled is
 	 * irrelevant (and there may not be a loaded vmcs12).
+	 *
+	 * If vmcs02 was last loaded on a different pCPU, then defer the flush
+	 * by invalidating the nested VPID tracking to ensure that KVM performs
+	 * the invalidation on the correct pCPU.
 	 */
-	if (type == VMX_VPID_EXTENT_INDIVIDUAL_ADDR)
+	cpu = get_cpu();
+	if (cpu != vmx->nested.vmcs02.cpu)
+		vmx->nested.last_vpid = 0;
+	else if (type == VMX_VPID_EXTENT_INDIVIDUAL_ADDR)
 		vpid_sync_vcpu_addr(nested_get_vpid02(vcpu), operand.gla);
 	else
 		vpid_sync_context(nested_get_vpid02(vcpu));
+	put_cpu();
 
 	/*
 	 * Sync the shadow page tables if EPT is disabled, L1 is invalidating
-- 
2.54.0.1136.gdb2ca164c4-goog

Re: [PATCH 3/3] KVM: nVM: Ensure INVVPID is emulated on the correct physical CPU

[Huang, Kai]

On Tue, 2026-06-16 at 21:46 +0000, Yosry Ahmed wrote:
> When emulating INVVPID, KVM executes INVVPID on the physical CPU using
> vpid02 (instead of the L1 assigned VPID), after doing some validations
> on the operands. However, it is possible that the physical CPU KVM
> executes INVVPID on is different from the CPU L2 is running on.
> 
> For example, in the following scenario:
> - L2 runs on CPU #1 and exits to L1 (vmx->nested.vmcs02.cpu=1)
> - L1 migrates to CPU #2 and executes INVVPID
> - KVM executes INVVPID on CPU #2
> - L1 migrates back to CPU #1 and runs L2 (vmx->nested.vmcs02.cpu=1)
> 
> The TLB entries on CPU #1 are never invalidated, because INVVPID was
> executed on CPU #2, and vmcs02 never ran on a different pCPU (i.e.
> vmx_vcpu_load_vmcs() will *not* request KVM_REQ_TLB_FLUSH).
> 
> Ensure that INVVPID is being executed on the same pCPU that L2 last ran
> on, and if not, fallback to clearing last_vpid=0 to trigger a full VPID
> flush on the next nested VM-Enter (as KVM will detect L1 using a
> different VPID for L2). If L2 ends up running on a different pCPU, KVM
> will flush the TLB anyway through vmx_vcpu_load_vmcs().
> 
> Cc: stable@vger.kernel.org
> Signed-off-by: Yosry Ahmed <yosry@kernel.org>
> 

Reviewed-by: Kai Huang <kai.huang@intel.com>