[PATCH v2] KVM: nVMX: Service local TLB flushes on failed nested VM-Enter

[PATCH v2] KVM: nVMX: Service local TLB flushes on failed nested VM-Enter

[Yosry Ahmed]

KVM services local TLB flushes on "full" nested VM-Exits (through
__nested_vmx_vmexit()), but not if a nested VM-Enter fails (e.g. due to
failed VMCS checks in nested_vmx_enter_non_root_mode()).

However, it is possible that KVM had queued TLB flushes that need to be
performed, even if the nested VM-Enter was not successful. For example,
if VPID is disabled for L2 (via nested_vmx_transition_tlb_flush(), or if
via the MSR load lists, as the SDM says:

  If any MSR is being loaded in such a way that would architecturally
  require a TLB flush, the TLBs are updated so that, after VM entry, the
  logical processor will not use any translations that were cached before
  the transition.

The SDM is unclear about when the TLB flush should occur, and whether or
not a failed VM entry would flush the TLB, so it is safer to always
do the TLB flush in this case.

More concretely, KVM also updates the last VPID L1 used for L2 in
nested_vmx_transition_tlb_flush() (i.e. last_vpid), even if the VM entry
ultimately fails. With the current code, KVM could miss a TLB flush if
L1 changes L2's VPID, then does a failed VM entry followed by a
successful one, as the failed VM entry would update last_vpid but not
actually flush the TLB. Servicing local TLB flushes on failed VM entries
makes sure that the TLB is always flushed when last_vpid is updated.

Fixes: 5c614b3583e7 ("KVM: nVMX: nested VPID emulation")
Cc: stable@vger.kernel.org
Reported-by: Sashiko <sashiko-bot@kernel.org> # Internal review
Suggested-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
---

v1: https://lore.kernel.org/kvm/20260717060505.2971514-1-yosry@kernel.org/

v1 -> v2:
- Service local TLB flushes on failed nested VM entries instead of
  delaying queuing the flushes, as TLB flushes could be queued in other
  ways (e.g. via the MSR load lists).

---
 arch/x86/kvm/vmx/nested.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index b5460de4b1a72..7e184a9644869 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -3755,6 +3755,14 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu,
 vmentry_fail_vmexit_guest_mode:
 	if (vmcs12->cpu_based_vm_exec_control & CPU_BASED_USE_TSC_OFFSETTING)
 		vcpu->arch.tsc_offset -= vmcs12->tsc_offset;
+
+	/*
+	 * Handle any TLB flush requests that were queued for L2 if KVM made it
+	 * far enough along to switch to L2 context.  Note, loading host state
+	 * will generate any flushes for L1 required by VM-Exit.
+	 */
+	kvm_service_local_tlb_flush_requests(vcpu);
+
 	leave_guest_mode(vcpu);
 
 vmentry_fail_vmexit:

base-commit: 6bc96b971766fbbbbdd9fb2642cedacaf02da957
-- 
2.55.0.229.g6434b31f56-goog

Re: [PATCH v2] KVM: nVMX: Service local TLB flushes on failed nested VM-Enter

[Yosry Ahmed]

On Wed, Jul 22, 2026 at 11:01:28PM +0000, Yosry Ahmed wrote:
> KVM services local TLB flushes on "full" nested VM-Exits (through
> __nested_vmx_vmexit()), but not if a nested VM-Enter fails (e.g. due to
> failed VMCS checks in nested_vmx_enter_non_root_mode()).
> 
> However, it is possible that KVM had queued TLB flushes that need to be
> performed, even if the nested VM-Enter was not successful. For example,
> if VPID is disabled for L2 (via nested_vmx_transition_tlb_flush(), or if
> via the MSR load lists, as the SDM says:
> 
>   If any MSR is being loaded in such a way that would architecturally
>   require a TLB flush, the TLBs are updated so that, after VM entry, the
>   logical processor will not use any translations that were cached before
>   the transition.
> 
> The SDM is unclear about when the TLB flush should occur, and whether or
> not a failed VM entry would flush the TLB, so it is safer to always
> do the TLB flush in this case.
> 
> More concretely, KVM also updates the last VPID L1 used for L2 in
> nested_vmx_transition_tlb_flush() (i.e. last_vpid), even if the VM entry
> ultimately fails. With the current code, KVM could miss a TLB flush if
> L1 changes L2's VPID, then does a failed VM entry followed by a
> successful one, as the failed VM entry would update last_vpid but not
> actually flush the TLB. Servicing local TLB flushes on failed VM entries
> makes sure that the TLB is always flushed when last_vpid is updated.
> 
> Fixes: 5c614b3583e7 ("KVM: nVMX: nested VPID emulation")
> Cc: stable@vger.kernel.org
> Reported-by: Sashiko <sashiko-bot@kernel.org> # Internal review
> Suggested-by: Sean Christopherson <seanjc@google.com>
> Signed-off-by: Yosry Ahmed <yosry@kernel.org>
> ---

For the record, my reproducer was basically the nested TLB flushes
selftest introduced here:
https://lore.kernel.org/kvm/20260728003557.1136583-29-yosry@kernel.org/

With this diff on top:

diff --git a/tools/testing/selftests/kvm/x86/nested_tlb_flush_test.c b/tools/testing/selftests/kvm/x86/nested_tlb_flush_test.c
index 55c9909bb085f..1659dd9c0044a 100644
--- a/tools/testing/selftests/kvm/x86/nested_tlb_flush_test.c
+++ b/tools/testing/selftests/kvm/x86/nested_tlb_flush_test.c
@@ -123,8 +123,26 @@ static void run_l2(void *nested_state, bool launch)
                vmx_tlb_flush(nested_state);
                if (launch)
                        GUEST_ASSERT(!vmlaunch());
-               else
-                       GUEST_ASSERT(!vmresume());
+               else {
+                       struct vmx_pages *vmx = nested_state;
+                       struct vmx_msr_entry *entry = vmx->msr;
+
+                       /* Inject VM-Entry failure via invalid MSR load */
+                       entry->index = 0xc0000100; /* MSR_FS_BASE, disallowed for loading */
+                       entry->reserved = 0;
+                       entry->value = 0;
+                       vmwrite(VM_ENTRY_MSR_LOAD_ADDR, vmx->msr_gpa);
+                       vmwrite(VM_ENTRY_MSR_LOAD_COUNT, 1);
+
+                       GUEST_ASSERT_EQ(vmresume(), 0);
+                       GUEST_ASSERT_EQ(vmreadz(VM_EXIT_REASON), (EXIT_REASON_FAILED_VMENTRY | EXIT_REASON_MSR_LOAD_FAIL));
+
+                       /* Fix failure and retry */
+                       vmwrite(VM_ENTRY_MSR_LOAD_COUNT, 0);
+                       memset(vmx->msr, 0, 4096);
+
+                       GUEST_ASSERT_EQ(vmresume(), 0);
+               }
                GUEST_ASSERT_EQ(vmreadz(VM_EXIT_REASON), EXIT_REASON_VMCALL);
                vmwrite(GUEST_RIP, vmreadz(GUEST_RIP) + 3); /* skip over VMCALL */
        } else {