[PATCH] KVM: VMX: Complete pending nested VM-Enter on invalid-state failure

[PATCH] KVM: VMX: Complete pending nested VM-Enter on invalid-state failure

[Hao Zhang]

From: Hao Zhang <zhanghao1@kylinos.cn>
Date: Fri, 17 Jul 2026 09:18:31 +0800

Commit 2bb8cafea80b ("KVM: vVMX: signal failure for nested VMEntry if
emulation_required") made KVM synthesize a failed VM-Entry with
EXIT_REASON_INVALID_STATE when L2 guest state requires invalid-state
emulation during nested VM-Enter.

That synthetic failure is generated in vmx_vcpu_run() before attempting
hardware VM-Enter.  As a result, it returns before the normal post-run
path that accounts and clears nested_run_pending after a real
VM-Enter/VM-Exit round trip.

If the synthetic invalid-state failure happens while a nested VM-Enter is
pending, __vmx_handle_exit() observes a trusted pending nested VM-Enter
and fires KVM_BUG_ON(), causing KVM_RUN to fail with -EIO instead of
reflecting the failed VM-Entry to L1.

Treat EXIT_REASON_INVALID_STATE with the failed VM-Entry bit set as
completion of the pending nested VM-Enter.  Clear nested_run_pending, keep
the existing BUG check for other trusted pending exits, and do not convert
this synthetic failed VM-Entry into a triple fault for L2.

Fixes: 2bb8cafea80b ("KVM: vVMX: signal failure for nested VMEntry if emulation_required")
Signed-off-by: Hao Zhang <zhanghao1@kylinos.cn>
---
 arch/x86/kvm/vmx/vmx.c | 33 +++++++++++++++++++++++++--------
 1 file changed, 25 insertions(+), 8 deletions(-)

diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index cc75feec05da..0ebd2e746ddd 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -6699,6 +6699,7 @@ static int __vmx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath)
 	union vmx_exit_reason exit_reason = vmx_get_exit_reason(vcpu);
 	u32 vectoring_info = vmx->idt_vectoring_info;
 	u16 exit_handler_index;
+	bool failed_nested_vmentry = false;
 
 	/*
 	 * Flush logged GPAs PML buffer, this will make dirty_bitmap more
@@ -6715,13 +6716,29 @@ static int __vmx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath)
 		return 0;
 
 	/*
-	 * KVM should never reach this point with a pending nested VM-Enter.
-	 * More specifically, short-circuiting VM-Entry to emulate L2 due to
-	 * invalid guest state should never happen as that means KVM knowingly
-	 * allowed a nested VM-Enter with an invalid vmcs12.  More below.
-	 */
-	if (KVM_BUG_ON(vcpu->arch.nested_run_pending, vcpu->kvm))
-		return -EIO;
+	 * vmx_vcpu_run() may synthesize EXIT_REASON_INVALID_STATE with the
+	 * failed VM-Entry bit set before attempting hardware VM-Enter, and thus
+	 * return before the normal post-run path clears nested_run_pending. Treat
+	 * that case as completion of the pending nested VM-Enter and reflect the
+	 * failed VM-Entry to L1.
+	 *
+	 * For all other exits, KVM should never reach this point with a trusted
+	 * pending nested VM-Enter. If userspace gained control while VM-Enter was
+	 * pending, the pending state is marked untrusted because userspace may
+	 * have modified vCPU state and induced an architecturally impossible
+	 * VM-Exit.
+	 */
+	if (unlikely(vcpu->arch.nested_run_pending)) {
+		if (exit_reason.basic == EXIT_REASON_INVALID_STATE &&
+		    exit_reason.failed_vmentry) {
+			failed_nested_vmentry = true;
+		} else if (KVM_BUG_ON(vcpu->arch.nested_run_pending ==
+				      KVM_NESTED_RUN_PENDING, vcpu->kvm)) {
+			return -EIO;
+		}
+
+		vcpu->arch.nested_run_pending = 0;
+	}
 
 	if (is_guest_mode(vcpu)) {
 		/*
@@ -6755,7 +6772,7 @@ static int __vmx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath)
 		 * the least awful solution for the userspace case without
 		 * risking false positives.
 		 */
-		if (vmx->vt.emulation_required) {
+		if (vmx->vt.emulation_required && !failed_nested_vmentry) {
 			nested_vmx_vmexit(vcpu, EXIT_REASON_TRIPLE_FAULT, 0, 0);
 			return 1;
 		}

base-commit: 58717b2a1365d06c8c64b72aa948541b53fe31eb
-- 
2.15.0

Re: [PATCH] KVM: VMX: Complete pending nested VM-Enter on invalid-state failure

[Sean Christopherson]

On Mon, Jul 20, 2026, Hao Zhang wrote:
> From: Hao Zhang <zhanghao1@kylinos.cn>
> Date: Fri, 17 Jul 2026 09:18:31 +0800
> 
> Commit 2bb8cafea80b ("KVM: vVMX: signal failure for nested VMEntry if
> emulation_required") made KVM synthesize a failed VM-Entry with
> EXIT_REASON_INVALID_STATE when L2 guest state requires invalid-state
> emulation during nested VM-Enter.
> 
> That synthetic failure is generated in vmx_vcpu_run() before attempting
> hardware VM-Enter.  As a result, it returns before the normal post-run
> path that accounts and clears nested_run_pending after a real
> VM-Enter/VM-Exit round trip.
> 
> If the synthetic invalid-state failure happens while a nested VM-Enter is
> pending, __vmx_handle_exit() observes a trusted pending nested VM-Enter
> and fires KVM_BUG_ON(), causing KVM_RUN to fail with -EIO instead of
> reflecting the failed VM-Entry to L1.

This KVM_BUG_ON() is working as intended.  prepare_vmcs02() is supposed to guard
against emulating VMLAUNCH/VMRESUME with invalid guest state.

Ugh this appears to be reachable by clobbering SMRAM state prior to a RSM to L2.
Is that how you triggered the KVM_BUG_ON()?  If so, I'd much figure out a way to
fix the RSM flow.  Or maybe just treat those runs as untrusted?  Because for all
intents and purposes, stuffing vCPU state via SMRAM is the same as stuffing vCPU
state via KVM ioctls.

diff --git arch/x86/kvm/svm/svm.c arch/x86/kvm/svm/svm.c
index 8c5018c65dc5..1f4604a9ae58 100644
--- arch/x86/kvm/svm/svm.c
+++ arch/x86/kvm/svm/svm.c
@@ -5091,7 +5091,7 @@ static int svm_leave_smm(struct kvm_vcpu *vcpu, const union kvm_smram *smram)
                goto unmap_save;
 
        ret = 0;
-       vcpu->arch.nested_run_pending = KVM_NESTED_RUN_PENDING;
+       vcpu->arch.nested_run_pending = KVM_NESTED_RUN_PENDING_UNTRUSTED;
 
 unmap_save:
        kvm_vcpu_unmap(vcpu, &map_save);
diff --git arch/x86/kvm/vmx/vmx.c arch/x86/kvm/vmx/vmx.c
index e4b9ac7fed9f..5c410f68a716 100644
--- arch/x86/kvm/vmx/vmx.c
+++ arch/x86/kvm/vmx/vmx.c
@@ -8453,7 +8453,7 @@ int vmx_leave_smm(struct kvm_vcpu *vcpu, const union kvm_smram *smram)
                if (ret != NVMX_VMENTRY_SUCCESS)
                        return 1;
 
-               vcpu->arch.nested_run_pending = KVM_NESTED_RUN_PENDING;
+               vcpu->arch.nested_run_pending = KVM_NESTED_RUN_PENDING_UNTRUSTED;
                vmx->nested.smm.guest_mode = false;
        }
        return 0;