Re: [Bug 217304] New: KVM does not handle NMI blocking correctly in nested virtualization

[Sean Christopherson <seanjc@google.com>]

On Thu, Apr 06, 2023, bugzilla-daemon@kernel.org wrote:
> Assume KVM runs in L0, LHV runs in L1, the nested guest runs in L2.
> 
> The code in LHV performs an experiment (called "Experiment 13" in serial
> output) on CPU 0 to test the behavior of NMI blocking. The experiment steps
> are:
> 1. Prepare state such that the CPU is currently in L1 (LHV), and NMI is blocked
> 2. Modify VMCS12 to make sure that L2 has virtual NMIs enabled (NMI exiting =
> 1, Virtual NMIs = 1), and L2 does not block NMI (Blocking by NMI = 0)
> 3. VM entry to L2
> 4. L2 performs VMCALL, get VM exit to L1
> 5. L1 checks whether NMI is blocked.
> 
> The expected behavior is that NMI should be blocked, which is reproduced on
> real hardware. According to Intel SDM, NMIs should be unblocked after VM entry
> to L2 (step 3). After VM exit to L1 (step 4), NMI blocking does not change, so
> NMIs are still unblocked. This behavior is reproducible on real hardware.
> 
> However, when running on KVM, the experiment shows that at step 5, NMIs are
> blocked in L1. Thus, I think NMI blocking is not implemented correctly in KVM's
> nested virtualization.

Ya, KVM blocks NMIs on nested NMI VM-Exits, but doesn't unblock NMIs for all other
exit types.  I believe this is the fix (untested):

---
 arch/x86/kvm/vmx/nested.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index 96ede74a6067..4240a052628a 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -4164,12 +4164,7 @@ static int vmx_check_nested_events(struct kvm_vcpu *vcpu)
 		nested_vmx_vmexit(vcpu, EXIT_REASON_EXCEPTION_NMI,
 				  NMI_VECTOR | INTR_TYPE_NMI_INTR |
 				  INTR_INFO_VALID_MASK, 0);
-		/*
-		 * The NMI-triggered VM exit counts as injection:
-		 * clear this one and block further NMIs.
-		 */
 		vcpu->arch.nmi_pending = 0;
-		vmx_set_nmi_mask(vcpu, true);
 		return 0;
 	}
 
@@ -4865,6 +4860,13 @@ void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,
 				INTR_INFO_VALID_MASK | INTR_TYPE_EXT_INTR;
 		}
 
+		/*
+		 * NMIs are blocked on VM-Exit due to NMI, and unblocked by all
+		 * other VM-Exit types.
+		 */
+		vmx_set_nmi_mask(vcpu, (u16)vm_exit_reason == EXIT_REASON_EXCEPTION_NMI &&
+				       !is_nmi(vmcs12->vm_exit_intr_info));
+
 		if (vm_exit_reason != -1)
 			trace_kvm_nested_vmexit_inject(vmcs12->vm_exit_reason,
 						       vmcs12->exit_qualification,

base-commit: 0b87a6bfd1bdb47b766aa0641b7cf93f3d3227e9
-- 

 
> I am happy to explain how the experiment code works in detail. c.img also
> reveals other NMI-related bugs in KVM. I am also happy to explain the other
> bugs.

I'm not sure I want to know ;-)  If you can give a quick rundown of each bug, it
would be quite helpful.

Thanks!