Re: CPU Lockups in KVM with deferred hrtimer rearming

[Sean Christopherson <seanjc@google.com>]

On Tue, Apr 21, 2026, Thomas Gleixner wrote:
> On Tue, Apr 21 2026 at 10:20, Jim Mattson wrote:
> > On Tue, Apr 21, 2026 at 10:14 AM Thomas Gleixner <tglx@kernel.org> wrote:
> >>
> >> On Tue, Apr 21 2026 at 13:49, Peter Zijlstra wrote:
> >> > On Tue, Apr 21, 2026 at 01:34:07PM +0200, Peter Zijlstra wrote:
> >> >> > > > KVM invokes regular interrupts with pt_regs which have interrupts
> >> >> > > > disabled. That's correct from the KVM point of view, but completely
> >> >> > > > violates the obviously correct expectations of the interrupt entry/exit
> >> >> > > > code.
> >> >> > >
> >> >> > > Mooo :-(
> >> >>
> >> >> Also, is this a x86/KVM 'special' or is this true for all arch/KVM that
> >> >> use GENERIC_ENTRY?
> >> >
> >> > Should we not make asm_fred_entry_from_kvm()/VMX_DO_EVENT_IRQOFF fix IF
> >> > on the fake frame instead? We know it will enable IRQs after doing
> >> > handle_exit_irqoff() in vcpu_enter_guest().
> >>
> >> Doesn't work :)
> >>
> >> > SVM does not seem affected with this particular insanity.
> >>
> >> Looks like. It will take the interrupt after local_irq_enable().
> >
> > FWIW, VMX should work just like SVM if we clear VM_EXIT_ACK_INTR_ON_EXIT.

Hell no.

> I know. What's the point of that VM_EXIT_ACK_INTR_ON_EXIT exercise? Is
> there any performance benefit or is it just used because it's there?

There are performance benefits, and it preserves ordering: the first IRQ that's
serviced by the host is guaranteed to be _the_ IRQ that triggered the VM-Exit.
E.g. with AMD's approach, any IRQs that arrive between the VM-Exit and STI (which
is a pretty big swath of code) could be serviced before the IRQ that triggered
the exit, depending on priority.

VM_EXIT_ACK_INTR_ON_EXIT also provides symmetry with Intel's handing of NMIs, as
NMIs are unconditionally "acked" on VM-Exit.

Even if performance is "fine", changing decades of fundamental KVM behavior is
terrifying.

Pulling in an earlier idea:

 : Now for VMX, that hrtimer_rearm_deferred() call should really go into
 : handle_external_interrupt_irqoff(), which in turn requires to export
 : __hrtimer_rearm_deferred().

IMO, that's the way to go.  But instead of exporting __hrtimer_rearm_deferred(),
move vmx_do_nmi_irqoff() and vmx_do_interrupt_irqoff() into core kernel entry code
(along with the assembly glue), and then EXPORT_SYMBOL_FOR_KVM those.  It'd mean
some extra surgery, e.g. to provide an equivalent to KVM's IDT lookup:

	gate_offset((gate_desc *)host_idt_base + vector)

But I suspect it would be a big net positive in the end.i  E.g. the entry code
would *know* it's dealing with a direct call from KVM, and thus shouldn't need
to play pt_regs games.

Actually, even better would be to bury the FRED vs. not-FRED details in entry
code.  E.g. on the KVM invocation side, we could get to something like the below,
and I'm pretty sure _reduce_ the number of for-KVM exports in the process.

diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index a29896a9ef14..f6f5c124ed3b 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -7127,17 +7127,9 @@ static void handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu,
            "unexpected VM-Exit interrupt info: 0x%x", intr_info))
                return;
 
-       /*
-        * Invoke the kernel's IRQ handler for the vector.  Use the FRED path
-        * when it's available even if FRED isn't fully enabled, e.g. even if
-        * FRED isn't supported in hardware, in order to avoid the indirect
-        * CALL in the non-FRED path.
-        */
+       /* For the IRQ to the core kernel for processing. */
        kvm_before_interrupt(vcpu, KVM_HANDLING_IRQ);
-       if (IS_ENABLED(CONFIG_X86_FRED))
-               fred_entry_from_kvm(EVENT_TYPE_EXTINT, vector);
-       else
-               vmx_do_interrupt_irqoff(gate_offset((gate_desc *)host_idt_base + vector));
+       x86_entry_from_kvm(EVENT_TYPE_EXTINT, vector);
        kvm_after_interrupt(vcpu);
 
        vcpu->arch.at_instruction_boundary = true;
@@ -7447,10 +7439,7 @@ noinstr void vmx_handle_nmi(struct kvm_vcpu *vcpu)
                return;
 
        kvm_before_interrupt(vcpu, KVM_HANDLING_NMI);
-       if (cpu_feature_enabled(X86_FEATURE_FRED))
-               fred_entry_from_kvm(EVENT_TYPE_NMI, NMI_VECTOR);
-       else
-               vmx_do_nmi_irqoff();
+       x86_entry_from_kvm(EVENT_TYPE_NMI, NMI_VECTOR);
        kvm_after_interrupt(vcpu);
 }