Re: [PATCH] KVM: drop the kvm_has_noapic_vcpu optimization

[Sean Christopherson <seanjc@google.com>]

On Thu, Oct 31, 2024, Bernhard Kauer wrote:
> > > > In practice, this almost never happens though.  Do you have a use case for
> > > > creating VMs without in-kernel local APICs?
> > >
> > > I switched from "full irqchip" to "no irqchip" due to a significant
> > > performance gain
> >
> > Signifcant performance gain for what path?  I'm genuinely curious.
> 
> I have this really slow PREEMPT_RT kernel (Debian 6.11.4-rt-amd64).
> The hello-world benchmark takes on average 100ms.  With IRQCHIP it goes
> up to 220ms.  An strace gives 83ms for the extra ioctl:
> 
>         ioctl(4, KVM_CREATE_IRQCHIP, 0)         = 0 <0.083242>
> 
> My current theory is that RCU takes ages on this kernel.  And creating an
> IOAPIC uses SRCU to synchronize the bus array...
> 
> However, in my latest benchmark runs the overhead for IRQCHIP is down to 15
> microseconds.  So no big deal anymore.

Assuming you're running a recent kernel, that's likely thanks to commit
fbe4a7e881d4 ("KVM: Setup empty IRQ routing when creating a VM").

> > Unless your VM doesn't need a timer and doesn't need interrupts of
> > any kind, emulating the local APIC in userspace is going to be much
> > less performant.
> 
> Do you have any performance numbers?

Heh, nope.  I actually tried to grab some, mostly out of curiosity again, but
recent (last few years) versions of QEMU don't even support a userspace APIC.

A single EOI is a great example though.  On a remotely modern CPU, an in-kernel
APIC allows KVM to enable hardware acceleration so that the EOI is virtualized by
hardware, i.e. doesn't take a VM-Exit and so the latency is basically the same as
a native EOI (tens of cycles, maybe less).

With a userspace APIC, the roundtrip to userspace to emulate the EOI is measured
in tens of thousands of cycles.  IIRC, last I played around with userspace exits
the average turnaround time was ~50k cycles.