Re: [RFC PATCH 3/4] KVM: x86: Add EOI exit bitmap inference

[Alex Williamson <alex.williamson@redhat.com>]

On Thu, 2015-05-14 at 15:10 -0700, Steve Rutherford wrote:
> On Thu, May 14, 2015 at 10:04:40AM -0600, Alex Williamson wrote:
> > On Thu, 2015-05-14 at 17:46 +0200, Paolo Bonzini wrote:
> > > 
> > > On 14/05/2015 17:23, Alex Williamson wrote:
> > > > > Would irq_ack_notifiers be used at all with this patch set?  Resampling
> > > > > of IOAPIC level-triggered interrupts would be implemented in userspace.
> > > > >  For the same reason, assigned devices using legacy device assignment
> > > > > probably would not be able to use INTX (so this feature should depend on
> > > > > !KVM_DEVICE_ASSIGNMENT).  Add the emulated i8254 and the last user of
> > > > > irq_ack_notifiers goes away.
> > > > > 
> > > > > Alex, how does VFIO do INTX resampling if you're using TCG or -machine
> > > > > kernel_irqchip=off?  (Context: this series keeps the local APIC
> > > > > emulation in the kernel, thus including MSI, but moves the IOAPIC
> > > > > emualtion to userspace).
> > > > 
> > > > Without KVM irqchip, we use a rudimentary approach where we disable
> > > > mmaps to the device when an interrupt occurs.  This makes us trap all
> > > > accesses to the device.  We then handle any device access from the guest
> > > > as a potential EOI and unmask the interrupt.  If the interrupt re-fires,
> > > > we've at least rate-limited it.  If it doesn't, a timer eventually
> > > > re-enables devices mmaps.  It's not very efficient, but it works (for
> > > > both PCI and platform devices) and avoids IRQ APIs through QEMU that get
> > > > very platform/machine specific.  Thanks,
> > > 
> > > If we move the IOAPIC back to userspace, it probably would be easier to
> > > implement irqfd/resamplefd in userspace as well.  Let the bikeshedding
> > > start!
> > 
> > The current solution is merely functional with the benefit of being
> > architecture and machine-type independent.  If someone actually cared
> > about vfio assigned device performance without KVM irqchip, we'd
> > certainly need a more deterministic and efficient EOI path.  Thanks,
> > 
> > Alex
> > 
> 
> I'd like this feature to not preclude fast /modern/ device assignment
> (i.e. devices using MSIs). A perf hit (or even incompatibility) for
> legacy devices is fine [that's sort of the premise of this patchset].
> What's necessary for the device assignment to work with the split
> irqchip?

The current code will attempt to use kvm_irqchip_add_msi_route() and
kvm_irqchip_add_irqfd_notifier() to route the MSI through the KVM
irqchip.  Without that, QEMU will receive the MSI eventfd and call
msi_notify() or msix_notify(), which would be significantly slower.
This should be pretty much the same as virtio, which you've hopefully
already taken into consideration.  Thanks,

Alex