Re: [RFC PATCH 3/4] KVM: x86: Add EOI exit bitmap inference

[Steve Rutherford <srutherford@google.com>]

On Thu, May 14, 2015 at 04:35:19PM -0600, Alex Williamson wrote:
> On Thu, 2015-05-14 at 15:10 -0700, Steve Rutherford wrote:
> > On Thu, May 14, 2015 at 10:04:40AM -0600, Alex Williamson wrote:
> > > On Thu, 2015-05-14 at 17:46 +0200, Paolo Bonzini wrote:
> > > > 
> > > > On 14/05/2015 17:23, Alex Williamson wrote:
> > > > > > Would irq_ack_notifiers be used at all with this patch set?  Resampling
> > > > > > of IOAPIC level-triggered interrupts would be implemented in userspace.
> > > > > >  For the same reason, assigned devices using legacy device assignment
> > > > > > probably would not be able to use INTX (so this feature should depend on
> > > > > > !KVM_DEVICE_ASSIGNMENT).  Add the emulated i8254 and the last user of
> > > > > > irq_ack_notifiers goes away.
> > > > > > 
> > > > > > Alex, how does VFIO do INTX resampling if you're using TCG or -machine
> > > > > > kernel_irqchip=off?  (Context: this series keeps the local APIC
> > > > > > emulation in the kernel, thus including MSI, but moves the IOAPIC
> > > > > > emualtion to userspace).
> > > > > 
> > > > > Without KVM irqchip, we use a rudimentary approach where we disable
> > > > > mmaps to the device when an interrupt occurs.  This makes us trap all
> > > > > accesses to the device.  We then handle any device access from the guest
> > > > > as a potential EOI and unmask the interrupt.  If the interrupt re-fires,
> > > > > we've at least rate-limited it.  If it doesn't, a timer eventually
> > > > > re-enables devices mmaps.  It's not very efficient, but it works (for
> > > > > both PCI and platform devices) and avoids IRQ APIs through QEMU that get
> > > > > very platform/machine specific.  Thanks,
> > > > 
> > > > If we move the IOAPIC back to userspace, it probably would be easier to
> > > > implement irqfd/resamplefd in userspace as well.  Let the bikeshedding
> > > > start!
> > > 
> > > The current solution is merely functional with the benefit of being
> > > architecture and machine-type independent.  If someone actually cared
> > > about vfio assigned device performance without KVM irqchip, we'd
> > > certainly need a more deterministic and efficient EOI path.  Thanks,
> > > 
> > > Alex
> > > 
> > 
> > I'd like this feature to not preclude fast /modern/ device assignment
> > (i.e. devices using MSIs). A perf hit (or even incompatibility) for
> > legacy devices is fine [that's sort of the premise of this patchset].
> > What's necessary for the device assignment to work with the split
> > irqchip?
> 
> The current code will attempt to use kvm_irqchip_add_msi_route() and
> kvm_irqchip_add_irqfd_notifier() to route the MSI through the KVM
> irqchip.  Without that, QEMU will receive the MSI eventfd and call
> msi_notify() or msix_notify(), which would be significantly slower.
> This should be pretty much the same as virtio, which you've hopefully
> already taken into consideration.  Thanks,
> 
> Alex
> 

Those will be easy to update, given that the routing table is staying in the kernel. The notion of "kvm_irqchip_in_kernel" will need to be updated when support for this is added to QEMU, but that's not a huge deal.