Re: [PATCH v3 0/2] kvm: level irqfd and new eoifd

[Alex Williamson <alex.williamson@redhat.com>]

On Thu, 2012-07-12 at 12:35 +0300, Avi Kivity wrote:
> On 07/11/2012 10:57 PM, Alex Williamson wrote:
> >> 
> >> > We still have classic KVM device assignment to provide fast-path INTx.
> >> > But if we want to replace it midterm, I think it's necessary for VFIO to
> >> > be able to provide such a path as well.
> >> 
> >> I would like VFIO to have no regressions vs. kvm device assignment,
> >> except perhaps in uncommon corner cases.  So I agree.
> > 
> > I ran a few TCP_RR netperf tests forcing a 1Gb tg3 nic to use INTx.
> > Without irqchip support vfio gets a bit more than 60% of KVM device
> > assignment.  That's a little bit of an unfair comparison since it's more
> > than just the I/O path.  With the proposed interfaces here, enabling
> > irqchip, vfio is within 10% of KVM device assignment for INTx.  For MSI,
> > I can actually make vfio come out more than 30% better than KVM device
> > assignment if I send the eventfd from the hard irq handler.  Using a
> > threaded handler as the code currently does, vfio is still behind KVM.
> > It's hard to beat a direct call chain.
> 
> We can have a direct call chain with vfio too, using a custom eventfd
> poll function, no?  Assuming we set up a fast path for unicast msi.

You'll have to help me out a little, eventfd_signal walks the wait_queue
and calls each function.  On the injection path that includes
irqfd_wakeup.  For an MSI that seems to already provide direct
injection.  For level we'll schedule_work, so that explains the overhead
in that path, but it's not too dissimilar to a a threaded irq.  vfio
does something very similar, so there's a schedule_work both on inject
and on eoi.  I'll have to check whether anything prevents the unmask
from the wait_queue function in vfio, that could be a significant chunk
of the gap.  Where's the custom poll function come into play?  Thanks,

Alex