Re: [PATCH v3 0/2] kvm: level irqfd and new eoifd

[Avi Kivity <avi@redhat.com>]

On 07/12/2012 07:19 PM, Alex Williamson wrote:
> On Thu, 2012-07-12 at 12:35 +0300, Avi Kivity wrote:
>> On 07/11/2012 10:57 PM, Alex Williamson wrote:
>> >> 
>> >> > We still have classic KVM device assignment to provide fast-path INTx.
>> >> > But if we want to replace it midterm, I think it's necessary for VFIO to
>> >> > be able to provide such a path as well.
>> >> 
>> >> I would like VFIO to have no regressions vs. kvm device assignment,
>> >> except perhaps in uncommon corner cases.  So I agree.
>> > 
>> > I ran a few TCP_RR netperf tests forcing a 1Gb tg3 nic to use INTx.
>> > Without irqchip support vfio gets a bit more than 60% of KVM device
>> > assignment.  That's a little bit of an unfair comparison since it's more
>> > than just the I/O path.  With the proposed interfaces here, enabling
>> > irqchip, vfio is within 10% of KVM device assignment for INTx.  For MSI,
>> > I can actually make vfio come out more than 30% better than KVM device
>> > assignment if I send the eventfd from the hard irq handler.  Using a
>> > threaded handler as the code currently does, vfio is still behind KVM.
>> > It's hard to beat a direct call chain.
>> 
>> We can have a direct call chain with vfio too, using a custom eventfd
>> poll function, no?  Assuming we set up a fast path for unicast msi.
> 
> You'll have to help me out a little, eventfd_signal walks the wait_queue
> and calls each function.  On the injection path that includes
> irqfd_wakeup.  

This is what I meant, except I forgot that we already do direct path for
MSI.

> For an MSI that seems to already provide direct
> injection.  

Ugh, even for a broadcast MSI into 254 vcpu guests.  That's going to be
one slow interrupt.

> For level we'll schedule_work, so that explains the overhead
> in that path, but it's not too dissimilar to a a threaded irq.  vfio
> does something very similar, so there's a schedule_work both on inject
> and on eoi.  I'll have to check whether anything prevents the unmask
> from the wait_queue function in vfio, that could be a significant chunk
> of the gap.  Where's the custom poll function come into play?  Thanks,

So I don't understand where the gap comes from.  The number of context
switches for kvm and vfio is the same, as long as both use MSI (and
either both use threaded irq or both don't).

-- 
error compiling committee.c: too many arguments to function