[Qemu-devel] Re: [PATCH] virtio: Use ioeventfd for virtqueue notify

["Michael S. Tsirkin" <mst@redhat.com>]

On Sun, Oct 03, 2010 at 01:01:59PM +0200, Avi Kivity wrote:
> >
> >Guest to Host TCP_STREAM throughput(Mb/sec)
> >-------------------------------------------
> >Msg Size  vhost-net  virtio-net  virtio-net/ioeventfd
> >65536         12755        6430                  7590
> >16384          8499        3084                  5764
> >  4096          4723        1578                  3659
> >  1024          1827         981                  2060
> 
> Even more impressive (expected since the copying, which isn't
> present for block, is now shunted off into an iothread).
> 
> On the last test you even exceeded vhost-net.  Any theories how/why?
> 
> Again, efficiency numbers would be interesting.
> 
> >Host to Guest TCP_STREAM throughput(Mb/sec)
> >-------------------------------------------
> >Msg Size  vhost-net  virtio-net  virtio-net/ioeventfd
> >65536         11156        5790                  5853
> >16384         10787        5575                  5691
> >  4096         10452        5556                  4277
> >  1024          4437        3671                  5277
> 
> Here you exceed vhost-net, too.

This is with small packets- I suspect this is the extra
per interrupt overhead  that eventfd has.

> >+static int kvm_check_many_iobus_devs(void)
> >+{
> >+    /* Older kernels have a 6 device limit on the KVM io bus.  In that case
> >+     * creating many ioeventfds must be avoided.  This tests checks for the
> >+     * limitation.
> >+     */
> >+    EventNotifier notifiers[7];
> >+    int i, ret = 0;
> >+    for (i = 0; i<  ARRAY_SIZE(notifiers); i++) {
> >+        ret = event_notifier_init(&notifiers[i], 0);
> >+        if (ret<  0) {
> >+            break;
> >+        }
> >+        ret = kvm_set_ioeventfd_pio_word(event_notifier_get_fd(&notifiers[i]), 0, i, true);
> >+        if (ret<  0) {
> >+            event_notifier_cleanup(&notifiers[i]);
> >+            break;
> >+        }
> >+    }
> >+
> >+    /* Decide whether many devices are supported or not */
> >+    ret = i == ARRAY_SIZE(notifiers);
> >+
> >+    while (i-->  0) {
> >+        kvm_set_ioeventfd_pio_word(event_notifier_get_fd(&notifiers[i]), 0, i, false);
> >+        event_notifier_cleanup(&notifiers[i]);
> >+    }
> >+    return ret;
> >+}
> 
> Sorry about that.
> 
> IIRC there was a problem (shared by vhost-net) with interrupts
> remaining enabled in the window between the guest kicking the queue
> and the host waking up and disabling interrupts.  An even more vague
> IIRC mst had an idea to fix this?

This is one of the things that vring2 is supposed to fix.

> -- 
> error compiling committee.c: too many arguments to function