Re: [PATCH] virtio: Use ioeventfd for virtqueue notify

Michael S. Tsirkin
2010-10-03T13:57:50Z

On Sun, Oct 03, 2010 at 01:01:59PM +0200, Avi Kivity wrote:
> >
> >Guest to Host TCP_STREAM throughput(Mb/sec)
> >-------------------------------------------
> >Msg Size vhost-net virtio-net virtio-net/ioeventfd
> >65536 12755 6430 7590
> >16384 8499 3084 5764
> > 4096 4723 1578 3659
> > 1024 1827 981 2060
>
> Even more impressive (expected since the copying, which isn't
> present for block, is now shunted off into an iothread).
>
> On the last test you even exceeded vhost-net. Any theories how/why?
>
> Again, efficiency numbers would be interesting.
>
> >Host to Guest TCP_STREAM throughput(Mb/sec)
> >-------------------------------------------
> >Msg Size vhost-net virtio-net virtio-net/ioeventfd
> >65536 11156 5790 5853
> >16384 10787 5575 5691
> > 4096 10452 5556 4277
> > 1024 4437 3671 5277
>
> Here you exceed vhost-net, too.

This is with small packets- I suspect this is the extra
per interrupt overhead that eventfd has.

> >+static int kvm_check_many_iobus_devs(void)
> >+{
> >+ /* Older kernels have a 6 device limit on the KVM io bus. In that case
> >+ * creating many ioeventfds must be avoided. This tests checks for the
> >+ * limitation.
> >+ */
> >+ EventNotifier notifiers[7];
> >+ int i, ret = 0;
> >+ for (i = 0; i< ARRAY_SIZE(notifiers); i++) {
> >+ ret = event_notifier_init(&notifiers[i], 0);
> >+ if (ret< 0) {
> >+ break;
> >+ }
> >+ ret = kvm_set_ioeventfd_pio_word(event_notifier_get_fd(&notifiers[i]), 0, i, true);
> >+ if (ret< 0) {
> >+ event_notifier_cleanup(&notifiers[i]);
> >+ break;
> >+ }
> >+ }
> >+
> >+ /* Decide whether many devices are supported or not */
> >+ ret = i == ARRAY_SIZE(notifiers);
> >+
> >+ while (i--> 0) {
> >+ kvm_set_ioeventfd_pio_word(event_notifier_get_fd(&notifiers[i]), 0, i, false);
> >+ event_notifier_cleanup(&notifiers[i]);
> >+ }
> >+ return ret;
> >+}
>
> Sorry about that.
>
> IIRC there was a problem (shared by vhost-net) with interrupts
> remaining enabled in the window between the guest kicking the queue
> and the host waking up and disabling interrupts. An even more vague
> IIRC mst had an idea to fix this?

This is one of the things that vring2 is supposed to fix.

> --
> error compiling committee.c: too many arguments to function