From mboxrd@z Thu Jan  1 00:00:00 1970
From: "Michael S. Tsirkin" <mst@redhat.com>
Subject: Re: [PATCH] virtio: Use ioeventfd for virtqueue notify
Date: Sun, 3 Oct 2010 15:51:38 +0200
Message-ID: <20101003135138.GA19775@redhat.com>
References: <1285855312-11739-1-git-send-email-stefanha@linux.vnet.ibm.com>
 <4CA862A7.2080302@redhat.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Cc: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>,
	qemu-devel@nongnu.org, kvm@vger.kernel.org,
	Anthony Liguori <aliguori@us.ibm.com>,
	Khoa Huynh <khoa@us.ibm.com>,
	Sridhar Samudrala <sri@us.ibm.com>,
	Steve Dobbelstein <steved@us.ibm.com>
To: Avi Kivity <avi@redhat.com>
Return-path: <kvm-owner@vger.kernel.org>
Received: from mx1.redhat.com ([209.132.183.28]:44628 "EHLO mx1.redhat.com"
	rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
	id S1753849Ab0JCN5u (ORCPT <rfc822;kvm@vger.kernel.org>);
	Sun, 3 Oct 2010 09:57:50 -0400
Content-Disposition: inline
In-Reply-To: <4CA862A7.2080302@redhat.com>
Sender: kvm-owner@vger.kernel.org
List-ID: <kvm.vger.kernel.org>

On Sun, Oct 03, 2010 at 01:01:59PM +0200, Avi Kivity wrote:
> >
> >Guest to Host TCP_STREAM throughput(Mb/sec)
> >-------------------------------------------
> >Msg Size  vhost-net  virtio-net  virtio-net/ioeventfd
> >65536         12755        6430                  7590
> >16384          8499        3084                  5764
> >  4096          4723        1578                  3659
> >  1024          1827         981                  2060
> 
> Even more impressive (expected since the copying, which isn't
> present for block, is now shunted off into an iothread).
> 
> On the last test you even exceeded vhost-net.  Any theories how/why?
> 
> Again, efficiency numbers would be interesting.
> 
> >Host to Guest TCP_STREAM throughput(Mb/sec)
> >-------------------------------------------
> >Msg Size  vhost-net  virtio-net  virtio-net/ioeventfd
> >65536         11156        5790                  5853
> >16384         10787        5575                  5691
> >  4096         10452        5556                  4277
> >  1024          4437        3671                  5277
> 
> Here you exceed vhost-net, too.

This is with small packets- I suspect this is the extra
per interrupt overhead  that eventfd has.

> >+static int kvm_check_many_iobus_devs(void)
> >+{
> >+    /* Older kernels have a 6 device limit on the KVM io bus.  In that case
> >+     * creating many ioeventfds must be avoided.  This tests checks for the
> >+     * limitation.
> >+     */
> >+    EventNotifier notifiers[7];
> >+    int i, ret = 0;
> >+    for (i = 0; i<  ARRAY_SIZE(notifiers); i++) {
> >+        ret = event_notifier_init(&notifiers[i], 0);
> >+        if (ret<  0) {
> >+            break;
> >+        }
> >+        ret = kvm_set_ioeventfd_pio_word(event_notifier_get_fd(&notifiers[i]), 0, i, true);
> >+        if (ret<  0) {
> >+            event_notifier_cleanup(&notifiers[i]);
> >+            break;
> >+        }
> >+    }
> >+
> >+    /* Decide whether many devices are supported or not */
> >+    ret = i == ARRAY_SIZE(notifiers);
> >+
> >+    while (i-->  0) {
> >+        kvm_set_ioeventfd_pio_word(event_notifier_get_fd(&notifiers[i]), 0, i, false);
> >+        event_notifier_cleanup(&notifiers[i]);
> >+    }
> >+    return ret;
> >+}
> 
> Sorry about that.
> 
> IIRC there was a problem (shared by vhost-net) with interrupts
> remaining enabled in the window between the guest kicking the queue
> and the host waking up and disabling interrupts.  An even more vague
> IIRC mst had an idea to fix this?

This is one of the things that vring2 is supposed to fix.

> -- 
> error compiling committee.c: too many arguments to function