From mboxrd@z Thu Jan 1 00:00:00 1970 From: Stefan Hajnoczi Subject: Re: [PATCH] virtio: Use ioeventfd for virtqueue notify Date: Tue, 19 Oct 2010 14:07:11 +0100 Message-ID: <20101019130710.GA3906@stefan-thinkpad.transitives.com> References: <1285855312-11739-1-git-send-email-stefanha@linux.vnet.ibm.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: kvm@vger.kernel.org, "Michael S. Tsirkin" , Anthony Liguori , Khoa Huynh , Sridhar Samudrala , Steve Dobbelstein To: qemu-devel@nongnu.org Return-path: Received: from mtagate2.de.ibm.com ([195.212.17.162]:39186 "EHLO mtagate2.de.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758595Ab0JSNHU (ORCPT ); Tue, 19 Oct 2010 09:07:20 -0400 Received: from d12nrmr1607.megacenter.de.ibm.com (d12nrmr1607.megacenter.de.ibm.com [9.149.167.49]) by mtagate2.de.ibm.com (8.13.1/8.13.1) with ESMTP id o9JD7Dnp003196 for ; Tue, 19 Oct 2010 13:07:13 GMT Received: from d12av01.megacenter.de.ibm.com (d12av01.megacenter.de.ibm.com [9.149.165.212]) by d12nrmr1607.megacenter.de.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id o9JD7CqD3920078 for ; Tue, 19 Oct 2010 15:07:12 +0200 Received: from d12av01.megacenter.de.ibm.com (loopback [127.0.0.1]) by d12av01.megacenter.de.ibm.com (8.12.11.20060308/8.13.3) with ESMTP id o9JD7CA1016366 for ; Tue, 19 Oct 2010 15:07:12 +0200 Content-Disposition: inline In-Reply-To: <1285855312-11739-1-git-send-email-stefanha@linux.vnet.ibm.com> Sender: kvm-owner@vger.kernel.org List-ID: On Thu, Sep 30, 2010 at 03:01:52PM +0100, Stefan Hajnoczi wrote: > Virtqueue notify is currently handled synchronously in userspace virtio. > This prevents the vcpu from executing guest code while hardware > emulation code handles the notify. > > On systems that support KVM, the ioeventfd mechanism can be used to make > virtqueue notify a lightweight exit by deferring hardware emulation to > the iothread and allowing the VM to continue execution. This model is > similar to how vhost receives virtqueue notifies. > > The result of this change is improved performance for userspace virtio > devices. Virtio-blk throughput increases especially for multithreaded > scenarios and virtio-net transmit throughput increases substantially. > Full numbers are below. > > This patch employs ioeventfd virtqueue notify for all virtio devices. > Linux kernels pre-2.6.34 only allow for 6 ioeventfds per VM and care > must be taken so that vhost-net, the other ioeventfd user in QEMU, is > able to function. On such kernels ioeventfd virtqueue notify will not > be used. > > Khoa Huynh collected the following data for > virtio-blk with cache=none,aio=native: > > FFSB Test Threads Unmodified Patched > (MB/s) (MB/s) > Large file create 1 21.7 21.8 > 8 101.0 118.0 > 16 119.0 157.0 > > Sequential reads 1 21.9 23.2 > 8 114.0 139.0 > 16 143.0 178.0 > > Random reads 1 3.3 3.6 > 8 23.0 25.4 > 16 43.3 47.8 > > Random writes 1 22.2 23.0 > 8 93.1 111.6 > 16 110.5 132.0 > > Sridhar Samudrala collected the following data for > virtio-net with 2.6.36-rc1 on the host and 2.6.34 on the guest. > > Guest to Host TCP_STREAM throughput(Mb/sec) > ------------------------------------------- > Msg Size vhost-net virtio-net virtio-net/ioeventfd > 65536 12755 6430 7590 > 16384 8499 3084 5764 > 4096 4723 1578 3659 > 1024 1827 981 2060 > > Host to Guest TCP_STREAM throughput(Mb/sec) > ------------------------------------------- > Msg Size vhost-net virtio-net virtio-net/ioeventfd > 65536 11156 5790 5853 > 16384 10787 5575 5691 > 4096 10452 5556 4277 > 1024 4437 3671 5277 > > Guest to Host TCP_RR latency(transactions/sec) > ---------------------------------------------- > > Msg Size vhost-net virtio-net virtio-net/ioeventfd > 1 9903 3459 3425 > 4096 7185 1931 1899 > 16384 6108 2102 1923 > 65536 3161 1610 1744 > > Signed-off-by: Stefan Hajnoczi > --- > Small changes are required for qemu-kvm.git. I will send them once qemu.git > has virtio-ioeventfd support. > > hw/vhost.c | 6 ++-- > hw/virtio.c | 106 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > hw/virtio.h | 9 +---- > kvm-all.c | 39 +++++++++++++++++++++ > kvm-stub.c | 5 +++ > kvm.h | 1 + > 6 files changed, 156 insertions(+), 10 deletions(-) Is there anything stopping this patch from being merged? Thanks, Stefan