Re: [PATCH] virtio: Use ioeventfd for virtqueue notify

Stefan Hajnoczi
2010-10-19T13:07:20Z

On Thu, Sep 30, 2010 at 03:01:52PM +0100, Stefan Hajnoczi wrote:
> Virtqueue notify is currently handled synchronously in userspace virtio.
> This prevents the vcpu from executing guest code while hardware
> emulation code handles the notify.
>
> On systems that support KVM, the ioeventfd mechanism can be used to make
> virtqueue notify a lightweight exit by deferring hardware emulation to
> the iothread and allowing the VM to continue execution. This model is
> similar to how vhost receives virtqueue notifies.
>
> The result of this change is improved performance for userspace virtio
> devices. Virtio-blk throughput increases especially for multithreaded
> scenarios and virtio-net transmit throughput increases substantially.
> Full numbers are below.
>
> This patch employs ioeventfd virtqueue notify for all virtio devices.
> Linux kernels pre-2.6.34 only allow for 6 ioeventfds per VM and care
> must be taken so that vhost-net, the other ioeventfd user in QEMU, is
> able to function. On such kernels ioeventfd virtqueue notify will not
> be used.
>
> Khoa Huynh <khoa@us.ibm.com> collected the following data for
> virtio-blk with cache=none,aio=native:
>
> FFSB Test Threads Unmodified Patched
> (MB/s) (MB/s)
> Large file create 1 21.7 21.8
> 8 101.0 118.0
> 16 119.0 157.0
>
> Sequential reads 1 21.9 23.2
> 8 114.0 139.0
> 16 143.0 178.0
>
> Random reads 1 3.3 3.6
> 8 23.0 25.4
> 16 43.3 47.8
>
> Random writes 1 22.2 23.0
> 8 93.1 111.6
> 16 110.5 132.0
>
> Sridhar Samudrala <sri@us.ibm.com> collected the following data for
> virtio-net with 2.6.36-rc1 on the host and 2.6.34 on the guest.
>
> Guest to Host TCP_STREAM throughput(Mb/sec)
> -------------------------------------------
> Msg Size vhost-net virtio-net virtio-net/ioeventfd
> 65536 12755 6430 7590
> 16384 8499 3084 5764
> 4096 4723 1578 3659
> 1024 1827 981 2060
>
> Host to Guest TCP_STREAM throughput(Mb/sec)
> -------------------------------------------
> Msg Size vhost-net virtio-net virtio-net/ioeventfd
> 65536 11156 5790 5853
> 16384 10787 5575 5691
> 4096 10452 5556 4277
> 1024 4437 3671 5277
>
> Guest to Host TCP_RR latency(transactions/sec)
> ----------------------------------------------
>
> Msg Size vhost-net virtio-net virtio-net/ioeventfd
> 1 9903 3459 3425
> 4096 7185 1931 1899
> 16384 6108 2102 1923
> 65536 3161 1610 1744
>
> Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
> ---
> Small changes are required for qemu-kvm.git. I will send them once qemu.git
> has virtio-ioeventfd support.
>
> hw/vhost.c | 6 ++--
> hw/virtio.c | 106 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> hw/virtio.h | 9 +----
> kvm-all.c | 39 +++++++++++++++++++++
> kvm-stub.c | 5 +++
> kvm.h | 1 +
> 6 files changed, 156 insertions(+), 10 deletions(-)

Is there anything stopping this patch from being merged?

Thanks,
Stefan