[Qemu-devel] Re: [PATCH] virtio: Use ioeventfd for virtqueue notify

[Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>]

On Thu, Sep 30, 2010 at 03:01:52PM +0100, Stefan Hajnoczi wrote:
> Virtqueue notify is currently handled synchronously in userspace virtio.
> This prevents the vcpu from executing guest code while hardware
> emulation code handles the notify.
> 
> On systems that support KVM, the ioeventfd mechanism can be used to make
> virtqueue notify a lightweight exit by deferring hardware emulation to
> the iothread and allowing the VM to continue execution.  This model is
> similar to how vhost receives virtqueue notifies.
> 
> The result of this change is improved performance for userspace virtio
> devices.  Virtio-blk throughput increases especially for multithreaded
> scenarios and virtio-net transmit throughput increases substantially.
> Full numbers are below.
> 
> This patch employs ioeventfd virtqueue notify for all virtio devices.
> Linux kernels pre-2.6.34 only allow for 6 ioeventfds per VM and care
> must be taken so that vhost-net, the other ioeventfd user in QEMU, is
> able to function.  On such kernels ioeventfd virtqueue notify will not
> be used.
> 
> Khoa Huynh <khoa@us.ibm.com> collected the following data for
> virtio-blk with cache=none,aio=native:
> 
> FFSB Test          Threads  Unmodified  Patched
>                             (MB/s)      (MB/s)
> Large file create  1        21.7        21.8
>                    8        101.0       118.0
>                    16       119.0       157.0
> 
> Sequential reads   1        21.9        23.2
>                    8        114.0       139.0
>                    16       143.0       178.0
> 
> Random reads       1        3.3         3.6
>                    8        23.0        25.4
>                    16       43.3        47.8
> 
> Random writes      1        22.2        23.0
>                    8        93.1        111.6
>                    16       110.5       132.0
> 
> Sridhar Samudrala <sri@us.ibm.com> collected the following data for
> virtio-net with 2.6.36-rc1 on the host and 2.6.34 on the guest.
> 
> Guest to Host TCP_STREAM throughput(Mb/sec)
> -------------------------------------------
> Msg Size  vhost-net  virtio-net  virtio-net/ioeventfd
> 65536         12755        6430                  7590
> 16384          8499        3084                  5764
>  4096          4723        1578                  3659
>  1024          1827         981                  2060
> 
> Host to Guest TCP_STREAM throughput(Mb/sec)
> -------------------------------------------
> Msg Size  vhost-net  virtio-net  virtio-net/ioeventfd
> 65536         11156        5790                  5853
> 16384         10787        5575                  5691
>  4096         10452        5556                  4277
>  1024          4437        3671                  5277
> 
> Guest to Host TCP_RR latency(transactions/sec)
> ----------------------------------------------
> 
> Msg Size  vhost-net  virtio-net  virtio-net/ioeventfd
>     1          9903        3459                  3425
>  4096          7185        1931                  1899
> 16384          6108        2102                  1923
> 65536          3161        1610                  1744
> 
> Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
> ---
> Small changes are required for qemu-kvm.git.  I will send them once qemu.git
> has virtio-ioeventfd support.
> 
>  hw/vhost.c  |    6 ++--
>  hw/virtio.c |  106 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  hw/virtio.h |    9 +----
>  kvm-all.c   |   39 +++++++++++++++++++++
>  kvm-stub.c  |    5 +++
>  kvm.h       |    1 +
>  6 files changed, 156 insertions(+), 10 deletions(-)

Is there anything stopping this patch from being merged?

Thanks,
Stefan