Re: [Qemu-devel] [PATCH v4 00/13] aio: experimental virtio-blk polling mode

[Paolo Bonzini <pbonzini@redhat.com>]

> v4:
>  * Added poll time self-tuning algorithm [Christian and Paolo]
>  * Try a single iteration of polling to avoid non-blocking
>  ppoll(2)/epoll_wait(2) [Paolo]
>  * Reordered patches to make performance analysis easier - see below

Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>

> v3:
>  * Avoid ppoll(2)/epoll_wait(2) if polling succeeded [Paolo]
>  * Disable guest->host virtqueue notification during polling [Christian]
>  * Rebased on top of my virtio-blk/scsi virtqueue notification disable
>  patches
> 
> v2:
>  * Uninitialized node->deleted gone [Fam]
>  * Removed 1024 polling loop iteration qemu_clock_get_ns() optimization which
>    created a weird step pattern [Fam]
>  * Unified with AioHandler, dropped AioPollHandler struct [Paolo]
>    (actually I think Paolo had more in mind but this is the first step)
>  * Only poll when all event loop resources support it [Paolo]
>  * Added run_poll_handlers_begin/end trace events for perf analysis
>  * Sorry, Christian, no virtqueue kick suppression yet
> 
> Recent performance investigation work done by Karl Rister shows that the
> guest->host notification takes around 20 us.  This is more than the
> "overhead"
> of QEMU itself (e.g. block layer).
> 
> One way to avoid the costly exit is to use polling instead of notification.
> The main drawback of polling is that it consumes CPU resources.  In order to
> benefit performance the host must have extra CPU cycles available on physical
> CPUs that aren't used by the guest.
> 
> This is an experimental AioContext polling implementation.  It adds a polling
> callback into the event loop.  Polling functions are implemented for
> virtio-blk
> virtqueue guest->host kick and Linux AIO completion.
> 
> The -object iothread,poll-max-ns=NUM parameter sets the number of nanoseconds
> to poll before entering the usual blocking poll(2) syscall.  Try setting this
> parameter to the time from old request completion to new virtqueue kick.  By
> default no polling is done so you must set this parameter to get busy
> polling.
> 
> Patch series overview:
> 
> Here are convenient points in the patch series at which to do performance
> benchmarking to see how various pieces interact.
> 
> 1. Basic -iothread poll-max-ns=NUM parameter without self-tuning:
> 
> aio: add flag to skip fds to aio_dispatch()
> aio: add AioPollFn and io_poll() interface
> aio: add polling mode to AioContext
> virtio: poll virtqueues for new buffers
> linux-aio: poll ring for completions
> iothread: add polling parameters
> 
> 2. My "[PATCH 0/3] virtio: disable notifications in blk and scsi" series is
>    needed as a base.  This is unrelated to polling but we'll also disable
>    notifications during polling later:
> 
> virtio-blk: suppress virtqueue kick during processing
> virtio-scsi: suppress virtqueue kick during processing
> 
> 3. Disable virtqueue notifications (reduce vmexits) during polling:
> 
> virtio: turn vq->notification into a nested counter
> aio: add .io_poll_begin/end() callbacks
> virtio: disable virtqueue notifications during polling
> 
> 4. Self-tuning, adjusts polling time depending on wait times:
> 
> aio: self-tune polling time
> iothread: add poll-grow and poll-shrink parameters
> 
> Stefan Hajnoczi (13):
>   aio: add flag to skip fds to aio_dispatch()
>   aio: add AioPollFn and io_poll() interface
>   aio: add polling mode to AioContext
>   virtio: poll virtqueues for new buffers
>   linux-aio: poll ring for completions
>   iothread: add polling parameters
>   virtio-blk: suppress virtqueue kick during processing
>   virtio-scsi: suppress virtqueue kick during processing
>   virtio: turn vq->notification into a nested counter
>   aio: add .io_poll_begin/end() callbacks
>   virtio: disable virtqueue notifications during polling
>   aio: self-tune polling time
>   iothread: add poll-grow and poll-shrink parameters
> 
>  include/block/aio.h         |  53 +++++++-
>  include/sysemu/iothread.h   |   5 +
>  aio-posix.c                 | 308
>  +++++++++++++++++++++++++++++++++++++++-----
>  aio-win32.c                 |  32 ++++-
>  async.c                     |  21 ++-
>  block/curl.c                |   8 +-
>  block/iscsi.c               |   3 +-
>  block/linux-aio.c           |  19 ++-
>  block/nbd-client.c          |   8 +-
>  block/nfs.c                 |   7 +-
>  block/sheepdog.c            |  26 ++--
>  block/ssh.c                 |   4 +-
>  block/win32-aio.c           |   4 +-
>  hw/block/virtio-blk.c       |  18 ++-
>  hw/scsi/virtio-scsi.c       |  36 +++---
>  hw/virtio/virtio.c          |  54 ++++++--
>  iohandler.c                 |   2 +-
>  iothread.c                  |  84 ++++++++++++
>  nbd/server.c                |   9 +-
>  stubs/set-fd-handler.c      |   1 +
>  tests/test-aio.c            |   4 +-
>  util/event_notifier-posix.c |   2 +-
>  trace-events                |   6 +
>  23 files changed, 604 insertions(+), 110 deletions(-)
> 
> --
> 2.9.3
> 
>