[Qemu-devel] [PATCH] virtio-blk: use blk_io_plug/unplug for Linux AIO batching

[Qemu-devel] [PATCH] virtio-blk: use blk_io_plug/unplug for Linux AIO batching

[Stefan Hajnoczi]

The raw-posix block driver implements Linux AIO batching so multiple
requests can be submitted with a single io_submit(2) system call.
Batching is currently only used by virtio-scsi and
virtio-blk-data-plane.

Enable batching for regular virtio-blk so the number of io_submit(2)
system calls is reduced for workloads with queue depth > 1.

In 4KB random read performance tests with queue depth 32, the CPU
utilization on the host is reduced by 9.4%.  The fio job is as follows:

  [global]
  bs=4k
  ioengine=libaio
  iodepth=32
  direct=1
  sync=0
  time_based=1
  runtime=30
  clocksource=gettimeofday
  ramp_time=5

  [job1]
  rw=randread
  filename=/dev/vdb
  size=4096M
  write_bw_log=fio
  write_iops_log=fio
  write_lat_log=fio
  log_avg_msec=1000

This benchmark was run on an raw image on LVM.  The disk was an SSD
drive and -drive cache=none,aio=native was used.

Tested-by: Pradeep Surisetty <psuriset@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 hw/block/virtio-blk.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index 6aefda4..a2137c8 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -600,6 +600,8 @@ static void virtio_blk_handle_output(VirtIODevice *vdev, VirtQueue *vq)
         return;
     }
 
+    blk_io_plug(s->blk);
+
     while ((req = virtio_blk_get_request(s))) {
         virtio_blk_handle_request(req, &mrb);
     }
@@ -607,6 +609,8 @@ static void virtio_blk_handle_output(VirtIODevice *vdev, VirtQueue *vq)
     if (mrb.num_reqs) {
         virtio_blk_submit_multireq(s->blk, &mrb);
     }
+
+    blk_io_unplug(s->blk);
 }
 
 static void virtio_blk_dma_restart_bh(void *opaque)
-- 
2.4.3

Re: [Qemu-devel] [PATCH] virtio-blk: use blk_io_plug/unplug for Linux AIO batching

[Fam Zheng]

On Mon, 07/20 16:54, Stefan Hajnoczi wrote:
> The raw-posix block driver implements Linux AIO batching so multiple
> requests can be submitted with a single io_submit(2) system call.
> Batching is currently only used by virtio-scsi and
> virtio-blk-data-plane.
> 
> Enable batching for regular virtio-blk so the number of io_submit(2)
> system calls is reduced for workloads with queue depth > 1.
> 
> In 4KB random read performance tests with queue depth 32, the CPU
> utilization on the host is reduced by 9.4%.  The fio job is as follows:
> 
>   [global]
>   bs=4k
>   ioengine=libaio
>   iodepth=32
>   direct=1
>   sync=0
>   time_based=1
>   runtime=30
>   clocksource=gettimeofday
>   ramp_time=5
> 
>   [job1]
>   rw=randread
>   filename=/dev/vdb
>   size=4096M
>   write_bw_log=fio
>   write_iops_log=fio
>   write_lat_log=fio
>   log_avg_msec=1000
> 
> This benchmark was run on an raw image on LVM.  The disk was an SSD
> drive and -drive cache=none,aio=native was used.
> 
> Tested-by: Pradeep Surisetty <psuriset@redhat.com>
> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
> ---
>  hw/block/virtio-blk.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
> index 6aefda4..a2137c8 100644
> --- a/hw/block/virtio-blk.c
> +++ b/hw/block/virtio-blk.c
> @@ -600,6 +600,8 @@ static void virtio_blk_handle_output(VirtIODevice *vdev, VirtQueue *vq)
>          return;
>      }
>  
> +    blk_io_plug(s->blk);
> +
>      while ((req = virtio_blk_get_request(s))) {
>          virtio_blk_handle_request(req, &mrb);
>      }
> @@ -607,6 +609,8 @@ static void virtio_blk_handle_output(VirtIODevice *vdev, VirtQueue *vq)
>      if (mrb.num_reqs) {
>          virtio_blk_submit_multireq(s->blk, &mrb);
>      }
> +
> +    blk_io_unplug(s->blk);
>  }
>  
>  static void virtio_blk_dma_restart_bh(void *opaque)
> -- 
> 2.4.3
> 
> 

Reviewed-by: Fam Zheng <famz@redhat.com>

Re: [Qemu-devel] [Qemu-block] [PATCH] virtio-blk: use blk_io_plug/unplug for Linux AIO batching

[Stefan Hajnoczi]

On Mon, Jul 20, 2015 at 04:54:16PM +0100, Stefan Hajnoczi wrote:
> The raw-posix block driver implements Linux AIO batching so multiple
> requests can be submitted with a single io_submit(2) system call.
> Batching is currently only used by virtio-scsi and
> virtio-blk-data-plane.
> 
> Enable batching for regular virtio-blk so the number of io_submit(2)
> system calls is reduced for workloads with queue depth > 1.
> 
> In 4KB random read performance tests with queue depth 32, the CPU
> utilization on the host is reduced by 9.4%.  The fio job is as follows:
> 
>   [global]
>   bs=4k
>   ioengine=libaio
>   iodepth=32
>   direct=1
>   sync=0
>   time_based=1
>   runtime=30
>   clocksource=gettimeofday
>   ramp_time=5
> 
>   [job1]
>   rw=randread
>   filename=/dev/vdb
>   size=4096M
>   write_bw_log=fio
>   write_iops_log=fio
>   write_lat_log=fio
>   log_avg_msec=1000
> 
> This benchmark was run on an raw image on LVM.  The disk was an SSD
> drive and -drive cache=none,aio=native was used.
> 
> Tested-by: Pradeep Surisetty <psuriset@redhat.com>
> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
> ---
>  hw/block/virtio-blk.c | 4 ++++
>  1 file changed, 4 insertions(+)

Thanks, applied to my master tree:
https://github.com/stefanha/qemu/commits/master

Stefan