From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60677) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bAZjk-0002Pk-Hl for qemu-devel@nongnu.org; Wed, 08 Jun 2016 05:25:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bAZjh-0003kl-B7 for qemu-devel@nongnu.org; Wed, 08 Jun 2016 05:25:20 -0400 Received: from mx1.redhat.com ([209.132.183.28]:57080) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bAZjh-0003kf-2V for qemu-devel@nongnu.org; Wed, 08 Jun 2016 05:25:17 -0400 From: Stefan Hajnoczi Date: Tue, 7 Jun 2016 17:28:24 +0100 Message-Id: <1465316911-20052-1-git-send-email-stefanha@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: [Qemu-devel] [PATCH v3 0/7] virtio-blk: multiqueue support List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: Kevin Wolf , Roman Pen , Fam Zheng , Christian Borntraeger , Paolo Bonzini , Ming Lei , Stefan Hajnoczi v3: * Drop Patch 1 to batch guest notify for non-dataplane The Linux AIO completion BH and the virtio-blk batch notify BH changed= order in the AioContext->first_bh list as a side-effect of moving the BH fro= m hw/block/dataplane/virtio-blk.c to hw/block/virtio-blk.c. This caused= a serious performance regression for both dataplane and non-dataplane. I've decided not to move the BH in this series and work on a separate solution for making batch notify generic. The remaining patches have been reordered and cleaned up. * See performance data below. v2: * Simplify s->rq live migration [Paolo] * Use more efficient bitmap ops for batch notification [Paolo] * Fix perf regression due to batch notify BH in wrong AioContext [Christ= ian] The virtio_blk guest driver has supported multiple virtqueues since Linux= 3.17. This patch series adds multiple virtqueues to QEMU's virtio-blk emulated device. Ming Lei sent patches previously but these were not merged. This series implements virtio-blk multiqueue for QEMU from scratch since the codebase= has changed. Live migration support for s->rq was also missing from the prev= ious series and has been added. It's important to note that QEMU's block layer does not support multiqueu= e yet. Therefore virtio-blk device processes all virtqueues in the same AioConte= xt (IOThread). Further work is necessary to take advantage of multiqueue su= pport in QEMU's block layer once it becomes available. Performance results: Using virtio-blk-pci,num-queues=3D4 can produce a speed-up but -smp 4 introduces a lot of variance across runs. No pinning was performed. Results show that there is no regression anymore, thanks to dropping the batch notify BH patch. RHEL 7.2 guest on RHEL 7.2 host with 1 vcpu and 1 GB RAM unless otherwise noted. The default configuration of the Linux null_blk driver is used as /dev/vdb. $ cat files/fio.job [global] filename=3D/dev/vdb ioengine=3Dlibaio direct=3D1 runtime=3D60 ramp_time=3D5 gtod_reduce=3D1 [job1] numjobs=3D4 iodepth=3D16 rw=3Drandread bs=3D4K $ ./analyze.py runs/ Name IOPS Error unpatched-d6550e9ed2 19269820.2 =C2=B1 1.36% unpatched-dataplane-d6550e9ed2 22351400.4 =C2=B1 1.07% v3-dataplane 22318511.2 =C2=B1 0.77% v3-no-dataplane 18936103.8 =C2=B1 1.12% v3-queues-4-no-dataplane 19177021.8 =C2=B1 1.45% v3-smp-4-no-dataplane 25509585.2 =C2=B1 29.50% v3-smp-4-no-dataplane-no-mq 12466177.2 =C2=B1 7.88% Configuration: Name Patched? Dataplane? SMP? MQ? unpatched-d6550e9ed2 N N N N unpatched-dataplane-d6550e9ed2 N Y N N v3-dataplane Y Y N N v3-no-dataplane Y N N N v3-queues-4-no-dataplane Y N N Y v3-smp-4-no-dataplane Y N Y Y v3-smp-4-no-dataplane-no-mq Y N Y N SMP means -smp 4. MQ means virtio-blk-pci,num-queues=3D4. Stefan Hajnoczi (7): virtio-blk: add VirtIOBlockConf->num_queues virtio-blk: multiqueue batch notify virtio-blk: tell dataplane which vq to notify virtio-blk: associate request with a virtqueue virtio-blk: live migrate s->rq with multiqueue virtio-blk: dataplane multiqueue support virtio-blk: add num-queues device property hw/block/dataplane/virtio-blk.c | 81 +++++++++++++++++++++++++++++------= ------ hw/block/dataplane/virtio-blk.h | 2 +- hw/block/virtio-blk.c | 52 +++++++++++++++++++++----- include/hw/virtio/virtio-blk.h | 6 ++- 4 files changed, 105 insertions(+), 36 deletions(-) --=20 2.5.5