From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35747) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bExb2-0004ZP-Gl for qemu-devel@nongnu.org; Mon, 20 Jun 2016 07:42:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bExax-00012p-Fo for qemu-devel@nongnu.org; Mon, 20 Jun 2016 07:42:27 -0400 Received: from mail-wm0-x242.google.com ([2a00:1450:400c:c09::242]:32919) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bExax-00012j-5p for qemu-devel@nongnu.org; Mon, 20 Jun 2016 07:42:23 -0400 Received: by mail-wm0-x242.google.com with SMTP id r201so13378367wme.0 for ; Mon, 20 Jun 2016 04:42:23 -0700 (PDT) Sender: Paolo Bonzini References: <1465316911-20052-1-git-send-email-stefanha@redhat.com> <20160620103612.GF3314@stefanha-x1.localdomain> From: Paolo Bonzini Message-ID: <245424eb-2ef5-20ab-32d3-bf64510b4686@redhat.com> Date: Mon, 20 Jun 2016 13:42:14 +0200 MIME-Version: 1.0 In-Reply-To: <20160620103612.GF3314@stefanha-x1.localdomain> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Subject: Re: [Qemu-devel] [PATCH v3 0/7] virtio-blk: multiqueue support List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi , qemu-devel@nongnu.org Cc: Kevin Wolf , Fam Zheng , Ming Lei , Christian Borntraeger , Roman Pen On 20/06/2016 12:36, Stefan Hajnoczi wrote: > On Tue, Jun 07, 2016 at 05:28:24PM +0100, Stefan Hajnoczi wrote: >> v3: >> * Drop Patch 1 to batch guest notify for non-dataplane >> >> The Linux AIO completion BH and the virtio-blk batch notify BH changed order >> in the AioContext->first_bh list as a side-effect of moving the BH from >> hw/block/dataplane/virtio-blk.c to hw/block/virtio-blk.c. This caused a >> serious performance regression for both dataplane and non-dataplane. >> >> I've decided not to move the BH in this series and work on a separate >> solution for making batch notify generic. >> >> The remaining patches have been reordered and cleaned up. >> >> * See performance data below. >> >> v2: >> * Simplify s->rq live migration [Paolo] >> * Use more efficient bitmap ops for batch notification [Paolo] >> * Fix perf regression due to batch notify BH in wrong AioContext [Christian] >> >> The virtio_blk guest driver has supported multiple virtqueues since Linux 3.17. >> This patch series adds multiple virtqueues to QEMU's virtio-blk emulated >> device. >> >> Ming Lei sent patches previously but these were not merged. This series >> implements virtio-blk multiqueue for QEMU from scratch since the codebase has >> changed. Live migration support for s->rq was also missing from the previous >> series and has been added. >> >> It's important to note that QEMU's block layer does not support multiqueue yet. >> Therefore virtio-blk device processes all virtqueues in the same AioContext >> (IOThread). Further work is necessary to take advantage of multiqueue support >> in QEMU's block layer once it becomes available. >> >> Performance results: >> >> Using virtio-blk-pci,num-queues=4 can produce a speed-up but -smp 4 >> introduces a lot of variance across runs. No pinning was performed. >> >> Results show that there is no regression anymore, thanks to dropping the >> batch notify BH patch. >> >> RHEL 7.2 guest on RHEL 7.2 host with 1 vcpu and 1 GB RAM unless otherwise >> noted. The default configuration of the Linux null_blk driver is used as >> /dev/vdb. >> >> $ cat files/fio.job >> [global] >> filename=/dev/vdb >> ioengine=libaio >> direct=1 >> runtime=60 >> ramp_time=5 >> gtod_reduce=1 >> >> [job1] >> numjobs=4 >> iodepth=16 >> rw=randread >> bs=4K >> >> $ ./analyze.py runs/ >> Name IOPS Error >> unpatched-d6550e9ed2 19269820.2 ± 1.36% >> unpatched-dataplane-d6550e9ed2 22351400.4 ± 1.07% >> v3-dataplane 22318511.2 ± 0.77% >> v3-no-dataplane 18936103.8 ± 1.12% >> v3-queues-4-no-dataplane 19177021.8 ± 1.45% >> v3-smp-4-no-dataplane 25509585.2 ± 29.50% >> v3-smp-4-no-dataplane-no-mq 12466177.2 ± 7.88% >> >> Configuration: >> Name Patched? Dataplane? SMP? MQ? >> unpatched-d6550e9ed2 N N N N >> unpatched-dataplane-d6550e9ed2 N Y N N >> v3-dataplane Y Y N N >> v3-no-dataplane Y N N N >> v3-queues-4-no-dataplane Y N N Y >> v3-smp-4-no-dataplane Y N Y Y >> v3-smp-4-no-dataplane-no-mq Y N Y N >> >> SMP means -smp 4. >> MQ means virtio-blk-pci,num-queues=4. >> >> Stefan Hajnoczi (7): >> virtio-blk: add VirtIOBlockConf->num_queues >> virtio-blk: multiqueue batch notify >> virtio-blk: tell dataplane which vq to notify >> virtio-blk: associate request with a virtqueue >> virtio-blk: live migrate s->rq with multiqueue >> virtio-blk: dataplane multiqueue support >> virtio-blk: add num-queues device property >> >> hw/block/dataplane/virtio-blk.c | 81 +++++++++++++++++++++++++++++------------ >> hw/block/dataplane/virtio-blk.h | 2 +- >> hw/block/virtio-blk.c | 52 +++++++++++++++++++++----- >> include/hw/virtio/virtio-blk.h | 6 ++- >> 4 files changed, 105 insertions(+), 36 deletions(-) > > Ping? > Looks good to me, but I'll never understand who's supposed to apply virtio-blk patches. :) Paolo