From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58804) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bFKkK-0003GF-9g for qemu-devel@nongnu.org; Tue, 21 Jun 2016 08:25:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bFKkF-0000R4-5e for qemu-devel@nongnu.org; Tue, 21 Jun 2016 08:25:36 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:10221 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bFKkF-0000Qr-0D for qemu-devel@nongnu.org; Tue, 21 Jun 2016 08:25:31 -0400 Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.11/8.16.0.11) with SMTP id u5LCOY5a057807 for ; Tue, 21 Jun 2016 08:25:30 -0400 Received: from e06smtp13.uk.ibm.com (e06smtp13.uk.ibm.com [195.75.94.109]) by mx0b-001b2d01.pphosted.com with ESMTP id 23n2eksx07-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Tue, 21 Jun 2016 08:25:30 -0400 Received: from localhost by e06smtp13.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 21 Jun 2016 13:25:28 +0100 Received: from b06cxnps3074.portsmouth.uk.ibm.com (d06relay09.portsmouth.uk.ibm.com [9.149.109.194]) by d06dlp03.portsmouth.uk.ibm.com (Postfix) with ESMTP id 5EBD31B0806E for ; Tue, 21 Jun 2016 13:26:37 +0100 (BST) Received: from d06av08.portsmouth.uk.ibm.com (d06av08.portsmouth.uk.ibm.com [9.149.37.249]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id u5LCPP0R65339568 for ; Tue, 21 Jun 2016 12:25:25 GMT Received: from d06av08.portsmouth.uk.ibm.com (localhost [127.0.0.1]) by d06av08.portsmouth.uk.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id u5LCPPPt018333 for ; Tue, 21 Jun 2016 06:25:25 -0600 References: <1466511196-12612-1-git-send-email-stefanha@redhat.com> From: Christian Borntraeger Date: Tue, 21 Jun 2016 14:25:24 +0200 MIME-Version: 1.0 In-Reply-To: <1466511196-12612-1-git-send-email-stefanha@redhat.com> Content-Type: text/plain; charset=utf-8 Message-Id: <57693234.9070701@de.ibm.com> Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [PATCH v4 0/7] virtio-blk: multiqueue support List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi , qemu-devel@nongnu.org Cc: Kevin Wolf , Roman Pen , Fam Zheng , Paolo Bonzini , Ming Lei On 06/21/2016 02:13 PM, Stefan Hajnoczi wrote: > v4: > * Rebased onto qemu.git/master > * Included latest performance results >=20 > v3: > * Drop Patch 1 to batch guest notify for non-dataplane >=20 > The Linux AIO completion BH and the virtio-blk batch notify BH chang= ed order > in the AioContext->first_bh list as a side-effect of moving the BH f= rom > hw/block/dataplane/virtio-blk.c to hw/block/virtio-blk.c. This caus= ed a > serious performance regression for both dataplane and non-dataplane. >=20 > I've decided not to move the BH in this series and work on a separat= e > solution for making batch notify generic. >=20 > The remaining patches have been reordered and cleaned up. >=20 > * See performance data below. >=20 > v2: > * Simplify s->rq live migration [Paolo] > * Use more efficient bitmap ops for batch notification [Paolo] > * Fix perf regression due to batch notify BH in wrong AioContext [Chri= stian] >=20 > The virtio_blk guest driver has supported multiple virtqueues since Lin= ux 3.17. > This patch series adds multiple virtqueues to QEMU's virtio-blk emulate= d > device. >=20 > Ming Lei sent patches previously but these were not merged. This serie= s > implements virtio-blk multiqueue for QEMU from scratch since the codeba= se has > changed. Live migration support for s->rq was also missing from the pr= evious > series and has been added. >=20 > It's important to note that QEMU's block layer does not support multiqu= eue yet. > Therefore virtio-blk device processes all virtqueues in the same AioCon= text > (IOThread). Further work is necessary to take advantage of multiqueue = support > in QEMU's block layer once it becomes available. >=20 > Performance results: >=20 > Using virtio-blk-pci,num-queues=3D4 can produce a speed-up but -smp 4 > introduces a lot of variance across runs. No pinning was performed. >=20 > RHEL 7.2 guest on RHEL 7.2 host with 1 vcpu and 1 GB RAM unless otherwi= se > noted. The default configuration of the Linux null_blk driver is used = as > /dev/vdb. >=20 > $ cat files/fio.job > [global] > filename=3D/dev/vdb > ioengine=3Dlibaio > direct=3D1 > runtime=3D60 > ramp_time=3D5 > gtod_reduce=3D1 >=20 > [job1] > numjobs=3D4 > iodepth=3D16 > rw=3Drandread > bs=3D4K >=20 > $ ./analyze.py runs/ > Name IOPS Error > v4-smp-4-dataplane 13326598.0 =C2=B1 6.31% > v4-smp-4-dataplane-no-mq 11483568.0 =C2=B1 3.42% > v4-smp-4-no-dataplane 18108611.6 =C2=B1 1.53% > v4-smp-4-no-dataplane-no-mq 13951225.6 =C2=B1 7.81% This differs from the previous numbers. What is with and what is without patch? I am surprised to see dataplane to be slower than no-dataplane - this contradicts everything that I have seen in the past. >=20 > Stefan Hajnoczi (7): > virtio-blk: add VirtIOBlockConf->num_queues > virtio-blk: multiqueue batch notify > virtio-blk: tell dataplane which vq to notify > virtio-blk: associate request with a virtqueue > virtio-blk: live migrate s->rq with multiqueue > virtio-blk: dataplane multiqueue support > virtio-blk: add num-queues device property >=20 > hw/block/dataplane/virtio-blk.c | 81 +++++++++++++++++++++++++++++----= -------- > hw/block/dataplane/virtio-blk.h | 2 +- > hw/block/virtio-blk.c | 52 +++++++++++++++++++++----- > include/hw/virtio/virtio-blk.h | 6 ++- > 4 files changed, 105 insertions(+), 36 deletions(-) >=20