From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:53345) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TZ6j4-0000iM-96 for qemu-devel@nongnu.org; Thu, 15 Nov 2012 16:11:57 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TZ6j1-0001E8-6q for qemu-devel@nongnu.org; Thu, 15 Nov 2012 16:11:54 -0500 Received: from e28smtp07.in.ibm.com ([122.248.162.7]:34454) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TZ6j0-0001Dt-J9 for qemu-devel@nongnu.org; Thu, 15 Nov 2012 16:11:51 -0500 Received: from /spool/local by e28smtp07.in.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 16 Nov 2012 02:41:43 +0530 Received: from d28av04.in.ibm.com (d28av04.in.ibm.com [9.184.220.66]) by d28relay05.in.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id qAFLBdQX721306 for ; Fri, 16 Nov 2012 02:41:39 +0530 Received: from d28av04.in.ibm.com (loopback [127.0.0.1]) by d28av04.in.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id qAG2fUZH031144 for ; Fri, 16 Nov 2012 13:41:30 +1100 From: Anthony Liguori In-Reply-To: References: <1352992746-8767-1-git-send-email-stefanha@redhat.com> <1352992746-8767-8-git-send-email-stefanha@redhat.com> <20121115184849.GA30544@redhat.com> Date: Thu, 15 Nov 2012 15:11:32 -0600 Message-ID: <87a9uiy3jv.fsf@codemonkey.ws> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: Re: [Qemu-devel] [PATCH 7/7] virtio-blk: add x-data-plane=on|off performance feature List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Khoa Huynh , "Michael S. Tsirkin" Cc: Kevin Wolf , Paolo Bonzini , Asias He , qemu-devel@nongnu.org, Stefan Hajnoczi Khoa Huynh writes: > "Michael S. Tsirkin" wrote on 11/15/2012 12:48:49 PM: > >> From: "Michael S. Tsirkin" >> To: Stefan Hajnoczi , >> Cc: qemu-devel@nongnu.org, Anthony Liguori/Austin/IBM@IBMUS, Paolo >> Bonzini , Kevin Wolf , Asias >> He , Khoa Huynh/Austin/IBM@IBMUS >> Date: 11/15/2012 12:46 PM >> Subject: Re: [PATCH 7/7] virtio-blk: add x-data-plane=on|off >> performance feature >> >> On Thu, Nov 15, 2012 at 04:19:06PM +0100, Stefan Hajnoczi wrote: >> > The virtio-blk-data-plane feature is easy to integrate into >> > hw/virtio-blk.c. The data plane can be started and stopped similar to >> > vhost-net. >> > >> > Users can take advantage of the virtio-blk-data-plane feature using the >> > new -device virtio-blk-pci,x-data-plane=on property. >> > >> > The x-data-plane name was chosen because at this stage the feature is >> > experimental and likely to see changes in the future. >> > >> > If the VM configuration does not support virtio-blk-data-plane an error >> > message is printed. Although we could fall back to regular virtio-blk, >> > I prefer the explicit approach since it prompts the user to fix their >> > configuration if they want the performance benefit of >> > virtio-blk-data-plane. >> > >> > Limitations: >> > * Only format=raw is supported >> > * Live migration is not supported >> > * Block jobs, hot unplug, and other operations fail with -EBUSY >> > * I/O throttling limits are ignored >> > * Only Linux hosts are supported due to Linux AIO usage >> > >> > Signed-off-by: Stefan Hajnoczi >> >> >> Would be very interested in learning about the performance >> impact of this. How does this compare to current model and >> to vhost-blk? > > I plan to do a complete evaluation of this patchset in the coming days. > However, I have done quite a bit of performance testing on earlier versions > of the data-plane and vhost-blk code bits. Here's what I found: > > 1) The existing kvm/qemu code can only handle up to about 150,000 IOPS for > a single KVM guest. The bottleneck here is the global qemu mutex. > > 2) With performance tuning, I was able to achieve 1.33 million IOPS for a > single KVM guest with data-plane. This is very close to the > 1.4-million-IOPS > limit of my storage setup. >>From my POV, if we can get this close to bare metal with virtio-blk-dataplane, there's absolutely no reason to merge vhost-blk support. We simply lose too much with a kernel-based solution. I'm sure there's more we can do to improve the userspace implementation too like a hypercall-based notify and adaptive polling. Regards, Anthony Liguori