From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:35001) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Sf8pv-00074M-RH for qemu-devel@nongnu.org; Thu, 14 Jun 2012 08:07:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Sf8pq-0005sC-Rv for qemu-devel@nongnu.org; Thu, 14 Jun 2012 08:07:39 -0400 Received: from e06smtp11.uk.ibm.com ([195.75.94.107]:59842) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Sf8pq-0005p8-En for qemu-devel@nongnu.org; Thu, 14 Jun 2012 08:07:34 -0400 Received: from /spool/local by e06smtp11.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 14 Jun 2012 13:07:29 +0100 Received: from d06av09.portsmouth.uk.ibm.com (d06av09.portsmouth.uk.ibm.com [9.149.37.250]) by d06nrmr1806.portsmouth.uk.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q5EC7PQq2732274 for ; Thu, 14 Jun 2012 13:07:25 +0100 Received: from d06av09.portsmouth.uk.ibm.com (loopback [127.0.0.1]) by d06av09.portsmouth.uk.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q5EC7NWW005988 for ; Thu, 14 Jun 2012 06:07:24 -0600 Date: Thu, 14 Jun 2012 13:07:22 +0100 From: Stefan Hajnoczi Message-ID: <20120614120722.GA7128@stefanha-thinkpad.localdomain> References: <1339582392.24309.15.camel@mengcong> <1339667122.28851.8.camel@mengcong> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1339667122.28851.8.camel@mengcong> Subject: Re: [Qemu-devel] IO performance test on the tcm-vhost scsi List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Cong Meng Cc: Stefan Hajnoczi , linuxram@us.ibm.com, qemu-devel@nongnu.org, "Nicholas A. Bellinger" , target-devel@vger.kernel.org, Anthony Liguori , Paolo Bonzini , Asias He On Thu, Jun 14, 2012 at 05:45:22PM +0800, Cong Meng wrote: > On Thu, 2012-06-14 at 09:30 +0100, Stefan Hajnoczi wrote: > > On Wed, Jun 13, 2012 at 11:13 AM, mengcong wrote: > > > seq-read seq-write rand-read rand-write > > > 8k 256k 8k 256k 8k 256k 8k 256k > > > ---------------------------------------------------------------------------- > > > bare-metal 67951 69802 67064 67075 1758 29284 1969 26360 > > > tcm-vhost-iblock 61501 66575 51775 67872 1011 22533 1851 28216 > > > tcm-vhost-pscsi 66479 68191 50873 67547 1008 22523 1818 28304 > > > virtio-blk 26284 66737 23373 65735 1724 28962 1805 27774 > > > scsi-disk 36013 60289 46222 62527 1663 12992 1804 27670 > > > > > > > > unit: KB/s > > > seq-read/write = sequential read/write > > > rand-read/write = random read/write > > > 8k,256k are blocksize of the IO > > > > What strikes me is how virtio-blk performs significantly worse than > > bare metal and tcm_vhost for seq-read/seq-write 8k. The good > > tcm_vhost results suggest that the overhead is not the virtio > > interface itself, since tcm_vhost implements virtio-scsi. > > > > To drill down on the tcm_vhost vs userspace performance gap we need > > virtio-scsi userspace results. QEMU needs to use the same block > > device as the tcm-vhost-iblock benchmark. > > > > Cong: Is it possible to collect the virtio-scsi userspace results > > using the same block device as tcm-vhost-iblock and -drive > > format=raw,aio=native,cache=none? > > > > virtio-scsi-raw 43065 69729 52052 67378 1757 29419 2024 28135 > > qemu ....\ > -drive file=/dev/sdb,format=raw,if=none,id=sdb,cache=none,aio=native \ > -device virtio-scsi-pci,id=mcbus \ > -device scsi-disk,drive=sdb > > there is only one scsi HBA. > /dev/sdb is the disk on which all tests have been done. > > Is this what you want? Perfect, thanks. virtio-scsi userspace is much better than virtio-blk here. That's unexpected since they both use the QEMU block layer. If anything, I would have expected virtio-blk to be faster! I wonder if the request patterns being sent through virtio-blk and virtio-scsi are different. Asias discovered that the guest I/O scheduler and request merging makes a big difference between QEMU and native KVM tool performance. It could be the same thing here which causes virtio-blk and virtio-scsi userspace to produce quite different results. The second question is why is tcm_vhost faster than virtio-scsi userspace. Stefan