From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from eggs.gnu.org ([2001:4830:134:3::10]:38724)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mlin@kernel.org>) id 1Zyvad-0006DT-5i
	for qemu-devel@nongnu.org; Wed, 18 Nov 2015 00:47:32 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mlin@kernel.org>) id 1ZyvaZ-0005Jl-Vg
	for qemu-devel@nongnu.org; Wed, 18 Nov 2015 00:47:31 -0500
Received: from mail.kernel.org ([198.145.29.136]:33284)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mlin@kernel.org>) id 1ZyvaZ-0005Jh-Q7
	for qemu-devel@nongnu.org; Wed, 18 Nov 2015 00:47:27 -0500
From: Ming Lin <mlin@kernel.org>
Date: Tue, 17 Nov 2015 21:47:02 -0800
Message-Id: <1447825624-17011-1-git-send-email-mlin@kernel.org>
Subject: [Qemu-devel] [RFC PATCH 0/2] Google extension to improve qemu-nvme
	performance
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=subscribe>
To: linux-nvme@lists.infradead.org, qemu-devel@nongnu.org
Cc: fes@google.com, keith.busch@intel.com, tytso@mit.edu, nab@linux-iscsi.org, virtualization@lists.linux-foundation.org, axboe@fb.com, digitaleric@google.com, Rob Nelson <rlnelson@google.com>, Christoph Hellwig <hch@lst.de>, Mihai Rusu <dizzy@google.com>

Hi Rob & Mihai,

I wrote vhost-nvme patches on top of Christoph's NVMe target.
vhost-nvme still uses mmio. So the guest OS can run unmodified NVMe
driver. But the tests I have done didn't show competitive performance
compared to virtio-blk/virtio-scsi. The bottleneck is in mmio. Your nvme
vendor extension patches reduces greatly the number of MMIO writes.
So I'd like to push it upstream.

I port these 2 patches to newer kernel and qemu.
I use ram disk as backend to compare performance.

qemu-nvme: 29MB/s
qemu-nvme+google-ext: 100MB/s
virtio-blk: 174MB/s
virtio-scsi: 118MB/s

I'll show you qemu-vhost-nvme+google-ext number later.

root@guest:~# cat test.job 
[global]
bs=4k
ioengine=libaio
iodepth=64
direct=1
runtime=120
time_based
rw=randread
norandommap
group_reporting
gtod_reduce=1
numjobs=2

[job1]
filename=/dev/nvme0n1
#filename=/dev/vdb
#filename=/dev/sda
rw=read

Patches also available at:
kernel:
https://git.kernel.org/cgit/linux/kernel/git/mlin/linux.git/log/?h=nvme-google-ext
qemu:
http://www.minggr.net/cgit/cgit.cgi/qemu/log/?h=nvme-google-ext 

Thanks,
Ming