From mboxrd@z Thu Jan 1 00:00:00 1970 From: Chris Wright Subject: Re: [RFC] vhost-blk implementation Date: Mon, 29 Mar 2010 16:56:48 -0700 Message-ID: <20100329235648.GM29241@sequoia.sous-sol.org> References: <1269306023.7931.72.camel@badari-desktop> <20100324200402.GA22272@infradead.org> <1269877312.7931.93.camel@badari-desktop> <20100329182010.GM1744@sequoia.sous-sol.org> <4BB10F75.2000701@redhat.com> <1269903085.7931.99.camel@badari-desktop> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Avi Kivity , Chris Wright , Christoph Hellwig , kvm@vger.kernel.org To: Badari Pulavarty Return-path: Received: from sous-sol.org ([216.99.217.87]:38290 "EHLO sequoia.sous-sol.org" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752144Ab0C2X5R (ORCPT ); Mon, 29 Mar 2010 19:57:17 -0400 Content-Disposition: inline In-Reply-To: <1269903085.7931.99.camel@badari-desktop> Sender: kvm-owner@vger.kernel.org List-ID: * Badari Pulavarty (pbadari@us.ibm.com) wrote: > On Mon, 2010-03-29 at 23:37 +0300, Avi Kivity wrote: > > On 03/29/2010 09:20 PM, Chris Wright wrote: > > > Your io wait time is twice as long and your throughput is about half. > > > I think the qmeu block submission does an extra attempt at merging > > > requests. Does blktrace tell you anything interesting? > > Yes. I see that in my testcase (2M writes) - QEMU is pickup 512K > requests from the virtio ring and merging them back to 2M before > submitting them. > > Unfortunately, I can't do that quite easily in vhost-blk. QEMU > does re-creates iovecs for the merged IO. I have to come up with > a scheme to do this :( Is close cooperator logic kicking in at all? Alternatively, using same io_context. > > It does. I suggest using fio O_DIRECT random access patterns to avoid > > such issues. > > Well, I am not trying to come up with a test case where vhost-blk > performs better than virtio-blk. I am trying to understand where > and why vhost-blk performnce worse than virtio-blk. It would just level the playing field. Alternatively, commenting out merging in qemu to further validate it's the source, something as simple as this in block.c: - num_reqs = multiwrite_merge(bs, reqs, num_reqs, mcb); +// num_reqs = multiwrite_merge(bs, reqs, num_reqs, mcb); Although, from above, sounds like you've already verified this is the difference. IIRC, the sync write path is at odds w/ cfq elevator merging (so cache=none would have trouble, but you were doing cache=writeback, right?), but if you can hack your worker threads to share an io_context, like you had done CLONE_IO, you might get some merging back (again, just a hack to pinpoint merging as the culprit). thanks, -chris