Re: [RFC] vhost-blk implementation

[Chris Wright <chrisw@sous-sol.org>]

* Badari Pulavarty (pbadari@us.ibm.com) wrote:
> On Mon, 2010-03-29 at 23:37 +0300, Avi Kivity wrote:
> > On 03/29/2010 09:20 PM, Chris Wright wrote:
> > > Your io wait time is twice as long and your throughput is about half.
> > > I think the qmeu block submission does an extra attempt at merging
> > > requests.  Does blktrace tell you anything interesting?
> 
> Yes. I see that in my testcase (2M writes) - QEMU is pickup 512K
> requests from the virtio ring and merging them back to 2M before
> submitting them. 
> 
> Unfortunately, I can't do that quite easily in vhost-blk. QEMU
> does re-creates iovecs for the merged IO. I have to come up with
> a scheme to do this :(

Is close cooperator logic kicking in at all?  Alternatively, using same
io_context.

> > It does.  I suggest using fio O_DIRECT random access patterns to avoid 
> > such issues.
> 
> Well, I am not trying to come up with a test case where vhost-blk
> performs better than virtio-blk. I am trying to understand where
> and why vhost-blk performnce worse than virtio-blk.

It would just level the playing field.  Alternatively, commenting out
merging in qemu to further validate it's the source, something as simple
as this in block.c:

-    num_reqs = multiwrite_merge(bs, reqs, num_reqs, mcb);
+//    num_reqs = multiwrite_merge(bs, reqs, num_reqs, mcb);


Although, from above, sounds like you've already verified this is the
difference.  IIRC, the sync write path is at odds w/ cfq elevator merging
(so cache=none would have trouble, but you were doing cache=writeback,
right?), but if you can hack your worker threads to share an io_context,
like you had done CLONE_IO,  you might get some merging back (again,
just a hack to pinpoint merging as the culprit).

thanks,
-chris