From mboxrd@z Thu Jan 1 00:00:00 1970 From: Badari Pulavarty Subject: Re: [RFC] vhost-blk implementation Date: Mon, 29 Mar 2010 15:51:25 -0700 Message-ID: <1269903085.7931.99.camel@badari-desktop> References: <1269306023.7931.72.camel@badari-desktop> <20100324200402.GA22272@infradead.org> <1269877312.7931.93.camel@badari-desktop> <20100329182010.GM1744@sequoia.sous-sol.org> <4BB10F75.2000701@redhat.com> Mime-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 7bit Cc: Chris Wright , Christoph Hellwig , kvm@vger.kernel.org To: Avi Kivity Return-path: Received: from e31.co.us.ibm.com ([32.97.110.149]:34666 "EHLO e31.co.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751631Ab0C2Wu6 (ORCPT ); Mon, 29 Mar 2010 18:50:58 -0400 Received: from d03relay01.boulder.ibm.com (d03relay01.boulder.ibm.com [9.17.195.226]) by e31.co.us.ibm.com (8.14.3/8.13.1) with ESMTP id o2TMfl5Z012632 for ; Mon, 29 Mar 2010 16:41:47 -0600 Received: from d03av01.boulder.ibm.com (d03av01.boulder.ibm.com [9.17.195.167]) by d03relay01.boulder.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id o2TMooaE154998 for ; Mon, 29 Mar 2010 16:50:51 -0600 Received: from d03av01.boulder.ibm.com (loopback [127.0.0.1]) by d03av01.boulder.ibm.com (8.14.3/8.13.1/NCO v10.0 AVout) with ESMTP id o2TMonVU014360 for ; Mon, 29 Mar 2010 16:50:50 -0600 In-Reply-To: <4BB10F75.2000701@redhat.com> Sender: kvm-owner@vger.kernel.org List-ID: On Mon, 2010-03-29 at 23:37 +0300, Avi Kivity wrote: > On 03/29/2010 09:20 PM, Chris Wright wrote: > > * Badari Pulavarty (pbadari@us.ibm.com) wrote: > > > >> I modified my vhost-blk implementation to offload work to > >> work_queues instead of doing synchronously. Infact, I tried > >> to spread the work across all the CPUs. But to my surprise, > >> this did not improve the performance compared to virtio-blk. > >> > >> I see vhost-blk taking more interrupts and context switches > >> compared to virtio-blk. What is virtio-blk doing which I > >> am not able to from vhost-blk ??? > >> > > Your io wait time is twice as long and your throughput is about half. > > I think the qmeu block submission does an extra attempt at merging > > requests. Does blktrace tell you anything interesting? > > Yes. I see that in my testcase (2M writes) - QEMU is pickup 512K requests from the virtio ring and merging them back to 2M before submitting them. Unfortunately, I can't do that quite easily in vhost-blk. QEMU does re-creates iovecs for the merged IO. I have to come up with a scheme to do this :( > It does. I suggest using fio O_DIRECT random access patterns to avoid > such issues. Well, I am not trying to come up with a test case where vhost-blk performs better than virtio-blk. I am trying to understand where and why vhost-blk performnce worse than virtio-blk. Thanks, Badari