From mboxrd@z Thu Jan 1 00:00:00 1970 From: Chris Wright Subject: Re: [RFC] vhost-blk implementation Date: Mon, 29 Mar 2010 11:20:10 -0700 Message-ID: <20100329182010.GM1744@sequoia.sous-sol.org> References: <1269306023.7931.72.camel@badari-desktop> <20100324200402.GA22272@infradead.org> <1269877312.7931.93.camel@badari-desktop> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Christoph Hellwig , kvm@vger.kernel.org To: Badari Pulavarty Return-path: Received: from sous-sol.org ([216.99.217.87]:49154 "EHLO sequoia.sous-sol.org" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750925Ab0C2SUZ (ORCPT ); Mon, 29 Mar 2010 14:20:25 -0400 Content-Disposition: inline In-Reply-To: <1269877312.7931.93.camel@badari-desktop> Sender: kvm-owner@vger.kernel.org List-ID: * Badari Pulavarty (pbadari@us.ibm.com) wrote: > I modified my vhost-blk implementation to offload work to > work_queues instead of doing synchronously. Infact, I tried > to spread the work across all the CPUs. But to my surprise, > this did not improve the performance compared to virtio-blk. > > I see vhost-blk taking more interrupts and context switches > compared to virtio-blk. What is virtio-blk doing which I > am not able to from vhost-blk ??? Your io wait time is twice as long and your throughput is about half. I think the qmeu block submission does an extra attempt at merging requests. Does blktrace tell you anything interesting? > procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu----- > r b swpd free buff cache si so bi bo in cs us sy id wa st > 3 1 8920 56076 20760 5603556 0 104 196 79826 17164 13912 0 5 65 30 0 > 2 4 9488 57216 20744 5605616 0 114 195 81120 17397 13824 0 5 65 30 0 > 2 2 10028 68476 20728 5594764 0 108 206 80318 17162 13845 0 5 65 30 0 > 0 4 10560 70856 20708 5593088 0 106 205 82363 17402 13904 0 5 65 30 0 > 1 3 10948 80380 20672 5584452 0 78 178 79714 17113 13875 0 5 66 29 0 > > qemu virtio-blk: > > procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu----- > r b swpd free buff cache si so bi bo in cs us sy id wa st > 0 1 14124 57456 5144 4924060 0 0 139 142546 11287 9312 1 4 80 15 0 > 0 2 14124 56736 5148 4927396 0 0 146 142968 11283 9248 1 4 80 15 0 > 0 1 14124 56712 5384 4927020 0 0 74 150738 11182 9327 1 4 80 16 0 > 1 1 14124 55496 5392 4927904 0 0 2 159902 11172 9401 1 3 79 17 0 > 0 1 14124 55968 5408 4927232 0 0 0 159202 11212 9325 1 3 80 16 0