From mboxrd@z Thu Jan  1 00:00:00 1970
From: Sasha Levin <levinsasha928@gmail.com>
Subject: Re: [PATCH 2/2] kvm tools: Submit multiple virtio-blk requests in
 parallel
Date: Sun, 18 Dec 2011 21:42:48 +0200
Message-ID: <1324237368.4062.1.camel@lappy>
References: <1323951311-16155-1-git-send-email-levinsasha928@gmail.com>
	 <1323951311-16155-2-git-send-email-levinsasha928@gmail.com>
	 <4EEB5C62.3010903@gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Cc: penberg@kernel.org, mingo@elte.hu, gorcunov@gmail.com,
	kvm@vger.kernel.org
To: Asias He <asias.hejun@gmail.com>
Return-path: <kvm-owner@vger.kernel.org>
Received: from mail-ee0-f46.google.com ([74.125.83.46]:51294 "EHLO
	mail-ee0-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
	with ESMTP id S1751841Ab1LRRnZ (ORCPT <rfc822;kvm@vger.kernel.org>);
	Sun, 18 Dec 2011 12:43:25 -0500
Received: by eekc4 with SMTP id c4so4746382eek.19
        for <kvm@vger.kernel.org>; Sun, 18 Dec 2011 09:43:24 -0800 (PST)
In-Reply-To: <4EEB5C62.3010903@gmail.com>
Sender: kvm-owner@vger.kernel.org
List-ID: <kvm.vger.kernel.org>

On Fri, 2011-12-16 at 22:57 +0800, Asias He wrote:
> On 12/15/2011 08:15 PM, Sasha Levin wrote:
> > When using AIO, submit all requests which exists in the vring in a
> single
> > io_submit instead of one io_submit for each descriptor.
> > 
> > Benchmarks:
> > 
> > Short version: 15%+ increase in IOPS, small increase in BW.
> > 
> > Read IOPS:
> > Before:
> >   vda: ios=291792/0, merge=0/0, ticks=35229/0, in_queue=31025,
> util=61.30%
> 
> I guess you are reading the wrong IOPS number, the 'ios' is the number
> of ios performed by all groups, not the IOPS result. Find the
> 'iops' ;-)
> 
> So, Here is the number without/with this patch.
> 
> (seq-read, seq-write, rand-read, rand-write)
> 
> Before:
>   read : io=98304KB, bw=63015KB/s, iops=15753, runt=  1560msec
>   write: io=98304KB, bw=56823KB/s, iops=14205, runt=  1730msec
>   read : io=98304KB, bw=62139KB/s, iops=15534, runt=  1582msec
>   write: io=98304KB, bw=53836KB/s, iops=13458, runt=  1826msec
> 
> After:
>   read : io=98304KB, bw=63096KB/s, iops=15774, runt=  1558msec
>   write: io=98304KB, bw=55823KB/s, iops=13955, runt=  1761msec
>   read : io=98304KB, bw=59148KB/s, iops=14787, runt=  1662msec
>   write: io=98304KB, bw=55072KB/s, iops=13768, runt=  1785msec
> 
> Submit more io requests in one time is not supposed to increase the
> iops
> or bw so dramatically.
> 
> I even tried to submit all read/write ops in one io_submit which still
> ends up with very little iops or bw improvement. 

Did you test it vs /dev/shm?

These are the results I see from the same test:
Before:
	read : io=1157.7MB, bw=118110KB/s, iops=29527 , runt= 10037msec
	write: io=1083.6MB, bw=110867KB/s, iops=27716 , runt= 10008msec
After:
	read : io=1304.4MB, bw=133337KB/s, iops=33334 , runt= 10017msec
	write: io=1292.4MB, bw=132087KB/s, iops=33021 , runt= 10019msec

-- 

Sasha.