Re: Time sliced CFQ io scheduler

[Jens Axboe <axboe@suse.de>]

On Thu, Dec 02 2004, Andrew Morton wrote:
> Jens Axboe <axboe@suse.de> wrote:
> >
> > > So what are you doing different?
> > 
> > Doing sync io, most likely. My results above are 64k O_DIRECT reads and
> > writes, see the mention of the test cases in the first mail.
> 
> OK.
> 
> Writer:
> 
> 	while true
> 	do
> 	write-and-fsync -o -m 100 -c 65536 foo 
> 	done
> 
> Reader:
> 
> 	time-read -o -b 65536 -n 256 x      (This is O_DIRECT)
> or:	time-read -b 65536 -n 256 x	    (This is buffered)
> 
> `vmstat 1':
> 
> procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
>  r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa
>  1  1   1032 137412   4276  84388   32    0 15456 25344 1659  1538  0  3 50 47
>  0  1   1032 137468   4276  84388    0    0     0 32128 1521  1027  0  2 51 48
>  0  1   1032 137476   4276  84388    0    0     0 32064 1519  1026  0  1 50 49
>  0  1   1032 137476   4276  84388    0    0     0 33920 1556  1102  0  2 50 49
>  0  1   1032 137476   4276  84388    0    0     0 33088 1541  1074  0  1 50 49
>  0  2   1032 135676   4284  85944    0    0  1656 29732 1868  2506  0  3 49 47
>  1  1   1032  96532   4292 125172    0    0 39220   128 10813 39313  0 31 35 34
>  0  2   1032  57724   4332 163892    0    0 38828   128 10716 38907  0 28 38 35
>  0  2   1032  18860   4368 202684    0    0 38768   128 10701 38845  1 28 38 35
>  0  2   1032   3672   4248 217764    0    0 39188   128 10803 39327  0 28 37 34
>  0  1   1032   2832   4260 218840    0    0 16812 17932 5504 17457  0 14 46 40

Well there you go, exactly what I saw. The writer(s) basically make no
progress as long as the reader is going. Since 'as' treats the sync
writes like reads internally and given the really bad fairness problems
demonstrated for same direction clients, that might be the same problem.

> Ugly.
> 
> (write-and-fsync and time-read are from
> http://www.zip.com.au/~akpm/linux/patches/stuff/ext3-tools.tar.gz)

I'll try and post my cruddy test programs tomorrow as well. Pretty handy
for getting a good feel for N client read/write performance.

-- 
Jens Axboe