Re: Slow file transfer speeds with CFQ IO scheduler in some cases

[Wu Fengguang <wfg@linux.intel.com>]

On Tue, Feb 17, 2009 at 10:01:13PM +0300, Vladislav Bolkhovitin wrote:
> Vladislav Bolkhovitin, on 02/12/2009 09:35 PM wrote:
>> Additional interesting observation is how badly simultaneous read IO  
>> streams are handled, if they aren't grouped in the corresponding IO  
>> contexts. In test 3 the result was as low as 4(!)MB/s. Wu, Jens, do you 
>> have any explanation on this? Why the inner tracks have so big 
>> preference?
>
> I realized, there is another explanation: access becomes about to be  
> completely random. I checked and it is true. Here is a sample "iostat -x  
> 3" output on the server:

Yes it's all about 64K sized reads. Is this the stripe size of md0?

Thanks,
Fengguang

> avg-cpu:  %user   %nice %system %iowait  %steal   %idle
>            0.00    0.00    0.57   26.62    0.00   72.81
>
> Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s  
> avgrq-sz avgqu-sz   await  svctm  %util
> sda             710.00     0.00   47.33    0.00  6058.67     0.00 128.00  
>    0.12    2.59   2.51  11.87
> sdb             710.00     0.00   47.33    0.00  6058.67     0.00 128.00  
>    3.99   84.34  21.13 100.00
> sdc               0.00     0.00    0.00    0.00     0.00     0.00 0.00    
>  0.00    0.00   0.00   0.00
> sdc1              0.00     0.00    0.00    0.00     0.00     0.00 0.00    
>  0.00    0.00   0.00   0.00
> sdc2              0.00     0.00    0.00    0.00     0.00     0.00 0.00    
>  0.00    0.00   0.00   0.00
> md0               0.00     0.00 1514.67    0.00 12117.33     0.00 8.00    
>  0.00    0.00   0.00   0.00
> dm-0              0.00     0.00  757.33    0.00  6058.67     0.00 8.00    
> 32.87   43.41   1.32 100.00
> dm-1              0.00     0.00    0.00    0.00     0.00     0.00 0.00    
>  0.00    0.00   0.00   0.00
> dm-2              0.00     0.00  757.33    0.00  6058.67     0.00 8.00    
> 32.79   43.33   1.32 100.00
>
> Thanks,
> Vlad