Re: [RESEND] [PATCH] readahead:add blk_run_backing_dev

[Vladislav Bolkhovitin <vst@vlnb.net>]

Ronald Moesbergen, on 07/28/2009 01:51 PM wrote:
> 2009/7/27 Vladislav Bolkhovitin <vst@vlnb.net>:
>> Hmm, it's really weird, why the case of 2 threads is faster. There must be
>> some commands reordering somewhere in SCST, which I'm missing, like
>> list_add() instead of list_add_tail().
>>
>> Can you apply the attached patch and repeat tests 5, 8 and 11 with 1 and 2
>> threads, please. The patch will enable forced commands order protection,
>> i.e. with it all the commands will be executed in exactly the same order as
>> they were received.
> 
> The patched source doesn't compile. I changed the code to this:
> 
> @ line 3184:
> 
>         case SCST_CMD_QUEUE_UNTAGGED:
> #if 1 /* left for future performance investigations */
>                 goto ordered;
> #endif
> 
> The results:
> 
> Overall performance seems lower.
> 
> client kernel: 2.6.26-15lenny3 (debian)
> server kernel: 2.6.29.5 with readahead-context, blk_run_backing_dev
> and io_context, forced_order
> 
> With one IO thread:
> 5) client: default, server: default (cfq)
> blocksize       R        R        R   R(avg,    R(std        R
>   (bytes)     (s)      (s)      (s)    MB/s)   ,MB/s)   (IOPS)
>  67108864  16.484   16.417   16.068   62.741    0.706    0.980
>  33554432  15.684   16.348   16.011   63.961    1.083    1.999
>  16777216  16.044   16.239   15.938   63.710    0.493    3.982
> 
> 8) client: default, server: 64 max_sectors_kb, RA 2MB (cfq)
> blocksize       R        R        R   R(avg,    R(std        R
>   (bytes)     (s)      (s)      (s)    MB/s)   ,MB/s)   (IOPS)
>  67108864  16.127   15.784   16.210   63.847    0.740    0.998
>  33554432  16.103   16.072   16.106   63.627    0.061    1.988
>  16777216  16.637   16.058   16.154   62.902    0.970    3.931
> 
> 11) client: 64 max_sectors_kb, 2MB. RA server: 64 max_sectors_kb, RA 2MB (cfq)
> blocksize       R        R        R   R(avg,    R(std        R
>   (bytes)     (s)      (s)      (s)    MB/s)   ,MB/s)   (IOPS)
>  67108864  13.417   15.219   13.912   72.405    3.785    1.131
>  33554432  13.868   13.789   14.110   73.558    0.718    2.299
>  16777216  13.691   13.784   10.280   82.898   11.822    5.181
> 
> 11) client: 64 max_sectors_kb, 2MB. RA server: 64 max_sectors_kb, RA
> 2MB (deadline)
> blocksize       R        R        R   R(avg,    R(std        R
>   (bytes)     (s)      (s)      (s)    MB/s)   ,MB/s)   (IOPS)
>  67108864  13.604   13.532   13.978   74.733    1.055    1.168
>  33554432  13.523   13.166   13.504   76.443    0.945    2.389
>  16777216  13.434   13.409   13.632   75.902    0.557    4.744
> 
> With two threads:
> 5) client: default, server: default (cfq)
> blocksize       R        R        R   R(avg,    R(std        R
>   (bytes)     (s)      (s)      (s)    MB/s)   ,MB/s)   (IOPS)
>  67108864  16.206   16.001   15.908   63.851    0.493    0.998
>  33554432  16.927   16.033   15.991   62.799    1.631    1.962
>  16777216  16.566   15.968   16.212   63.035    0.950    3.940
> 
> 8) client: default, server: 64 max_sectors_kb, RA 2MB (cfq)
> blocksize       R        R        R   R(avg,    R(std        R
>   (bytes)     (s)      (s)      (s)    MB/s)   ,MB/s)   (IOPS)
>  67108864  16.017   15.849   15.748   64.521    0.450    1.008
>  33554432  16.652   15.542   16.259   63.454    1.823    1.983
>  16777216  16.456   16.071   15.943   63.392    0.849    3.962
> 
> 11) client: 64 max_sectors_kb, 2MB. RA server: 64 max_sectors_kb, RA 2MB (cfq)
> blocksize       R        R        R   R(avg,    R(std        R
>   (bytes)     (s)      (s)      (s)    MB/s)   ,MB/s)   (IOPS)
>  67108864  14.109    9.985   13.548   83.572   13.478    1.306
>  33554432  13.698   14.236   13.754   73.711    1.267    2.303
>  16777216  13.610   12.090   14.136   77.458    5.244    4.841
> 
> 11) client: 64 max_sectors_kb, 2MB. RA server: 64 max_sectors_kb, RA
> 2MB (deadline)
> blocksize       R        R        R   R(avg,    R(std        R
>   (bytes)     (s)      (s)      (s)    MB/s)   ,MB/s)   (IOPS)
>  67108864  13.542   13.975   13.978   74.049    1.110    1.157
>  33554432   9.921   13.272   13.321   85.746   12.349    2.680
>  16777216  13.850   13.600   13.344   75.324    1.144    4.708

Can you perform the tests 5 and 8 the deadline? I asked for deadline..

What I/O scheduler do you use on the initiator? Can you check if 
changing it to deadline or noop makes any difference?

Thanks,
Vlad