Re: Odd (slow) RAID performance

[Bill Davidsen <davidsen@tmr.com>]

Neil Brown wrote:
> On Friday December 8, neilb@suse.de wrote:
>   
>> I have measured very slow write throughput for raid5 as well, though
>> 2.6.18 does seem to have the same problem.  I'll double check and do a
>> git bisect and see what I can come up with.
>>     
>
> Correction... it isn't 2.6.18 that fixes the problem.  It is compiling
> without LOCKDEP or PROVE_LOCKING.  I remove those and suddenly a
> 3 drive raid5 is faster than a single drive rather than much slower.
>
> Bill:  Do you have LOCKDEP or PROVE_LOCKING enabled in your .config ??

YES and NO respectively. I did try increasing the stripe_cache_size and 
got better but not anywhere near max performance, perhaps the 
PROVE_LOCKING is still at fault, although performance of RAID-0 is as 
expected, so I'm dubious. In any case, by pushing the size from 256 to 
1024, 4096, and finally 10240 I was able to raise the speed to 82MB/s, 
which is right at the edge of what I need. I want to read the doc on 
stripe_cache_size before going huge, if that's K 10MB is a LOT of cache 
when 256 works perfectly in RAID-0.

I noted that the performance really was bad using 2k write, before 
increasing the stripe_cache, I will repeat that after doing some other 
"real work" things.

Any additional input appreciated, I would expect the speed to be (Ndisk 
- 1)*SingleDiskSpeed without a huge buffer, so the fact that it isn't 
makes me suspect there's unintended serialization or buffering, even 
when not need (and NOT wanted).

Thanks for the feedback, I'm updating the files as I type.
http://www.tmr.com/~davidsen/RAID_speed
http://www.tmr.com/~davidsen/FC6-config

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979