Degraded RAID performance - Was : Re: [OOPS] 2.6.21-rc6-git5 in cfq_dispatch_insert

[Brad Campbell <brad@wasp.net.au>]

Neil Brown wrote:
> I wonder if we should avoid bypassing the stripe cache if the needed stripes
> are already in the cache... or if at least one needed stripe is.... or
> if the array is degraded...
> Probably in the degraded case we should never bypass the cache, as if
> we do, then a sequential read of a full stripe will read every block
> twice.  I'd better to some performance measurements.

Ok, that would explain some odd performance issues I've noticed.
Let's say I run

dstat -D sda,sdb,sdc,sdd,md0 5
----total-cpu-usage---- --disk/sda----disk/sdb----disk/sdc----disk/sdd----disk/md0- -net/total- 
---paging-- ---system--
usr sys idl wai hiq siq|_read write _read write _read write _read write _read write|_recv 
_send|__in_ _out_|_int_ _csw_
  25  22   0  47   0   6|20.1M    0 :20.2M    0 :20.1M    0 :   0     0 :40.2M    0 | 146B  662B| 
0     0 |1186   661
  26  20   0  46   0   8|19.4M    0 :19.4M    0 :19.4M    0 :   0     0 :38.9M    0 | 160B  549B| 
0     0 |1365   650

Given I'm doing a read, I would have expected a read to consist of 2 direct reads, one parity read 
and some calculation. The numbers I'm seeing however show 3 reads for 2 reads worth of bandwidth.

root@storage2:~# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sda[0] sdc[2] sdb[1]
       585934080 blocks level 5, 64k chunk, algorithm 2 [4/3] [UUU_]

(Dropped Jens and Chuck from the cc as this likely has little interest for them)

Brad
-- 
"Human beings, who are almost unique in having the ability
to learn from the experience of others, are also remarkable
for their apparent disinclination to do so." -- Douglas Adams