Re: Linux SATA RAID FAQ

["J. Ryan Earl" <heretic@clanhk.org>]

Alan Cox wrote:

>Be cautious what you measure. One of he problems until you reach PCI-X
>is PCI bandwidth. Thus software md5 can look good but the moment its
>combined with other PCI activity goes down the pan entirely.
>  
>
Right, which is why you'd think hardware based RAID would fair better, 
the parity information or mirror'd writes would only require one 
transfer/transaction across the PCI bus.  However, when benchmarking a 
3Ware 4-port ide raid controller (7000) we saw 40-50MB/sec read/write on 
4x160GB drives raid5.  With md raid5 (same controller, HDs, FS, etc)  we 
saw 100MB/sec read, 60MB/sec write.  This was using max 15% CPU on a 
2.4GHz Pentium4, with a 32bit/33MHz PCI bus.

>  
>
>>When the libata Marvell drivers come out, you'll have a cheap upgrade 
>>path for PCI-X boards if you want fast md raid: 
>>    
>>
>
>Agreed. PCI-X will change a lot of this for boxes that are not very
>cpu/memory limited.
>  
>
 From the testing I've done, interconnect bandwidth has always been the 
limiting factor for the md driver.  Using cheap ($~230) PCI-X 
motherboards--http://www.supermicro.com/products/motherboard/P4/E7210/P4SCT+.cfm--with 
dedicated gigE channels, you can make high density/price and 
performance/price NAS type appliances with no bottlenecks.  For <$3K you 
can build a 2TB NAS server that'll keep a 250MB/s gigE link saturated.

-ryan