Re: Linux Raid performance

[Learner Study <learner.study@gmail.com>]

Happy Easter!!!

So, 550-600MB/s is the best we have seen with Linux raid using 16-24 SAS drives.

Not sure if its appropriate to ask on this list - has someone seen
better numbers with non-linux raid stack? Perhaps freebsd/lustre..

Thanks for your time!

On Sun, Apr 4, 2010 at 8:00 AM, MRK <mrk@shiftmail.org> wrote:
> Richard Scobie wrote:
>>
>> MRK wrote:
>>
>>> I spent some time trying to optimize it but that was the best I could
>>> get. Anyway both my benchmark and Richard's one imply a very significant
>>> bottleneck somehwere.
>>
>> This bottleneck is the SAS controller, at least in my case. I did the same
>> math regarding streaming performance of one drive times number of drive and
>> wondered where the shortfall was, after tests showed I could only streaming
>> read at 850MB/s on the same array.
>>
>> A query to an LSI engineer got the following response, which basically
>> boils down to "you get what you pay for" - SAS vs SATA drives.
>>
>> "Yes, you're at the "practical" limit.
>>
>> With that setup and SAS disks, you will exceed 1200 MB/s.  Could go
>> higher than 1,400 MB/s given the right server chipset.
>>
>> However with SATA disks, and the way they break up data transfers, 815
>> to 850 MB/s is the best you can do.
>>
>> Under SATA, there are multiple connections per I/O request.
>>  * Command Initiator -> HDD
>>  * DMA Setup  Initiator -> HDD
>>  * DMA Activate  HDD -> Initiator
>>  * Data   HDD -> Initiator
>>  * Status    HDD -> Initiator
>> And there is little ability with typical SATA disks to combine traffic
>> from different I/Os on the same connection.  So you get lots of
>> individual connections being made, used, & broken.
>>
>> Contrast that with SAS which has typically 2 connections per I/O, and
>> will combine traffic from more than 1 I/O per connection.  It uses the
>> SAS links much more efficiently."
>
> Firstly: Happy Easter!  :-)
>
> Secondly:
>
> If this is true then one won't achieve higher speeds even on RAID-0. If
> anybody can test this... I cannot right now
>
> I am a bit surprised though. The SATA "link" is one per drive, so if 1 drive
> is able to do 90MB/sec, N drives on N cables should do Nx90MB/sec.
> If this is not so, then the chipset of the controller must be the
> bottleneck.
> If this is so, the newer LSI controllers at 6.0gbit/sec could be able to do
> better (they supposedly have a faster chip). Also maybe one could buy more
> controller cards and divide drives among those. These two workarounds would
> still be cheaper than SAS drives.
>
>
>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html