Re: Linux Raid performance

[John Robinson <john.robinson@anonymous.org.uk>]

On 05/04/2010 11:07, Learner Study wrote:
> Hi Mark, all:
> 
> My apologies if I sounded like a pest by asking too many
> questions...

Don't worry about it, but do take care - I'm afraid I was similarly 
sceptical about the email alias you've chosen, and got the impression 
from some of your questions that you might be a kid asking people here 
to do your homework for you, which does happen from time to time.

> Since I don't have access to array of disks, I sought to
> the list to see if I could use the info already out there instead of
> reinventing it...how well suited is linux raid for 10G kind of
> traffic..

If you mean 10G bits per second, or at least enough for a 10GigE 
network, then yes I think it is, as evidenced by people here saying 
they've got up to 1400MB/s, as long as you've got enough really fast SAS 
drives hooked up via high-class SAS interfaces over wide PCI Express 
buses to Xeon CPUs with fast RAM, though none of this is exactly 
commodity-class hardware.

As far as I know, 10G bytes per second is beyond PC hardware at the 
moment, and I have no knowledge or experience of the kinds of hardware 
that could reach those speeds.

You got me thinking about the bottlenecks, though. Just for fun (so you 
can see what an odd idea of fun I have), I just restarted my big storage 
server box (Core 2 Quad 3.2GHz on 1600MHz FSB, dual-channel DDR2-800 
memory, Intel P45/ICH10R motherboard) into memtest86+, and it's telling 
me that my memory bandwidth is 4.5GB/s. That's probably about the limit 
of this architecture, though the newer Core i5/7 chips with dual- and 
triple-channel memory controllers integrated can probably manage more.

Now, if I'm running RAID-5 or 6, and Linux md has to copy every page I 
want to write to disc (therefore read it from RAM and write it back 
again), and then read it again to calculate the parity block(s), then 
the SATA/SAS controllers have to read it, that's at least 4 memory 
operations for every write to disc, which is going to mean there's a 
memory subsystem bottleneck/throttle on this machine of about 
1.125GB/sec. I skipped over the bit where I have to write the parity to 
RAM and also have the SATA/SAS controllers read that to write it to the 
disc, but that's going to make a fairly modest difference, maybe down to 
1GB/sec. Since my CPU can calculate RAID-5 or 6 parity on a single core 
at 8GB/sec, that's not the bottleneck.

And I do understand that I'd be rather optimistic to think my RAM 
bandwidth was as much as 4.5GB/s for much of the time.

Reads should be quicker though :-)

Cheers,

John.