RAID10 far (f2) read throughput on random and sequential / read-ahead

RAID10 far (f2) read throughput on random and sequential / read-ahead

[Nat Makarevitch]

'md' performs wonderfully. Thanks to every contributor!

I pitted it against a 3ware 9650 and 'md' won on nearly every account (albeit on
RAID5 for sequential I/O the 3ware is a distant winner):
http://www.makarevitch.org/rant/raid/#3wmd

On RAID10 f2 a small read-ahead reduces the throughput on sequential read, but
even a low value (768 for the whole 'md' block device, 0 for the underlying
spindles) enables very good sequential read performance (300 MB/s on 6 low-end
Hitachi 500 GB spindles).

What baffles me is that, on a 1.4TB array served by a box having 12 GB RAM (low
cache-hit ratio), the random access performance remains stable and high (450
IOPS with 48 threads, 20% writes - 10% fsync'ed), even with a fairly high
read-ahead (16k). How comes?!

Re: RAID10 far (f2) read throughput on random and sequential / read-ahead

[Nat Makarevitch]

Nat Makarevitch <nat <at> makarevitch.org> writes:

> 'md' performs wonderfully

> random access performance remains stable and high (450
> IOPS with 48 threads, 20% writes - 10% fsync'ed), even with a fairly high
> read-ahead (16k).

Mystery solved, sorry for the noise

Explanation: I use the 'randomio' tool and searched in its sourcecode for the
various 'advise' calls (posix_fadvise(2), madvise(2)...) and forgot to check the
open call (it uses O_DIRECT!)

Re: RAID10 far (f2) read throughput on random and sequential / read-ahead

[Keld Jørn Simonsen]

I made a reference to your work in the wiki howto on performance.
Thanks!

Keld

On Fri, Feb 22, 2008 at 04:14:05AM +0000, Nat Makarevitch wrote:
> 'md' performs wonderfully. Thanks to every contributor!
> 
> I pitted it against a 3ware 9650 and 'md' won on nearly every account (albeit on
> RAID5 for sequential I/O the 3ware is a distant winner):
> http://www.makarevitch.org/rant/raid/#3wmd
> 
> On RAID10 f2 a small read-ahead reduces the throughput on sequential read, but
> even a low value (768 for the whole 'md' block device, 0 for the underlying
> spindles) enables very good sequential read performance (300 MB/s on 6 low-end
> Hitachi 500 GB spindles).
> 
> What baffles me is that, on a 1.4TB array served by a box having 12 GB RAM (low
> cache-hit ratio), the random access performance remains stable and high (450
> IOPS with 48 threads, 20% writes - 10% fsync'ed), even with a fairly high
> read-ahead (16k). How comes?!
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html