raid10 far layout outperforms offset at writing? (was: Help with chunksize on raid10 -p o3 array)

raid10 far layout outperforms offset at writing? (was: Help with chunksize on raid10 -p o3 array)

[Peter Rabbitson]

Peter Rabbitson wrote:
> I have been trying to figure out the best chunk size for raid10 before 
> migrating my server to it (currently raid1). I am looking at 3 offset 
> stripes, as I want to have two drive failure redundancy, and offset 
> striping is said to have the best write performance, with read 
> performance equal to far.

Incorporating suggestions from previous posts (thank you everyone), I 
used this modified script at http://rabbit.us/pool/misc/raid_test2.txt 
To negate effects of caching memory was jammed below 200mb free by using 
a full tmpfs mount with no swap. Here is what I got with far layout (-p 
f3): http://rabbit.us/pool/misc/raid_far.html The clear winner is 1M 
chunks, and is very consistent at any block size. I was surprised even 
more to see that my read speed was identical to that of a raid0 getting 
near the _maximum_ physical speed of 4 drives (roughly 55MB sustained 
across 1.2G). Unlike offset layout, far really shines at reading stuff 
back. The write speed did not suffer noticeably compared to offset 
striping. Here are the results (-p o3) for comparison: 
http://rabbit.us/pool/misc/raid_offset.html, and they roughly seem to 
correlate with my earlier testing using dd.

So I guess the way to go for this system will be f3, although the md(4) 
says that offset layout should be more beneficial. Is there anything I 
missed while setting my o3 array, so that I got worse performance for 
both read and write compared to f3?

Once again thanks everyone for the help.
Peter

raid10 far layout outperforms offset at writing? (was: Help with chunksize on raid10 -p o3 array)

[Keld Jørn Simonsen]

I found this old message:

> Peter Rabbitson
> Mon, 19 Mar 2007 06:14:38 -0800
> 
> Peter Rabbitson wrote:
> 
>     I have been trying to figure out the best chunk size for raid10
> before migrating my server to it (currently raid1). I am looking at 3
> offset stripes, as I want to have two drive failure redundancy, and
> offset striping is said to have the best write performance, with read
> performance equal to far. 
> 
> Incorporating suggestions from previous posts (thank you everyone), I
> used this modified script at http://rabbit.us/pool/misc/raid_test2.txt
> To negate effects of caching memory was jammed below 200mb free by using
> a full tmpfs mount with no swap. Here is what I got with far layout (-p
> f3): http://rabbit.us/pool/misc/raid_far.html The clear winner is 1M
> chunks, and is very consistent at any block size. I was surprised even
> more to see that my read speed was identical to that of a raid0 getting
> near the _maximum_ physical speed of 4 drives (roughly 55MB sustained
> across 1.2G). Unlike offset layout, far really shines at reading stuff
> back. The write speed did not suffer noticeably compared to offset
> striping. Here are the results (-p o3) for comparison:
> http://rabbit.us/pool/misc/raid_offset.html, and they roughly seem to
> correlate with my earlier testing using dd.
> 
> So I guess the way to go for this system will be f3, although the md(4)
> says that offset layout should be more beneficial. Is there anything I
> missed while setting my o3 array, so that I got worse performance for
> both read and write compared to f3?
> 
> Once again thanks everyone for the help.
> Peter

The links were not valid anymore. I wanted to see the results and 
possibly include the results in the performance wiki page
I would appreciate some new links here.

Furthermore some comments to the post: My take on o3 vs f3 is that both
in theory and practice f3 should be much faster for sequential reading,
as the layout is equivalent to raid0. For random reading and sequential
and random writing f3 and o3 (and the same goes for the more normal f2
vs o2) should be about the same, especially when a filesystem and
its associated elevator algorithm is employed.

Best regards
keld

Re: raid10 far layout outperforms offset at writing? (was: Help with chunksize on raid10 -p o3 array)

[Peter Rabbitson]

Keld Jørn Simonsen wrote:
> I found this old message:
> 
>> Peter Rabbitson
>> Mon, 19 Mar 2007 06:14:38 -0800
>>
> 
> The links were not valid anymore. I wanted to see the results and 
> possibly include the results in the performance wiki page
> I would appreciate some new links here.

I apologize, I don't have the data available anymore.


> Furthermore some comments to the post: My take on o3 vs f3 is that both
> in theory and practice f3 should be much faster for sequential reading,
> as the layout is equivalent to raid0. For random reading and sequential
> and random writing f3 and o3 (and the same goes for the more normal f2
> vs o2) should be about the same, especially when a filesystem and
> its associated elevator algorithm is employed.

Yes, this is what I also concluded since I wrote this email. I am in the
process of upgrading my raid setup, and while I am at it I am leaving
5GB blank partitions at the start of all my workstations spindles, so I
can get some real testing at night. I will share my methodology with the
list before I commence testing (which should take about 20 days the way
I am planing it).

But first comes the vacation - happy holidays to you too guys.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: raid10 far layout outperforms offset at writing? (was: Help with chunksize on raid10 -p o3 array)

[Keld Jørn Simonsen]

On Wed, Dec 17, 2008 at 01:50:24PM +0100, Peter Rabbitson wrote:
> Keld Jørn Simonsen wrote:
> > I found this old message:
> > 
> >> Peter Rabbitson
> >> Mon, 19 Mar 2007 06:14:38 -0800
> >>
> > 
> > The links were not valid anymore. I wanted to see the results and 
> > possibly include the results in the performance wiki page
> > I would appreciate some new links here.
> 
> I apologize, I don't have the data available anymore.
'
OK.

> > Furthermore some comments to the post: My take on o3 vs f3 is that both
> > in theory and practice f3 should be much faster for sequential reading,
> > as the layout is equivalent to raid0. For random reading and sequential
> > and random writing f3 and o3 (and the same goes for the more normal f2
> > vs o2) should be about the same, especially when a filesystem and
> > its associated elevator algorithm is employed.
> 
> Yes, this is what I also concluded since I wrote this email. I am in the
> process of upgrading my raid setup, and while I am at it I am leaving
> 5GB blank partitions at the start of all my workstations spindles, so I
> can get some real testing at night. I will share my methodology with the
> list before I commence testing (which should take about 20 days the way
> I am planing it).

I have tried to persuade Neil to change the description for MD to
reflect the above, but until now with no luck.

I look forward to see your new tests!

> But first comes the vacation - happy holidays to you too guys.

Yes, happy holidays to all!

Best regards
Keld
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html