Raid settings

Raid settings

[o1bigtenor]

Greetings

I have been doing some research thinking toward the future.

Is there a 'best' raid setup?

It seems to me (a noob) that each of the options carries some negatives with it.

Is there a good option for say:

2 - 5 disks
4 - 8 disks
6 - 12 disks
10 - 30 disks
etc.

I looked at raid 5/6/10/50/60/100 and I am wondering where is the
'best' use of each of these options?

TIA

Dee

Re: Raid settings

[Wols Lists]

On 28/08/16 22:43, o1bigtenor wrote:
> Greetings
> 
> I have been doing some research thinking toward the future.
> 
> Is there a 'best' raid setup?

What do you want to achieve? There's no such thing as "best" - there's
only "most suitable for the circumstances".
> 
> It seems to me (a noob) that each of the options carries some negatives with it.
> 
> Is there a good option for say:
> 
> 2 - 5 disks
> 4 - 8 disks
> 6 - 12 disks
> 10 - 30 disks
> etc.
> 
> I looked at raid 5/6/10/50/60/100 and I am wondering where is the
> 'best' use of each of these options?
> 
Ignoring linear or stripe (which you seem to have done), with 2 disks
the only option is raid 1 (mirror). 3 disks gives you raid 5, and 4
disks gives you raid 6.

But do you want to make maximum use of the disk space (raid 6 is your
friend) or do you want maximum redundancy (raid 1)?

For my home system I've got 2 x 3TB in a raid1 config. I had intended to
add a 3rd drive and go raid5, but with two Barracudas I'd be an idiot
:-( If I want to go that route, I need three new proper raid drives :-(
I want maximum disk capacity with some redundancy, so raid 5 or 6 makes
most sense for me.

Without knowing what you want, we can't know what's best for you.

Cheers,
Wol

Re: Raid settings

[o1bigtenor]

On Sun, Aug 28, 2016 at 4:59 PM, Wols Lists <antlists@youngman.org.uk> wrote:
> On 28/08/16 22:43, o1bigtenor wrote:
>> Greetings
>>
>> I have been doing some research thinking toward the future.
>>
>> Is there a 'best' raid setup?
>
> What do you want to achieve? There's no such thing as "best" - there's
> only "most suitable for the circumstances".
>>
>> It seems to me (a noob) that each of the options carries some negatives with it.
>>
>> Is there a good option for say:
>>
>> 2 - 5 disks
>> 4 - 8 disks
>> 6 - 12 disks
>> 10 - 30 disks
>> etc.
>>
>> I looked at raid 5/6/10/50/60/100 and I am wondering where is the
>> 'best' use of each of these options?
>>
> Ignoring linear or stripe (which you seem to have done), with 2 disks
> the only option is raid 1 (mirror). 3 disks gives you raid 5, and 4
> disks gives you raid 6.
>
> But do you want to make maximum use of the disk space (raid 6 is your
> friend) or do you want maximum redundancy (raid 1)?
>
> For my home system I've got 2 x 3TB in a raid1 config. I had intended to
> add a 3rd drive and go raid5, but with two Barracudas I'd be an idiot
> :-( If I want to go that route, I need three new proper raid drives :-(
> I want maximum disk capacity with some redundancy, so raid 5 or 6 makes
> most sense for me.
>
> Without knowing what you want, we can't know what's best for you.

That's what it seems like - - - its possible to justify any setup.

I have 2 - 4 disc setups both running raid 10 trying to get s fairly high level
of security yet also some through put.

I'm asking because what do I do if I need to have say 10 to 25 TB of online
storage.

Do I go for my raid 10 with 2 sets of 6 TB discs or is there a better way to
achieve high levels of security AND throughput?

What about for the next level of storage (my thinking here) or at 35 to 100 TB?

Maybe the better question is - - - how do I decide what I want?

TIA

Dee

Re: Raid settings

[Adam Goryachev]

On 29/08/16 12:28, o1bigtenor wrote:
> On Sun, Aug 28, 2016 at 4:59 PM, Wols Lists <antlists@youngman.org.uk> wrote:
>> On 28/08/16 22:43, o1bigtenor wrote:
>>> Greetings
>>>
>>> I have been doing some research thinking toward the future.
>>>
>>> Is there a 'best' raid setup?
>> What do you want to achieve? There's no such thing as "best" - there's
>> only "most suitable for the circumstances".
>>> It seems to me (a noob) that each of the options carries some negatives with it.
>>>
>>> Is there a good option for say:
>>>
>>> 2 - 5 disks
>>> 4 - 8 disks
>>> 6 - 12 disks
>>> 10 - 30 disks
>>> etc.
>>>
>>> I looked at raid 5/6/10/50/60/100 and I am wondering where is the
>>> 'best' use of each of these options?
>>>
>> Ignoring linear or stripe (which you seem to have done), with 2 disks
>> the only option is raid 1 (mirror). 3 disks gives you raid 5, and 4
>> disks gives you raid 6.
>>
>> But do you want to make maximum use of the disk space (raid 6 is your
>> friend) or do you want maximum redundancy (raid 1)?
>>
>> For my home system I've got 2 x 3TB in a raid1 config. I had intended to
>> add a 3rd drive and go raid5, but with two Barracudas I'd be an idiot
>> :-( If I want to go that route, I need three new proper raid drives :-(
>> I want maximum disk capacity with some redundancy, so raid 5 or 6 makes
>> most sense for me.
>>
>> Without knowing what you want, we can't know what's best for you.
> That's what it seems like - - - its possible to justify any setup.
>
> I have 2 - 4 disc setups both running raid 10 trying to get s fairly high level
> of security yet also some through put.
>
> I'm asking because what do I do if I need to have say 10 to 25 TB of online
> storage.
>
> Do I go for my raid 10 with 2 sets of 6 TB discs or is there a better way to
> achieve high levels of security AND throughput?
>
> What about for the next level of storage (my thinking here) or at 35 to 100 TB?
>
> Maybe the better question is - - - how do I decide what I want?
>

Hi,
I think you are forgetting that there are multiple factors, you are only 
looking at the storage capacity, and to some degree protection. You also 
need to consider what sort of performance you want to achieve, and this 
is usually the deciding factor.

To get 25TB, you can easily use 3 x 10TB drives in RAID0 .... but if you 
want some level of protection, then you could choose 4 in RAID5, or 5 in 
RAID6, or 6 in RAID10.
You of course have the same options when you want 35, or 100, or 1000TB 
etc...
RAID10 will scale linearly, simply keep adding drives in pairs, and you 
will continue to have the similar level of protection (I guess the 
chance of a pair of drives failing increases as you have more pairs)...
RAID5/6 you will likely want to use RAID50/60 with no more than X drives 
in each RAID5/6 part, where X is determined by your 
performance/storage/reliability decisions.

After all that, you then need to look at another dozen options, (bitmap, 
chunk size, etc etc), which will also have a significant impact on 
performance (and reliability).

To get 35TB, you might do either 16 x 8TB drives in RAID10 (40TB) or you 
can do 7 x 8TB drives in RAID6 (at these capacities, I'd strongly 
suggest you skip RAID5). You could also consider doing 14 x 8TB drives 
in RAID60 (two sets of 7 drives).

PS, you will want to use this range of drives:
http://www.wdc.com/en/products/internal/nas/

Or equivalent, as long as it definitely supports SCT/ERC.

Regards,
Adam


-- 
Adam Goryachev Website Managers www.websitemanagers.com.au

Re: Raid settings

[o1bigtenor]

On Sun, Aug 28, 2016 at 9:46 PM, Adam Goryachev
<mailinglists@websitemanagers.com.au> wrote:
> On 29/08/16 12:28, o1bigtenor wrote:
snip
>>>
>>> Without knowing what you want, we can't know what's best for you.
>>
>> That's what it seems like - - - its possible to justify any setup.
>>
>> I have 2 - 4 disc setups both running raid 10 trying to get s fairly high
>> level
>> of security yet also some through put.
>>
>> I'm asking because what do I do if I need to have say 10 to 25 TB of
>> online
>> storage.
>>
>> Do I go for my raid 10 with 2 sets of 6 TB discs or is there a better way
>> to
>> achieve high levels of security AND throughput?
>>
>> What about for the next level of storage (my thinking here) or at 35 to
>> 100 TB?
>>
>> Maybe the better question is - - - how do I decide what I want?
>>
>
> Hi,
> I think you are forgetting that there are multiple factors, you are only
> looking at the storage capacity, and to some degree protection. You also
> need to consider what sort of performance you want to achieve, and this is
> usually the deciding factor.

Your examples also point to cost being a possible major factor.
>
> To get 25TB, you can easily use 3 x 10TB drives in RAID0 .... but if you
> want some level of protection, then you could choose 4 in RAID5, or 5 in
> RAID6, or 6 in RAID10.
> You of course have the same options when you want 35, or 100, or 1000TB
> etc...
> RAID10 will scale linearly, simply keep adding drives in pairs, and you will
> continue to have the similar level of protection (I guess the chance of a
> pair of drives failing increases as you have more pairs)...
> RAID5/6 you will likely want to use RAID50/60 with no more than X drives in
> each RAID5/6 part, where X is determined by your
> performance/storage/reliability decisions.
>
> After all that, you then need to look at another dozen options, (bitmap,
> chunk size, etc etc), which will also have a significant impact on
> performance (and reliability).

I've looked but haven't been able to find anything that discusses the
points that
have been raised so far. Would you point me to a, or some, documents that
discuss these options?
What are the options besides the 2 you mentioned?

What I'm looking for is a methodology for the decision rather than a blanket
answer at this point. Thought initially that there might be a 'best' answer but
now its looking like a much much more fluid thing.
>
> To get 35TB, you might do either 16 x 8TB drives in RAID10 (40TB) or you can
> do 7 x 8TB drives in RAID6 (at these capacities, I'd strongly suggest you
> skip RAID5). You could also consider doing 14 x 8TB drives in RAID60 (two
> sets of 7 drives).
>
> PS, you will want to use this range of drives:
> http://www.wdc.com/en/products/internal/nas/

Thanks for the tip.
>
> Or equivalent, as long as it definitely supports SCT/ERC.
>

Regards

Dee

Re: Raid settings

[Wols Lists]

On 29/08/16 10:26, o1bigtenor wrote:
> I've looked but haven't been able to find anything that discusses the
> points that
> have been raised so far. Would you point me to a, or some, documents that
> discuss these options?
> What are the options besides the 2 you mentioned?
> 
> What I'm looking for is a methodology for the decision rather than a blanket
> answer at this point. Thought initially that there might be a 'best' answer but
> now its looking like a much much more fluid thing.

Apropos all this, I should be getting write access to the linux raid
wiki sometime soon. At the moment, as I mentioned before, it seems to be
a cobweb site, with the admin MIA and stuck in the days of kernel 2.6,
LILO, and parallel ATA drives ... :-)

So I'm planning on a major update/rewrite, and I'll probably be bugging
the list for info :-) but this is exactly the sort of thing that might
well go on there, if I can find the info or get someone to write it for me.

Cheers,
Wol

Re: Raid settings

[o1bigtenor]

On Mon, Aug 29, 2016 at 6:16 AM, Wols Lists <antlists@youngman.org.uk> wrote:
> On 29/08/16 10:26, o1bigtenor wrote:
>> I've looked but haven't been able to find anything that discusses the
>> points that
>> have been raised so far. Would you point me to a, or some, documents that
>> discuss these options?
>> What are the options besides the 2 you mentioned?
>>
>> What I'm looking for is a methodology for the decision rather than a blanket
>> answer at this point. Thought initially that there might be a 'best' answer but
>> now its looking like a much much more fluid thing.
>
> Apropos all this, I should be getting write access to the linux raid
> wiki sometime soon. At the moment, as I mentioned before, it seems to be
> a cobweb site, with the admin MIA and stuck in the days of kernel 2.6,
> LILO, and parallel ATA drives ... :-)
>
> So I'm planning on a major update/rewrite, and I'll probably be bugging
> the list for info :-) but this is exactly the sort of thing that might
> well go on there, if I can find the info or get someone to write it for me.
>

Would be willing to assist in the document crafting if I were given good
information (see it as a way to 'help' knowing bupkis about code!).

Wol, do you have any pointers to documents that discuss the variable
 to be considered?

What kind of shape would you think the document should have?

What should be included/excluded?

Regards

Dee

Re: Raid settings

[Wols Lists]

On 29/08/16 13:11, o1bigtenor wrote:
> Would be willing to assist in the document crafting if I were given good
> information (see it as a way to 'help' knowing bupkis about code!).
> 
> Wol, do you have any pointers to documents that discuss the variable
>  to be considered?
> 
> What kind of shape would you think the document should have?
> 
> What should be included/excluded?

Take a look at the website -
https://raid.wiki.kernel.org/index.php/Linux_Raid

In particular look at "Raid setup" and the "Advanced Options" section,
which is where this should probably go. Something to fit in there, I guess.

Once I've got access, I can sort out access for you, and if it works out
well, we can work on this together :-) It'd be nice to get an up-to-date
resource out there :-) Times move on, and the two original guys have
moved on in their lives - now's as good a time as any to pass the baton :-)

Cheers,
Wol

Re: Raid settings

[o1bigtenor]

On Mon, Aug 29, 2016 at 7:19 AM, Wols Lists <antlists@youngman.org.uk> wrote:
> On 29/08/16 13:11, o1bigtenor wrote:
>> Would be willing to assist in the document crafting if I were given good
>> information (see it as a way to 'help' knowing bupkis about code!).
>>
>> Wol, do you have any pointers to documents that discuss the variable
>>  to be considered?
>>
>> What kind of shape would you think the document should have?
>>
>> What should be included/excluded?
>
> Take a look at the website -
> https://raid.wiki.kernel.org/index.php/Linux_Raid
>
> In particular look at "Raid setup" and the "Advanced Options" section,
> which is where this should probably go. Something to fit in there, I guess.
>
> Once I've got access, I can sort out access for you, and if it works out
> well, we can work on this together :-) It'd be nice to get an up-to-date
> resource out there :-) Times move on, and the two original guys have
> moved on in their lives - now's as good a time as any to pass the baton :-)
>
I'm starting to think that documents should be checked for
'currentness' at least
annually if not more often. The software is constantly being worked on and
hardware stuff is changing likely on a semi-annual basis.

TTYL

Dee

Re: Raid settings

[Wols Lists]

On 29/08/16 19:13, o1bigtenor wrote:
>> Once I've got access, I can sort out access for you, and if it works out
>> > well, we can work on this together :-) It'd be nice to get an up-to-date
>> > resource out there :-) Times move on, and the two original guys have
>> > moved on in their lives - now's as good a time as any to pass the baton :-)
>> >
> I'm starting to think that documents should be checked for
> 'currentness' at least
> annually if not more often. The software is constantly being worked on and
> hardware stuff is changing likely on a semi-annual basis.

Good sentiment. Snag is, the web is full of cobweb sites - and given the
propensity of the web to never forget, I don't think you're going to win
... It's like documentation - how many programmers keep documentation up
to date :-)

But yes, if it's (allegedly) hosted at kernel.org, yes it looks official
and should be kept up-to-date. And the reality is, a lot of it still is
good relevant advice. It's just the technical stuff, which isn't
actually the majority of the site. Unfortunately, if the tech stuff is
out of date, techies will distrust the rest.

Cheers,
Wol

Re: Raid settings

[Brad Campbell]

On 29/08/16 05:59, Wols Lists wrote:

> For my home system I've got 2 x 3TB in a raid1 config. I had intended to
> add a 3rd drive and go raid5, but with two Barracudas I'd be an idiot
> :-( If I want to go that route, I need three new proper raid drives :-(
> I want maximum disk capacity with some redundancy, so raid 5 or 6 makes
> most sense for me.
>

I'll get roasted for suggesting this, but for a home RAID where 
potential response times in cases of errors are not an issue those 
Barracudas will be fine. Just make sure you set the appropriate 
timeouts. It's not like the drives are going to explode in a ball of 
flames because they don't support ERC, they just won't play nice with 
the default linux stack timeouts.

Sure, if you are buying new drives spend the extra 20 bucks and get 
drives that do support ERC, but if what you have works then just keep on 
keeping on. Add another drive, stretch it out to RAID5 and be happy.

I don't do RAID5 anymore, but for 3 disks with the right configuration 
it's not awful.

I have a couple of RAID10 here and a couple of RAID6. The RAID10 is an 
interesting case, because it'll survive a double drive failure some of 
the time. If the wrong pair fails though it's toast, whereas a RAID6 
will survive a double drive failure *all* of the time.

I like your "most suitable for the circumstances" quote though.

I saw some pretty interesting configurations on Sun X4500's a few years 
ago. 48 drives in varying configurations depending entirely on the 
projected workload. No right tool for every job.