raid 5 and different device size

raid 5 and different device size

[Hendrik Friedel]

Hello,

I need your advice for the following scenario:
I do have three -identical- 3TB HDD. On these I store:
-Backups (Images of my Families Laptop; Backup of the Machines (the one 
with the HDD) running System).
-Videos (recordings, not essential)
-HomeVideos (Familiy, essential)
-Photos
-Documents

Now I intend to:
* increase the available disc space
* stop worrying how much space is available on which drive
* create redundancy
* have low operating cost (power-consumption)


So, I need to buy one additional HDD.
And I want to combine the drives to one big volume.

I am aware that raid is no backup.
Thus, I backup the HomeVideos and the Photos on an external drive not 
connected to the power supply and USB.

I think, that I do not need raid5, depending on the definition of 'not 
essential'. Nevertheless, raid5 would efficiently (at low cost) decrease 
the likelyhood of loss of the 'non-essential' data.
Furthermore, raid5 would fill the gap that I have between backups (the 
fact, that I do these backup manually means that I do not do them every 
day) and at least safe me from hardware failures.

So, yes, I think I do want raid5 and I want to combine.
But what size of HDD do I want to buy? The best value (cost per TB) I 
get for 3TB drives. So I could by two 3TB drives, one for additional 
space, the other for the parity information.
Or I could by one bigger drive, with potentially lower power 
consumption. But how would the raid5 look in this case? I fear, only 3TB 
of the 6 would be redundant and the other 3 would be not redundant, right?


Now the last point: Power consumption: Under which conditions can the 
drives spin down in case of raid5? I assume that all drives have to run 
as in case the data is written on any one of the drives, right?
Is that also true during reading of data, i.e. is the parity also 
checked for read operations?

Thanks for your advice in advance,
Hendrik





---
Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft.
http://www.avast.com

Re: raid 5 and different device size

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 2646 bytes --]

On Mon, May 11, 2015 at 10:03:32PM +0200, Hendrik Friedel wrote:
> Hello,
> 
> I need your advice for the following scenario:
> I do have three -identical- 3TB HDD. On these I store:
> -Backups (Images of my Families Laptop; Backup of the Machines (the
> one with the HDD) running System).
> -Videos (recordings, not essential)
> -HomeVideos (Familiy, essential)
> -Photos
> -Documents
> 
> Now I intend to:
> * increase the available disc space
> * stop worrying how much space is available on which drive
> * create redundancy
> * have low operating cost (power-consumption)
> 
> 
> So, I need to buy one additional HDD.
> And I want to combine the drives to one big volume.
> 
> I am aware that raid is no backup.
> Thus, I backup the HomeVideos and the Photos on an external drive
> not connected to the power supply and USB.
> 
> I think, that I do not need raid5, depending on the definition of
> 'not essential'. Nevertheless, raid5 would efficiently (at low cost)
> decrease the likelyhood of loss of the 'non-essential' data.
> Furthermore, raid5 would fill the gap that I have between backups
> (the fact, that I do these backup manually means that I do not do
> them every day) and at least safe me from hardware failures.
> 
> So, yes, I think I do want raid5 and I want to combine.
> But what size of HDD do I want to buy? The best value (cost per TB)
> I get for 3TB drives. So I could by two 3TB drives, one for
> additional space, the other for the parity information.
> Or I could by one bigger drive, with potentially lower power
> consumption. But how would the raid5 look in this case? I fear, only
> 3TB of the 6 would be redundant and the other 3 would be not
> redundant, right?

   No, 3 TB of the new drive would be usable, giving you a grand total
of (4-1) * 3TB = 9 TB of usable space on the array, with 3 TB of raw
space unusable (the remainder of the 6 TB device).

   http://carfax.org.uk/btrfs-usage may be helpful to you here.

> Now the last point: Power consumption: Under which conditions can
> the drives spin down in case of raid5? I assume that all drives have
> to run as in case the data is written on any one of the drives,
> right?
> Is that also true during reading of data, i.e. is the parity also
> checked for read operations?

   Parity is spread evenly across all devices, so a read of more than
a trivial quantity of data would almost certainly spin up all devices
anyway.

   Hugo.

-- 
Hugo Mills             | A gentleman doesn't do damage unless he's paid for
hugo@... carfax.org.uk | it.
http://carfax.org.uk/  |
PGP: E2AB1DE4          |                                            Juri Papay

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: raid 5 and different device size

[ronnie sahlberg]

For that kind of data where you mainly store lots of (big?) files but
rarely modify the files once stored,
I would suggest using snapraid  it is mor stable and reliable than
current btrfs raid5 state.
And it allows you to to go to raid6 and beyond as your data and
redundancy factors grow.


On Mon, May 11, 2015 at 1:03 PM, Hendrik Friedel <hendrik@friedels.name> wrote:
> Hello,
>
> I need your advice for the following scenario:
> I do have three -identical- 3TB HDD. On these I store:
> -Backups (Images of my Families Laptop; Backup of the Machines (the one with
> the HDD) running System).
> -Videos (recordings, not essential)
> -HomeVideos (Familiy, essential)
> -Photos
> -Documents
>
> Now I intend to:
> * increase the available disc space
> * stop worrying how much space is available on which drive
> * create redundancy
> * have low operating cost (power-consumption)
>
>
> So, I need to buy one additional HDD.
> And I want to combine the drives to one big volume.
>
> I am aware that raid is no backup.
> Thus, I backup the HomeVideos and the Photos on an external drive not
> connected to the power supply and USB.
>
> I think, that I do not need raid5, depending on the definition of 'not
> essential'. Nevertheless, raid5 would efficiently (at low cost) decrease the
> likelyhood of loss of the 'non-essential' data.
> Furthermore, raid5 would fill the gap that I have between backups (the fact,
> that I do these backup manually means that I do not do them every day) and
> at least safe me from hardware failures.
>
> So, yes, I think I do want raid5 and I want to combine.
> But what size of HDD do I want to buy? The best value (cost per TB) I get
> for 3TB drives. So I could by two 3TB drives, one for additional space, the
> other for the parity information.
> Or I could by one bigger drive, with potentially lower power consumption.
> But how would the raid5 look in this case? I fear, only 3TB of the 6 would
> be redundant and the other 3 would be not redundant, right?
>
>
> Now the last point: Power consumption: Under which conditions can the drives
> spin down in case of raid5? I assume that all drives have to run as in case
> the data is written on any one of the drives, right?
> Is that also true during reading of data, i.e. is the parity also checked
> for read operations?
>
> Thanks for your advice in advance,
> Hendrik
>
>
>
>
>
> ---
> Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft.
> http://www.avast.com
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: raid 5 and different device size

[Andrew E. Mileski]

On 11/05/15 04:03 PM, Hendrik Friedel wrote:
> I do have three -identical- 3TB HDD.
>
> Now I intend to:
> * increase the available disc space
> * stop worrying how much space is available on which drive
> * create redundancy
> * have low operating cost (power-consumption)
>
> So, I need to buy one additional HDD.
> And I want to combine the drives to one big volume.
>
> I am aware that raid is no backup.
> Thus, I backup the HomeVideos and the Photos on an external drive not
> connected to the power supply and USB.

In my experience, people tend to GREATLY undervalue their data, until it 
is lost!

Your proposed 12 TB (4x3TB) of storage is > 10^14 bits, so you are very 
likely to to encounter an unrecoverable sector on a full-read with most 
typically error-rated drives.

Also consider drive wear (average lifespan of 3 years), spares, and how 
you will deal with a single or multi-drive failure.

You may want to rethink your plan, and the value of your data.

> Now the last point: Power consumption: Under which conditions can the
> drives spin down in case of raid5? I assume that all drives have to run
> as in case the data is written on any one of the drives, right?
> Is that also true during reading of data, i.e. is the parity also
> checked for read operations?

In RAID-5/6, parity is only checked when an error or failure is reported 
by the drive.  This can lead to undetected errors from "bit rot".  Hence 
regular scrubbing (reading of all data) is recommended.

Note that BTRFS checksums data, and verifies data read, so even if the 
drive doesn't detect an error BTRFS is likely to, but BTRFS can't 
recover without additional redundancy.

Personally, I use BTRFS formatted with "-mdup -dsingle" on top of either 
hardware RAID-6 (primary array) or software RAID-5 (backup array), so I 
don't really rely upon BTRFS for recovery.

Most external consumer-grade enclosures are best used with "Green" 
drives, which tend to be 5400 RPM max, as these enclosures often have 
very limited cooling.  Make sure your drives are compatible with the 
enclosure.

I have even seen server-grade rack enclosures with reduced temperature 
ratings when using 7200 RPM or faster drives.  Heat and vibration are 
important considerations with arrays.

I prefer Seagate enterprise drives and WD consumer drives.

Note that WD "Red Label" drives are identical to their "Green Label" 
drives, except "Red Label" drives have time-limited error recovery 
(TLER) firmware for use in arrays.  However, it is possible to increase 
the error timeout in Linux, so even non-TLER drives can be used in 
arrays, but it isn't as convenient nor recommended.  Both are 
error-rated 1 unrecoverable sector in 10^14 bits read.

~~Andrew E. Mileski