Looking for the best way to do this

Looking for the best way to do this

[maurice]

Good day, and thanks in advance for any ideas and advice.

I am working on a server.
The server is normally in a remote location, not outside accessible by 
network,
and there is no easy method to do backups on it.
It is running Fedora 14

It has 2 hard drives at present, and there are 5 partitions  on each.
 From this we have 3 RAID1 md devices.
These are for /boot, /data and /

I wish to add a 3rd disk, for enhancing the safety of the data.
There is no space to add a 4th disk.

Things I thought of doing include:
Add 3rd disk to make a 3 disk RAID1
Make a RAID10, missing one disk.

Which do you think is the better means to enhance data safety / chances 
of surviving a hardware problem?
One of the above, or an other configuration?

And, assuming I want to preserve the existing data, what is the best way 
to do this?


-- 
Cheers,
Maurice Hilarius
eMail: /mhilarius@gmail.com/

RE: Looking for the best way to do this

[Leslie Rhorer]

> -----Original Message-----
> From: linux-raid-owner@vger.kernel.org [mailto:linux-raid-
> owner@vger.kernel.org] On Behalf Of maurice
> Sent: Tuesday, June 21, 2011 1:13 PM
> To: linux-raid@vger.kernel.org
> Subject: Looking for the best way to do this
> 
> Good day, and thanks in advance for any ideas and advice.
> 
> I am working on a server.
> The server is normally in a remote location, not outside accessible by
> network,
> and there is no easy method to do backups on it.
> It is running Fedora 14
> 
> It has 2 hard drives at present, and there are 5 partitions  on each.
>  From this we have 3 RAID1 md devices.
> These are for /boot, /data and /

	Presumably the fourth partition is swap, and the fifth is...?

> I wish to add a 3rd disk, for enhancing the safety of the data.
> There is no space to add a 4th disk.
> 
> Things I thought of doing include:
> Add 3rd disk to make a 3 disk RAID1

	That's the easiest and most robust.

> Make a RAID10, missing one disk.

	No, I don't think I would do that.  If you need more storage, you
could fail one drive, create a 3 drive RAID5 array with one missing, copy
the data over from the remaining RAID1 member, and then make the remaining
RAID1 drive into the third member for the RAID5 array.  This is a bit less
robust than the 3 member RAID1 array, but gives you twice the storage.  If
you don't need the extra storage, go for the 3 member RAID1.  Whatever you
do, make an off-site backup of (at a minimum) /data.

> Which do you think is the better means to enhance data safety / chances
> of surviving a hardware problem?
> One of the above, or an other configuration?
> 
> And, assuming I want to preserve the existing data, what is the best way
> to do this?

	Like I said, make your backup, and then either add a 3rd member to
the RAID1 array or else fail 1 of the drives, create a RAID5 array with one
missing member from the new drive and the failed drive, copy the data from
the remaining live RAID1 drive to the RAID5 array, shut down the RAID1
array, and add the drive to the RAID5 array.  Since this system is going to
be bootable, you probably need to use 0.90 metadata for the /boot array.
The rest can be 1.x.

Re: Looking for the best way to do this

[Iordan Iordanov]

Hi Maurice,

On 06/21/11 14:13, maurice wrote:
> Add 3rd disk to make a 3 disk RAID1
> Make a RAID10, missing one disk.

Perhaps I misunderstand your supposition, and if I do, please correct 
me. RAID10 with mdadm does not require a minimum of 4 disks. For 
example, mdadm can do RAID10 with 2 copies on 3 disks or RAID10 with 3 
copies on 3 disks, yielding the equivalent of RAID1 in terms of data 
security. It is not confined to mirroring two pairs of disks and then 
striping across them as is the "conventional" way of doing RAID10.

You can read the man-page of mdadm, where the options for "layout" of 
RAID10 are discussed. Pay particular attention to the end of the 
"--layout" option explanation.

Cheers!
Iordan

Re: Looking for the best way to do this

[maurice]

On 6/23/2011 1:23 PM, Iordan Iordanov wrote:
> Hi Maurice,
>
> On 06/21/11 14:13, maurice wrote:
>> Add 3rd disk to make a 3 disk RAID1
>> Make a RAID10, missing one disk.
>
> Perhaps I misunderstand your supposition, and if I do, please correct 
> me. RAID10 with mdadm does not require a minimum of 4 disks.
Yes, that was my understanding.

> For example, mdadm can do RAID10 with 2 copies on 3 disks or RAID10 
> with 3 copies on 3 disks, yielding the equivalent of RAID1 in terms of 
> data security. It is not confined to mirroring two pairs of disks and 
> then striping across them as is the "conventional" way of doing RAID10.
Yes, I understand the principle.
>
> You can read the man-page of mdadm, where the options for "layout" of 
> RAID10 are discussed. Pay particular attention to the end of the 
> "--layout" option explanation.

What I do NOT know is the following:
A) Which is more RELIABLE: R1 with 3 disks or R10 with 3 disks.
B) HOW to create either the 3 disk R1 or R10, assuming an existing 2 
disk R1.



-- 
Cheers,
Maurice Hilarius
eMail: /mhilarius@gmail.com/

Re: Looking for the best way to do this

[Phil Turmel]

Hi maurice,

On 06/23/2011 06:06 PM, maurice wrote:
> On 6/23/2011 1:23 PM, Iordan Iordanov wrote:
>> Hi Maurice,
>>
>> On 06/21/11 14:13, maurice wrote:
>>> Add 3rd disk to make a 3 disk RAID1
>>> Make a RAID10, missing one disk.
>>
>> Perhaps I misunderstand your supposition, and if I do, please correct me. RAID10 with mdadm does not require a minimum of 4 disks.
> Yes, that was my understanding.
> 
>> For example, mdadm can do RAID10 with 2 copies on 3 disks or RAID10 with 3 copies on 3 disks, yielding the equivalent of RAID1 in terms of data security. It is not confined to mirroring two pairs of disks and then striping across them as is the "conventional" way of doing RAID10.
> Yes, I understand the principle.
>>
>> You can read the man-page of mdadm, where the options for "layout" of RAID10 are discussed. Pay particular attention to the end of the "--layout" option explanation.
> 
> What I do NOT know is the following:
> A) Which is more RELIABLE: R1 with 3 disks or R10 with 3 disks.

R1 with 3 disks is three copies of all sectors.  R10 with three disks can be either two copies of each sector or three copies.  A two-copy layout would have capacity equal to 1.5 * the member disk size.  For reliability, three copies is better than two copies.  R10 and R1 use different algorithms, so you will have different performance pros and cons.

> B) HOW to create either the 3 disk R1 or R10, assuming an existing 2 disk R1.

You can "--grow" the number of disks (copies) in an R1 while running.  R10 does not (yet) support layout or capacity changes, whether running or not.  So if you want R10, you'll have to start from scratch.

If you are willing to give up your redundancy temporarily, you could create a degraded R10 from the new drive and one of the existing drives, copy the partition from the remaining drive in the R1, stop and destroy the R1, then add that disk into the R10.

Phil

Re: Looking for the best way to do this

[Phil Turmel]

Hi Maurice,

[restored CC list....  use reply-to-all on kernel.org lists]

On 06/23/2011 07:26 PM, maurice wrote:
> On 6/23/2011 4:55 PM, Phil Turmel wrote:
>> Hi maurice,
> 
> Hi Phil
> 
> That was great, thank you!
> 
> I gather that the R10 results ion a faster disk IO, in general?
> Probably with the  "far" version, if we are mostly reading?

For most applications, yes.  I haven't benchmarked it myself, though.

> I would be willing to lose redundancy for the duration of the creation process.
> Both disks presently seem to be OK.

You'll also be offline while copying and switching to the new filesystem.  Did you want layout=f2 or =f3 ?

> Any chance you could "wrap up" for me with the correct syntax for this?
> I find that the "trial and error " approach is less than optimal.

First, run a check on your array, to minimize the chance of a read error during the copy:

echo "check" >/sys/block/md*/md/sync_action

While that's running, send me the output of "lsdrv", which will let me give advice with real device names:

wget -O /usr/sbin/lsdrv http://github.com/pturmel/lsdrv/raw/master/lsdrv ; chmod 755 /usr/sbin/lsdrv ; lsdrv

Phil

Re: Looking for the best way to do this

[Iordan Iordanov]

Hi Maurice,

On 06/23/11 18:06, maurice wrote:
> What I do NOT know is the following:
> A) Which is more RELIABLE: R1 with 3 disks or R10 with 3 disks.

R10 has different layouts. There are two components to the layout - 
where the chunks are located on the different devices (near, offset, 
far), and number of copies of the data. The near/offset/far part of the 
layout does not play a role in reliability, just in performance. So, the 
number of copies is the only thing that matters to this discussion.

If you have R10 with 3 disks and 3 copies, it is equally reliable to R1 
with 3 disks, as they have an equal number of copies. In both R10 and R1 
in this case, two disks can fail, and you still have 1 intact copy 
remaining.

> B) HOW to create either the 3 disk R1 or R10, assuming an existing 2
> disk R1.

If there is an existing R1, and you just want to add an extra device to 
it, it's easy - you just have to "grow" the array to increase the number 
of devices, and then "add" a device, and it'll become a 3 device R1. I 
don't think you'd want to switch to R10 if you are looking to just add a 
3rd device to an existing R1!

Example:

mdadm --grow -n 3 /dev/md0
mdadm /dev/md0 --add /dev/sdc1

Does that help?

Cheers,
Iordan

Re: Looking for the best way to do this

[maurice]

On 6/24/2011 10:17 AM, Iordan Iordanov wrote:
> Hi Maurice,
> ..
> R10 has different layouts. There are two components to the layout - 
> where the chunks are located on the different devices (near, offset, 
> far), and number of copies of the data. The near/offset/far part of 
> the layout does not play a role in reliability, just in performance. 
> So, the number of copies is the only thing that matters to this 
> discussion.
>
> If you have R10 with 3 disks and 3 copies, it is equally reliable to 
> R1 with 3 disks, as they have an equal number of copies. In both R10 
> and R1 in this case, two disks can fail, and you still have 1 intact 
> copy remaining.
>
 From what I read, assuming I want to optimize this for READ 
performance, I would want "R10 far".
As I understand this, if I wanted the R10, I could set aside one disk of 
the R1 with my existing data,
and use on of the old disks, along with the new one, to create the R10 ?


I believe the way to accomplish this is as follows:

Set aside (disconnect) 1 of the existing 2 disks.
Boot from emergency recovery media, making sure the md version is 
relatively current ( newer than 3.0 ?)

Create the R10 with the other 2 disks ( 1 old and 1 new) with 1 of the 3 
disks missing.
I think I have to wipe the superblock on the disk that was formerly part 
of the RAID1 set.

Reconnect the old R1 reserved disk.
Copy the data from the single member of the old R1 to the new R10
On the old disk of the R1 set wipe the superblock.
Lastly add the 3rd disk to the R10 set.
Run grub to put the boot block on all 3 of the disks.
Assuming that this sequence is correct, what I need now is the correct 
syntx for these steps.

Or, perhaps, I have it wrong!

-- 
Cheers,
Maurice Hilarius
eMail: /mhilarius@gmail.com/

Re: Looking for the best way to do this

[Iordan Iordanov]

Hi Maurice,

On 06/24/11 12:35, maurice wrote:
> Set aside (disconnect) 1 of the existing 2 disks.
> Boot from emergency recovery media, making sure the md version is
> relatively current ( newer than 3.0 ?)
>
> Create the R10 with the other 2 disks ( 1 old and 1 new) with 1 of the 3
> disks missing.
> I think I have to wipe the superblock on the disk that was formerly part
> of the RAID1 set.

I think your procedure can be made less risky by making a R10, with 
layout f3 with *two* missing disks. Since there are *3* copies in the 
R10, there is no problem with having just one "real" disk and two 
missing ones. This way, you don't have to live with just one of the R1 
disks and hope it doesn't go bad during the copy operation.

Then, you copy the data from your old R1 to your new R10 with the two 
missing disks. When the data is copied over, you can remove *one* of the 
disks from the R1, zero the superblock, and add it to the R10. When the 
rebuild is complete, and only then, zero the superblock of your last R1 
disk, and add it to the R10.

With this mod, your data never really lost redundancy during the entire 
procedure (except when you remove the first disk from the R1 and move it 
to the R10, but instead of redundancy, you have 2 copies - one on the 
R10 and one on the R1 at that point, so it's still redundant in a way).

Whether or not you use this revised procedure, write yourself a 
step-by-step guide with all the commands (a TO-DO list), and if you 
want, you can share it with the list if you're not confident.

Cheers,
Iordan

Re: Looking for the best way to do this

[maurice]

On 6/24/2011 10:59 AM, Iordan Iordanov wrote:
>
> On 06/24/11 12:35, maurice wrote:
>> Set aside (disconnect) 1 of the existing 2 disks.
>> Boot from emergency recovery media, making sure the md version is
>> relatively current ( newer than 3.0 ?)
>>
>> Create the R10 with the other 2 disks ( 1 old and 1 new) with 1 of the 3
>> disks missing.
>> I think I have to wipe the superblock on the disk that was formerly part
>> of the RAID1 set.
>
> I think your procedure can be made less risky by making a R10, with 
> layout f3 with *two* missing disks. Since there are *3* copies in the 
> R10, there is no problem with having just one "real" disk and two 
> missing ones. This way, you don't have to live with just one of the R1 
> disks and hope it doesn't go bad during the copy operation.
>
> Then, you copy the data from your old R1 to your new R10 with the two 
> missing disks. When the data is copied over, you can remove *one* of 
> the disks from the R1, zero the superblock, and add it to the R10. 
> When the rebuild is complete, and only then, zero the superblock of 
> your last R1 disk, and add it to the R10.
>
> With this mod, your data never really lost redundancy during the 
> entire procedure (except when you remove the first disk from the R1 
> and move it to the R10, but instead of redundancy, you have 2 copies - 
> one on the R10 and one on the R1 at that point, so it's still 
> redundant in a way).
>
> Whether or not you use this revised procedure, write yourself a 
> step-by-step guide with all the commands (a TO-DO list), and if you 
> want, you can share it with the list if you're not confident.
>

Thanks, this makes really good sense, and sounds VERY safe.

As for the "TO-DO list" or "recipe" I still lack the skills to produce 
the syntax.


-- 
Cheers,
Maurice Hilarius
eMail: /mhilarius@gmail.com/