Reshape 3-drive RAID1 into 3-drive RAID-5

Reshape 3-drive RAID1 into 3-drive RAID-5

[Mark Knecht]

Hi all,
   I have an existing 3-drive RAID-1 (md6) that I use as a short term
backup device for a 5-drive RAID-6 (md7) on this machine. The _only_
purpose of this internal backup RAID-1 is to protect against a
fat-finger 'oops' event when using VMWare or Virtualbox VMs on md7.
Real data backups happen over the network to other machines but that's
slow and only gets done about once a week. With this local storage I
can do it in just a few minutes every day.

   I recently reshaped the RAID-6 from a 5-drive RAID-1and now it's
larger than this local backup so I can't get everything backed up any
more. I.e. - I need /dev/md6 large enough to back up /dev/md7. With 3
existing 235GB partitions I believe the RAID-5 will be about 470GB
when complete and therefore large enough to hold my 400GB RAID-6 data.

   I would like to verify that the right commands to do this would be:

umount /dev/md6
mdadm --stop /dev/md6
mdadm --create /dev/md6 --level=5 --raid-devices=3 /dev/sda6 /dev/sdb6 /dev/sdc6

   If these commands are wrong or anyone thinks there are other things
I should be doing at this time please speak up as I'm interested.

   The data that's currently on the backup is not terribly important
but if possible I'd like to keep it through this process unless doing
so makes things more difficult. The time it takes to do this reshape
is not terribly important to me.

Thanks,
Mark


c2stable ~ # cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5]
[raid4]
md6 : active raid1 sdc6[2] sdb6[1] sda6[0]
      247416933 blocks super 1.1 [3/3] [UUU]

md7 : active raid6 sdc7[2] sdb7[1] sda7[0] sdd2[3] sde2[4]
      395387904 blocks super 1.2 level 6, 16k chunk, algorithm 2 [5/5]
[UUUUU]

unused devices: <none>
c2stable ~ #


c2stable ~ # mdadm -D /dev/md6
/dev/md6:
        Version : 1.1
  Creation Time : Thu Apr 15 10:45:35 2010
     Raid Level : raid1
     Array Size : 247416933 (235.96 GiB 253.35 GB)
  Used Dev Size : 247416933 (235.96 GiB 253.35 GB)
   Raid Devices : 3
  Total Devices : 3
    Persistence : Superblock is persistent

    Update Time : Tue Nov 15 06:20:06 2011
          State : clean
 Active Devices : 3
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 0

           Name : c2stable:6  (local to host c2stable)
           UUID : 249c7331:a8203540:c8f3b020:fb30a66b
         Events : 1034

    Number   Major   Minor   RaidDevice State
       0       8        6        0      active sync   /dev/sda6
       1       8       22        1      active sync   /dev/sdb6
       2       8       38        2      active sync   /dev/sdc6
c2stable ~ #

Re: Reshape 3-drive RAID1 into 3-drive RAID-5

[Phil Turmel]

Hi Mark,

On 11/15/2011 10:13 AM, Mark Knecht wrote:
> Hi all,
>    I have an existing 3-drive RAID-1 (md6) that I use as a short term
> backup device for a 5-drive RAID-6 (md7) on this machine. The _only_
> purpose of this internal backup RAID-1 is to protect against a
> fat-finger 'oops' event when using VMWare or Virtualbox VMs on md7.
> Real data backups happen over the network to other machines but that's
> slow and only gets done about once a week. With this local storage I
> can do it in just a few minutes every day.
> 
>    I recently reshaped the RAID-6 from a 5-drive RAID-1and now it's
> larger than this local backup so I can't get everything backed up any
> more. I.e. - I need /dev/md6 large enough to back up /dev/md7. With 3
> existing 235GB partitions I believe the RAID-5 will be about 470GB
> when complete and therefore large enough to hold my 400GB RAID-6 data.
> 
>    I would like to verify that the right commands to do this would be:
> 
> umount /dev/md6
> mdadm --stop /dev/md6
> mdadm --create /dev/md6 --level=5 --raid-devices=3 /dev/sda6 /dev/sdb6 /dev/sdc6

This will blow away the contents, but will otherwise work.  After the
--create, MD will spend time generating parity for the third disk from
the other two.

>    If these commands are wrong or anyone thinks there are other things
> I should be doing at this time please speak up as I'm interested.
> 
>    The data that's currently on the backup is not terribly important
> but if possible I'd like to keep it through this process unless doing
> so makes things more difficult. The time it takes to do this reshape
> is not terribly important to me.

In your case, the data can be retained, and even remain in use, as follows:

1) Drop the third device and wipe its superblock.

mdadm /dev/md6 --fail /dev/sdc6 --remove /dev/sdc6
mdadm --zero-superblock /dev/sdc6

2) Convert 2-drive raid1 to 2-drive raid5.  Due to a quirk of the parity math, the second drive contains the same data as the first, and Neil has optimized this conversion to be instant.  (I don't know what version that became true, but I believe it has been a while.)

mdadm /dev/md6 --grow --level=5

3) Add third drive as a spare:

mdadm /dev/md6 --add /dev/sdc6

4) Reshape/grow to use the new space.  MD will re-arrange the blocks and compute parity as it goes, without disturbing the running filesystem.  You probably need a backup-file option as well...  mdadm will tell you if it needs it.

mdadm /dev/md6 --grow --raid-devices=3

5) Resize your filesystem to use the additional space.

.... (varies, some FS can do it on the run.)

> 
> Thanks,
> Mark

HTH,

Phil

Re: Reshape 3-drive RAID1 into 3-drive RAID-5

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 3198 bytes --]

On Tue, 15 Nov 2011 16:11:43 -0500 Phil Turmel <philip@turmel.org> wrote:

> Hi Mark,
> 
> On 11/15/2011 10:13 AM, Mark Knecht wrote:
> > Hi all,
> >    I have an existing 3-drive RAID-1 (md6) that I use as a short term
> > backup device for a 5-drive RAID-6 (md7) on this machine. The _only_
> > purpose of this internal backup RAID-1 is to protect against a
> > fat-finger 'oops' event when using VMWare or Virtualbox VMs on md7.
> > Real data backups happen over the network to other machines but that's
> > slow and only gets done about once a week. With this local storage I
> > can do it in just a few minutes every day.
> > 
> >    I recently reshaped the RAID-6 from a 5-drive RAID-1and now it's
> > larger than this local backup so I can't get everything backed up any
> > more. I.e. - I need /dev/md6 large enough to back up /dev/md7. With 3
> > existing 235GB partitions I believe the RAID-5 will be about 470GB
> > when complete and therefore large enough to hold my 400GB RAID-6 data.
> > 
> >    I would like to verify that the right commands to do this would be:
> > 
> > umount /dev/md6
> > mdadm --stop /dev/md6
> > mdadm --create /dev/md6 --level=5 --raid-devices=3 /dev/sda6 /dev/sdb6 /dev/sdc6
> 
> This will blow away the contents, but will otherwise work.  After the
> --create, MD will spend time generating parity for the third disk from
> the other two.
> 
> >    If these commands are wrong or anyone thinks there are other things
> > I should be doing at this time please speak up as I'm interested.
> > 
> >    The data that's currently on the backup is not terribly important
> > but if possible I'd like to keep it through this process unless doing
> > so makes things more difficult. The time it takes to do this reshape
> > is not terribly important to me.
> 
> In your case, the data can be retained, and even remain in use, as follows:
> 
> 1) Drop the third device and wipe its superblock.
> 
> mdadm /dev/md6 --fail /dev/sdc6 --remove /dev/sdc6
> mdadm --zero-superblock /dev/sdc6

1a/ Convert the 2-drive RAID1 to a 2-drive RAID1:
  mdadm --grow /dev/md6 --raid-devices=2

> 
> 2) Convert 2-drive raid1 to 2-drive raid5.  Due to a quirk of the parity math, the second drive contains the same data as the first, and Neil has optimized this conversion to be instant.  (I don't know what version that became true, but I believe it has been a while.)
When RAID1->RAID5 conversion was first implemented it was instant - 2.6.30.

> 
> mdadm /dev/md6 --grow --level=5
> 
> 3) Add third drive as a spare:
> 
> mdadm /dev/md6 --add /dev/sdc6
> 
> 4) Reshape/grow to use the new space.  MD will re-arrange the blocks and compute parity as it goes, without disturbing the running filesystem.  You probably need a backup-file option as well...  mdadm will tell you if it needs it.

You won't.  Backups is only needed for the first 6 stripes, and the spare can
be used for that.

> 
> mdadm /dev/md6 --grow --raid-devices=3
> 
> 5) Resize your filesystem to use the additional space.
> 
> .... (varies, some FS can do it on the run.)
> 
> > 
> > Thanks,
> > Mark
> 
> HTH,
> 
> Phil

NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: Reshape 3-drive RAID1 into 3-drive RAID-5

[Mark Knecht]

On Tue, Nov 15, 2011 at 3:40 PM, NeilBrown <neilb@suse.de> wrote:
> On Tue, 15 Nov 2011 16:11:43 -0500 Phil Turmel <philip@turmel.org> wrote:
<SNIP>
>>
>> In your case, the data can be retained, and even remain in use, as follows:
>>
>> 1) Drop the third device and wipe its superblock.
>>
>> mdadm /dev/md6 --fail /dev/sdc6 --remove /dev/sdc6
>> mdadm --zero-superblock /dev/sdc6
>
> 1a/ Convert the 2-drive RAID1 to a 2-drive RAID1:
>  mdadm --grow /dev/md6 --raid-devices=2
>
>>
>> 2) Convert 2-drive raid1 to 2-drive raid5.  Due to a quirk of the parity math, the second drive contains the same data as the first, and Neil has optimized this conversion to be instant.  (I don't know what version that became true, but I believe it has been a while.)
> When RAID1->RAID5 conversion was first implemented it was instant - 2.6.30.
>
>>
>> mdadm /dev/md6 --grow --level=5
>>
>> 3) Add third drive as a spare:
>>
>> mdadm /dev/md6 --add /dev/sdc6
>>
>> 4) Reshape/grow to use the new space.  MD will re-arrange the blocks and compute parity as it goes, without disturbing the running filesystem.  You probably need a backup-file option as well...  mdadm will tell you if it needs it.
>
> You won't.  Backups is only needed for the first 6 stripes, and the spare can
> be used for that.
>
>>
>> mdadm /dev/md6 --grow --raid-devices=3
>>
>> 5) Resize your filesystem to use the additional space.
>>
>> .... (varies, some FS can do it on the run.)
>>
>> >
>> > Thanks,
>> > Mark
>>
>> HTH,
>>
>> Phil
>
> NeilBrown
>

Hi all,
   After reading through your responses and going over the man pages I
got started. Here's the set of commands I have executed so far:

mdadm /dev/md6 --fail /dev/sdc6 --remove /dev/sdc6
mdadm --zero-superblock /dev/sdc6
mdadm --grow /dev/md6 --raid-devices=2

At this point I appear to have a good 2-drive RAID-1:

md6 : active raid1 sda6[0] sdb6[1]
      247416933 blocks super 1.1 [2/2] [UU]

The next command was supposed to convert it to RAID-5 but doesn't work
for me so far:

c2stable ~ # mdadm --grow /dev/md6 --level=5
mdadm: /dev/md6: could not set level to raid5
c2stable ~ #

   I currently have mdadm-3.1.4 install. Gentoo portage has 3.1.5
marked as testing which I can install if required. Kernel is
3.1.1-gentoo.

   Can you see what I'm doing wrong? I was wondering whether mdadm
needs something like --raid-devices=2 to know this is a special
conversion case but don't want to make any stupid mistakes.

Thanks,
Mark

c2stable ~ # mdadm -D /dev/md6
/dev/md6:
        Version : 1.1
  Creation Time : Thu Apr 15 10:45:35 2010
     Raid Level : raid1
     Array Size : 247416933 (235.96 GiB 253.35 GB)
  Used Dev Size : 247416933 (235.96 GiB 253.35 GB)
   Raid Devices : 2
  Total Devices : 2
    Persistence : Superblock is persistent

    Update Time : Thu Nov 17 13:27:20 2011
          State : clean
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0

           Name : c2stable:6  (local to host c2stable)
           UUID : 249c7331:a8203540:c8f3b020:fb30a66b
         Events : 1039

    Number   Major   Minor   RaidDevice State
       0       8        6        0      active sync   /dev/sda6
       1       8       22        1      active sync   /dev/sdb6
c2stable ~ #
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Reshape 3-drive RAID1 into 3-drive RAID-5

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 1736 bytes --]

On Thu, 17 Nov 2011 13:44:04 -0800 Mark Knecht <markknecht@gmail.com> wrote:


> Hi all,
>    After reading through your responses and going over the man pages I
> got started. Here's the set of commands I have executed so far:
> 
> mdadm /dev/md6 --fail /dev/sdc6 --remove /dev/sdc6
> mdadm --zero-superblock /dev/sdc6
> mdadm --grow /dev/md6 --raid-devices=2
> 
> At this point I appear to have a good 2-drive RAID-1:
> 
> md6 : active raid1 sda6[0] sdb6[1]
>       247416933 blocks super 1.1 [2/2] [UU]
> 
> The next command was supposed to convert it to RAID-5 but doesn't work
> for me so far:
> 
> c2stable ~ # mdadm --grow /dev/md6 --level=5
> mdadm: /dev/md6: could not set level to raid5
> c2stable ~ #

This is because:

> /dev/md6:
>         Version : 1.1
>   Creation Time : Thu Apr 15 10:45:35 2010
>      Raid Level : raid1
>      Array Size : 247416933 (235.96 GiB 253.35 GB)
                    ^^^^^^^^^


This number is not a multiple of a usable chunk size.  I keep forgetting
about that issue.  I'll get mdadm to be more helpful in a future release.

You need to reduce the size of the array a little bit first.  That might mean
shrinking a filesystem that you have on it.

If you reduce it by 101K it will be a multiple of 1M which is good.

If the filesystem is extX, you can use resize2fs to make the filesystem
smaller first.  If it is something else ... you might have difficulty.

Once you think you are ready I would:

 mdadm -G /dev/md6 --size 241618M

 fsck /dev/md6  # just to me sure - if something is wrong you can
                # still increase the size again and nothing will be
                # lost
 mdadm -G /dev/md6 --level 5

NeilBrown



[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]