Growing a RAID array

Growing a RAID array

[Leslie Rhorer]

	I am shortly going to be growing a 10 drive RAID6 array with two
additional drives.  Can this be done in a single grow operation by setting
n=12, or do I have to grow it 1 drive at a time?

Re: Growing a RAID array

[NeilBrown]

On Thu, 12 May 2011 20:02:01 -0500 "Leslie Rhorer" <lrhorer@satx.rr.com>
wrote:

> 
> 	I am shortly going to be growing a 10 drive RAID6 array with two
> additional drives.  Can this be done in a single grow operation by setting
> n=12, or do I have to grow it 1 drive at a time?
> 

Experiment with loop-back devices created on modest sized files.


(but yes, you can grow by any number you like.  You should test the drives
fairly heavily first, because if they both die during the reshape, you lose
your data).

NeilBrown

RE: Growing a RAID array

[Leslie Rhorer]

> -----Original Message-----
> From: NeilBrown [mailto:neilb@suse.de]
> Sent: Thursday, May 12, 2011 8:23 PM
> To: lrhorer@satx.rr.com
> Cc: linux-raid@vger.kernel.org
> Subject: Re: Growing a RAID array
> 
> On Thu, 12 May 2011 20:02:01 -0500 "Leslie Rhorer" <lrhorer@satx.rr.com>
> wrote:
> 
> >
> > 	I am shortly going to be growing a 10 drive RAID6 array with two
> > additional drives.  Can this be done in a single grow operation by
> setting
> > n=12, or do I have to grow it 1 drive at a time?
> >
> 
> Experiment with loop-back devices created on modest sized files.
> 
> 
> (but yes, you can grow by any number you like.  You should test the drives
> fairly heavily first, because if they both die during the reshape, you
> lose
> your data).

	Actually, the "drives" won't be physical drives, but RAID0 volumes
made of 2 drives each.  'Ugly, I know, but I really have little choice for
the time being.  As to losing the data, this is a backup system in the first
place, and all its data is backed up to offline storage, as well, so a loss
of data would not be the most hideous thing ever.  All four drives have been
operating continuously without error for over a year, having been completely
read and written over several times, so I'm pretty confidant of their
reliability, enough so I don't think any additional testing is worth the
time and effort.

RE: Growing a RAID array

[Leslie Rhorer]

> -----Original Message-----
> From: NeilBrown [mailto:neilb@suse.de]
> Sent: Thursday, May 12, 2011 8:23 PM
> To: lrhorer@satx.rr.com
> Cc: linux-raid@vger.kernel.org
> Subject: Re: Growing a RAID array
> 
> On Thu, 12 May 2011 20:02:01 -0500 "Leslie Rhorer" <lrhorer@satx.rr.com>
> wrote:
> 
> >
> > 	I am shortly going to be growing a 10 drive RAID6 array with two
> > additional drives.  Can this be done in a single grow operation by
> setting
> > n=12, or do I have to grow it 1 drive at a time?
> >
> 
> Experiment with loop-back devices created on modest sized files.

	OK, I tried this, but for some reason the re-shape is failing, even
with 1 "disk".  I've grown arrays under mdadm any number of times, and I'm
not sure why this is failing.  I created 8 loop devices of 10M each.  I then
created a RAID6 array with 6 members and 2 spares.  When I try to add one or
both drives, I get an error:

RAID-Server:/RAID# mdadm -G /dev/md6 -n 8 --backup-file=/RAID/raid6.bak
mdadm: Need to backup 12288K of critical section..
mdadm: /dev/md6: Something wrong - reshape aborted

	Here is the reported layout:

RAID-Server:/RAID# mdadm -D /dev/md6
/dev/md6:
        Version : 1.2
  Creation Time : Fri May 13 23:15:39 2011
     Raid Level : raid6
     Array Size : 36864 (36.01 MiB 37.75 MB)
  Used Dev Size : 9216 (9.00 MiB 9.44 MB)
   Raid Devices : 6
  Total Devices : 8
    Persistence : Superblock is persistent

  Intent Bitmap : Internal

    Update Time : Fri May 13 23:23:04 2011
          State : active
 Active Devices : 6
Working Devices : 8
 Failed Devices : 0
  Spare Devices : 2

         Layout : left-symmetric
     Chunk Size : 1024K

           Name : RAID-Server:6  (local to host RAID-Server)
           UUID : 332f387c:6c301089:22904f6c:8089b9b4
         Events : 30

    Number   Major   Minor   RaidDevice State
       0       7        0        0      active sync   /dev/loop0
       1       7        1        1      active sync   /dev/loop1
       2       7        2        2      active sync   /dev/loop2
       3       7        3        3      active sync   /dev/loop3
       4       7        4        4      active sync   /dev/loop4
       5       7        5        5      active sync   /dev/loop5

       6       7        6        -      spare   /dev/loop6
       7       7        7        -      spare   /dev/loop7


	The mdadm version is 3.1.4 under kernel 2.6.32-5-amd64

Re: Growing a RAID array

[NeilBrown]

On Sat, 14 May 2011 02:17:27 -0500 "Leslie Rhorer" <lrhorer@satx.rr.com>
wrote:

> > -----Original Message-----
> > From: NeilBrown [mailto:neilb@suse.de]
> > Sent: Thursday, May 12, 2011 8:23 PM
> > To: lrhorer@satx.rr.com
> > Cc: linux-raid@vger.kernel.org
> > Subject: Re: Growing a RAID array
> > 
> > On Thu, 12 May 2011 20:02:01 -0500 "Leslie Rhorer" <lrhorer@satx.rr.com>
> > wrote:
> > 
> > >
> > > 	I am shortly going to be growing a 10 drive RAID6 array with two
> > > additional drives.  Can this be done in a single grow operation by
> > setting
> > > n=12, or do I have to grow it 1 drive at a time?
> > >
> > 
> > Experiment with loop-back devices created on modest sized files.
> 
> 	OK, I tried this, but for some reason the re-shape is failing, even
> with 1 "disk".  I've grown arrays under mdadm any number of times, and I'm
> not sure why this is failing.  I created 8 loop devices of 10M each.  I then
> created a RAID6 array with 6 members and 2 spares.  When I try to add one or
> both drives, I get an error:
> 
> RAID-Server:/RAID# mdadm -G /dev/md6 -n 8 --backup-file=/RAID/raid6.bak
> mdadm: Need to backup 12288K of critical section..
> mdadm: /dev/md6: Something wrong - reshape aborted
> 
> 	Here is the reported layout:
> 
> RAID-Server:/RAID# mdadm -D /dev/md6
> /dev/md6:
>         Version : 1.2
>   Creation Time : Fri May 13 23:15:39 2011
>      Raid Level : raid6
>      Array Size : 36864 (36.01 MiB 37.75 MB)
>   Used Dev Size : 9216 (9.00 MiB 9.44 MB)
>    Raid Devices : 6
>   Total Devices : 8
>     Persistence : Superblock is persistent
> 
>   Intent Bitmap : Internal
> 
>     Update Time : Fri May 13 23:23:04 2011
>           State : active
>  Active Devices : 6
> Working Devices : 8
>  Failed Devices : 0
>   Spare Devices : 2
> 
>          Layout : left-symmetric
>      Chunk Size : 1024K

10Meg devices with a 1Meg chunk size means just 10 stripes.

To grow from 4 to 5 data disks it calculates that it needs to back up the
first 20 stripes (I think).  It sees this is more than half the device and
gets confused and assumes something is wrong.

Try with 4k chunks, or 100Meg devices.

NeilBrown


> 
>            Name : RAID-Server:6  (local to host RAID-Server)
>            UUID : 332f387c:6c301089:22904f6c:8089b9b4
>          Events : 30
> 
>     Number   Major   Minor   RaidDevice State
>        0       7        0        0      active sync   /dev/loop0
>        1       7        1        1      active sync   /dev/loop1
>        2       7        2        2      active sync   /dev/loop2
>        3       7        3        3      active sync   /dev/loop3
>        4       7        4        4      active sync   /dev/loop4
>        5       7        5        5      active sync   /dev/loop5
> 
>        6       7        6        -      spare   /dev/loop6
>        7       7        7        -      spare   /dev/loop7
> 
> 
> 	The mdadm version is 3.1.4 under kernel 2.6.32-5-amd64
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Growing a RAID array

[Leslie Rhorer]

> 10Meg devices with a 1Meg chunk size means just 10 stripes.
> 
> To grow from 4 to 5 data disks it calculates that it needs to back up the
> first 20 stripes (I think).  It sees this is more than half the device and
> gets confused and assumes something is wrong.
> 
> Try with 4k chunks, or 100Meg devices.

	Silly machine.  :-)

	'Works like a champ, thanks.  No errors, and the process completes
in the blink of an eye.  Now if growing an array with 3T spindles were only
that fast...

RE: Growing a RAID array

[Leslie Rhorer]

> -----Original Message-----
> From: linux-raid-owner@vger.kernel.org [mailto:linux-raid-
> owner@vger.kernel.org] On Behalf Of Leslie Rhorer
> Sent: Saturday, May 14, 2011 9:44 AM
> To: 'NeilBrown'
> Cc: linux-raid@vger.kernel.org
> Subject: RE: Growing a RAID array
> 
> > 10Meg devices with a 1Meg chunk size means just 10 stripes.
> >
> > To grow from 4 to 5 data disks it calculates that it needs to back up
> the
> > first 20 stripes (I think).  It sees this is more than half the device
> and
> > gets confused and assumes something is wrong.
> >
> > Try with 4k chunks, or 100Meg devices.
> 
> 	Silly machine.  :-)
> 
> 	'Works like a champ, thanks.  No errors, and the process completes
> in the blink of an eye.  Now if growing an array with 3T spindles were
> only
> that fast...

	Well, this is interesting.  I was playing around a bit more, and I
tried to start a resync.  It failed:

RAID-Server:/RAID# echo repair > /sys/block/md6/md/sync_action
bash: echo: write error: Device or resource busy

	The device was unmounted at the time, so I mounted it and tried
again.  This time it worked.  Is this normal behaviour?