Growing a raid 6 array

Growing a raid 6 array

[Laurent CARON]

Hi,

As our storage needs are growing i'm in the process of growing a 6TB
array to 9TB by changing the disks one by one (500GB to 750GB).

I'll have to partition the new drives with bigger partitions and then
fail the array, remove a drive, add a new 750GB disk.... repeat it as
long as 500GB disks are remaining in the system.

The question now is to know if i'm right, or ... simply wrong in
thinking that this operation is then just as simple as:

server:~# mdadm -G /dev/mdX -n number_of_disks(same as original array)

once done, i'll have to grow the filesystem (ext3) by issuing

server:~# resize2fs -p /dev/mdX



Does the fact that the ext3 partition is nearly full involves a bigger
risk of screwing the data ?

Would it be better/worse to have a bitmap (internal one) on the mdX
device while growing it ?

Thanks

Laurent

Re: Growing a raid 6 array

[Justin Piszcz]

You can only grow a RAID5 array in Linux as of 2.6.20 AFAIK.

Justin.

On Thu, 1 Mar 2007, Laurent CARON wrote:

> Hi,
>
> As our storage needs are growing i'm in the process of growing a 6TB
> array to 9TB by changing the disks one by one (500GB to 750GB).
>
> I'll have to partition the new drives with bigger partitions and then
> fail the array, remove a drive, add a new 750GB disk.... repeat it as
> long as 500GB disks are remaining in the system.
>
> The question now is to know if i'm right, or ... simply wrong in
> thinking that this operation is then just as simple as:
>
> server:~# mdadm -G /dev/mdX -n number_of_disks(same as original array)
>
> once done, i'll have to grow the filesystem (ext3) by issuing
>
> server:~# resize2fs -p /dev/mdX
>
>
>
> Does the fact that the ext3 partition is nearly full involves a bigger
> risk of screwing the data ?
>
> Would it be better/worse to have a bitmap (internal one) on the mdX
> device while growing it ?
>
> Thanks
>
> Laurent
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Growing a raid 6 array

[Laurent CARON]

Justin Piszcz wrote:
> You can only grow a RAID5 array in Linux as of 2.6.20 AFAIK.


From the man page:

Grow   Grow (or shrink) an array, or otherwise reshape it in some way.
Currently supported growth options including changing the active size of
component devices in RAID level 1/4/5/6 and changing the number of
active devices in RAID1.

Re: Growing a raid 6 array

[Justin Piszcz]

On Fri, 2 Mar 2007, Laurent CARON wrote:

> Justin Piszcz wrote:
>> You can only grow a RAID5 array in Linux as of 2.6.20 AFAIK.
>
>
>> From the man page:
>
> Grow   Grow (or shrink) an array, or otherwise reshape it in some way.
> Currently supported growth options including changing the active size of
> component devices in RAID level 1/4/5/6 and changing the number of
> active devices in RAID1.
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Interesting, give it a try? :)  The kernel says raid-5 only.

   x x    [*]     RAID-4/RAID-5/RAID-6 mode
   x x    [*]       Support adding drives to a raid-5 array

   x CONFIG_MD_RAID5_RESHAPE:                                                x
   x                                                                         x
   x A RAID-5 set can be expanded by adding extra drives. This               x
   x requires "restriping" the array which means (almost) every              x
   x block must be written to a different place.                             x

Re: Growing a raid 6 array

[Neil Brown]

On Thursday March 1, jpiszcz@lucidpixels.com wrote:
> You can only grow a RAID5 array in Linux as of 2.6.20 AFAIK.

There are two dimensions for growth.
You can increase the amount of each device that is used, or you can
increase the number of devices.

You are correct that increasing the number of devices only works for
RAID5 (and RAID1, but you don't get extra space) in 2.6.20 (RAID6
coming in 2.6.21).

However this question is about growing an array the first way:
increasing the amount of space used on each devices, and that is
supported for RAID1/4/5/6.

And Laurent:
  1/ Yes, it is that easy
  2/ I doubt a nearly-full ext3 array increases the risk
  3/ The effect of adding a bitmap is that if you suffer a crash while
     the array is degraded, it will resync faster so you have less
     exposure to multiple failure.

NeilBrown

Re: Growing a raid 6 array

[Laurent CARON]

Neil Brown a écrit :
> And Laurent:
>   1/ Yes, it is that easy
>   2/ I doubt a nearly-full ext3 array increases the risk
>   3/ The effect of adding a bitmap is that if you suffer a crash while
>      the array is degraded, it will resync faster so you have less
>      exposure to multiple failure.
> 

Ok,

Thanks for your advice.

Laurent
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Growing a raid 6 array

[Laurent CARON]

Neil Brown a écrit :
> On Thursday March 1, jpiszcz@lucidpixels.com wrote:
>> You can only grow a RAID5 array in Linux as of 2.6.20 AFAIK.
> 
> There are two dimensions for growth.
> You can increase the amount of each device that is used, or you can
> increase the number of devices.
> 
> You are correct that increasing the number of devices only works for
> RAID5 (and RAID1, but you don't get extra space) in 2.6.20 (RAID6
> coming in 2.6.21).
> 
> However this question is about growing an array the first way:
> increasing the amount of space used on each devices, and that is
> supported for RAID1/4/5/6.
> 
> And Laurent:
>   1/ Yes, it is that easy
>   2/ I doubt a nearly-full ext3 array increases the risk
>   3/ The effect of adding a bitmap is that if you suffer a crash while
>      the array is degraded, it will resync faster so you have less
>      exposure to multiple failure.

I just finished changing disks, growing the array, and then the filesystem.

It worked flawlessy.

Just a little notice: I had to unmount my ext3 filesystem to be able to
resize it. (Took ~8 hours to fsck + resize the 15 disks array from 6 to
9TB on a dual Xeon with 4GB RAM).

Thanks

Laurent
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Growing a raid 6 array

[Louis-David Mitterrand]

On Fri, Apr 13, 2007 at 10:15:05AM +0200, Laurent CARON wrote:
> Neil Brown a écrit :
> > On Thursday March 1, jpiszcz@lucidpixels.com wrote:
> >> You can only grow a RAID5 array in Linux as of 2.6.20 AFAIK.
> > 
> > There are two dimensions for growth.
> > You can increase the amount of each device that is used, or you can
> > increase the number of devices.
> > 
> > You are correct that increasing the number of devices only works for
> > RAID5 (and RAID1, but you don't get extra space) in 2.6.20 (RAID6
> > coming in 2.6.21).
> > 
> > However this question is about growing an array the first way:
> > increasing the amount of space used on each devices, and that is
> > supported for RAID1/4/5/6.
> > 
> > And Laurent:
> >   1/ Yes, it is that easy
> >   2/ I doubt a nearly-full ext3 array increases the risk
> >   3/ The effect of adding a bitmap is that if you suffer a crash while
> >      the array is degraded, it will resync faster so you have less
> >      exposure to multiple failure.
> 
> I just finished changing disks, growing the array, and then the filesystem.
> 
> It worked flawlessy.
> 
> Just a little notice: I had to unmount my ext3 filesystem to be able to
> resize it. (Took ~8 hours to fsck + resize the 15 disks array from 6 to
> 9TB on a dual Xeon with 4GB RAM).

FWIW:

I changed a 6 x 400G system to 500G disks and grew the raid5 array. It 
worked fine save for these warnings:

	Apr  8 16:54:33 sylla mdadm: RebuildStarted event detected on md device /dev/md1
	Apr  8 16:54:33 sylla mdadm: Rebuild20 event detected on md device /dev/md1
	Apr  8 16:54:33 sylla mdadm: Rebuild40 event detected on md device /dev/md1
	Apr  8 16:54:33 sylla mdadm: Rebuild60 event detected on md device /dev/md1
	Apr  8 16:54:33 sylla mdadm: Rebuild80 event detected on md device /dev/md1
	Apr  8 16:54:33 sylla kernel: md1: invalid bitmap page request: 187 (> 186)
	Apr  8 16:54:33 sylla kernel: md1: invalid bitmap page request: 187 (> 186)

	etc...

So after rebooting I removed the bitmap and recreated it.

Online resizing of the 2.0T xfs to 2.4T only took a few _seconds_.

A similar operation on a 4 x 250G raid5 system upgraded to 400G disks 
with reiser3 also took a few seconds of _online_ time.

Kernel 2.6.20.6
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html