reshape raid5 to raid6

reshape raid5 to raid6

[Michael Ole Olsen]

[-- Attachment #1: Type: text/plain, Size: 852 bytes --]

Is it possible to reshape my /dev/md0 raid5 into raid6?

I found the following recipe in the docs:
mdadm --create /dev/md0 -l6 -n4 /dev/sdb1 /dev/sdc1 /dev/sdd1 missing

I have 9 disks in my raid5, would i then do:
[1] mdadm --create /dev/md0 -l6 -n10 /dev/sda /dev/sdb ... missing

i.e. only the first 9 drives and then hot add a new device to get it to
reshape and add the extra parity to all drives?

mdadm --add /dev/md0 /dev/sdk ?

I would guess  command [1] would give problems as md0 is an active array
will it work if i --stop the array first so that i can reshape to raid6?

I couldn't really find much information on this, some say it is impossible
without recreating the array? (So i ask here first before i try, as I dont
want to risk my data, even though I have backups of the important
data)

Any advice would be nice

/Michael Ole Olsen

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 835 bytes --]

Re: reshape raid5 to raid6

[NeilBrown]

On Wed, June 24, 2009 8:27 pm, Michael Ole Olsen wrote:
> Is it possible to reshape my /dev/md0 raid5 into raid6?

If you are are using Linux 2.6.30, then you can

  echo raid6 > /sys/block/md0/md/level

and it will instantly be sort-of-raid6.
It is exactly like raid6 except that the Q blocks are all one
the one drive, and drive that previously didn't exist.
If you have a spare, it will start building the Q blocks
on that drive and when it finishes you will have true raid6
redundancy, though possibly a little less than raid6 performance,
as a real raid6 has the Q block distributed.

When mdadm-3.1 is released, you will be able to tell the raid6
to re-stripe with a more traditional layout.  This will take quite
a while, but you can continue to use the array (though a bit more
slowly) will it progresses.
Of course you don't need to do that step if you don't want to.



>
> I found the following recipe in the docs:
> mdadm --create /dev/md0 -l6 -n4 /dev/sdb1 /dev/sdc1 /dev/sdd1 missing
>
> I have 9 disks in my raid5, would i then do:
> [1] mdadm --create /dev/md0 -l6 -n10 /dev/sda /dev/sdb ... missing

If you did this, an the drives contained raid5 data, then that data
would not be available on /dev/md0.

NeilBrown


>
> i.e. only the first 9 drives and then hot add a new device to get it to
> reshape and add the extra parity to all drives?
>
> mdadm --add /dev/md0 /dev/sdk ?
>
> I would guess  command [1] would give problems as md0 is an active array
> will it work if i --stop the array first so that i can reshape to raid6?
>
> I couldn't really find much information on this, some say it is impossible
> without recreating the array? (So i ask here first before i try, as I dont
> want to risk my data, even though I have backups of the important
> data)
>
> Any advice would be nice
>
> /Michael Ole Olsen
>

Re: reshape raid5 to raid6

[Beolach]

On Wed, Jun 24, 2009 at 05:20, NeilBrown <neilb@suse.de> wrote:
>
> On Wed, June 24, 2009 8:27 pm, Michael Ole Olsen wrote:
> > Is it possible to reshape my /dev/md0 raid5 into raid6?
>
> If you are are using Linux 2.6.30, then you can
>
>  echo raid6 > /sys/block/md0/md/level
>
> and it will instantly be sort-of-raid6.
> It is exactly like raid6 except that the Q blocks are all one
> the one drive, and drive that previously didn't exist.
> If you have a spare, it will start building the Q blocks
> on that drive and when it finishes you will have true raid6
> redundancy, though possibly a little less than raid6 performance,
> as a real raid6 has the Q block distributed.
>
> When mdadm-3.1 is released, you will be able to tell the raid6
> to re-stripe with a more traditional layout.  This will take quite
> a while, but you can continue to use the array (though a bit more
> slowly) will it progresses.
> Of course you don't need to do that step if you don't want to.

If a raid5 is changed to raid6, and then the raid6 is grown w/
additional disks, will that switch it to a more traditional layout,
since it has to be re-striped anyway?


TIA,
Conway S. Smith
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: reshape raid5 to raid6

[Michael Ole Olsen]

[-- Attachment #1: Type: text/plain, Size: 3496 bytes --]

Hello I just compiled the 2.6.30 kernel from kernel.org
I was running 29.3 and 2.6.7.2 mdadm (Nov 08) Debian lenny, but kernel from
kernel.org.

I added the new device with echoing into /sys as you described

mfs:/home/michael# mdadm --zero-superblock /dev/sdc (It was an old disk from
the raid that was taken out long ago and not used - array was not degraded,
so I can just zero it)

mfs:/home/michael# mdadm --add /dev/md0 /dev/sdc

mfs:/home/michael# cat /proc/mdstat 
Personalities : [raid6] [raid5] [raid4] 
md0 : active raid6 sdc[10] sdi[0] sda[8] sdj[7] sdb[6] sde[5] sdg[4] sdd[3]
sdh[2] sdf[1]
      11721107968 blocks level 6, 64k chunk, algorithm 18 [10/9] [UUUUUUUUU_]
            [>....................]  recovery =  0.9% (14365824/1465138496)
    finish=414.1min speed=58382K/sec

the raid6 seems to be working, at least proc says so.

after stopping my reads and writes to the disk the reshape went 
from approximately 1000kB/s to 55000kB/s (i think i was only reading a bit from it)

440 minutes till reshape, so seems to go just fine.

I will try to re-stripe (and of course backup data first) with the new mdadm 
when it comes out, looking forward to it, thanks

/Michael Ole Olsen


NeilBrown schrieb am Wednesday, den 24. June 2009:

> On Wed, June 24, 2009 8:27 pm, Michael Ole Olsen wrote:
> > Is it possible to reshape my /dev/md0 raid5 into raid6?
> 
> If you are are using Linux 2.6.30, then you can
> 
>   echo raid6 > /sys/block/md0/md/level
> 
> and it will instantly be sort-of-raid6.
> It is exactly like raid6 except that the Q blocks are all one
> the one drive, and drive that previously didn't exist.
> If you have a spare, it will start building the Q blocks
> on that drive and when it finishes you will have true raid6
> redundancy, though possibly a little less than raid6 performance,
> as a real raid6 has the Q block distributed.
> 
> When mdadm-3.1 is released, you will be able to tell the raid6
> to re-stripe with a more traditional layout.  This will take quite
> a while, but you can continue to use the array (though a bit more
> slowly) will it progresses.
> Of course you don't need to do that step if you don't want to.
> 
> 
> 
> >
> > I found the following recipe in the docs:
> > mdadm --create /dev/md0 -l6 -n4 /dev/sdb1 /dev/sdc1 /dev/sdd1 missing
> >
> > I have 9 disks in my raid5, would i then do:
> > [1] mdadm --create /dev/md0 -l6 -n10 /dev/sda /dev/sdb ... missing
> 
> If you did this, an the drives contained raid5 data, then that data
> would not be available on /dev/md0.
> 
> NeilBrown
> 
> 
> >
> > i.e. only the first 9 drives and then hot add a new device to get it to
> > reshape and add the extra parity to all drives?
> >
> > mdadm --add /dev/md0 /dev/sdk ?
> >
> > I would guess  command [1] would give problems as md0 is an active array
> > will it work if i --stop the array first so that i can reshape to raid6?
> >
> > I couldn't really find much information on this, some say it is impossible
> > without recreating the array? (So i ask here first before i try, as I dont
> > want to risk my data, even though I have backups of the important
> > data)
> >
> > Any advice would be nice
> >
> > /Michael Ole Olsen
> >
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 835 bytes --]

Re: reshape raid5 to raid6

[Billy Crook]

On Wed, Jun 24, 2009 at 06:20, NeilBrown<neilb@suse.de> wrote:
> On Wed, June 24, 2009 8:27 pm, Michael Ole Olsen wrote:
>> Is it possible to reshape my /dev/md0 raid5 into raid6?
>
> If you are are using Linux 2.6.30, then you can
>
>  echo raid6 > /sys/block/md0/md/level
>
> and it will instantly be sort-of-raid6.
> It is exactly like raid6 except that the Q blocks are all one
> the one drive, and drive that previously didn't exist.
> If you have a spare, it will start building the Q blocks
> on that drive and when it finishes you will have true raid6
> redundancy, though possibly a little less than raid6 performance,
> as a real raid6 has the Q block distributed.
>
> When mdadm-3.1 is released, you will be able to tell the raid6
> to re-stripe with a more traditional layout.  This will take quite
> a while, but you can continue to use the array (though a bit more
> slowly) will it progresses.
> Of course you don't need to do that step if you don't want to.

I have a raid5 array on 2.6.18 that I'd like to grow like this.  I
might wait until mdadm-3.1 so I can stripe Q from the git-go.  I'd
like to --stop the array on the 2.6.18 machine, and export the
individual disks over iscsi to a 2.6.30 machine, and use the newer
mdadm there to grow the array from raid5 to raid6.  Then --stop it on
the 2.6.30 machine, unexport the disks, and --start the array again on
the 2.6.18 machine.  Disclaimers aside, should that work?  My main
concern is 2.6.18's ability to work with this 'creative' raid6
implementation that currently results from the grow from raid5 to
raid6.

I've also got a few disks to add, so maybe the better solution would
be to add one and get the unstriped Q, then add another and let Q
stripe with everything else during the reshape.  That is, if it will
stripe Q during the reshape.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: reshape raid5 to raid6

[Neil Brown]

On Wednesday June 24, billycrook@gmail.com wrote:
> On Wed, Jun 24, 2009 at 06:20, NeilBrown<neilb@suse.de> wrote:
> > On Wed, June 24, 2009 8:27 pm, Michael Ole Olsen wrote:
> >> Is it possible to reshape my /dev/md0 raid5 into raid6?
> >
> > If you are are using Linux 2.6.30, then you can
> >
> >  echo raid6 > /sys/block/md0/md/level
> >
> > and it will instantly be sort-of-raid6.
> > It is exactly like raid6 except that the Q blocks are all one
> > the one drive, and drive that previously didn't exist.
> > If you have a spare, it will start building the Q blocks
> > on that drive and when it finishes you will have true raid6
> > redundancy, though possibly a little less than raid6 performance,
> > as a real raid6 has the Q block distributed.
> >
> > When mdadm-3.1 is released, you will be able to tell the raid6
> > to re-stripe with a more traditional layout.  This will take quite
> > a while, but you can continue to use the array (though a bit more
> > slowly) will it progresses.
> > Of course you don't need to do that step if you don't want to.
> 
> I have a raid5 array on 2.6.18 that I'd like to grow like this.  I
> might wait until mdadm-3.1 so I can stripe Q from the git-go.  I'd
> like to --stop the array on the 2.6.18 machine, and export the
> individual disks over iscsi to a 2.6.30 machine, and use the newer
> mdadm there to grow the array from raid5 to raid6.  Then --stop it on
> the 2.6.30 machine, unexport the disks, and --start the array again on
> the 2.6.18 machine.  Disclaimers aside, should that work?  My main
> concern is 2.6.18's ability to work with this 'creative' raid6
> implementation that currently results from the grow from raid5 to
> raid6.

2.6.18 will not understand the raid6 created by simply echoing 'raid6'
in to the 'level' file.  It will need to be restriped with the help of
mdadm-3.1 first.

> 
> I've also got a few disks to add, so maybe the better solution would
> be to add one and get the unstriped Q, then add another and let Q
> stripe with everything else during the reshape.  That is, if it will
> stripe Q during the reshape.

Your best bet would be to wait for mdadm-3.1 and do it all at once,
something like:
  mdadm --grow /dev/md0 --level=raid6 --raid-disks=8

NeilBrown
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: reshape raid5 to raid6

[Mikael Abrahamsson]

On Wed, 24 Jun 2009, Beolach wrote:

> If a raid5 is changed to raid6, and then the raid6 is grown w/ 
> additional disks, will that switch it to a more traditional layout, 
> since it has to be re-striped anyway?

I'd greatly appreciate if someone in the know could answer this?

Also, this raid6 with all Q blocks on a single drive, will older kernels 
read this as well or is 2.6.30 needed to handle this "new" layout?

I'd like to convert a raid5 to raid6 with 2.6.30, then go back to 2.6.28 
and run it, then when I grow it next time, I'd like to get the "correct" 
layout with the Q blocks striped just as if I had created the raid6 from 
scratch. Is this possible?

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

Re: reshape raid5 to raid6

[NeilBrown]

On Thu, July 9, 2009 6:36 pm, Mikael Abrahamsson wrote:
> On Wed, 24 Jun 2009, Beolach wrote:
>
>> If a raid5 is changed to raid6, and then the raid6 is grown w/
>> additional disks, will that switch it to a more traditional layout,
>> since it has to be re-striped anyway?
>
> I'd greatly appreciate if someone in the know could answer this?

Sorry, I thought I had... must have only intended to.

The answer is "only if you explicitly tell it to", which mdadm-3.1
will do for you.
If you switch to raid6 by writing to a sysfs file, then grow with an
earlier version of mdadm, it will keep to unusual layout.

>
> Also, this raid6 with all Q blocks on a single drive, will older kernels
> read this as well or is 2.6.30 needed to handle this "new" layout?

You will need 2.6.30.  Older kernels will not understand it.

>
> I'd like to convert a raid5 to raid6 with 2.6.30, then go back to 2.6.28
> and run it, then when I grow it next time, I'd like to get the "correct"
> layout with the Q blocks striped just as if I had created the raid6 from
> scratch. Is this possible?

Nope, sorry.

NeilBrown

Re: reshape raid5 to raid6

[Michael Ole Olsen]

Is there any way to get anything below 2.6.30 to recognize this
'fake' raid6 with all Q blocks on the last disk?

I reshaped my raid5 to raid6 using this echo into /sys

The 2.6.30 and 2.6.30.1 is terribly unstable with xfs+nfs 
(1-3 kernel oopses a day and complete resync much of the time) 
(I have sent a bug report to xfs mailing list, it seems to be xfs/nfs)

Best regards,
Michael Ole Olsen

Neil Brown schrieb am Friday, den 26. June 2009:

> On Wednesday June 24, billycrook@gmail.com wrote:
> > On Wed, Jun 24, 2009 at 06:20, NeilBrown<neilb@suse.de> wrote:
> > > On Wed, June 24, 2009 8:27 pm, Michael Ole Olsen wrote:
> > >> Is it possible to reshape my /dev/md0 raid5 into raid6?
> > >
> > > If you are are using Linux 2.6.30, then you can
> > >
> > >  echo raid6 > /sys/block/md0/md/level
> > >
> > > and it will instantly be sort-of-raid6.
> > > It is exactly like raid6 except that the Q blocks are all one
> > > the one drive, and drive that previously didn't exist.
> > > If you have a spare, it will start building the Q blocks
> > > on that drive and when it finishes you will have true raid6
> > > redundancy, though possibly a little less than raid6 performance,
> > > as a real raid6 has the Q block distributed.
> > >
> > > When mdadm-3.1 is released, you will be able to tell the raid6
> > > to re-stripe with a more traditional layout.  This will take quite
> > > a while, but you can continue to use the array (though a bit more
> > > slowly) will it progresses.
> > > Of course you don't need to do that step if you don't want to.
> > 
> > I have a raid5 array on 2.6.18 that I'd like to grow like this.  I
> > might wait until mdadm-3.1 so I can stripe Q from the git-go.  I'd
> > like to --stop the array on the 2.6.18 machine, and export the
> > individual disks over iscsi to a 2.6.30 machine, and use the newer
> > mdadm there to grow the array from raid5 to raid6.  Then --stop it on
> > the 2.6.30 machine, unexport the disks, and --start the array again on
> > the 2.6.18 machine.  Disclaimers aside, should that work?  My main
> > concern is 2.6.18's ability to work with this 'creative' raid6
> > implementation that currently results from the grow from raid5 to
> > raid6.
> 
> 2.6.18 will not understand the raid6 created by simply echoing 'raid6'
> in to the 'level' file.  It will need to be restriped with the help of
> mdadm-3.1 first.
> 
> > 
> > I've also got a few disks to add, so maybe the better solution would
> > be to add one and get the unstriped Q, then add another and let Q
> > stripe with everything else during the reshape.  That is, if it will
> > stripe Q during the reshape.
> 
> Your best bet would be to wait for mdadm-3.1 and do it all at once,
> something like:
>   mdadm --grow /dev/md0 --level=raid6 --raid-disks=8
> 
> NeilBrown
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: reshape raid5 to raid6

[NeilBrown]

On Wed, July 15, 2009 1:29 pm, Michael Ole Olsen wrote:
> Is there any way to get anything below 2.6.30 to recognize this
> 'fake' raid6 with all Q blocks on the last disk?

You could back-port a selection of patches.  if you
  git log drivers/md/raid5.c
it should list all the patches you need, but it will list
quite a few that you don't want as well.

Or you could change the array back to raid5 by echoing "raid5" to
the same place you echoed "raid6".

mdadm-3.1 is making good progress but if you are getting
frequent reboots, then you will really need to restart-in-the-middle
-of-a-reshape functionality, and I'm not even sure the kernel
side of that works yet.  It'll be at least 2 weeks before I
could suggest you try that.

NeilBrown


>
> I reshaped my raid5 to raid6 using this echo into /sys
>
> The 2.6.30 and 2.6.30.1 is terribly unstable with xfs+nfs
> (1-3 kernel oopses a day and complete resync much of the time)
> (I have sent a bug report to xfs mailing list, it seems to be xfs/nfs)
>
> Best regards,
> Michael Ole Olsen
>
> Neil Brown schrieb am Friday, den 26. June 2009:
>
>> On Wednesday June 24, billycrook@gmail.com wrote:
>> > On Wed, Jun 24, 2009 at 06:20, NeilBrown<neilb@suse.de> wrote:
>> > > On Wed, June 24, 2009 8:27 pm, Michael Ole Olsen wrote:
>> > >> Is it possible to reshape my /dev/md0 raid5 into raid6?
>> > >
>> > > If you are are using Linux 2.6.30, then you can
>> > >
>> > >  echo raid6 > /sys/block/md0/md/level
>> > >
>> > > and it will instantly be sort-of-raid6.
>> > > It is exactly like raid6 except that the Q blocks are all one
>> > > the one drive, and drive that previously didn't exist.
>> > > If you have a spare, it will start building the Q blocks
>> > > on that drive and when it finishes you will have true raid6
>> > > redundancy, though possibly a little less than raid6 performance,
>> > > as a real raid6 has the Q block distributed.
>> > >
>> > > When mdadm-3.1 is released, you will be able to tell the raid6
>> > > to re-stripe with a more traditional layout.  This will take quite
>> > > a while, but you can continue to use the array (though a bit more
>> > > slowly) will it progresses.
>> > > Of course you don't need to do that step if you don't want to.
>> >
>> > I have a raid5 array on 2.6.18 that I'd like to grow like this.  I
>> > might wait until mdadm-3.1 so I can stripe Q from the git-go.  I'd
>> > like to --stop the array on the 2.6.18 machine, and export the
>> > individual disks over iscsi to a 2.6.30 machine, and use the newer
>> > mdadm there to grow the array from raid5 to raid6.  Then --stop it on
>> > the 2.6.30 machine, unexport the disks, and --start the array again on
>> > the 2.6.18 machine.  Disclaimers aside, should that work?  My main
>> > concern is 2.6.18's ability to work with this 'creative' raid6
>> > implementation that currently results from the grow from raid5 to
>> > raid6.
>>
>> 2.6.18 will not understand the raid6 created by simply echoing 'raid6'
>> in to the 'level' file.  It will need to be restriped with the help of
>> mdadm-3.1 first.
>>
>> >
>> > I've also got a few disks to add, so maybe the better solution would
>> > be to add one and get the unstriped Q, then add another and let Q
>> > stripe with everything else during the reshape.  That is, if it will
>> > stripe Q during the reshape.
>>
>> Your best bet would be to wait for mdadm-3.1 and do it all at once,
>> something like:
>>   mdadm --grow /dev/md0 --level=raid6 --raid-disks=8
>>
>> NeilBrown
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html