[Question] Fail event during reshape raid5 -> raid6

[Question] Fail event during reshape raid5 -> raid6

[Roland 'ValiDOM' Jungnickel]

hi!

I started reshaping a 3-device raid5 to a 4-device raid6. After some 
hours, I got a fail event on a harddrive which was already used in the 
raid5.

There are two different raid-groups on these harddrives, a reshape of 
the second one has not yet started.

What would you suggest?
* stop the reshape, go back to raid5 and start rebuild to the new disk 
(how?)
* just wait, hope and pray... ( there is no backup, I do not have 
harddrives to backup to as this is just to much...)

md0 : active raid6 sdc2[0] sde2[4](F) sdb2[3] sdd2[1]
       2858420224 blocks super 0.91 level 6, 64k chunk, algorithm 18 
[4/2] [UU__]
       [=>...................]  reshape =  7.8% (112662528/1429210112) 
finish=4938.2min speed=4442K/sec

md1 : active raid5 sdc1[0] sde1[2] sdb1[3](S) sdd1[1]
       1048575744 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]

(md1 not mounted, sde1 will fail once used/checked and the raid5 will 
automatically start rebuild on sdb1)

Thanks!
Vali

Re: [Question] Fail event during reshape raid5 -> raid6

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 1654 bytes --]

On Tue, 03 Sep 2013 09:20:00 +0200 Roland 'ValiDOM' Jungnickel
<vali2013@validom.de> wrote:

> hi!
> 
> I started reshaping a 3-device raid5 to a 4-device raid6. After some 
> hours, I got a fail event on a harddrive which was already used in the 
> raid5.
> 
> There are two different raid-groups on these harddrives, a reshape of 
> the second one has not yet started.
> 
> What would you suggest?
> * stop the reshape, go back to raid5 and start rebuild to the new disk 
> (how?)
> * just wait, hope and pray... ( there is no backup, I do not have 
> harddrives to backup to as this is just to much...)
> 
> md0 : active raid6 sdc2[0] sde2[4](F) sdb2[3] sdd2[1]
>        2858420224 blocks super 0.91 level 6, 64k chunk, algorithm 18 
> [4/2] [UU__]
>        [=>...................]  reshape =  7.8% (112662528/1429210112) 
> finish=4938.2min speed=4442K/sec
> 
> md1 : active raid5 sdc1[0] sde1[2] sdb1[3](S) sdd1[1]
>        1048575744 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]
> 
> (md1 not mounted, sde1 will fail once used/checked and the raid5 will 
> automatically start rebuild on sdb1)
> 

I would probably add the new good drive to the array (after some basic
testing to ensure that it really is good).  Then stop the array and restart
it.
I think it will recovery and reshape at the same time though that might
depend on what kernel you are using.
Testing with loop-back devices should be fairly easy to confirm what will
happen.

mdadm-3.3 (just released) can reverse a reshape for you.
   mdadm --assemble /dev/md0 --update=revert-reshape /dev/sd[dbc]2
   --backup=/whatever

NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]