Re: raid 5 to raid 6 reshape gone bad

[NeilBrown <neilb@suse.de>]

[-- Attachment #1: Type: text/plain, Size: 3404 bytes --]

On Sat, 12 Nov 2011 21:56:56 -0500 Travis Brown <teb@jetcom.org> wrote:

> I was reshaping my 5 drive raid 5 with spare to a raid 6 array when the drive I was using for my backup went offline.  If that's not murphy's law, I don't know what is.  The array is still up and usable, but I'm afraid to reboot or doing anything to it, really.  Suggestions on getting this thing back to usable are very welcome. 
> 
> Thanks,
> Travis
> 
> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
> md126 : active raid6 sde3[0] sdf3[3] sdb3[1] sdd3[4] sdc3[2]
>       5856728064 blocks super 0.91 level 6, 512k chunk, algorithm 18 [5/4] [UUUU_]
>       [>....................]  reshape =  0.9% (19267584/1952242688) finish=623878.3min speed=51K/sec

1/ Don't Panic.  
   You seem to have achieved this step quite effectively - congratulations.

2/ Stop the array cleanly.  Not having a backup will only cause possible
   corruption if the machine crashes while the reshape is happening.  The
   reshape has stopped so there is no chance for corruption.  But you still
   need to cleanly stop the array.
   (A subsequent version of mdadm may allow you to continue the reshape
   without the stop/restart step, but we aren't there yet).

3/ Make sure you have a version of mdadm which is at least 3.2.   I would
   suggest the latest:3.2.2.  You particularly need the --invalid-backup
   flag.

4/ Reassemble the array with e.g.

    mdadm --assemble /dev/md126 --backup=/some/file  \
    --invalid-backup /dev/sd[bcdef]3


   The  backup file does not need to exist (I think).  Maybe create an empty
   file and use that just to be safe.
   The "--invalid-backup" flag says to mdadm "Yes, I know the backup file is
   currently invalid and you cannot restore anything from it.  I happen to
   know that there is no need to restore anything because I did a clean
   shutdown.  Just use the backup file for making new backups as you continue
   the reshape".

NeilBrown


> 
> /dev/md126:
>         Version : 0.91
>   Creation Time : Wed Nov 10 20:19:03 2010
>      Raid Level : raid6
>      Array Size : 5856728064 (5585.41 GiB 5997.29 GB)
>   Used Dev Size : 1952242688 (1861.80 GiB 1999.10 GB)
>    Raid Devices : 5
>   Total Devices : 5
> Preferred Minor : 126
>     Persistence : Superblock is persistent
> 
>     Update Time : Sat Nov 12 21:55:46 2011
>           State : clean, degraded, recovering
>  Active Devices : 4
> Working Devices : 5
>  Failed Devices : 0
>   Spare Devices : 1
> 
>          Layout : left-symmetric-6
>      Chunk Size : 512K
> 
>  Reshape Status : 0% complete
>      New Layout : left-symmetric
> 
>            UUID : 3fd8b303:7727aa3b:c5d110f2:f9137e1d
>          Events : 0.124051
> 
>     Number   Major   Minor   RaidDevice State
>        0       8       67        0      active sync   /dev/sde3
>        1       8       19        1      active sync   /dev/sdb3
>        2       8       35        2      active sync   /dev/sdc3
>        3       8       83        3      active sync   /dev/sdf3
>        4       8       51        4      spare rebuilding   /dev/sdd3--
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]