Re: Unable to restart reshape

[Michael Busby <michael.a.busby@gmail.com>]

>>>> I have a system the was doing a reshape from RAID5 to 6, the system
>>>> had to be powered off this morning and moved, upon restarting the
>>>> server i issued the following command to continue the reshape
>>>>
>>>>  mdadm -A /dev/md0 --backup-file=/home/md.backup
>>>>
>>>> i get back to following error
>>>>
>>>> mdadm: Failed to restore critical section for reshape, sorry.
>>>>
>>>> any idea why?
>>>>
>>>> before shutting down cat /proc/mdstat showed
>>>>
>>>> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
>>>> [raid4] [raid10]
>>>> md0 : active raid6 sdf[0] sdb[6](S) sda[4] sdc[3] sde[2] sdd[1]
>>>>     7814055936 blocks super 1.0 level 6, 512k chunk, algorithm 18
>>>> [6/5] [UUUUU_]
>>>>     [==============>......]  reshape = 70.8% (1384415232/1953513984)
>>>> finish=3658.6min speed=2592K/sec
>>>>
>>>> but now it shows
>>>>
>>>> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
>>>> [raid4] [raid10]
>>>> md0 : inactive sdc[3] sdb[6](S) sde[2] sdd[1] sdf[0]
>>>>      9767572240 blocks super 1.0
>>>>
>>>> i am totally confused, it seems to have lost a drive from the raid,
>>>> and the number of blocks is incorrect
>>>>
>>>
>>> issuing the following
>>>
>>>  mdadm -Avv --backup-file=/home/md.backup /dev/md0
>>>
>>> returns
>>>
>>>
>>> mdadm: looking for devices for /dev/md0
>>> mdadm: cannot open device /dev/sda5: Device or resource busy
>>> mdadm: /dev/sda5 has wrong uuid.
>>> mdadm: no RAID superblock on /dev/sda2
>>> mdadm: /dev/sda2 has wrong uuid.
>>> mdadm: cannot open device /dev/sda1: Device or resource busy
>>> mdadm: /dev/sda1 has wrong uuid.
>>> mdadm: cannot open device /dev/sda: Device or resource busy
>>> mdadm: /dev/sda has wrong uuid.
>>> mdadm: /dev/sdg is identified as a member of /dev/md0, slot -1.
>>> mdadm: /dev/sdf is identified as a member of /dev/md0, slot 4.
>>> mdadm: /dev/sdd is identified as a member of /dev/md0, slot 2.
>>> mdadm: /dev/sde is identified as a member of /dev/md0, slot 0.
>>> mdadm: /dev/sdc is identified as a member of /dev/md0, slot 1.
>>> mdadm: /dev/sdb is identified as a member of /dev/md0, slot 3.
>>> mdadm:/dev/md0 has an active reshape - checking if critical section
>>> needs to be restored
>>> mdadm: backup-metadata found on /home/md.backup but is not needed
>>> mdadm: Failed to find backup of critical section
>>> mdadm: Failed to restore critical section for reshape, sorry.
>>>
>>
>> seem the above was trying at use the wrong disks to assemble, so using
>> the following
>>
>> mdadm -Avv /dev/md0 --backup-file=/home/md.backup /dev/sd[abcdef]
>>
>>  mdadm: looking for devices for /dev/md0
>> mdadm: /dev/sda is identified as a member of /dev/md0, slot 4.
>> mdadm: /dev/sdb is identified as a member of /dev/md0, slot -1.
>> mdadm: /dev/sdc is identified as a member of /dev/md0, slot 3.
>> mdadm: /dev/sdd is identified as a member of /dev/md0, slot 1.
>> mdadm: /dev/sde is identified as a member of /dev/md0, slot 2.
>> mdadm: /dev/sdf is identified as a member of /dev/md0, slot 0.
>> mdadm:/dev/md0 has an active reshape - checking if critical section
>> needs to be restored
>> mdadm: backup-metadata found on /home/md.backup but is not needed
>> mdadm: Failed to find backup of critical section
>> mdadm: Failed to restore critical section for reshape, sorry.
>>
>
> have now upgraded to mdadm 3.2.2
>
> and get a little more info
>
> mdadm -Avv /dev/md0 --backup-file=/home/md.backup /dev/sd[abcdef]
>
> mdadm: looking for devices for /dev/md0
> mdadm: /dev/sda is identified as a member of /dev/md0, slot 4.
> mdadm: /dev/sdb is identified as a member of /dev/md0, slot -1.
> mdadm: /dev/sdc is identified as a member of /dev/md0, slot 3.
> mdadm: /dev/sdd is identified as a member of /dev/md0, slot 1.
> mdadm: /dev/sde is identified as a member of /dev/md0, slot 2.
> mdadm: /dev/sdf is identified as a member of /dev/md0, slot 0.
> mdadm: device 6 in /dev/md0 has wrong state in superblock, but /dev/sdb seems ok
> mdadm:/dev/md0 has an active reshape - checking if critical section
> needs to be restored
> mdadm: backup-metadata found on /home/md.backup but is not needed
> mdadm: Failed to find backup of critical section
> mdadm: Failed to restore critical section for reshape, sorry.
>


Ok, i dont know if this is the right thing to have done

~# mdadm -Avv --force /dev/md0 --backup-file=/home/md.backup /dev/sd[abcdef]

mdadm: looking for devices for /dev/md0
mdadm: /dev/sda is identified as a member of /dev/md0, slot 4.
mdadm: /dev/sdb is identified as a member of /dev/md0, slot -1.
mdadm: /dev/sdc is identified as a member of /dev/md0, slot 3.
mdadm: /dev/sdd is identified as a member of /dev/md0, slot 1.
mdadm: /dev/sde is identified as a member of /dev/md0, slot 2.
mdadm: /dev/sdf is identified as a member of /dev/md0, slot 0.
mdadm: clearing FAULTY flag for device 1 in /dev/md0 for /dev/sdb
mdadm: Marking array /dev/md0 as 'clean'
mdadm:/dev/md0 has an active reshape - checking if critical section
needs to be restored
mdadm: backup-metadata found on /home/md.backup but is not needed
mdadm: Failed to find backup of critical section
mdadm: Failed to restore critical section for reshape, sorry.


~# mdadm -Avv /dev/md0 --backup-file=/home/md.backup /dev/sd[abcdef]

mdadm: looking for devices for /dev/md0
mdadm: /dev/sda is identified as a member of /dev/md0, slot 4.
mdadm: /dev/sdb is identified as a member of /dev/md0, slot -1.
mdadm: /dev/sdc is identified as a member of /dev/md0, slot 3.
mdadm: /dev/sdd is identified as a member of /dev/md0, slot 1.
mdadm: /dev/sde is identified as a member of /dev/md0, slot 2.
mdadm: /dev/sdf is identified as a member of /dev/md0, slot 0.
mdadm:/dev/md0 has an active reshape - checking if critical section
needs to be restored
mdadm: restoring critical section
mdadm: added /dev/sdd to /dev/md0 as 1
mdadm: added /dev/sde to /dev/md0 as 2
mdadm: added /dev/sdc to /dev/md0 as 3
mdadm: added /dev/sda to /dev/md0 as 4
mdadm: no uptodate device for slot 5 of /dev/md0
mdadm: added /dev/sdb to /dev/md0 as -1
mdadm: added /dev/sdf to /dev/md0 as 0
mdadm: /dev/md0 has been started with 4 drives (out of 6) and 1 spare.

~# cat /proc/mdstat

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
[raid4] [raid10]
md0 : active raid6 sdf[0] sdb[6](S) sdc[3] sde[2] sdd[1]
      7814055936 blocks super 1.0 level 6, 512k chunk, algorithm 18
[6/4] [UUUU__]
      [==============>......]  reshape = 74.3% (1452929024/1953513984)
finish=2545.2min speed=3276K/sec

unused devices: <none>

so looks like its carrying on now but with 4 disks and a spare, maybe
i can add the other disk once the reshape has finished
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html