Re: RAID5 - Disk failed during re-shape

[NeilBrown <neilb@suse.de>]

[-- Attachment #1: Type: text/plain, Size: 3440 bytes --]

On Mon, 13 Aug 2012 18:14:30 +0200 "Sam Clark" <sclark_77@hotmail.com> wrote:

> Thanks Neil, really appreciate the assistance.
> 
> Would love to give that a try - at least to catch the data that has changed 
> since last backup, however I don't know the chunk size.  I created the array 
> so long ago, and of course didn't document anything.  I would guess they are 
> 64K, but not sure.  Is there any way to check from the disks themselves?
> 
> I've captured the 128K chunks as follows - hope it's correct:
> 
> I got the disk size in Bytes from fdisk -l, and subtracted 131072.. then 
> ran:
> sam@nas:~$ sudo dd if=/dev/sd[b-f] of=test.128k bs=1 skip=xxx count=128k,
> The 5 files are attached.
> 
> The disk sizes are as follows:
> sam@nas:~$ sudo blockdev --getsize /dev/sd[b-f]
> sdb: 2930277168
> sdc: 2930277168
> sdd: 2930277168
> sde: 2930277168
> sdf: 3907029168
> 

Unfortunately the metadata doesn't contain any trace of the the reshape
position, so we'll make do with 11.4%

The following script will assemble the array read-only.  You can then try
"fsck -n /dev/md_restore" to see if it is credible.  Then try to mount it.

Most of the details I'm confident of.

'chunk' is probably right, but there is no way to know for sure until you
have access to your data.  If you try changing it you'll need to also change
reshape to be an appropriate multiple of it.

'reshape' is approximately 11.4% of the array.  Maybe try other suitable
multiples.

'devs' is probably wrong.  I chose that order because the metadata seems to
suggest that order - yes, with sdf in the middle.  Maybe you know better.
You can try different orders until it seems to work.

Everything else should be correct.  component_size is definitely correct, I
found that in the metadata.  'layout' is the default and is hardly ever
changed.

As it assembles read-only, there is no risk in getting it wrong, changing
some values and trying again.  The script disassembles and old array before
creating the new.

good luck.

NeilBrown



# Script to try to assemble a RAID5 which got it's metadata corrupted
# in the middle of a reshape (ouch).
# We assemble as externally-managed-metadata in read-only mode
# by writing magic values to sysfs.

# devices in correct order.
devs='sdb sdd sdf sde sdc'

# number of devices, both before and after reshape
before=4
after=5

# reshape position as sectors per array.  It must be a multiple
# of one stripe, so chunk*old_data_disks*new_data_disks
# This number is 0.114 * 2930276992 * 3, rounded up to
# a multiple of 128*3*4.   Other multiples could be tried.
reshape=1002155520

# array parameters
level=raid5
chunk=65536
layout=2
component_size=2930276992

# always creates /dev/md_restore
mdadm -S /dev/md_restore
echo md_restore >  /sys/module/md_mod/parameters/new_array
cd /sys/class/block/md_restore/md

echo external:readonly > metadata_version
echo $level > level
echo $chunk > chunk_size
echo $component_size > component_size
echo 2 > layout
echo $before > raid_disks

echo $reshape > reshape_position
echo $after > raid_disks

slot=0
for i in $devs
do
 cat /sys/class/block/$i/dev > new_dev
 echo 0 > dev-$i/offset
 echo $component_size > dev-$i/size
 echo insync > dev-$i/state
 echo $slot > dev-$i/slot

 slot=$[slot+1]
done

echo readonly > array_state 

grep md_restore /proc/partitions

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]