Re: Recover array after I panicked

[Brad Campbell <lists2009@fnarfbargle.com>]

On 23/04/17 22:09, Patrik Dahlström wrote:
>
> On 04/23/2017 04:06 PM, Brad Campbell wrote:
>> On 23/04/17 17:47, Patrik Dahlström wrote:
>>> Hello,
>>>
>>> Here's the story:
>>>
>>> I started with a 5x6 TB raid5 array. I added another 6 TB drive and
>>> started to grow the array. However, one of my SATA cables were bad and
>>> the reshape gave me lots of I/O errors.
>>>
>>> Instead of fixing the SATA cable issue directly, I shutdown the server
>>> and swapped places of 2 drives. My reasoning was that putting the new
>>> drive in a good slot would reduce the I/O errors. Bad move, I know. I
>>> tried a few commands but was not able to continue the reshape.
>>>
>> Nobody seems to have mentioned the reshape issue. What sort of reshape
>> were you running? How far into the reshape did it get? Do you have any
>> logs of the errors (which might at least indicate whereabouts in the
>> array things were before you pushed it over the edge)?
> These were the grow commands I ran:
> mdadm --add /dev/md1 /dev/sdf
> mdadm --grow --raid-devices=6 /dev/md1
>
> It got to roughly 15-17 % before I decided that the I/O errors were more
> scary than stopping the reshape.
>
You might be very lucky. If my reading of the code is correct (and my 
memory is any good), simply adding a disk to a raid5 on a recent enough 
kernel should make the resync go backwards. So it should have started at 
the end and worked towards the start. This would mean the majority of 
your data should be on your 5 original disks.

If this is the case, then permuting the array with 6 disks is going to 
fail always as 1/5th of every stripe will be bogus. Doing it with the 
original 5 disks may ultimately yield something in the order of 85% of 
your data if your estimate of 15-17% is correct.

No harm testing it with some overlays anyway. Someone more familiar with 
the code will correct me if I'm wrong.

Regards,
Brad

-- 
Dolphins are so intelligent that within a few weeks they can
train Americans to stand at the edge of the pool and throw them
fish.