mdadm create to existing raid5

mdadm create to existing raid5

[Jon Collette]

I wasn't thinking and did a mdadm --create to my existing raid5 instead 
of --assemble.  The syncing process ran and now its not mountable.  Is 
there anyway to recover from this?

Thanks

RE: mdadm create to existing raid5

[Guy Watkins]

} -----Original Message-----
} From: linux-raid-owner@vger.kernel.org [mailto:linux-raid-
} owner@vger.kernel.org] On Behalf Of Jon Collette
} Sent: Thursday, July 12, 2007 5:29 PM
} To: linux-raid@vger.kernel.org
} Subject: mdadm create to existing raid5
} 
} I wasn't thinking and did a mdadm --create to my existing raid5 instead
} of --assemble.  The syncing process ran and now its not mountable.  Is
} there anyway to recover from this?
} 
} Thanks
} -
} To unsubscribe from this list: send the line "unsubscribe linux-raid" in
} the body of a message to majordomo@vger.kernel.org
} More majordomo info at  http://vger.kernel.org/majordomo-info.html

Maybe.  Not really sure.  But don't do anything until someone that really
knows answers!

What I think...
If you did a create with the exact same parameters the data should not have
changed.  But you can't mount so you must have used different parameters.

Only 1 disk was written to during the create.  Only that disk was changed.
If you remove the 1 disk and do another create with the original parameters
and put "missing" for the 1 disk your array will be back to normal, but
degraded.  Once you confirm this you can add back the 1 disk.  You must be
able to determine which disk was written to.  I don't know how to do that
unless you have the output from "mdadm -D" during the create/syncing.

But please don't proceed until someone else confirms what I say or gives
better advice!

Guy

Re: mdadm create to existing raid5

[David Greaves]

Guy Watkins wrote:
> } owner@vger.kernel.org] On Behalf Of Jon Collette
> } I wasn't thinking and did a mdadm --create to my existing raid5 instead
> } of --assemble.  The syncing process ran and now its not mountable.  Is
> } there anyway to recover from this?
> Maybe.  Not really sure.  But don't do anything until someone that really
> knows answers!
I agree - Yes, maybe.


> What I think...
> If you did a create with the exact same parameters the data should not have
> changed.  But you can't mount so you must have used different parameters.
I'd agree.


> Only 1 disk was written to during the create.
Yep.

>  Only that disk was changed.
Yep.

> If you remove the 1 disk and do another create with the original parameters
> and put "missing" for the 1 disk your array will be back to normal, but
> degraded.  Once you confirm this you can add back the 1 disk.
Yep.
**WARNING**
**WARNING**
**WARNING**
At this point you are relatively safe (!) but as soon as you do an 'add' and 
initiate another resync then if you got it wrong you will have toasted your data 
completely!!!
**WARNING**
**WARNING**
**WARNING**

>  You must be
> able to determine which disk was written to.  I don't know how to do that
> unless you have the output from "mdadm -D" during the create/syncing.

Do you know the *exact* command you issued when you did the initial --create?
Do you know the *exact* command you issued when you did the bogus --create?

And what version of mdadm you are using?

Neil said that it's mdadm, not the kernel, that determines which device is 
initially degraded during a create. We can look at the code and your command 
line and guess which device mdadm chose. (Getting this wrong won't matter but it 
may make recovery quicker.)

assuming you have a 4 device raid using /dev/sda1, /dev/sdb1, /dev/sdc1, /dev/sdd1

you'll then do something like:
mdadm --create --verbose /dev/md0 --level=5 --raid-devices=4 /dev/sda1 /dev/sdb1 
/dev/sdc1 missing
try a mount
mdadm --create --verbose /dev/md0 --level=5 --raid-devices=4 /dev/sda1 missing 
/dev/sdc1 /dev/sdb1
try a mount
mdadm --create --verbose /dev/md0 --level=5 --raid-devices=4 /dev/sdb1 /dev/sda1 
/dev/sdc1 missing
try a mount
mdadm --create --verbose /dev/md0 --level=5 --raid-devices=4 /dev/sdc1 /dev/sdb1 
/dev/sda1 missing
try a mount

etc etc,

So you'll still need to do a trial and error assemble....
For a simple 4 device array I there are 24 permutations - doable by hand, if you 
have 5 devices then it's 120, 6 is 720 - getting tricky ;)

I'm bored so I'm going to write a script based on something like this:
http://www.unix.org.ua/orelly/perl/cookbook/ch04_20.htm

Feel free to beat me to it ...

The critical thing is that you *must* use 'missing' when doing these trial 
--create calls.

If we've not explained something very well and you don't understand then please 
ask before trying it out...

David

Re: mdadm create to existing raid5

[David Greaves]

David Greaves wrote:
> For a simple 4 device array I there are 24 permutations - doable by 
> hand, if you have 5 devices then it's 120, 6 is 720 - getting tricky ;)

Oh, wait, for 4 devices there are 24 permutations - and you need to do it 4 
times, substituting 'missing' for each device - so 96 trials.

4320 trials for a 6 device array.

Hmm. I've got a 7 device raid 6 - I think I'll go an make a note of how it's put 
together... <grin>


Have a look at this section and the linked script.
I can't test it until later

http://linux-raid.osdl.org/index.php/RAID_Recovery

http://linux-raid.osdl.org/index.php/Permute_array.pl


David

Re: mdadm create to existing raid5

[Jon Collette]

The mdadm --create with missing instead of a drive is a good idea.  Do 
you actually say missing or just leave out a drive?  However doesn't it 
do a sync everytime you create?  So wouldn't you run the risk of 
corrupting another drive each time?  Or does it not sync because of the 
saying missing?

To bad I am intent on learning things the hard way.

/etc/mdadm.conf from before I recreated
ARRAY /dev/md2 level=raid5 num-devices=4 spares=1 
UUID=4f935928:2b7a1633:71d575d6:dab4d6bc

/etc/mdadm.conf after I recreated
ARRAY /dev/md1 level=raid5 num-devices=4 
UUID=81bdd737:901c0a8f:af38cb94:41c4e3da

Well before I heard back from you guys .  I noticed this problem and in 
my fountain of infinite wisdom I did mdadm --zero-superblock to all my 
raid drives and created them again thinking if I got it to look the same 
it woud just fix it.  Well they do look the same now, I am at work or I 
would give you the new mdadm.conf.

I really need to learn patients :(


David Greaves wrote:
> David Greaves wrote:
>> For a simple 4 device array I there are 24 permutations - doable by 
>> hand, if you have 5 devices then it's 120, 6 is 720 - getting tricky ;)
>
> Oh, wait, for 4 devices there are 24 permutations - and you need to do 
> it 4 times, substituting 'missing' for each device - so 96 trials.
>
> 4320 trials for a 6 device array.
>
> Hmm. I've got a 7 device raid 6 - I think I'll go an make a note of 
> how it's put together... <grin>
>
>
> Have a look at this section and the linked script.
> I can't test it until later
>
> http://linux-raid.osdl.org/index.php/RAID_Recovery
>
> http://linux-raid.osdl.org/index.php/Permute_array.pl
>
>
> David
>
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html