Re: Help with corrupted MDADM Raid6

[NeilBrown <neilb@suse.de>]

[-- Attachment #1: Type: text/plain, Size: 2437 bytes --]

On Fri, 13 Jun 2014 10:34:55 +0200 "ptschack ." <ptschack@googlemail.com>
wrote:

> Hi,
> 
> I fear I may have messed up my MDADM RAID 6. Some info:
> 
> The RAID consists of 11 hard drives (3 TB each), 2 of which are
> spares. It used to be 6 drives, no spares.
> 
> This happened: one drive (/dev/sdg) kept giving me SMART warnings
> (failure imminent). As I was trying to do a fresh install at the time,
> i restored a backup of my system drive (an ssd which is is wholly
> independent of the RAID) and booted off of that.
> I hadn't realized that the backup was too old, from the time before i
> grew the raid... Apparently mdadm tried to mount the raid as it used
> to be (6 drives), and now it says the raid consists of 6 drives, all
> spares!

That doesn't make sense.  md store the important information about an array
on the drive of the array.  The information in /etc/mdadm.conf is largely
advisory.
The mdadm.conf that you attached looks perfectly OK and would work with
either the old 6-drive array or the new 9+2 drive array.

> Plus the drive /dev/sdg seems to have totally failed now :(
> 
> I guess I somehow need to re-assemble (or re-create) the raid6 in a
> degraded state, since one drive failed, but I'm not entirely clear on
> how to do that. One question is if I should include the spares in a
> reassembly or add them later.
> 
> Can anyone help?

Hopefully.

> 
> I have attached the outputs of dumpe2fs and mdadm --examine of the
> drives, plus my mdadm.conf.

sdk1 is the only device to show an md superblock, and it was last updated on
6th April, so it is rather old.  That is presumably before you made the array
larger.

sda1, sdb1 sdc1 sdd1 sde1 sdf1 sdh1 sdi1 sdj1 sdk1 all appear to contain
ext3fs or ext4fs superblocks.  I think some of those are from before the
devices were added into the array.  Quite possibly md doesn't over-write the
ext3 superblock.

I wonder if maybe the array is actually on the whole devices rather than on
the partitions.  What does

  mdadm --examine /dev/sd[abcdefghijk]

report?


Certainly *don't* try to create the array until you have tried all other
options.

If you do find mdadm superblocks on the whole devices, then you could try

 mdadm --assemble /dev/sd[a-k]

possibly remove devices from the list which don't work.  Possibly add
"--force" if not including "--force" doesn't work.

NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]