md tries to assemble md_d0 instead of md0

md tries to assemble md_d0 instead of md0

[Tony Arcieri]

I have a fileserver with a 5 disk RAID-5.  Things were going along
swimmingly with it for about a year, until one day it rebooted (I
assume because of a power blip/outage).

md0 was never assembled.  Instead, it tried to assemble md_d0.

I have no idea what md_d0 is.  I never tried to create md_d0.  I don't
even know what md_d0 is supposed to represent.

Upon boot, md would add one *random* drive to md_d0.  This means
different drive letters, different drive contents, no rhyme or reason
to the whole thing.  It would literally pick one out of the 5 disks in
my array and add it to md_d0.  I am not joking when I say it's a
different disk every time.  It is, and only one disk.  It would not
bother to try to assemble md0 at all.  It just tosses one and only one
random drive into md_d0:

[   77.968028] md: bind<sdc1>

I have verified it's a different disk every time by md5sum of the
first kilobyte on each disk.  Every time I reboot it is a different
drive with a different drive letter.

mdstat looks like this:

--snip--
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
[raid4] [raid10]
md_d0 : inactive sdc1[1](S)
      976759936 blocks

unused devices: <none>
--snip--

In the past I fixed this by stopping md_d0, removing the offending
drive from it, assembling md0, and adding the offending drive from
md_d0 back to the array (after zeroing its superblock with mdadm
--zero-superblock).  This would result in a RAID resync, but after
that completed all was well.  My fileserver was up for about a month
after performing this procedure without problems.

This morning I woke up and my clock was blinking, so I had a bit of a
power blip at my place.  My fileserver had rebooted, and I find it
again in the exact same state: no attempts to assemble md0, and one
random drive added to md_d0.

Is there a permanent solution to this problem?  Where is md_d0 even
coming from?  How do I permanently get rid of it?

--
Tony Arcieri
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: md tries to assemble md_d0 instead of md0

[Tony Arcieri]

I booted with a LiveCD, which brought md0 up in the following configuration:

md0 : active raid5 sdg1[4] sdf1[3] sde1[2] sdb1[0]
      3907039744 blocks level 5, 64k chunk, algorithm 2 [5/4] [U_UUU]

Unfortunately it also mounted it read/write.  On a subsequent reboot
back to my original install I'm getting:

md_d0 : inactive sde1[2](S)
      976759936 blocks

And when I mdadm --assemble --scan -fv:

md0 : active raid5 sdb1[0] sdg1[4] sdf1[3] sdc1[1]
      3907039744 blocks level 5, 64k chunk, algorithm 2 [5/4] [UU_UU]

md_d0 : inactive sde1[2](S)
      976759936 blocks

(note the odd drive out changed from sdc to sde)

Any idea WTF is going on here?  I already need to repair the
filesystem on there because of this.  When I mount it all my data is
gone.  This is extremely annoying to say the very least.

-- 
Tony Arcieri

Re: md tries to assemble md_d0 instead of md0

[Luca Berra]

On Sun, Aug 23, 2009 at 10:12:43AM -0600, Tony Arcieri wrote:
>Is there a permanent solution to this problem?  Where is md_d0 even
>coming from?  How do I permanently get rid of it?
check your udev rules

L.

-- 
Luca Berra -- bluca@comedia.it
         Communication Media & Services S.r.l.
  /"\
  \ /     ASCII RIBBON CAMPAIGN
   X        AGAINST HTML MAIL
  / \
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: md tries to assemble md_d0 instead of md0

[Tony Arcieri]

On Sun, Aug 23, 2009 at 11:50 PM, Luca Berra <bluca@comedia.it> wrote:
>
> On Sun, Aug 23, 2009 at 10:12:43AM -0600, Tony Arcieri wrote:
>>
>> Is there a permanent solution to this problem?  Where is md_d0 even
>> coming from?  How do I permanently get rid of it?
>
> check your udev rules

For what it's worth, reinstalling Ubuntu solved the problem.  Upon my
first boot into the new install it automatically started resyncing the
RAID.

Dunno what that's about, and reinstalling Linux is not my preferred
way to fix a problem, but at this point my RAID is resyncing and
everything seems to be fine.

--
Tony Arcieri
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html