md array numbering is messed up

md array numbering is messed up

[Peb]

Hi,

I have 2 arrays whose numbers get inverted, creating havoc, when booting
under different kernels.

I have md0 (raid1) made up of ide drives and md1 (raid5) made up of five
sata drives, when booting with my current ubuntu 2.6.12-9 kernel. When I
try to boot a more recent kernel (2.6.15-26 or 2.6.15-27) the
order is inversed and my sata raid5 array shows up as md0.

My arrays are part of evms volumes that just stop working if the
numbering is inverted.

any clues ?

thanks,
Pierre

Re: md array numbering is messed up

[Neil Brown]

On Sunday October 29, peb-misc@peeble.net wrote:
> Hi,
> 
> I have 2 arrays whose numbers get inverted, creating havoc, when booting
> under different kernels.
> 
> I have md0 (raid1) made up of ide drives and md1 (raid5) made up of five
> sata drives, when booting with my current ubuntu 2.6.12-9 kernel. When I
> try to boot a more recent kernel (2.6.15-26 or 2.6.15-27) the
> order is inversed and my sata raid5 array shows up as md0.
> 
> My arrays are part of evms volumes that just stop working if the
> numbering is inverted.
> 
> any clues ?

Your arrays are being started the wrong way.
Do you have an mdadm.conf that lists the arrays?  Can you show us what
it looked like?
If not, do you know how the arrays are started in ubuntu?

NeilBrown

Re: md array numbering is messed up

[Michael Tokarev]

Neil Brown wrote:
> On Sunday October 29, peb-misc@peeble.net wrote:
>> Hi,
>>
>> I have 2 arrays whose numbers get inverted, creating havoc, when booting
>> under different kernels.
>>
>> I have md0 (raid1) made up of ide drives and md1 (raid5) made up of five
>> sata drives, when booting with my current ubuntu 2.6.12-9 kernel. When I
>> try to boot a more recent kernel (2.6.15-26 or 2.6.15-27) the
>> order is inversed and my sata raid5 array shows up as md0.
>>
>> My arrays are part of evms volumes that just stop working if the
>> numbering is inverted.
>>
>> any clues ?
> 
> Your arrays are being started the wrong way.
> Do you have an mdadm.conf that lists the arrays?  Can you show us what
> it looked like?
> If not, do you know how the arrays are started in ubuntu?

My guess is that it's using mdrun shell script - the same as on Debian.
It's a long story, the thing is quite ugly and messy and does messy things
too, but they says it's compatibility stuff and continue shipping it.

For the OP, the solution is to *create* mdadm.conf file - in that case
mdrun should hopefully NOT run.

/mjt

Re: md array numbering is messed up

[Brad Campbell]

Michael Tokarev wrote:
> Neil Brown wrote:
>> On Sunday October 29, peb-misc@peeble.net wrote:
>>> Hi,
>>>
>>> I have 2 arrays whose numbers get inverted, creating havoc, when booting
>>> under different kernels.
>>>
>>> I have md0 (raid1) made up of ide drives and md1 (raid5) made up of five
>>> sata drives, when booting with my current ubuntu 2.6.12-9 kernel. When I
>>> try to boot a more recent kernel (2.6.15-26 or 2.6.15-27) the
>>> order is inversed and my sata raid5 array shows up as md0.
>>>
>>> My arrays are part of evms volumes that just stop working if the
>>> numbering is inverted.
>>>
>>> any clues ?
>> Your arrays are being started the wrong way.
>> Do you have an mdadm.conf that lists the arrays?  Can you show us what
>> it looked like?
>> If not, do you know how the arrays are started in ubuntu?
> 
> My guess is that it's using mdrun shell script - the same as on Debian.
> It's a long story, the thing is quite ugly and messy and does messy things
> too, but they says it's compatibility stuff and continue shipping it.
> 
> For the OP, the solution is to *create* mdadm.conf file - in that case
> mdrun should hopefully NOT run.

I'd suggest you are probably correct. By default on Ubuntu 6.06

brad@bklaptop2:~$ cat /etc/init.d/mdadm-raid
#!/bin/sh
#
# Start any arrays which are described in /etc/mdadm/mdadm.conf and which are
# not running already.
#
# Copyright (c) 2001-2004 Mario Jou/3en <joussen@debian.org>
# Distributable under the terms of the GNU GPL version 2.

MDADM=/sbin/mdadm
MDRUN=/sbin/mdrun
CONFIG=/etc/mdadm/mdadm.conf
DEBIANCONFIG=/etc/default/mdadm

. /lib/lsb/init-functions

test -x $MDADM || exit 0

AUTOSTART=true
test -f $DEBIANCONFIG && . $DEBIANCONFIG

case "$1" in
     start)
         if [ "x$AUTOSTART" = "xtrue" ] ; then
             if [ ! -f /proc/mdstat ] && [ -x /sbin/modprobe ] ; then
                 /sbin/modprobe -k md > /dev/null 2>&1
             fi
             test -f /proc/mdstat || exit 0
             log_begin_msg "Starting RAID devices..."
             if [ -f $CONFIG ] && [ -x $MDADM ] ; then
                 $MDADM -A -s
             elif [ -x $MDRUN ] ; then
                 $MDRUN
             fi
             log_end_msg $?
         fi
         ;;
     stop|restart|reload|force-reload)
         ;;
     *)
         log_success_msg "Usage: $0 {start|stop|restart|reload|force-reload}"
         exit 1
         ;;
esac

exit 0


Brad
-- 
"Human beings, who are almost unique in having the ability
to learn from the experience of others, are also remarkable
for their apparent disinclination to do so." -- Douglas Adams

Re: md array numbering is messed up

[dean gaudet]

On Mon, 30 Oct 2006, Brad Campbell wrote:

> Michael Tokarev wrote:
> > My guess is that it's using mdrun shell script - the same as on Debian.
> > It's a long story, the thing is quite ugly and messy and does messy things
> > too, but they says it's compatibility stuff and continue shipping it.
...
> 
> I'd suggest you are probably correct. By default on Ubuntu 6.06
> 
> brad@bklaptop2:~$ cat /etc/init.d/mdadm-raid
> #!/bin/sh
> #
> # Start any arrays which are described in /etc/mdadm/mdadm.conf and which are
> # not running already.
> #
> # Copyright (c) 2001-2004 Mario Jou/3en <joussen@debian.org>
> # Distributable under the terms of the GNU GPL version 2.
> 
> MDADM=/sbin/mdadm
> MDRUN=/sbin/mdrun

fwiw mdrun is finally on its way out.  the debian "unstable" mdadm package 
is full of new goodness (initramfs goodness, 2.5.x mdadm featurefulness, 
monthly full array check goodness).  ubuntu folks should copy it again 
before they finalize edgy.

-dean

Re: md array numbering is messed up

[Peb]

Michael Tokarev wrote:
> Neil Brown wrote:
>> On Sunday October 29, peb-misc@peeble.net wrote:
>>> Hi,
>>>
>>> I have 2 arrays whose numbers get inverted, creating havoc, when booting
>>> under different kernels.
>>>
>>> I have md0 (raid1) made up of ide drives and md1 (raid5) made up of five
>>> sata drives, when booting with my current ubuntu 2.6.12-9 kernel. When I
>>> try to boot a more recent kernel (2.6.15-26 or 2.6.15-27) the
>>> order is inversed and my sata raid5 array shows up as md0.
>>>
>>> My arrays are part of evms volumes that just stop working if the
>>> numbering is inverted.
>>>
>>> any clues ?
>> Your arrays are being started the wrong way.
>> Do you have an mdadm.conf that lists the arrays?  Can you show us what
>> it looked like?
>> If not, do you know how the arrays are started in ubuntu?
> 
> My guess is that it's using mdrun shell script - the same as on Debian.
> It's a long story, the thing is quite ugly and messy and does messy things
> too, but they says it's compatibility stuff and continue shipping it.
> 
> For the OP, the solution is to *create* mdadm.conf file - in that case
> mdrun should hopefully NOT run.
> 
> /mjt

Well I have the following mdadm.conf:

DEVICE /dev/hda /dev/hdc /dev/sd*
ARRAY /dev/md1 level=raid5 num-devices=4 UID=8ed64073:04d21e1c:33660158:
a5bc892f
ARRAY /dev/md0 level=raid1 num-devices=2 UID=cab9de58:d20bffae:654d1910:
6f440136

I 've tried inverting the two ARRAY lines and placing the sd* device
before the ide drives, but that didn't change anything.

So I guess my mdadm.conf isn't correct since mdrun is still running even
though the file exists ???

--
Peb

Re: md array numbering is messed up

[Neil Brown]

On Tuesday October 31, peb-misc@peeble.net wrote:
> 
> Well I have the following mdadm.conf:
> 
> DEVICE /dev/hda /dev/hdc /dev/sd*
> ARRAY /dev/md1 level=raid5 num-devices=4 UID=8ed64073:04d21e1c:33660158:
> a5bc892f
> ARRAY /dev/md0 level=raid1 num-devices=2 UID=cab9de58:d20bffae:654d1910:
> 6f440136
> 
> I 've tried inverting the two ARRAY lines and placing the sd* device
> before the ide drives, but that didn't change anything.
> 
> So I guess my mdadm.conf isn't correct since mdrun is still running even
> though the file exists ???

Try 
   sh -x /etc/init.d/mdadm-raid start
and see what it does.
If have created /etc/mdadm/mdadm.conf rather than /etc/mdadm.conf have
you?  Debian decided the config file should be in a directory
/etc/mdadm, and so are different from everyone else...

NeilBrown

Re: md array numbering is messed up

[Peb]

Neil Brown wrote:
> On Tuesday October 31, peb-misc@peeble.net wrote:
>> Well I have the following mdadm.conf:
>>
>> DEVICE /dev/hda /dev/hdc /dev/sd*
>> ARRAY /dev/md1 level=raid5 num-devices=4 UID=8ed64073:04d21e1c:33660158:
>> a5bc892f
>> ARRAY /dev/md0 level=raid1 num-devices=2 UID=cab9de58:d20bffae:654d1910:
>> 6f440136
>>
> Try 
>    sh -x /etc/init.d/mdadm-raid start
> and see what it does.
> If have created /etc/mdadm/mdadm.conf rather than /etc/mdadm.conf have
> you?  Debian decided the config file should be in a directory
> /etc/mdadm, and so are different from everyone else...
> 
> NeilBrown

Neil,
I ran the sh -x /etc/init.d/mdadm-raid start command and it is starting
the arrays through mdadm -A -s and not by running mdrun.

All in all, I'd tried the following before checking mdadm-raid:
removing /sbin/mdrun
removing my mdadm.conf
placing my mdadm.conf in /etc instead of /etc/mdadm/

It just seems to be linked to the kernel.. since when booting from
2.6.12 my initscripts and config just work, while when using 2.6.15
without changing anything else.

--
Peb