raid=noautodetect is apparently ignored?

raid=noautodetect is apparently ignored?

[Ian Dall]

When I try and disable auto detection, with kernel boot parameters, it
goes ahead and auto assembles and runs anyway. The md= parameters seem
to be noticed, but don't seem to have any other effect (beyond resulting
in a dmesg).

Here is the result

        $ dmesg | egrep 'raid|md:'
        Kernel command line: ro root=/dev/sda8 raid=noautodetect md=1,/dev/sdb2,/dev/sdc2,/dev/sdd2 md=2,/dev/sdb3,/dev/sdc3,/dev/sdd3 md=3,/dev/md1,/dev/hda2 md=4,/dev/md2,/dev/hda3 single
        md: Will configure md1 (super-block) from /dev/sdb2,/dev/sdc2,/dev/sdd2, below.
        md: Will configure md2 (super-block) from /dev/sdb3,/dev/sdc3,/dev/sdd3, below.
        md: Will configure md3 (super-block) from /dev/md1,/dev/hda2, below.
        md: Will configure md4 (super-block) from /dev/md2,/dev/hda3, below.
        md: Autodetecting RAID arrays.
        md: autorun ...
        md: considering sdd4 ...
        md:  adding sdd4 ...
        md: sdd2 has different UUID to sdd4
        md: sdd1 has different UUID to sdd4
        md:  adding sdc4 ...
        md: sdc2 has different UUID to sdd4
        md: sdc1 has different UUID to sdd4
        md:  adding sdb4 ...
        md: sdb2 has different UUID to sdd4
        md: sdb1 has different UUID to sdd4
        md: hda3 has different UUID to sdd4
        md: hda2 has different UUID to sdd4
        md: created md2
        md: bind<sdb4>
        md: bind<sdc4>
        md: bind<sdd4>
        md: running: <sdd4><sdc4><sdb4>
        md: raid0 personality registered for level 0
        raid0: looking at sdd4
        raid0:   comparing sdd4(8409856) with sdd4(8409856)
        raid0:   END
        raid0:   ==> UNIQUE
        raid0: 1 zones
        raid0: looking at sdc4
        raid0:   comparing sdc4(8409856) with sdd4(8409856)
        raid0:   EQUAL
        raid0: looking at sdb4
        raid0:   comparing sdb4(8409856) with sdd4(8409856)
        raid0:   EQUAL
        raid0: FINAL 1 zones
        raid0: done.
        raid0 : md_size is 25229568 blocks.
        raid0 : conf->hash_spacing is 25229568 blocks.
        raid0 : nb_zone is 1.
        raid0 : Allocating 4 bytes for hash.
        md: considering sdd2 ...
        md:  adding sdd2 ...
        md: sdd1 has different UUID to sdd2
        md:  adding sdc2 ...
        md: sdc1 has different UUID to sdd2
        md:  adding sdb2 ...
        md: sdb1 has different UUID to sdd2
        md: hda3 has different UUID to sdd2
        md: hda2 has different UUID to sdd2
        md: created md1
        md: bind<sdb2>
        md: bind<sdc2>
        md: bind<sdd2>
        md: running: <sdd2><sdc2><sdb2>
        raid0: looking at sdd2
        raid0:   comparing sdd2(8409856) with sdd2(8409856)
        raid0:   END
        raid0:   ==> UNIQUE
        raid0: 1 zones
        raid0: looking at sdc2
        raid0:   comparing sdc2(8409856) with sdd2(8409856)
        raid0:   EQUAL
        raid0: looking at sdb2
        raid0:   comparing sdb2(8409856) with sdd2(8409856)
        raid0:   EQUAL
        raid0: FINAL 1 zones
        raid0: done.
        raid0 : md_size is 25229568 blocks.
        raid0 : conf->hash_spacing is 25229568 blocks.
        raid0 : nb_zone is 1.
        raid0 : Allocating 4 bytes for hash.
        md: considering sdd1 ...
        md:  adding sdd1 ...
        md:  adding sdc1 ...
        md:  adding sdb1 ...
        md: hda3 has different UUID to sdd1
        md: hda2 has different UUID to sdd1
        md: created md0
        md: bind<sdb1>
        md: bind<sdc1>
        md: bind<sdd1>
        md: running: <sdd1><sdc1><sdb1>
        md: raid1 personality registered for level 1
        raid1: raid set md0 active with 3 out of 3 mirrors
        md: considering hda3 ...
        md:  adding hda3 ...
        md: hda2 has different UUID to hda3
        md: created md4
        md: bind<hda3>
        md: running: <hda3>
        raid1: raid set md4 active with 1 out of 2 mirrors
        md: considering hda2 ...
        md:  adding hda2 ...
        md: created md3
        md: bind<hda2>
        md: running: <hda2>
        raid1: raid set md3 active with 1 out of 2 mirrors
        md: ... autorun DONE.

By way of context, this is kernel 2.6.20. I have set two striped (raid0)
arrays, md1 and md2. Each of these is mirrored by with a single
partition (hda2 and hda3) to form two raid 1 devices md3 and md4. This
works perfectly if I start and stop the md3 and md4 arrays from mdadm.
However the boot time autodetection always assembles degraded arrays
(see the 1 out of 2 messages above). I presume this is because it only
considers real physical partitions of type "Linux raid autodetect" and
is not considering md1 and md2 as candidate devices to form further
arrays. 

This is why I am trying to disable the autodetection, to see if directly
specifying the raid configuration would help.

This is driving me nuts. Maybe changing the type of the hda partitions
would help?

Any clues appreciated.

Ian

-- 
Ian Dall <ian@beware.dropbear.id.au>

Re: raid=noautodetect is apparently ignored?

[Neil Brown]

On Tuesday June 26, ian@beware.dropbear.id.au wrote:
> When I try and disable auto detection, with kernel boot parameters, it
> goes ahead and auto assembles and runs anyway. The md= parameters seem
> to be noticed, but don't seem to have any other effect (beyond resulting
> in a dmesg).

Odd....
 Maybe you have an initrd which is loading md as a module, then
 running "raidautorun" or similar?

> 
> This is driving me nuts. Maybe changing the type of the hda partitions
> would help?

Definitely.  Change the partition types.

NeilBrown

Re: raid=noautodetect is apparently ignored?

[Ian Dall]

On Tue, 2007-06-26 at 16:38 +1000, Neil Brown wrote:
> On Tuesday June 26, ian@beware.dropbear.id.au wrote:
> > When I try and disable auto detection, with kernel boot parameters, it
> > goes ahead and auto assembles and runs anyway. The md= parameters seem
> > to be noticed, but don't seem to have any other effect (beyond resulting
> > in a dmesg).
> 
> Odd....
>  Maybe you have an initrd which is loading md as a module, then
>  running "raidautorun" or similar?

I don't think so. lsmod doesn't show a md module (even though md is
clearly available).

Also if I break open the current initrd, init (the script) looks like
this:

#!/bin/nash

mount -t proc /proc /proc
setquiet
echo Mounting proc filesystem
echo Mounting sysfs filesystem
mount -t sysfs /sys /sys
echo Creating /dev
mount -o mode=0755 -t tmpfs /dev /dev
mkdir /dev/pts
mount -t devpts -o gid=5,mode=620 /dev/pts /dev/pts
mkdir /dev/shm
mkdir /dev/mapper
echo Creating initial device nodes
mknod /dev/null c 1 3
mknod /dev/zero c 1 5
mknod /dev/systty c 4 0
mknod /dev/tty c 5 0
mknod /dev/console c 5 1
mknod /dev/ptmx c 5 2
mknod /dev/rtc c 10 135
mknod /dev/tty0 c 4 0
mknod /dev/tty1 c 4 1
mknod /dev/tty2 c 4 2
mknod /dev/tty3 c 4 3
mknod /dev/tty4 c 4 4
mknod /dev/tty5 c 4 5
mknod /dev/tty6 c 4 6
mknod /dev/tty7 c 4 7
mknod /dev/tty8 c 4 8
mknod /dev/tty9 c 4 9
mknod /dev/tty10 c 4 10
mknod /dev/tty11 c 4 11
mknod /dev/tty12 c 4 12
mknod /dev/ttyS0 c 4 64
mknod /dev/ttyS1 c 4 65
mknod /dev/ttyS2 c 4 66
mknod /dev/ttyS3 c 4 67
echo Setting up hotplug.
hotplug
echo Creating block device nodes.
mkblkdevs
echo "Loading scsi_mod.ko module"
insmod /lib/scsi_mod.ko 
echo "Loading sd_mod.ko module"
insmod /lib/sd_mod.ko 
echo "Loading qla1280.ko module"
insmod /lib/qla1280.ko 
echo "Loading jbd.ko module"
insmod /lib/jbd.ko 
echo "Loading ext3.ko module"
insmod /lib/ext3.ko 
echo "Loading dm-mod.ko module"
insmod /lib/dm-mod.ko 
echo "Loading dm-mirror.ko module"
insmod /lib/dm-mirror.ko 
echo "Loading dm-zero.ko module"
insmod /lib/dm-zero.ko 
echo "Loading dm-snapshot.ko module"
insmod /lib/dm-snapshot.ko 
mkblkdevs
resume /dev/sda7
echo Creating root device.
mkrootdev -t ext3 -o defaults,ro /dev/sda8
echo Mounting root filesystem.
mount /sysroot
echo Setting up other filesystems.
setuproot
echo Switching to new root and running init.
switchroot


-- 
Ian Dall <ian@beware.dropbear.id.au>

Re: raid=noautodetect is apparently ignored?

[Bill Davidsen]

Ian Dall wrote:
> On Tue, 2007-06-26 at 16:38 +1000, Neil Brown wrote:
>   
>> On Tuesday June 26, ian@beware.dropbear.id.au wrote:
>>     
>>> When I try and disable auto detection, with kernel boot parameters, it
>>> goes ahead and auto assembles and runs anyway. The md= parameters seem
>>> to be noticed, but don't seem to have any other effect (beyond resulting
>>> in a dmesg).
>>>       
>> Odd....
>>  Maybe you have an initrd which is loading md as a module, then
>>  running "raidautorun" or similar?
>>     
>
> I don't think so. lsmod doesn't show a md module (even though md is
> clearly available).
>   

I suspect that the last comment is the clue, after pivotroot I bet it 
runs another init, not from the boot/initrd images, but from the init.d 
in the root filesystem. I seem to remember setting up my systems to 
start all the arrays found, by using PARTITIONS in my config file. Look 
at the setup in /etc /rc.d files for more info.
> Also if I break open the current initrd, init (the script) looks like
> this:
>
> #!/bin/nash
>
> mount -t proc /proc /proc
> setquiet
> echo Mounting proc filesystem
> echo Mounting sysfs filesystem
> mount -t sysfs /sys /sys
> echo Creating /dev
> mount -o mode=0755 -t tmpfs /dev /dev
> mkdir /dev/pts
> mount -t devpts -o gid=5,mode=620 /dev/pts /dev/pts
> mkdir /dev/shm
> mkdir /dev/mapper
> echo Creating initial device nodes
> mknod /dev/null c 1 3
> mknod /dev/zero c 1 5
> mknod /dev/systty c 4 0
> mknod /dev/tty c 5 0
> mknod /dev/console c 5 1
> mknod /dev/ptmx c 5 2
> mknod /dev/rtc c 10 135
> mknod /dev/tty0 c 4 0
> mknod /dev/tty1 c 4 1
> mknod /dev/tty2 c 4 2
> mknod /dev/tty3 c 4 3
> mknod /dev/tty4 c 4 4
> mknod /dev/tty5 c 4 5
> mknod /dev/tty6 c 4 6
> mknod /dev/tty7 c 4 7
> mknod /dev/tty8 c 4 8
> mknod /dev/tty9 c 4 9
> mknod /dev/tty10 c 4 10
> mknod /dev/tty11 c 4 11
> mknod /dev/tty12 c 4 12
> mknod /dev/ttyS0 c 4 64
> mknod /dev/ttyS1 c 4 65
> mknod /dev/ttyS2 c 4 66
> mknod /dev/ttyS3 c 4 67
> echo Setting up hotplug.
> hotplug
> echo Creating block device nodes.
> mkblkdevs
> echo "Loading scsi_mod.ko module"
> insmod /lib/scsi_mod.ko 
> echo "Loading sd_mod.ko module"
> insmod /lib/sd_mod.ko 
> echo "Loading qla1280.ko module"
> insmod /lib/qla1280.ko 
> echo "Loading jbd.ko module"
> insmod /lib/jbd.ko 
> echo "Loading ext3.ko module"
> insmod /lib/ext3.ko 
> echo "Loading dm-mod.ko module"
> insmod /lib/dm-mod.ko 
> echo "Loading dm-mirror.ko module"
> insmod /lib/dm-mirror.ko 
> echo "Loading dm-zero.ko module"
> insmod /lib/dm-zero.ko 
> echo "Loading dm-snapshot.ko module"
> insmod /lib/dm-snapshot.ko 
> mkblkdevs
> resume /dev/sda7
> echo Creating root device.
> mkrootdev -t ext3 -o defaults,ro /dev/sda8
> echo Mounting root filesystem.
> mount /sysroot
> echo Setting up other filesystems.
> setuproot
> echo Switching to new root and running init.
> switchroot
>
>
>   


-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: raid=noautodetect is apparently ignored?

[Ian Dall]

Ha! I think I have almost figured this out and it is to do with initrd.
I went back to 2.16.17 kernel, which I happened to have the source tree
for, and put in some printks.

First, if there is a ramdisk, the kernel doesn't auto detect
raids /regardless/ of whether raid=noautodetect is specified or not.

This is because md_run_setup() is never called, which in turn is because
of the following code in init/main.c  

	 * check if there is an early userspace init.  If yes, let it do all
	 * the work
	 */

	if (!ramdisk_execute_command)
		ramdisk_execute_command = "/init";

	if (sys_access((const char __user *) ramdisk_execute_command, 0) != 0) {
		ramdisk_execute_command = NULL;
		prepare_namespace();   /* which eventually calls md_run_setup() */
	}

The only other way the kernel's raid autodetection can be triggered is
from userspace with a RAID_AUTORUN ioctl. Something in the initrd is
sending the ioctl. It must be one of the functions of nash, but I don't
have the source handy to check.

This all seems reasonable enough, but it would be good if the
documentation for md mentioned that the options would be ignored if you
use an initrd.

One thing, changing the partition type seems to have fixed my immediate
problem.

Ian

On Tue, 2007-06-26 at 16:38 +1000, Neil Brown wrote:
> On Tuesday June 26, ian@beware.dropbear.id.au wrote:
> > When I try and disable auto detection, with kernel boot parameters, it
> > goes ahead and auto assembles and runs anyway. The md= parameters seem
> > to be noticed, but don't seem to have any other effect (beyond resulting
> > in a dmesg).
> 
> Odd....
>  Maybe you have an initrd which is loading md as a module, then
>  running "raidautorun" or similar?
> 
> > 
> > This is driving me nuts. Maybe changing the type of the hda partitions
> > would help?
> 
> Definitely.  Change the partition types.
> 
> NeilBrown
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
-- 
Ian Dall <ian@beware.dropbear.id.au>

Re: raid=noautodetect is apparently ignored?

[Andrew Burgess]

>>> Odd....
>>>  Maybe you have an initrd which is loading md as a module, then
>>>  running "raidautorun" or similar?
..
>I suspect that the last comment is the clue, after pivotroot I bet it 
>runs another init, not from the boot/initrd images, but from the init.d 
>in the root filesystem. 

One quick way to test this is to boot with "init=/bin/sh"
This lets all the initrd stuff run but nothing from the
root filesystem.

I had a similar problem and found that Fedora's /etc/rc.d/rc.sysinit was
starting up the raid arrays -- that script should probably look
for "raid=noautodetect" in /proc/cmdline.

HTH

Re: raid=noautodetect is apparently ignored?

[Ian Dall]

On Wed, 2007-06-27 at 08:48 -0700, Andrew Burgess wrote:
> >>> Odd....
> >>>  Maybe you have an initrd which is loading md as a module, then
> >>>  running "raidautorun" or similar?
> ..
> >I suspect that the last comment is the clue, after pivotroot I bet it 
> >runs another init, not from the boot/initrd images, but from the init.d 
> >in the root filesystem.

You are absolutely correct. On Fedora core5, in rc.sysinit

        echo "raidautorun /dev/md0" | nash --quiet
        if [ -f /etc/mdadm.conf ]; then
            /sbin/mdadm -A -s
        fi
        

But my original observation was correct. The noautodetect was/is being
ignored because (whenever) there is an initrd. FC5 doesn't support raid
root partitions (mkinitrd doesn't put the right stuff in initrd), but
FC7 tries to. I have upgraded and things are mostly correct. Albeit, FC7
doesn't support my nested raid configuration and so it took some coaxing
to get the the upgrade done and a hack to coax mkinitrd into doing the
right thing.

Putting mdadm.conf on a floppy disk plus a little intervention with a
virtual console early in the upgrade process worked wonders.

> One quick way to test this is to boot with "init=/bin/sh"
> This lets all the initrd stuff run but nothing from the
> root filesystem.

Neat idea. I'll try and remember that for the future.

-- 
Ian Dall <ian@beware.dropbear.id.au>