RAID superblock confusion

RAID superblock confusion

[Richard Gooch]

  Hi, all. I've found the following (vexing) problem with the way the
raid code handles superblocks/migrating devices. This was with
2.4.19-pre6. This machine has 7 SCSI controllers: 4 isp2x00 (qlogicfc)
and 3 sym53c8xxx. On 3 of the isp2x00 controllers, there are 2 drives
each. The second partition on each of these 6 drives forms part of the
RAID0 set. The sym53c8xx driver is built into the kernel, always (it
has the boot drive).

If the isp2x00 driver is built-in to the kernel, then host numbers
[0-3] are assigned to the isp2x00 controllers, and [4-6] to the
sym53c8xx controllers. The devices on the isp2x00 controllers are
detected first, which means they have lower minor numbers. This is
configuration used when the RAID set was configured many moons ago,
and continues to work.

If the isp2x00 driver is built as a module, and the scsihosts boot
parameter is passed to the kernel to reserve host numbers [0-3] for
the isp2x00 controllers, then the RAID code gets confused. Even though
the host numbers are the same, the device detection order is changed,
and thus the minor numbers are changed. Even though I'm using
persistent superblockss, which is supposed to allow one to move
devices from one controller to another, I can't use my RAID) set in
this configuration. Looks like a bug.

Below is some relevant information.

Kernel log for first (working) configuration:
===============================================================================
 [events: 0000004a]
 [events: 0000004a]
 [events: 0000004a]
 [events: 0000004a]
 [events: 0000004a]
 [events: 0000004a]
md: autorun ...
md: considering scsi/host2/bus0/target2/lun0/part2 ...
md:  adding scsi/host2/bus0/target2/lun0/part2 ...
md:  adding scsi/host2/bus0/target1/lun0/part2 ...
md:  adding scsi/host1/bus0/target2/lun0/part2 ...
md:  adding scsi/host1/bus0/target1/lun0/part2 ...
md:  adding scsi/host0/bus0/target2/lun0/part2 ...
md:  adding scsi/host0/bus0/target1/lun0/part2 ...
md: created md0
md: bind<scsi/host0/bus0/target1/lun0/part2,1>
md: bind<scsi/host0/bus0/target2/lun0/part2,2>
md: bind<scsi/host1/bus0/target1/lun0/part2,3>
md: bind<scsi/host1/bus0/target2/lun0/part2,4>
md: bind<scsi/host2/bus0/target1/lun0/part2,5>
md: bind<scsi/host2/bus0/target2/lun0/part2,6>
md: running: <scsi/host2/bus0/target2/lun0/part2><scsi/host2/bus0/target1/lun0/part2><scsi/host1/bus0/target2/lun0/part2><scsi/host1/bus0/target1/lun0/part2><scsi/host0/bus0/target2/lun0/part2><scsi/host0/bus0/target1/lun0/part2>
md: scsi/host2/bus0/target2/lun0/part2's event counter: 0000004a
md: scsi/host2/bus0/target1/lun0/part2's event counter: 0000004a
md: scsi/host1/bus0/target2/lun0/part2's event counter: 0000004a
md: scsi/host1/bus0/target1/lun0/part2's event counter: 0000004a
md: scsi/host0/bus0/target2/lun0/part2's event counter: 0000004a
md: scsi/host0/bus0/target1/lun0/part2's event counter: 0000004a
md0: max total readahead window set to 1536k
md0: 6 data-disks, max readahead per data-disk: 256k
raid0: looking at scsi/host0/bus0/target1/lun0/part2
raid0:   comparing scsi/host0/bus0/target1/lun0/part2(11767488) with scsi/host0/bus0/target1/lun0/part2(11767488)
raid0:   END
raid0:   ==> UNIQUE
raid0: 1 zones
raid0: looking at scsi/host0/bus0/target2/lun0/part2
raid0:   comparing scsi/host0/bus0/target2/lun0/part2(11767488) with scsi/host0/bus0/target1/lun0/part2(11767488)
raid0:   EQUAL
raid0: looking at scsi/host1/bus0/target1/lun0/part2
raid0:   comparing scsi/host1/bus0/target1/lun0/part2(11767488) with scsi/host0/bus0/target1/lun0/part2(11767488)
raid0:   EQUAL
raid0: looking at scsi/host1/bus0/target2/lun0/part2
raid0:   comparing scsi/host1/bus0/target2/lun0/part2(11767488) with scsi/host0/bus0/target1/lun0/part2(11767488)
raid0:   EQUAL
raid0: looking at scsi/host2/bus0/target1/lun0/part2
raid0:   comparing scsi/host2/bus0/target1/lun0/part2(11767488) with scsi/host0/bus0/target1/lun0/part2(11767488)
raid0:   EQUAL
raid0: looking at scsi/host2/bus0/target2/lun0/part2
raid0:   comparing scsi/host2/bus0/target2/lun0/part2(11767488) with scsi/host0/bus0/target1/lun0/part2(11767488)
raid0:   EQUAL
raid0: FINAL 1 zones
raid0: zone 0
raid0: checking scsi/host0/bus0/target1/lun0/part2 ... contained as device 0
  (11767488) is smallest!.
raid0: checking scsi/host0/bus0/target2/lun0/part2 ... contained as device 1
raid0: checking scsi/host1/bus0/target1/lun0/part2 ... contained as device 2
raid0: checking scsi/host1/bus0/target2/lun0/part2 ... contained as device 3
raid0: checking scsi/host2/bus0/target1/lun0/part2 ... contained as device 4
raid0: checking scsi/host2/bus0/target2/lun0/part2 ... contained as device 5
raid0: zone->nb_dev: 6, size: 70604928
raid0: current zone offset: 11767488
raid0: done.
raid0 : md_size is 70604928 blocks.
raid0 : conf->smallest->size is 70604928 blocks.
raid0 : nb_zone is 1.
raid0 : Allocating 8 bytes for hash.
md: updating md0 RAID superblock on device
md: scsi/host2/bus0/target2/lun0/part2 [events: 0000004b]<6>(write) scsi/host2/bus0/target2/lun0/part2's sb offset: 11767488
md: scsi/host2/bus0/target1/lun0/part2 [events: 0000004b]<6>(write) scsi/host2/bus0/target1/lun0/part2's sb offset: 11767488
md: scsi/host1/bus0/target2/lun0/part2 [events: 0000004b]<6>(write) scsi/host1/bus0/target2/lun0/part2's sb offset: 11767488
md: scsi/host1/bus0/target1/lun0/part2 [events: 0000004b]<6>(write) scsi/host1/bus0/target1/lun0/part2's sb offset: 11767488
md: scsi/host0/bus0/target2/lun0/part2 [events: 0000004b]<6>(write) scsi/host0/bus0/target2/lun0/part2's sb offset: 11767488
md: scsi/host0/bus0/target1/lun0/part2 [events: 0000004b]<6>(write) scsi/host0/bus0/target1/lun0/part2's sb offset: 11767488
md: ... autorun DONE.
===============================================================================

Kernel log for second (failing) configuration:
===============================================================================
 [events: 0000004a]
md: can not import scsi/host6/bus0/target0/lun0/part2, has active inodes!
md: could not import scsi/host6/bus0/target0/lun0/part2, trying to run array nevertheless.
 [events: 0000004a]
 [events: 0000004a]
 [events: 0000004a]
 [events: 0000004a]
md: autorun ...
md: considering scsi/host2/bus0/target1/lun0/part2 ...
md:  adding scsi/host2/bus0/target1/lun0/part2 ...
md:  adding scsi/host1/bus0/target2/lun0/part2 ...
md:  adding scsi/host1/bus0/target1/lun0/part2 ...
md:  adding scsi/host0/bus0/target2/lun0/part2 ...
md:  adding scsi/host0/bus0/target1/lun0/part2 ...
md: created md0
md: bind<scsi/host0/bus0/target1/lun0/part2,1>
md: bind<scsi/host0/bus0/target2/lun0/part2,2>
md: bind<scsi/host1/bus0/target1/lun0/part2,3>
md: bind<scsi/host1/bus0/target2/lun0/part2,4>
md: bind<scsi/host2/bus0/target1/lun0/part2,5>
md: running: <scsi/host2/bus0/target1/lun0/part2><scsi/host1/bus0/target2/lun0/part2><scsi/host1/bus0/target1/lun0/part2><scsi/host0/bus0/target2/lun0/part2><scsi/host0/bus0/target1/lun0/part2>
md: scsi/host2/bus0/target1/lun0/part2's event counter: 0000004a
md: scsi/host1/bus0/target2/lun0/part2's event counter: 0000004a
md: scsi/host1/bus0/target1/lun0/part2's event counter: 0000004a
md: scsi/host0/bus0/target2/lun0/part2's event counter: 0000004a
md: scsi/host0/bus0/target1/lun0/part2's event counter: 0000004a
md: device name has changed from scsi/host1/bus0/target2/lun0/part2 to scsi/host2/bus0/target1/lun0/part2 since last import!
md: device name has changed from scsi/host1/bus0/target1/lun0/part2 to scsi/host1/bus0/target2/lun0/part2 since last import!
md: device name has changed from scsi/host0/bus0/target2/lun0/part2 to scsi/host1/bus0/target1/lun0/part2 since last import!
md: device name has changed from scsi/host0/bus0/target1/lun0/part2 to scsi/host0/bus0/target2/lun0/part2 since last import!
md: device name has changed from scsi/host6/bus0/target0/lun0/part2 to scsi/host0/bus0/target1/lun0/part2 since last import!
md0: former device scsi/host2/bus0/target1/lun0/part2 is unavailable, removing from array!
md0: max total readahead window set to 1536k
md0: 6 data-disks, max readahead per data-disk: 256k
md: md0, array needs 6 disks, has 5, aborting.
raid0: disks are not ordered, aborting!
md: pers->run() failed ...
md :do_md_run() returned -22
md: md0 stopped.
md: unbind<scsi/host2/bus0/target1/lun0/part2,4>
md: export_rdev(scsi/host2/bus0/target1/lun0/part2)
md: unbind<scsi/host1/bus0/target2/lun0/part2,3>
md: export_rdev(scsi/host1/bus0/target2/lun0/part2)
md: unbind<scsi/host1/bus0/target1/lun0/part2,2>
md: export_rdev(scsi/host1/bus0/target1/lun0/part2)
md: unbind<scsi/host0/bus0/target2/lun0/part2,1>
md: export_rdev(scsi/host0/bus0/target2/lun0/part2)
md: unbind<scsi/host0/bus0/target1/lun0/part2,0>
md: export_rdev(scsi/host0/bus0/target1/lun0/part2)
md: ... autorun DONE.
===============================================================================
Note the following line from the kernel logs above:
md: can not import scsi/host6/bus0/target0/lun0/part2, has active inodes!

Well, that's no surprise, as this partition has /usr! And this
partition isn't even mentioned in the /etc/raidtab file. But I note
that it has the same device number in this (the broken) configuration
as /dev/sd/c0b0t1u0p2 has in the working configuration.

The /etc/raidtab file:
===============================================================================
raiddev /dev/md/0
        raid-level             0
        nr-raid-disks          6
        persistent-superblock  1
        chunk-size             4
        device                 /dev/sd/c0b0t1u0p2
        raid-disk              0
        device                 /dev/sd/c0b0t2u0p2
        raid-disk              1
        device                 /dev/sd/c1b0t1u0p2
        raid-disk              2
        device                 /dev/sd/c1b0t2u0p2
        raid-disk              3
        device                 /dev/sd/c2b0t1u0p2
        raid-disk              4
        device                 /dev/sd/c2b0t2u0p2
        raid-disk              5
===============================================================================

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: RAID superblock confusion

[Andreas Dilger]

On Apr 10, 2002  09:33 -0600, Richard Gooch wrote:
> Even though I'm using persistent superblockss, which is supposed to
> allow one to move devices from one controller to another, I can't
> use my RAID) set in this configuration. Looks like a bug.
> 
> md0: former device scsi/host2/bus0/target1/lun0/part2 is unavailable, removing from array!
> md: md0, array needs 6 disks, has 5, aborting.

Note that this appears to be your real problem.

> Note the following line from the kernel logs above:
> md: can not import scsi/host6/bus0/target0/lun0/part2, has active inodes!
> 
> Well, that's no surprise, as this partition has /usr! And this
> partition isn't even mentioned in the /etc/raidtab file. But I note
> that it has the same device number in this (the broken) configuration
> as /dev/sd/c0b0t1u0p2 has in the working configuration.

That is a red herring, I think.

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/

Re: RAID superblock confusion

[Richard Gooch]

Andreas Dilger writes:
> On Apr 10, 2002  09:33 -0600, Richard Gooch wrote:
> > Even though I'm using persistent superblockss, which is supposed to
> > allow one to move devices from one controller to another, I can't
> > use my RAID) set in this configuration. Looks like a bug.
> > 
> > md0: former device scsi/host2/bus0/target1/lun0/part2 is unavailable, removing from array!
> > md: md0, array needs 6 disks, has 5, aborting.
> 
> Note that this appears to be your real problem.

No. I tested all 6 partitions used in the RAID set. They are all
available.

> > Note the following line from the kernel logs above:
> > md: can not import scsi/host6/bus0/target0/lun0/part2, has active inodes!
> > 
> > Well, that's no surprise, as this partition has /usr! And this
> > partition isn't even mentioned in the /etc/raidtab file. But I note
> > that it has the same device number in this (the broken) configuration
> > as /dev/sd/c0b0t1u0p2 has in the working configuration.
> 
> That is a red herring, I think.

Then what *is* the problem?

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: RAID superblock confusion

[Andreas Dilger]

On Apr 10, 2002  13:24 -0600, Richard Gooch wrote:
> Andreas Dilger writes:
> > On Apr 10, 2002  09:33 -0600, Richard Gooch wrote:
> > > Even though I'm using persistent superblockss, which is supposed to
> > > allow one to move devices from one controller to another, I can't
> > > use my RAID) set in this configuration. Looks like a bug.
> > > 
> > > md0: former device scsi/host2/bus0/target1/lun0/part2 is unavailable,
> > >      removing from array!
> > > md: md0, array needs 6 disks, has 5, aborting.
> > 
> > Note that this appears to be your real problem.
> 
> No. I tested all 6 partitions used in the RAID set. They are all
> available.

Well, MD seems to think it is unavailable...  I would check the codepath
that generates this message and see why it is happening.  Maybe it is a
timing issue or something, that MD autostart is starting before this
device is set up or something?  I don't know.

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/

Re: RAID superblock confusion

[Richard Gooch]

Andreas Dilger writes:
> On Apr 10, 2002  13:24 -0600, Richard Gooch wrote:
> > Andreas Dilger writes:
> > > On Apr 10, 2002  09:33 -0600, Richard Gooch wrote:
> > > > Even though I'm using persistent superblockss, which is supposed to
> > > > allow one to move devices from one controller to another, I can't
> > > > use my RAID) set in this configuration. Looks like a bug.
> > > > 
> > > > md0: former device scsi/host2/bus0/target1/lun0/part2 is unavailable,
> > > >      removing from array!
> > > > md: md0, array needs 6 disks, has 5, aborting.
> > > 
> > > Note that this appears to be your real problem.
> > 
> > No. I tested all 6 partitions used in the RAID set. They are all
> > available.
> 
> Well, MD seems to think it is unavailable...  I would check the
> codepath that generates this message and see why it is happening.
> Maybe it is a timing issue or something, that MD autostart is
> starting before this device is set up or something?  I don't know.

The device is set up (i.e. SCSI host driver is loaded) long before I
do raidstart /dev/md/0

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: RAID superblock confusion

[Mike Fedyk]

On Wed, Apr 10, 2002 at 02:37:48PM -0600, Richard Gooch wrote:
> Andreas Dilger writes:
> > On Apr 10, 2002  13:24 -0600, Richard Gooch wrote:
> > > Andreas Dilger writes:
> > > > On Apr 10, 2002  09:33 -0600, Richard Gooch wrote:
> > > > > Even though I'm using persistent superblockss, which is supposed to
> > > > > allow one to move devices from one controller to another, I can't
> > > > > use my RAID) set in this configuration. Looks like a bug.
> > > > > 
> > > > > md0: former device scsi/host2/bus0/target1/lun0/part2 is unavailable,
> > > > >      removing from array!
> > > > > md: md0, array needs 6 disks, has 5, aborting.
> > > > 
> > > > Note that this appears to be your real problem.
> > > 
> > > No. I tested all 6 partitions used in the RAID set. They are all
> > > available.
> > 
> > Well, MD seems to think it is unavailable...  I would check the
> > codepath that generates this message and see why it is happening.
> > Maybe it is a timing issue or something, that MD autostart is
> > starting before this device is set up or something?  I don't know.
> 
> The device is set up (i.e. SCSI host driver is loaded) long before I
> do raidstart /dev/md/0

But kernel auto-detection doesn't depend on the raidstart command.  If
things are setup correctly, you can remove that from your init scripts.

Re: RAID superblock confusion

[Richard Gooch]

Mike Fedyk writes:
> On Wed, Apr 10, 2002 at 02:37:48PM -0600, Richard Gooch wrote:
> > Andreas Dilger writes:
> > > On Apr 10, 2002  13:24 -0600, Richard Gooch wrote:
> > > > Andreas Dilger writes:
> > > > > On Apr 10, 2002  09:33 -0600, Richard Gooch wrote:
> > > > > > Even though I'm using persistent superblockss, which is supposed to
> > > > > > allow one to move devices from one controller to another, I can't
> > > > > > use my RAID) set in this configuration. Looks like a bug.
> > > > > > 
> > > > > > md0: former device scsi/host2/bus0/target1/lun0/part2 is unavailable,
> > > > > >      removing from array!
> > > > > > md: md0, array needs 6 disks, has 5, aborting.
> > > > > 
> > > > > Note that this appears to be your real problem.
> > > > 
> > > > No. I tested all 6 partitions used in the RAID set. They are all
> > > > available.
> > > 
> > > Well, MD seems to think it is unavailable...  I would check the
> > > codepath that generates this message and see why it is happening.
> > > Maybe it is a timing issue or something, that MD autostart is
> > > starting before this device is set up or something?  I don't know.
> > 
> > The device is set up (i.e. SCSI host driver is loaded) long before I
> > do raidstart /dev/md/0
> 
> But kernel auto-detection doesn't depend on the raidstart command.  If
> things are setup correctly, you can remove that from your init scripts.

I'm not (explicitely) using auto-detection. When I insmod the raid0
module, there are no messages about finding devices. All I get is:
md: raid0 personality registered as nr 2

Only when I run raidstart do I get kernel messages about the devices.

In any case, I should be able to move my devices around (especially
if /etc/raidtab is still correct), whether or not autostart is
running. The behaviour I'm observing is a bug (I assume it's not a
mis-feature, since the raidstart man page tells me that moving devices
around should be safe).

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: RAID superblock confusion

[Mike Fedyk]

On Wed, Apr 10, 2002 at 03:39:09PM -0600, Richard Gooch wrote:
> Mike Fedyk writes:
> > On Wed, Apr 10, 2002 at 02:37:48PM -0600, Richard Gooch wrote:
> > > 
> > > The device is set up (i.e. SCSI host driver is loaded) long before I
> > > do raidstart /dev/md/0
> > 
> > But kernel auto-detection doesn't depend on the raidstart command.  If
> > things are setup correctly, you can remove that from your init scripts.
> 
> I'm not (explicitely) using auto-detection. When I insmod the raid0
> module, there are no messages about finding devices. All I get is:
> md: raid0 personality registered as nr 2
> 
> Only when I run raidstart do I get kernel messages about the devices.
> 
> In any case, I should be able to move my devices around (especially
> if /etc/raidtab is still correct), whether or not autostart is
> running. The behaviour I'm observing is a bug (I assume it's not a
> mis-feature, since the raidstart man page tells me that moving devices
> around should be safe).

Ehh, I ran into this a while ago.  When you compile raid as modules it
doesn't use the raid superblocks for anything except for verification.  I
took a quick glance at the source and the auto-detect code is ifdefed out if
you compiled as a module.

Ever since I have had raid compiled into my kernels.

Mike

Re: RAID superblock confusion

[Richard Gooch]

Mike Fedyk writes:
> On Wed, Apr 10, 2002 at 03:39:09PM -0600, Richard Gooch wrote:
> > Mike Fedyk writes:
> > > On Wed, Apr 10, 2002 at 02:37:48PM -0600, Richard Gooch wrote:
> > > > 
> > > > The device is set up (i.e. SCSI host driver is loaded) long before I
> > > > do raidstart /dev/md/0
> > > 
> > > But kernel auto-detection doesn't depend on the raidstart command.  If
> > > things are setup correctly, you can remove that from your init scripts.
> > 
> > I'm not (explicitely) using auto-detection. When I insmod the raid0
> > module, there are no messages about finding devices. All I get is:
> > md: raid0 personality registered as nr 2
> > 
> > Only when I run raidstart do I get kernel messages about the devices.
> > 
> > In any case, I should be able to move my devices around (especially
> > if /etc/raidtab is still correct), whether or not autostart is
> > running. The behaviour I'm observing is a bug (I assume it's not a
> > mis-feature, since the raidstart man page tells me that moving devices
> > around should be safe).
> 
> Ehh, I ran into this a while ago.  When you compile raid as modules
> it doesn't use the raid superblocks for anything except for
> verification.  I took a quick glance at the source and the
> auto-detect code is ifdefed out if you compiled as a module.

Exactly where is this? A scan with find and grep don't reveal this.

> Ever since I have had raid compiled into my kernels.

This is my relevant .config:
CONFIG_MD=y
CONFIG_BLK_DEV_MD=y
CONFIG_MD_LINEAR=m
CONFIG_MD_RAID0=m
CONFIG_MD_RAID1=m
CONFIG_MD_RAID5=m
CONFIG_MD_MULTIPATH=m

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: RAID superblock confusion

[Mike Fedyk]

On Wed, Apr 10, 2002 at 04:49:06PM -0600, Richard Gooch wrote:
> Mike Fedyk writes:
> > On Wed, Apr 10, 2002 at 03:39:09PM -0600, Richard Gooch wrote:
> > > Mike Fedyk writes:
> > > > On Wed, Apr 10, 2002 at 02:37:48PM -0600, Richard Gooch wrote:
> > > > > 
> > > > > The device is set up (i.e. SCSI host driver is loaded) long before I
> > > > > do raidstart /dev/md/0
> > > > 
> > > > But kernel auto-detection doesn't depend on the raidstart command.  If
> > > > things are setup correctly, you can remove that from your init scripts.
> > > 
> > > I'm not (explicitely) using auto-detection. When I insmod the raid0
> > > module, there are no messages about finding devices. All I get is:
> > > md: raid0 personality registered as nr 2
> > > 
> > > Only when I run raidstart do I get kernel messages about the devices.
> > > 
> > > In any case, I should be able to move my devices around (especially
> > > if /etc/raidtab is still correct), whether or not autostart is
> > > running. The behaviour I'm observing is a bug (I assume it's not a
> > > mis-feature, since the raidstart man page tells me that moving devices
> > > around should be safe).
> > 
> > Ehh, I ran into this a while ago.  When you compile raid as modules
> > it doesn't use the raid superblocks for anything except for
> > verification.  I took a quick glance at the source and the
> > auto-detect code is ifdefed out if you compiled as a module.
> 
> Exactly where is this? A scan with find and grep don't reveal this.
>

drivers/md/md.c

in the ifndef MODULE sectioin.

> > Ever since I have had raid compiled into my kernels.
> 
> This is my relevant .config:
> CONFIG_MD=y
> CONFIG_BLK_DEV_MD=y
> CONFIG_MD_LINEAR=m
> CONFIG_MD_RAID0=m
> CONFIG_MD_RAID1=m
> CONFIG_MD_RAID5=m
> CONFIG_MD_MULTIPATH=m
> 

Set this to =y and you're set.

I'd like to see this working from modules though.

Mike

Re: RAID superblock confusion

[Luigi Genoni]

> > >
> > > Ehh, I ran into this a while ago.  When you compile raid as modules
> > > it doesn't use the raid superblocks for anything except for
> > > verification.  I took a quick glance at the source and the
> > > auto-detect code is ifdefed out if you compiled as a module.
> >
> > Exactly where is this? A scan with find and grep don't reveal this.
> >
>
> drivers/md/md.c
>
> in the ifndef MODULE sectioin.
>
> > > Ever since I have had raid compiled into my kernels.
> >
> > This is my relevant .config:
> > CONFIG_MD=y
> > CONFIG_BLK_DEV_MD=y
> > CONFIG_MD_LINEAR=m
> > CONFIG_MD_RAID0=m
> > CONFIG_MD_RAID1=m
> > CONFIG_MD_RAID5=m
> > CONFIG_MD_MULTIPATH=m
> >
>
> Set this to =y and you're set.
>
> I'd like to see this working from modules though.

NO, please. There are hundreds of scenarios where that could be dangerous.
Suppose you load the RAID
module when all partitions are mounted, and two partiton in mirror are
mount on different mount point (you can do this, raid module is not
loaded, and so...). And now you load the module and md device is
registered. That would not be really nice, also if it is ulikely that you
could damnage your system

Re: RAID superblock confusion

[Richard Gooch]

Luigi Genoni writes:
> 
> > > >
> > > > Ehh, I ran into this a while ago.  When you compile raid as modules
> > > > it doesn't use the raid superblocks for anything except for
> > > > verification.  I took a quick glance at the source and the
> > > > auto-detect code is ifdefed out if you compiled as a module.
> > >
> > > Exactly where is this? A scan with find and grep don't reveal this.
> > >
> >
> > drivers/md/md.c
> >
> > in the ifndef MODULE sectioin.
> >
> > > > Ever since I have had raid compiled into my kernels.
> > >
> > > This is my relevant .config:
> > > CONFIG_MD=y
> > > CONFIG_BLK_DEV_MD=y
> > > CONFIG_MD_LINEAR=m
> > > CONFIG_MD_RAID0=m
> > > CONFIG_MD_RAID1=m
> > > CONFIG_MD_RAID5=m
> > > CONFIG_MD_MULTIPATH=m
> > >
> >
> > Set this to =y and you're set.
> >
> > I'd like to see this working from modules though.
> 
> NO, please. There are hundreds of scenarios where that could be
> dangerous.  Suppose you load the RAID module when all partitions are
> mounted, and two partiton in mirror are mount on different mount
> point (you can do this, raid module is not loaded, and so...). And
> now you load the module and md device is registered. That would not
> be really nice, also if it is ulikely that you could damnage your
> system

The RAID code checks to see if there are busy inodes for each device
in a RAID set. So your hundreds of scenarios are not a problem.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: RAID superblock confusion

[Mike Fedyk]

On Sat, Apr 13, 2002 at 01:29:05PM -0600, Richard Gooch wrote:
> Luigi Genoni writes:
> > 
> > > > >
> > > > > Ehh, I ran into this a while ago.  When you compile raid as modules
> > > > > it doesn't use the raid superblocks for anything except for
> > > > > verification.  I took a quick glance at the source and the
> > > > > auto-detect code is ifdefed out if you compiled as a module.
> > > >
> > > > Exactly where is this? A scan with find and grep don't reveal this.
> > > >
> > >
> > > drivers/md/md.c
> > >
> > > in the ifndef MODULE sectioin.
> > >
> > > > > Ever since I have had raid compiled into my kernels.
> > > >
> > > > This is my relevant .config:
> > > > CONFIG_MD=y
> > > > CONFIG_BLK_DEV_MD=y
> > > > CONFIG_MD_LINEAR=m
> > > > CONFIG_MD_RAID0=m
> > > > CONFIG_MD_RAID1=m
> > > > CONFIG_MD_RAID5=m
> > > > CONFIG_MD_MULTIPATH=m
> > > >
> > >
> > > Set this to =y and you're set.
> > >
> > > I'd like to see this working from modules though.
> > 
> > NO, please. There are hundreds of scenarios where that could be
> > dangerous.  Suppose you load the RAID module when all partitions are
> > mounted, and two partiton in mirror are mount on different mount
> > point (you can do this, raid module is not loaded, and so...). And
> > now you load the module and md device is registered. That would not
> > be really nice, also if it is ulikely that you could damnage your
> > system
> 
> The RAID code checks to see if there are busy inodes for each device
> in a RAID set. So your hundreds of scenarios are not a problem.
> 

I had a machine that had raid1 setup correctly but was accidentally
configured to root=/dev/hda1 (one member of the md0 raid1 set).

All was well until I noticed I wasn't rooting from md0, so reboot with new
root=/dev/md0 and now my filesystem is b0rked (maybe because hdc1 was the
primary mirror?).

Luckily I was still setting up that machine so I just reinstalled it.

This was with raid compiled into the kernel, so it's not a module checking
issue, and I consider it a user error.  But maybe someone else thinks
different...

Just reporting in case someone is intereted...

Mike

Re: RAID superblock confusion

[Richard Gooch]

Mike Fedyk writes:
> On Sat, Apr 13, 2002 at 01:29:05PM -0600, Richard Gooch wrote:
> > Luigi Genoni writes:
> > > 
> > > > > >
> > > > > > Ehh, I ran into this a while ago.  When you compile raid as modules
> > > > > > it doesn't use the raid superblocks for anything except for
> > > > > > verification.  I took a quick glance at the source and the
> > > > > > auto-detect code is ifdefed out if you compiled as a module.
> > > > >
> > > > > Exactly where is this? A scan with find and grep don't reveal this.
> > > > >
> > > >
> > > > drivers/md/md.c
> > > >
> > > > in the ifndef MODULE sectioin.
> > > >
> > > > > > Ever since I have had raid compiled into my kernels.
> > > > >
> > > > > This is my relevant .config:
> > > > > CONFIG_MD=y
> > > > > CONFIG_BLK_DEV_MD=y
> > > > > CONFIG_MD_LINEAR=m
> > > > > CONFIG_MD_RAID0=m
> > > > > CONFIG_MD_RAID1=m
> > > > > CONFIG_MD_RAID5=m
> > > > > CONFIG_MD_MULTIPATH=m
> > > > >
> > > >
> > > > Set this to =y and you're set.
> > > >
> > > > I'd like to see this working from modules though.
> > > 
> > > NO, please. There are hundreds of scenarios where that could be
> > > dangerous.  Suppose you load the RAID module when all partitions are
> > > mounted, and two partiton in mirror are mount on different mount
> > > point (you can do this, raid module is not loaded, and so...). And
> > > now you load the module and md device is registered. That would not
> > > be really nice, also if it is ulikely that you could damnage your
> > > system
> > 
> > The RAID code checks to see if there are busy inodes for each device
> > in a RAID set. So your hundreds of scenarios are not a problem.
> > 
> 
> I had a machine that had raid1 setup correctly but was accidentally
> configured to root=/dev/hda1 (one member of the md0 raid1 set).
> 
> All was well until I noticed I wasn't rooting from md0, so reboot with new
> root=/dev/md0 and now my filesystem is b0rked (maybe because hdc1 was the
> primary mirror?).
> 
> Luckily I was still setting up that machine so I just reinstalled it.
> 
> This was with raid compiled into the kernel, so it's not a module checking
> issue, and I consider it a user error.  But maybe someone else thinks
> different...

Yep, user error. Just like if you dd if=/dev/zero of=/dev/hda2 but
meant to write to /dev/hda1 instead, and /dev/hda2 has your OS while
/dev/hda1 had M$ which you wanted to erase and re-install. Not much to
be done about that. One learns best by fucking up :-)

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: RAID superblock confusion

[Mike Fedyk]

On Sat, Apr 13, 2002 at 06:00:13PM -0600, Richard Gooch wrote:
> Mike Fedyk writes:
> > On Sat, Apr 13, 2002 at 01:29:05PM -0600, Richard Gooch wrote:
> > > Luigi Genoni writes:
> > > > 
> > > > > > >
> > > > > > > Ehh, I ran into this a while ago.  When you compile raid as modules
> > > > > > > it doesn't use the raid superblocks for anything except for
> > > > > > > verification.  I took a quick glance at the source and the
> > > > > > > auto-detect code is ifdefed out if you compiled as a module.
> > > > > >
> > > > > > Exactly where is this? A scan with find and grep don't reveal this.
> > > > > >
> > > > >
> > > > > drivers/md/md.c
> > > > >
> > > > > in the ifndef MODULE sectioin.
> > > > >
> > > > > > > Ever since I have had raid compiled into my kernels.
> > > > > >
> > > > > > This is my relevant .config:
> > > > > > CONFIG_MD=y
> > > > > > CONFIG_BLK_DEV_MD=y
> > > > > > CONFIG_MD_LINEAR=m
> > > > > > CONFIG_MD_RAID0=m
> > > > > > CONFIG_MD_RAID1=m
> > > > > > CONFIG_MD_RAID5=m
> > > > > > CONFIG_MD_MULTIPATH=m
> > > > > >
> > > > >
> > > > > Set this to =y and you're set.
> > > > >
> > > > > I'd like to see this working from modules though.
> > > > 
> > > > NO, please. There are hundreds of scenarios where that could be
> > > > dangerous.  Suppose you load the RAID module when all partitions are
> > > > mounted, and two partiton in mirror are mount on different mount
> > > > point (you can do this, raid module is not loaded, and so...). And
> > > > now you load the module and md device is registered. That would not
> > > > be really nice, also if it is ulikely that you could damnage your
> > > > system
> > > 
> > > The RAID code checks to see if there are busy inodes for each device
> > > in a RAID set. So your hundreds of scenarios are not a problem.
> > > 
> > 
> > I had a machine that had raid1 setup correctly but was accidentally
> > configured to root=/dev/hda1 (one member of the md0 raid1 set).
> > 
> > All was well until I noticed I wasn't rooting from md0, so reboot with new
> > root=/dev/md0 and now my filesystem is b0rked (maybe because hdc1 was the
> > primary mirror?).
> > 
> > Luckily I was still setting up that machine so I just reinstalled it.
> > 
> > This was with raid compiled into the kernel, so it's not a module checking
> > issue, and I consider it a user error.  But maybe someone else thinks
> > different...
> 
> Yep, user error. Just like if you dd if=/dev/zero of=/dev/hda2 but
> meant to write to /dev/hda1 instead, and /dev/hda2 has your OS while
> /dev/hda1 had M$ which you wanted to erase and re-install. Not much to
> be done about that. One learns best by fucking up :-)
> 

One can also argue that raid (sh|c)ould lock the devices that it has been
started on also...

But it is the unix way to give the root user lots of rope to hang
him/herself, and what keeps the root user from trying to erase the entire
drive (hda instead of hda1) instead of just the locked partition?

Re: RAID superblock confusion

[Neil Brown]

On Wednesday April 10, rgooch@ras.ucalgary.ca wrote:
> 
> The device is set up (i.e. SCSI host driver is loaded) long before I
> do raidstart /dev/md/0

raidstart simply does not and cannot work reliably when your device
numbers change around.  It takes the first device listed in
/etc/raidtab and gives it to the kernel.
The kernel reads the superblock, finds some device numbers and tries
to attach those devices.  If device number have changed, you loose.

autodetect is the other alternative.  However, as has been mentioned,
it does not and cannot work with md as a module.  This is because
devices can only be register for autodetection after md.o is loaded,
and autodetection is done at the time that md is loaded.  So
autodetection can only work if the device driver and md are loaded at
simultaneously.  i.e. they are compiled into the kernel.

I use and recommend mdadm.
  http://www.cse.unsw.edu.au/~neilb/source/mdadm/
(I hope to release a 1.0 in the next week or so).

With mdadm you can assemble devices whenever you like based on uuid in
the superblock.

Assuming you use devfs :-)
Create /etc/mdadm.conf containing:

  DEVICE /dev/scsi/*/*/*/*/part*

[[ you should be able to say

   DEVICE /dev/discs/*/part*

  but glob(3) gets confused by that :-(
]]

and run

   mdadm -Es >> /etc/mdadm.conf

Then edit /etc/mdadm.conf removing bits that are wrong, and changing
bits that need changing.  The mdadm.conf.5 to understand what should
be there.

You will need to:
   get rid of the "devices=" parts
   Make sure no extra arrays were found.

Once you have done this, then

  mdadm -As

in your startup scripts will assemble your arrays.

Make sure you use mdadm-0.8.2.  I found a little bug in 0.8.1 as I was
testing this example.

NeilBrown

Re: RAID superblock confusion

[Mike Fedyk]

On Thu, Apr 11, 2002 at 11:38:19AM +1000, Neil Brown wrote:
> autodetect is the other alternative.  However, as has been mentioned,
> it does not and cannot work with md as a module.  This is because
> devices can only be register for autodetection after md.o is loaded,
> and autodetection is done at the time that md is loaded.  So
> autodetection can only work if the device driver and md are loaded at
> simultaneously.  i.e. they are compiled into the kernel.

Ahh, but if you use initrd you can even have the ide and scsi drivers as
modules. 

What is needed is to make the disk modules depend on the raid modules (only
if the raid code is enabled of course) so that modprobe can load the raid
modules first. 

Then you'd just need to make sure that if there are any block modules linked
into the kernel that raid is also linked into the kernle instead of a module.

Is there some reason why this wouldn't work (except for CML1
complications...)?

Kieth, will kbuild2.5 affect this in any way?  Or is this entirely a CML2
thing? 

Mike

Re: RAID superblock confusion

[Keith Owens]

On Wed, 10 Apr 2002 19:41:11 -0700, 
Mike Fedyk <mfedyk@matchmail.com> wrote:
>Then you'd just need to make sure that if there are any block modules linked
>into the kernel that raid is also linked into the kernle instead of a module.
>
>Is there some reason why this wouldn't work (except for CML1
>complications...)?
>
>Kieth, will kbuild2.5 affect this in any way?  Or is this entirely a CML2
>thing? 

AFAICT it is pure config.  kbuild 2.5 does not care how .config is
built, it just requires a clean .config.  I want a clean seperation
between configuring and building the kernel.  In particular, Makefile
rules should not try to adjust for broken config entries, this is a
config only problem.

Re: RAID superblock confusion

[Helge Hafting]

Mike Fedyk wrote:
[...] 
> Ahh, but if you use initrd you can even have the ide and scsi drivers as
> modules.
> 
> What is needed is to make the disk modules depend on the raid modules (only
> if the raid code is enabled of course) so that modprobe can load the raid
> modules first.
> 
> Then you'd just need to make sure that if there are any block modules linked
> into the kernel that raid is also linked into the kernle instead of a module.
> 
> Is there some reason why this wouldn't work (except for CML1
> complications...)?

Having block devices depend on raid modules isn't the solution.
A user may have many block devices with raid on only a few
of them.

I think this sort of setup-specific dependencies belong in userspace,
i.e. /etc/conf.modules:

pre-install <block-driver-with-raid-partitions> insmod
<raid-modules-needed>


Helge Hafting

Re: RAID superblock confusion

[Richard Gooch]

Neil Brown writes:
> On Wednesday April 10, rgooch@ras.ucalgary.ca wrote:
> > 
> > The device is set up (i.e. SCSI host driver is loaded) long before I
> > do raidstart /dev/md/0
> 
> raidstart simply does not and cannot work reliably when your device
> numbers change around.  It takes the first device listed in
> /etc/raidtab and gives it to the kernel.
> The kernel reads the superblock, finds some device numbers and tries
> to attach those devices.  If device number have changed, you loose.

Sounds to me like the flaw is in the ioctl(2) interface, in that it
doesn't allow passing all the block devices in the RAID set. If it
allowed you to pass all the block devices, then it could check if all
the signatures on each block device match.

I tried the alternative of setting persistent-superblock=0 in
/etc/raidtab, but the stupid thing complained because it found a
superblock. Sigh.

If there was only a "do as I say, regardless" mode, I would be happy.
This programmer-knows-best attitude smacks of M$.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: RAID superblock confusion

[Neil Brown]

On Saturday April 13, rgooch@ras.ucalgary.ca wrote:
> Neil Brown writes:
> > On Wednesday April 10, rgooch@ras.ucalgary.ca wrote:
> > > 
> > > The device is set up (i.e. SCSI host driver is loaded) long before I
> > > do raidstart /dev/md/0
> > 
> > raidstart simply does not and cannot work reliably when your device
> > numbers change around.  It takes the first device listed in
> > /etc/raidtab and gives it to the kernel.
> > The kernel reads the superblock, finds some device numbers and tries
> > to attach those devices.  If device number have changed, you loose.
> 
> Sounds to me like the flaw is in the ioctl(2) interface, in that it
> doesn't allow passing all the block devices in the RAID set. If it
> allowed you to pass all the block devices, then it could check if all
> the signatures on each block device match.

Exactly.  The flaw is in the ioctl interface.  2.4 comes with an
improved ioctl interface which allows you to tell the kernel exactly
what you want it to do.  mdadm used this interface.

> 
> I tried the alternative of setting persistent-superblock=0 in
> /etc/raidtab, but the stupid thing complained because it found a
> superblock. Sigh.
> 
> If there was only a "do as I say, regardless" mode, I would be happy.
> This programmer-knows-best attitude smacks of M$.

mdadm will do as you say, reguardless - if you ask it to.  Have you
tried mdadm?
   http://www.cse.unsw.edu.au/~neilb/source/mdadm/

NeilBrown

Re: RAID superblock confusion

[Mike Fedyk]

On Thu, Apr 18, 2002 at 11:54:13AM +1000, Neil Brown wrote:
> On Saturday April 13, rgooch@ras.ucalgary.ca wrote:
> > If there was only a "do as I say, regardless" mode, I would be happy.
> > This programmer-knows-best attitude smacks of M$.
> 
> mdadm will do as you say, reguardless - if you ask it to.  Have you
> tried mdadm?
>    http://www.cse.unsw.edu.au/~neilb/source/mdadm/

Niel, do you plan to merge mdadm into the raidtools package?  It sounds like
it belongs there.

Re: RAID superblock confusion

[Neil Brown]

On Wednesday April 17, mfedyk@matchmail.com wrote:
> On Thu, Apr 18, 2002 at 11:54:13AM +1000, Neil Brown wrote:
> > On Saturday April 13, rgooch@ras.ucalgary.ca wrote:
> > > If there was only a "do as I say, regardless" mode, I would be happy.
> > > This programmer-knows-best attitude smacks of M$.
> > 
> > mdadm will do as you say, reguardless - if you ask it to.  Have you
> > tried mdadm?
> >    http://www.cse.unsw.edu.au/~neilb/source/mdadm/
> 
> Niel, do you plan to merge mdadm into the raidtools package?  It sounds like
> it belongs there.

No.
If distributions want to distribute mdadm together with the stuff from
raidtools, then that is up to them.
But from a development perspective, I don't see any value in making a
single source distribution.

NeilBrown

Re: RAID superblock confusion

[Mike Fedyk]

On Thu, Apr 18, 2002 at 12:23:38PM +1000, Neil Brown wrote:
> On Wednesday April 17, mfedyk@matchmail.com wrote:
> > On Thu, Apr 18, 2002 at 11:54:13AM +1000, Neil Brown wrote:
> > > On Saturday April 13, rgooch@ras.ucalgary.ca wrote:
> > > > If there was only a "do as I say, regardless" mode, I would be happy.
> > > > This programmer-knows-best attitude smacks of M$.
> > > 
> > > mdadm will do as you say, reguardless - if you ask it to.  Have you
> > > tried mdadm?
> > >    http://www.cse.unsw.edu.au/~neilb/source/mdadm/
> > 
> > Niel, do you plan to merge mdadm into the raidtools package?  It sounds like
> > it belongs there.
> 
> No.
> If distributions want to distribute mdadm together with the stuff from
> raidtools, then that is up to them.
> But from a development perspective, I don't see any value in making a
> single source distribution.

Why's that?  Don't they compliment each other, or is mdadm a replacement?
If it's not a replacement, combining it with raidtools2 would probably get
it into distributions faster and people have already been taught with
countless how-tos that you need the raidtools package...

Hmm, checking now debian has already packaged mdadm for debian 3.0 (woody),
so raidtools just may become obsoleted so my argument is probably moot...

ehh, what's one more message on lkml anyway? ;)

Mike

Re: RAID superblock confusion

[Richard Gooch]

Neil Brown writes:
> On Wednesday April 17, mfedyk@matchmail.com wrote:
> > On Thu, Apr 18, 2002 at 11:54:13AM +1000, Neil Brown wrote:
> > > On Saturday April 13, rgooch@ras.ucalgary.ca wrote:
> > > > If there was only a "do as I say, regardless" mode, I would be happy.
> > > > This programmer-knows-best attitude smacks of M$.
> > > 
> > > mdadm will do as you say, reguardless - if you ask it to.  Have you
> > > tried mdadm?
> > >    http://www.cse.unsw.edu.au/~neilb/source/mdadm/
> > 
> > Niel, do you plan to merge mdadm into the raidtools package?  It sounds like
> > it belongs there.
> 
> No.
> If distributions want to distribute mdadm together with the stuff from
> raidtools, then that is up to them.
> But from a development perspective, I don't see any value in making a
> single source distribution.

But from a user perspective, it's a pain the in ass to have to
download yet another package to do what raidtools do, just do it
properly. And having multiple packages doing nearly the same thing is
really confusing. Users won't know which one to pick.

<rant>
Every wanker out there wants to write their own cool tool, and since
sound and graphics are both l33t, why not do both at once? The result
is that we have (to name one example) a plethora of audio mixer
control tools.
Since I'm not in the "in crowd", and I just want a tool that lets me
frob the mixer, I wasted lots of time downloading many different
tools, reading the README's and compiling ones that didn't depend on
some bloated library (glibc, KDE or Gnome). Then waste more time
finding out which ones actually worked properly.
If resources were pooled, rather than all this ego stroking, we might
have something that (most importantly) works, is lightweight and is
usable for everyone, not just the users of the One True Desktop[R].
</rant>

Please let's just have one set of MD utilities.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: RAID superblock confusion

[Luigi Genoni]

On Thu, 18 Apr 2002, Richard Gooch wrote:

> Since I'm not in the "in crowd", and I just want a tool that lets me
> frob the mixer, I wasted lots of time downloading many different
> tools, reading the README's and compiling ones that didn't depend on
> some bloated library (glibc, KDE or Gnome). Then waste more time
> finding out which ones actually worked properly.

I think you would admitt that it is quite difficoult di find a C source
code that does not depend on glibc ;).


Luigi

Re: RAID superblock confusion

[Richard Gooch]

Luigi Genoni writes:
> 
> 
> On Thu, 18 Apr 2002, Richard Gooch wrote:
> 
> > Since I'm not in the "in crowd", and I just want a tool that lets me
> > frob the mixer, I wasted lots of time downloading many different
> > tools, reading the README's and compiling ones that didn't depend on
> > some bloated library (glibc, KDE or Gnome). Then waste more time
> > finding out which ones actually worked properly.
> 
> I think you would admitt that it is quite difficoult di find a C source
> code that does not depend on glibc ;).

glibc != only libc available.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: RAID superblock confusion

[Luigi Genoni]

On Fri, 19 Apr 2002, Richard Gooch wrote:

> Luigi Genoni writes:
> >
> >
> > On Thu, 18 Apr 2002, Richard Gooch wrote:
> >
> > > Since I'm not in the "in crowd", and I just want a tool that lets me
> > > frob the mixer, I wasted lots of time downloading many different
> > > tools, reading the README's and compiling ones that didn't depend on
> > > some bloated library (glibc, KDE or Gnome). Then waste more time
> > > finding out which ones actually worked properly.
> >
> > I think you would admitt that it is quite difficoult di find a C source
> > code that does not depend on glibc ;).
>
> glibc != only libc available.
>
yes,
and I have 24 systems based on libc5, and every time I compile something
complex like gcc or XFree86 4.2.0, or something network related I can even
have an hard
time with the sources (the most of times are just minimal hacks).

Point is that actually C sources under Linux are developed 99% on glibc,
and as a result they are mostly developed to link against glibc and to be
compiled with gcc.

Luigi

p.s.
by the way, never had problems to compile linux 2.4 on those systems ;).

Re: RAID superblock confusion

[Neil Brown]

On Wednesday April 10, mfedyk@matchmail.com wrote:
> On Thu, Apr 11, 2002 at 11:38:19AM +1000, Neil Brown wrote:
> > autodetect is the other alternative.  However, as has been mentioned,
> > it does not and cannot work with md as a module.  This is because
> > devices can only be register for autodetection after md.o is loaded,
> > and autodetection is done at the time that md is loaded.  So
> > autodetection can only work if the device driver and md are loaded at
> > simultaneously.  i.e. they are compiled into the kernel.
> 
> Ahh, but if you use initrd you can even have the ide and scsi drivers as
> modules. 
> 
> What is needed is to make the disk modules depend on the raid modules (only
> if the raid code is enabled of course) so that modprobe can load the raid
> modules first. 
> 
> Then you'd just need to make sure that if there are any block modules linked
> into the kernel that raid is also linked into the kernle instead of
> a module.

Woah... I think you are going off the deep end here.  This sounds just
too complicated.

1/ If you wanted to do autodetect "right", you would make it look a
   lot like partition detection (split md into two bits. A partition
   detection personality that registers the component devices
   somewhere, and the main raid module that gets autoloaded when you
   try to actually access a raid device).
2/ Partition detection *should* be done in user-space.  So should
   autodetect. mdadm does that for you..

NeilBrown

Re: RAID superblock confusion

[Luigi Genoni]

On Thu, 11 Apr 2002, Neil Brown wrote:

> On Wednesday April 10, mfedyk@matchmail.com wrote:
> > On Thu, Apr 11, 2002 at 11:38:19AM +1000, Neil Brown wrote:
> > > autodetect is the other alternative.  However, as has been mentioned,
> > > it does not and cannot work with md as a module.  This is because
> > > devices can only be register for autodetection after md.o is loaded,
> > > and autodetection is done at the time that md is loaded.  So
> > > autodetection can only work if the device driver and md are loaded at
> > > simultaneously.  i.e. they are compiled into the kernel.
> >
> > Ahh, but if you use initrd you can even have the ide and scsi drivers as
> > modules.
> >
> > What is needed is to make the disk modules depend on the raid modules (only
> > if the raid code is enabled of course) so that modprobe can load the raid
> > modules first.
you are supposing that I load md modules and raid module together, mostly
during boot with initrd. In the reality I have some servers with more that
200 days of uptime, and I have to change external disks sometime. I do
usually have two external boxes, and something like 8/20 disks (two scsi
controllers),  and different raid on different disks. You see, it is not
that easy.

Luigi

Re: RAID superblock confusion

[Mike Fedyk]

On Thu, Apr 11, 2002 at 12:19:35PM +0200, Luigi Genoni wrote:
> > On Wednesday April 10, mfedyk@matchmail.com wrote:
> > > On Thu, Apr 11, 2002 at 11:38:19AM +1000, Neil Brown wrote:
> > > > autodetect is the other alternative.  However, as has been mentioned,
> > > > it does not and cannot work with md as a module.  This is because
> > > > devices can only be register for autodetection after md.o is loaded,
> > > > and autodetection is done at the time that md is loaded.  So
> > > > autodetection can only work if the device driver and md are loaded at
> > > > simultaneously.  i.e. they are compiled into the kernel.
> > >
> > > Ahh, but if you use initrd you can even have the ide and scsi drivers as
> > > modules.
> > >
> > > What is needed is to make the disk modules depend on the raid modules (only
> > > if the raid code is enabled of course) so that modprobe can load the raid
> > > modules first.
> you are supposing that I load md modules and raid module together, mostly

md and raid is the same and not split yet as Niel proposed.

> during boot with initrd. In the reality I have some servers with more that
> 200 days of uptime, and I have to change external disks sometime. I do
> usually have two external boxes, and something like 8/20 disks (two scsi
> controllers),  and different raid on different disks. You see, it is not
> that easy.

Currently, if you have raid compiled as modules the autodetection does not
work, and is disabled.  The issue here is enabling autodetection from the
modules, wheather that be at boot time or not.  If you use raid modules you
won't get the autodetection and the features that come with that.

When adding a new disk, you would have to add it to an array manually with
the raidtools2 anyway, so this is unchanged.

Mike

Re: RAID superblock confusion

[Neil Brown]

On Wednesday April 17, mfedyk@matchmail.com wrote:
> Why's that?  Don't they compliment each other, or is mdadm a replacement?

You should read the man page....

> 
> ehh, what's one more message on lkml anyway? ;)

Probably adds up to a person hour or so.

NeilBrown