Fwd: starting an array on Linux, advise needed

Fwd: starting an array on Linux, advise needed

[Arkadiusz Miskiewicz]

I've asked Neil about that previously but he didn't respond, so I'm asking here....

----------  Przekazana wiadomość  ----------

Subject: starting an array on Linux, advise needed
Date: pon 15. grudnia 2003 13:35
From: Arkadiusz Miskiewicz <arekm@pld-linux.org>
To: neilb@cse.unsw.edu.au

Hi,

I need smallest possible version of tool for starting md array on Linux.
 Right now I'm doing:

raid_start /dev/md0 /dev/hda1 /dev/hdb1 /dev/hdc1

and code does
fd_md = open(argv[1], O_RDWR, 0);

for (i = 2; i < argc; i++) {
       if (stat(argv[i], &st)) {
                       bb_perror_msg("error stating raid builing device
 `%s'", argv[i]);
                       continue;
        }

        if (ioctl(fd_md, START_ARRAY, (unsigned long) st.st_rdev) == -1) {
                       bb_perror_msg("error starting raid device `%s:%s'",
argv[1], argv[i]);
                       continue;
         }
         return 0;
}

return 1;

so it starts using first working device and tries to assemble array from
 other devices found in superblock.

Unfortunately there are some hard disk controllers which on 2.4
provide /dev/hdX devices while on 2.6 these are visible as /dev/sdX so even
raid_start /dev/md0 /dev/sda1 /dev/sdb1 /dev/sdc1 won't work because it will
try to use /dev/sda1 and hdb1+hdc1 (from superblock).

How simplest possible version should look like - is this below ok?

fd_md = open(argv[1], O_RDWR, 0);
ioctl(fd_md, SET_ARRAY_INFO, NULL);
for (i = 2; i < argc; i++) {
       stat(argv[i], &disk);
       ioctl(fd_md, ADD_NEW_DISK, &disk);
}
ioctl(fd_md, RUN_ARRAY, NULL);

and what if some of these ioctls fails - would be safe to do
ioctl(fd_md, STOP_ARRAY, NULL)?

Thanks for help.
--
Arkadiusz Miśkiewicz    CS at FoE, Wroclaw University of Technology
arekm.pld-linux.org AM2-6BONE, 1024/3DB19BBD, arekm(at)ircnet, PLD/Linux

-------------------------------------------------------

The other problem is that after switching from 2.4.21 to 2.6.1 I got:
FS: Mounted root (romfs filesystem) readonly.
+ insmod /lib/modumd: md driver 0.90.0 MAX_MD_DEVS=256, MD_SB_DISKS=27
les/2.6.1-1.csetraid5: measuring checksumming speed
20040116_0629smp   8regs     :   916.000 MB/sec
/kernel/drivers/   8regs_prefetch:   732.000 MB/sec
md/md.ko
Using    32regs    :   408.000 MB/sec
/lib/modules/2.6   32regs_prefetch:   400.000 MB/sec
.1-1.cset2004011   pIII_sse  :   996.000 MB/sec
6_0629smp/kernel   pII_mmx   :  1252.000 MB/sec
/drivers/md/md.k   p5_mmx    :  1308.000 MB/sec
o
+ insmod /libraid5: using function: pIII_sse (996.000 MB/sec)
/modules/2.6.1-1md: raid5 personality registered as nr 4
.cset20040116_06SCSI subsystem initialized
29smp/kernel/drivers/md/xor.ko
Using /lib/modules/2.6.1-1.cset20040116_0629smp/kernel/drivers/md/xor.ko
+ insmod /lib/modules/2.6.1-1.cset20040116_0629smp/kernel/drivers/md/raid5.ko
Using /lib/modules/2.6.1-1.cset20040116_0629smp/kernel/drivers/md/raid5.ko
+ insmod /lib/modules/2.6.1-1.cset20040116_0629smp/kernel/drivers/scsi/scsi_mod.ko
Using /lib/modules/2.6.1-1.cset20040116_0629smp/kernel/drivers/scsi/scsi_mod.ko
+ insmod /lib/modules/2.6.1-1.cset20040116_0629smp/kernel/drivers/scsi/sd_mod.ko
Using /lib/modules/2.6.1-1.cset20040116_0629smp/kernel/drivers/scsi/sd_mod.ko
+ insmod /lib/modules/2.6.1-1.cset20040116_0629smp/kernel/fs/jbd/jbd.ko
Using /lib/modules/2.6.1-1.cset20040116_0629smp/kernel/fs/jbd/jbd.ko
+ insmod /lib/modules/2.6.1-1.cset20040116_0629smp/kernel/fs/mbcache.kmd: could not lock unknown-block(8,34).
o
Using /lib/momd: could not import unknown-block(8,34)!
dules/2.6.1-1.csmd: autostart unknown-block(0,2082) failed!
et20040116_0629smd: could not lock unknown-block(8,18).
mp/kernel/fs/mbcmd: could not import unknown-block(8,18)!
ache.ko
+ insmomd: autostart unknown-block(0,2066) failed!
d /lib/modules/2md: could not lock unknown-block(8,2).
.6.1-1.cset20040md: could not import unknown-block(8,2)!
116_0629smp/kernmd: autostart unknown-block(0,2050) failed!
el/fs/ext3/ext3.ko
Using /lib/modules/2.6.1-1.cset20040116_0629smp/kernel/fs/ext3/ext3.ko
+ raid_start /dev/md0 /dev/sdc2 /dev/sdb2 /dev/sda2
raid_start: error starting raid device `/dev/md0:/dev/sdc2': Invalid argument
raid_start: error starting raid device `/dev/md0:/dev/sdb2': Invalid argument
raid_start: error starting raid device `/dev/md0:/dev/sda2': Invalid argument
romfs: unable to read superblock
EXT3-fs: unable to read superblock
Kernel panic: VFS: Unable to mount root fs on md0

(kernel messages are unfortunately mixed with messages from userspace apps)
-- 
Arkadiusz Miśkiewicz    CS at FoE, Wroclaw University of Technology
arekm.pld-linux.org AM2-6BONE, 1024/3DB19BBD, arekm(at)ircnet, PLD/Linux
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Fwd: starting an array on Linux, advise needed

[Luca Berra]

On Sat, Jan 17, 2004 at 12:46:45AM +0100, Arkadiusz Miskiewicz wrote:
>I need smallest possible version of tool for starting md array on Linux.
> Right now I'm doing:
>
i developed and use this patch to mdadm
http://cgi.cse.unsw.edu.au/~neilb/source/mdadm/patch/applied/001mdadm-1.3.0-diet.diff

-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

RE: starting an array on Linux, advise needed

[Guy]

I found this old message, it may help.
======================================
http://www.corpit.ru/mjt/mdctl.c - a very small utility similar to raidtools
suite.  I wrote this program to use it inside an initrd to bring raid arrays
up at boot time - where the size of executable and simplicity is at premium.
Differences between this implementation and raidtools are:

  - size.  When compiled with dietlibc, executable is about
    7kb on i386.

  - it does not have mkraid utility and other fun stuff.
    Supported are
    - raidstart
    - raidstop
    - raidhot{add,remove,generateerror)
    - raidsetfaulty
   i.e., only "simple" commands - just ioctls in fact.

  - it uses NO config file like all the raidtools suite.
    I.e., it is more like mdadm in this respect.  All
    parameters are accepted in the command line.  This
    makes the utility handy inside read-only boot-time
    environment (you may e.g. parse md=... kernel
    parameter and convert it into mdctl commandline)

  - `mdctl start' tries to bring an array using ALL components
    of the array given in the command line, until successeful
    (raidstart tries only first component, and if that is missing
    but other component(s) are available, operation will fail).

Usage:

  mdctl start /dev/mdN /dev/hda1 /dev/hdb1 /dev/hdc1
    equivalent to md=N,/dev/hda1,/dev/hdb1,/dev/hdc1 boot
    parameter, but you can't specify raid level (it will
    be read from the superblock), and kernel may choose to
    bring up another mdN (as indicated by super-minor)

  mdctl {stop,readonly,ro,readwrite,rw} /dev/mdN

  mdctl {add,remove,setfaulty,generaterror} /dev/mdN /dev/hda1

This tiny code released under GPL licence... in a hope it will be useful to
someone else too, not only to me ;)

Enjoy.

/mjt

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the
body of a message to majordomo@vger.kernel.org More majordomo info at
http://vger.kernel.org/majordomo-info.html

-----Original Message-----
From: linux-raid-owner@vger.kernel.org
[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Arkadiusz Miskiewicz
Sent: Friday, January 16, 2004 6:47 PM
To: linux-raid@vger.kernel.org
Subject: Fwd: starting an array on Linux, advise needed

I've asked Neil about that previously but he didn't respond, so I'm asking
here....

----------  Przekazana wiadomość  ----------

Subject: starting an array on Linux, advise needed
Date: pon 15. grudnia 2003 13:35
From: Arkadiusz Miskiewicz <arekm@pld-linux.org>
To: neilb@cse.unsw.edu.au

Hi,

I need smallest possible version of tool for starting md array on Linux.
 Right now I'm doing:

raid_start /dev/md0 /dev/hda1 /dev/hdb1 /dev/hdc1

and code does
fd_md = open(argv[1], O_RDWR, 0);

for (i = 2; i < argc; i++) {
       if (stat(argv[i], &st)) {
                       bb_perror_msg("error stating raid builing device
 `%s'", argv[i]);
                       continue;
        }

        if (ioctl(fd_md, START_ARRAY, (unsigned long) st.st_rdev) == -1) {
                       bb_perror_msg("error starting raid device `%s:%s'",
argv[1], argv[i]);
                       continue;
         }
         return 0;
}

return 1;

so it starts using first working device and tries to assemble array from
 other devices found in superblock.

Unfortunately there are some hard disk controllers which on 2.4
provide /dev/hdX devices while on 2.6 these are visible as /dev/sdX so even
raid_start /dev/md0 /dev/sda1 /dev/sdb1 /dev/sdc1 won't work because it will
try to use /dev/sda1 and hdb1+hdc1 (from superblock).

How simplest possible version should look like - is this below ok?

fd_md = open(argv[1], O_RDWR, 0);
ioctl(fd_md, SET_ARRAY_INFO, NULL);
for (i = 2; i < argc; i++) {
       stat(argv[i], &disk);
       ioctl(fd_md, ADD_NEW_DISK, &disk);
}
ioctl(fd_md, RUN_ARRAY, NULL);

and what if some of these ioctls fails - would be safe to do
ioctl(fd_md, STOP_ARRAY, NULL)?

Thanks for help.
--
Arkadiusz Miśkiewicz    CS at FoE, Wroclaw University of Technology
arekm.pld-linux.org AM2-6BONE, 1024/3DB19BBD, arekm(at)ircnet, PLD/Linux

-------------------------------------------------------

The other problem is that after switching from 2.4.21 to 2.6.1 I got:
FS: Mounted root (romfs filesystem) readonly.
+ insmod /lib/modumd: md driver 0.90.0 MAX_MD_DEVS=256, MD_SB_DISKS=27
les/2.6.1-1.csetraid5: measuring checksumming speed
20040116_0629smp   8regs     :   916.000 MB/sec
/kernel/drivers/   8regs_prefetch:   732.000 MB/sec
md/md.ko
Using    32regs    :   408.000 MB/sec
/lib/modules/2.6   32regs_prefetch:   400.000 MB/sec
.1-1.cset2004011   pIII_sse  :   996.000 MB/sec
6_0629smp/kernel   pII_mmx   :  1252.000 MB/sec
/drivers/md/md.k   p5_mmx    :  1308.000 MB/sec
o
+ insmod /libraid5: using function: pIII_sse (996.000 MB/sec)
/modules/2.6.1-1md: raid5 personality registered as nr 4
.cset20040116_06SCSI subsystem initialized
29smp/kernel/drivers/md/xor.ko
Using /lib/modules/2.6.1-1.cset20040116_0629smp/kernel/drivers/md/xor.ko
+ insmod
/lib/modules/2.6.1-1.cset20040116_0629smp/kernel/drivers/md/raid5.ko
Using /lib/modules/2.6.1-1.cset20040116_0629smp/kernel/drivers/md/raid5.ko
+ insmod
/lib/modules/2.6.1-1.cset20040116_0629smp/kernel/drivers/scsi/scsi_mod.ko
Using
/lib/modules/2.6.1-1.cset20040116_0629smp/kernel/drivers/scsi/scsi_mod.ko
+ insmod
/lib/modules/2.6.1-1.cset20040116_0629smp/kernel/drivers/scsi/sd_mod.ko
Using
/lib/modules/2.6.1-1.cset20040116_0629smp/kernel/drivers/scsi/sd_mod.ko
+ insmod /lib/modules/2.6.1-1.cset20040116_0629smp/kernel/fs/jbd/jbd.ko
Using /lib/modules/2.6.1-1.cset20040116_0629smp/kernel/fs/jbd/jbd.ko
+ insmod /lib/modules/2.6.1-1.cset20040116_0629smp/kernel/fs/mbcache.kmd:
could not lock unknown-block(8,34).
o
Using /lib/momd: could not import unknown-block(8,34)!
dules/2.6.1-1.csmd: autostart unknown-block(0,2082) failed!
et20040116_0629smd: could not lock unknown-block(8,18).
mp/kernel/fs/mbcmd: could not import unknown-block(8,18)!
ache.ko
+ insmomd: autostart unknown-block(0,2066) failed!
d /lib/modules/2md: could not lock unknown-block(8,2).
.6.1-1.cset20040md: could not import unknown-block(8,2)!
116_0629smp/kernmd: autostart unknown-block(0,2050) failed!
el/fs/ext3/ext3.ko
Using /lib/modules/2.6.1-1.cset20040116_0629smp/kernel/fs/ext3/ext3.ko
+ raid_start /dev/md0 /dev/sdc2 /dev/sdb2 /dev/sda2
raid_start: error starting raid device `/dev/md0:/dev/sdc2': Invalid
argument
raid_start: error starting raid device `/dev/md0:/dev/sdb2': Invalid
argument
raid_start: error starting raid device `/dev/md0:/dev/sda2': Invalid
argument
romfs: unable to read superblock
EXT3-fs: unable to read superblock
Kernel panic: VFS: Unable to mount root fs on md0

(kernel messages are unfortunately mixed with messages from userspace apps)
-- 
Arkadiusz Miśkiewicz    CS at FoE, Wroclaw University of Technology
arekm.pld-linux.org AM2-6BONE, 1024/3DB19BBD, arekm(at)ircnet, PLD/Linux
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: starting an array on Linux, advise needed

[Michael Tokarev]

Guy wrote:
> I found this old message, it may help.
> ======================================
> http://www.corpit.ru/mjt/mdctl.c - a very small utility similar to raidtools

Note it will not work correctly in the situation described.
Or at least it may not work.  For the same reason as original
code does not work: "renamed" devices in 2.4 vs 2.6.

With raidstart (and this utility follows it logic), it is the
kernel who is reading the raid superblock, opening devices
and assembles the array.  Most important part is the superblock:
kernel will pick up any devices found there (by major/minor
number).  When booting 2.4 and 2.6, or when your devices are
using dynamically-assigned numbers, that numbers may be different
on next reboot, so kernel will not be able to find all required
devices.

While mdadm tries to examine all superblocks and find all
devices in between the ones listed on command line, in the
superblock or ones that are really present on your system
at any given time (--scan; depends on your mdadm.conf also).

So when you expect your device names/numbers to be instable
between reboots, it's best to use mdadm together with --scan
and proper mdadm.conf or with all possible devices listed in
command line.  Mdassemble by Luca Berra will be useful here
too.

/mjt

P.S.  Please trim original message somehow when replying.

Re: starting an array on Linux, advise needed

[Michael]

> Guy wrote:
> > I found this old message, it may help.
> > ======================================
> > http://www.corpit.ru/mjt/mdctl.c - a very small utility similar to raidtools
> 
> Note it will not work correctly in the situation described.
> Or at least it may not work.  For the same reason as original
> code does not work: "renamed" devices in 2.4 vs 2.6.

<snip>

I've been following this thread for a while and have now become a bit 
confused. What are the implications for an autostart array marked 
'FD' in 2.6 ?? Will it start correctly? hmmm.... I think I missed 
something in the discussion..... How does one specify the raidtab 
under this circumstance if the device names are not stable.

Michael
Michael@Insulin-Pumpers.org

RE: starting an array on Linux, advise needed

[Guy]

raidtab? 
Your sad devotion to that ancient software has not helped you conjure up the
superblocks.

Read:
	man mdadm

I have this line in my mdadm.conf:
	DEVICE /dev/sd[abcdefghijklmnopqrstuvwxyz][12]
It handles the first 2 partitions of the first 26 SCSI disks!
I only have 17 disks, most only have 1 partition, some have 2.


-----Original Message-----
From: linux-raid-owner@vger.kernel.org
[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Michael
Sent: Friday, January 16, 2004 7:48 PM
To: linux-raid@vger.kernel.org
Subject: Re: starting an array on Linux, advise needed

> Guy wrote:
> > I found this old message, it may help.
> > ======================================
> > http://www.corpit.ru/mjt/mdctl.c - a very small utility similar to
raidtools
> 
> Note it will not work correctly in the situation described.
> Or at least it may not work.  For the same reason as original
> code does not work: "renamed" devices in 2.4 vs 2.6.

<snip>

I've been following this thread for a while and have now become a bit 
confused. What are the implications for an autostart array marked 
'FD' in 2.6 ?? Will it start correctly? hmmm.... I think I missed 
something in the discussion..... How does one specify the raidtab 
under this circumstance if the device names are not stable.

Michael
Michael@Insulin-Pumpers.org
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html