Raid 5 questions

Raid 5 questions

[David Greaves]

Hi

I've perused the archives and read the docs but I'm not sure on this 
point :)

If I create an array with 3 250Gb disks and use the 'missing' option I 
should get a degraded raid5 array with 750Gb
Now, when another 250Gb disk arrives I want to add that as the missing 
disk - will that work?

Now, since I actually have the 3 disks and I'm waiting for the 4th, lets 
try it with 2 and add a third...

Clear out the superblocks so we're starting from scratch
cu:~# mdadm --stop /dev/md0
cu:~# mdadm --zero-superblock /dev/sda
cu:~# mdadm --zero-superblock /dev/sdb
cu:~# mdadm --zero-superblock /dev/sdc

Make the array.
cu:~# mdadm -v -C /dev/md0 --chunk=128 --level=5 --raid-devices=3 
/dev/sda /dev/sdb missing
mdadm: layout defaults to left-symmetric
mdadm: size set to 245117312K
mdadm: array /dev/md0 started.

What have we got?
cu:~# mdadm --detail /dev/md0
/dev/md0:
        Version : 00.90.01
  Creation Time : Thu Jun  3 16:19:26 2004
     Raid Level : raid5
     Array Size : 490234624 (467.52 GiB 502.00 GB)
    Device Size : 245117312 (233.76 GiB 251.00 GB)
   Raid Devices : 3
  Total Devices : 2
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Thu Jun  3 16:19:26 2004
          State : clean, no-errors
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 128K

    Number   Major   Minor   RaidDevice State
       0       8        0        0      active sync   /dev/sda
       1       8       16        1      active sync   /dev/sdb
       2       0        0       -1      removed
           UUID : 8acbe990:b31287f7:32888f7d:f0edcf0c
         Events : 0.3887

Thats 2x 251Gb giving a 502Gb array - good

Add this to the config
cu:~# echo 'DEVICE /dev/sd[abcd]' >/etc/mdadm/mdadm.conf
cu:~# mdadm --detail --scan >>/etc/mdadm/mdadm.conf
cu:~# cat /etc/mdadm/mdadm.conf
DEVICE /dev/sd[abcd]
ARRAY /dev/md0 level=raid5 num-devices=3 
UUID=8acbe990:b31287f7:32888f7d:f0edcf0c
   devices=/dev/sda,/dev/sdb

Now, the new disk arrives...
cu:~# mdadm  /dev/md0 -a /dev/sdc
mdadm: hot added /dev/sdc

So it should be working fine...
cu:~# mdadm --detail /dev/md0
/dev/md0:
        Version : 00.90.01
  Creation Time : Thu Jun  3 16:19:26 2004
     Raid Level : raid5
     Array Size : 490234624 (467.52 GiB 502.00 GB)
    Device Size : 245117312 (233.76 GiB 251.00 GB)
   Raid Devices : 3
  Total Devices : 3
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Thu Jun  3 16:23:21 2004
          State : clean, no-errors
 Active Devices : 2
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 1

         Layout : left-symmetric
     Chunk Size : 128K

 Rebuild Status : 0% complete

    Number   Major   Minor   RaidDevice State
       0       8        0        0      active sync   /dev/sda
       1       8       16        1      active sync   /dev/sdb
       2       0        0       -1      removed
       3       8       32        2      spare   /dev/sdc
           UUID : 8acbe990:b31287f7:32888f7d:f0edcf0c
         Events : 0.3888


Nope it's a spare?
Help - how do I make my array resilient?


Environment:
Debian testing
Linux cu.dgreaves.com 2.6.6 #1 Thu Jun 3 14:12:01 BST 2004 i686 GNU/Linux
mdadm - v1.5.0 - 22 Jan 2004

Thanks

David

RE: Raid 5 questions

[Guy]

Did you notice:
	Rebuild Status : 0% complete

Do this:
	cat /proc/mdstat

Guy

-----Original Message-----
From: linux-raid-owner@vger.kernel.org
[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of David Greaves
Sent: Thursday, June 03, 2004 10:30 AM
To: linux-raid@vger.kernel.org
Subject: Raid 5 questions

Hi

I've perused the archives and read the docs but I'm not sure on this 
point :)

If I create an array with 3 250Gb disks and use the 'missing' option I 
should get a degraded raid5 array with 750Gb
Now, when another 250Gb disk arrives I want to add that as the missing 
disk - will that work?

Now, since I actually have the 3 disks and I'm waiting for the 4th, lets 
try it with 2 and add a third...

Clear out the superblocks so we're starting from scratch
cu:~# mdadm --stop /dev/md0
cu:~# mdadm --zero-superblock /dev/sda
cu:~# mdadm --zero-superblock /dev/sdb
cu:~# mdadm --zero-superblock /dev/sdc

Make the array.
cu:~# mdadm -v -C /dev/md0 --chunk=128 --level=5 --raid-devices=3 
/dev/sda /dev/sdb missing
mdadm: layout defaults to left-symmetric
mdadm: size set to 245117312K
mdadm: array /dev/md0 started.

What have we got?
cu:~# mdadm --detail /dev/md0
/dev/md0:
        Version : 00.90.01
  Creation Time : Thu Jun  3 16:19:26 2004
     Raid Level : raid5
     Array Size : 490234624 (467.52 GiB 502.00 GB)
    Device Size : 245117312 (233.76 GiB 251.00 GB)
   Raid Devices : 3
  Total Devices : 2
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Thu Jun  3 16:19:26 2004
          State : clean, no-errors
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 128K

    Number   Major   Minor   RaidDevice State
       0       8        0        0      active sync   /dev/sda
       1       8       16        1      active sync   /dev/sdb
       2       0        0       -1      removed
           UUID : 8acbe990:b31287f7:32888f7d:f0edcf0c
         Events : 0.3887

Thats 2x 251Gb giving a 502Gb array - good

Add this to the config
cu:~# echo 'DEVICE /dev/sd[abcd]' >/etc/mdadm/mdadm.conf
cu:~# mdadm --detail --scan >>/etc/mdadm/mdadm.conf
cu:~# cat /etc/mdadm/mdadm.conf
DEVICE /dev/sd[abcd]
ARRAY /dev/md0 level=raid5 num-devices=3 
UUID=8acbe990:b31287f7:32888f7d:f0edcf0c
   devices=/dev/sda,/dev/sdb

Now, the new disk arrives...
cu:~# mdadm  /dev/md0 -a /dev/sdc
mdadm: hot added /dev/sdc

So it should be working fine...
cu:~# mdadm --detail /dev/md0
/dev/md0:
        Version : 00.90.01
  Creation Time : Thu Jun  3 16:19:26 2004
     Raid Level : raid5
     Array Size : 490234624 (467.52 GiB 502.00 GB)
    Device Size : 245117312 (233.76 GiB 251.00 GB)
   Raid Devices : 3
  Total Devices : 3
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Thu Jun  3 16:23:21 2004
          State : clean, no-errors
 Active Devices : 2
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 1

         Layout : left-symmetric
     Chunk Size : 128K

 Rebuild Status : 0% complete

    Number   Major   Minor   RaidDevice State
       0       8        0        0      active sync   /dev/sda
       1       8       16        1      active sync   /dev/sdb
       2       0        0       -1      removed
       3       8       32        2      spare   /dev/sdc
           UUID : 8acbe990:b31287f7:32888f7d:f0edcf0c
         Events : 0.3888


Nope it's a spare?
Help - how do I make my array resilient?


Environment:
Debian testing
Linux cu.dgreaves.com 2.6.6 #1 Thu Jun 3 14:12:01 BST 2004 i686 GNU/Linux
mdadm - v1.5.0 - 22 Jan 2004

Thanks

David
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Raid 5 questions

[David Greaves]

Answering my own post:
<snip>
 > Rebuild Status : 0% complete
<snip>

Yeah, ok, I'm dim :)

Maybe the output could be a little more helpful? State = rebuilding?

and for anyone else, cat /proc/mdstat gave me the hint!!

Other things:
1)
in Manage mode it's documented to allow multiple commands and yet:

cu:~# mdadm  /dev/md0 -f /dev/sdc1 -r /dev/sdc1
mdadm: set /dev/sdc1 faulty in /dev/md0
mdadm: hot remove failed for /dev/sdc1: Device or resource busy
cu:~# mdadm  /dev/md0 -r /dev/sdc1
mdadm: hot removed /dev/sdc1


2)
To allow autodetection on startup you're supposed to set the partition 
type to 0xfd
What if you're using whole disks? Are you supposed to create massive 
partitions?

3)
I'm having some odd responses from the Monitor mode:

Here is the system:

cu:~# mdadm --detail /dev/md0
/dev/md0:
       Version : 00.90.01
 Creation Time : Thu Jun  3 17:03:11 2004
    Raid Level : raid5
    Array Size : 1975552 (1.88 GiB 2.02 GB)
   Device Size : 987776 (964.63 MiB 1011.48 MB)
  Raid Devices : 3
 Total Devices : 4
Preferred Minor : 0
   Persistence : Superblock is persistent

   Update Time : Thu Jun  3 17:32:45 2004
         State : clean, no-errors
Active Devices : 3
Working Devices : 4
Failed Devices : 0
 Spare Devices : 1

        Layout : left-symmetric
    Chunk Size : 128K

   Number   Major   Minor   RaidDevice State
      0       8        1        0      active sync   /dev/sda1
      1       8       17        1      active sync   /dev/sdb1
      2       8       33        2      active sync   /dev/sdc1
      3       8        2       -1      spare   /dev/sda2
          UUID : 9120f538:88dcabef:ba6f5f75:d2ce9e00
        Events : 0.3904

I run the monitor using a PROGRAM to ensure I see all events (#'s below 
are annotations :) )
cu:~# mdadm --monitor -d 1 -t /dev/md0
mdadm: Monitor using program "/bin/echo" from config file
TestMessage /dev/md0
# -f a device at this point
Fail /dev/md0
# Note that the device is not reported.
# A bit later (not long now I'm using 1Gb test partitions!)
SpareActive /dev/md0 /dev/sda2
# Note there were no RebuildStarted, RebuildNN, RebuildFinished events?
# -r the device - no events
# -a the device - no events
# mdadm --stop /dev/md0 here
DeviceDisappeared /dev/md0

Also stopping the monitor and degrading the array (-f a device) followed 
by a monitor restart doesn't give a DegradedArray event.


David

RE: Raid 5 questions

[Guy]

To allow autodetection on startup:
	Yes, 1 big partition.

Monitor:
	Need MAILADDR and/or PROGRAM keywords in /etc/mdadm.conf
	Example:
		MAILADDR  bugzilla@watkins-home.com
	And/or
		PROGRAM /root/bin/handle-mdadm-events

handle-mdadm-events is a script I created.  The default messages did not
include enough info for me.  And if I recall, without "PROGRAM" the event
types are limited to just a few.

For the most part the script looks like this:
(date;cat /proc/mdstat;mdadm --detail $2)|mail -s "md event: $1 $2 $3"
bugzilla@watkins-home.com

Guy


-----Original Message-----
From: linux-raid-owner@vger.kernel.org
[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of David Greaves
Sent: Thursday, June 03, 2004 11:51 AM
To: linux-raid@vger.kernel.org
Subject: Re: Raid 5 questions

Answering my own post:
<snip>
 > Rebuild Status : 0% complete
<snip>

Yeah, ok, I'm dim :)

Maybe the output could be a little more helpful? State = rebuilding?

and for anyone else, cat /proc/mdstat gave me the hint!!

Other things:
1)
in Manage mode it's documented to allow multiple commands and yet:

cu:~# mdadm  /dev/md0 -f /dev/sdc1 -r /dev/sdc1
mdadm: set /dev/sdc1 faulty in /dev/md0
mdadm: hot remove failed for /dev/sdc1: Device or resource busy
cu:~# mdadm  /dev/md0 -r /dev/sdc1
mdadm: hot removed /dev/sdc1


2)
To allow autodetection on startup you're supposed to set the partition 
type to 0xfd
What if you're using whole disks? Are you supposed to create massive 
partitions?

3)
I'm having some odd responses from the Monitor mode:

Here is the system:

cu:~# mdadm --detail /dev/md0
/dev/md0:
       Version : 00.90.01
 Creation Time : Thu Jun  3 17:03:11 2004
    Raid Level : raid5
    Array Size : 1975552 (1.88 GiB 2.02 GB)
   Device Size : 987776 (964.63 MiB 1011.48 MB)
  Raid Devices : 3
 Total Devices : 4
Preferred Minor : 0
   Persistence : Superblock is persistent

   Update Time : Thu Jun  3 17:32:45 2004
         State : clean, no-errors
Active Devices : 3
Working Devices : 4
Failed Devices : 0
 Spare Devices : 1

        Layout : left-symmetric
    Chunk Size : 128K

   Number   Major   Minor   RaidDevice State
      0       8        1        0      active sync   /dev/sda1
      1       8       17        1      active sync   /dev/sdb1
      2       8       33        2      active sync   /dev/sdc1
      3       8        2       -1      spare   /dev/sda2
          UUID : 9120f538:88dcabef:ba6f5f75:d2ce9e00
        Events : 0.3904

I run the monitor using a PROGRAM to ensure I see all events (#'s below 
are annotations :) )
cu:~# mdadm --monitor -d 1 -t /dev/md0
mdadm: Monitor using program "/bin/echo" from config file
TestMessage /dev/md0
# -f a device at this point
Fail /dev/md0
# Note that the device is not reported.
# A bit later (not long now I'm using 1Gb test partitions!)
SpareActive /dev/md0 /dev/sda2
# Note there were no RebuildStarted, RebuildNN, RebuildFinished events?
# -r the device - no events
# -a the device - no events
# mdadm --stop /dev/md0 here
DeviceDisappeared /dev/md0

Also stopping the monitor and degrading the array (-f a device) followed 
by a monitor restart doesn't give a DegradedArray event.


David

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RAID 5 questions

[Ninti Systems]

I'm about to embark on building a RAID 5 Linux software array with 4
disks (I've only ever built 2 disk RAID 1 arrays before).

Q1: Is it true that Linux can't/shouldn't boot off a RAID 5 array?

Q2: The four disks have identical manufacturer, model and size, but two
are brand new while two have had NTFS filesystems on them. Accordingly,
if I run hdparm -g on them, two disks have these specs:

geometry = 2498/255/63, sectors = 40132503, start = 0

while the other two disks have these specs:

geometry = 39813/16/63, sectors = 40132503, start = 0


Is this an issue with software RAID? 

Thanks
Michael Hall

Re: RAID 5 questions

[Daniel Pittman]

On 1 Aug 2004, Ninti Systems wrote:
> I'm about to embark on building a RAID 5 Linux software array with 4
> disks (I've only ever built 2 disk RAID 1 arrays before).
>
> Q1: Is it true that Linux can't/shouldn't boot off a RAID 5 array?

Grub certainly can't boot of a software RAID-5 array, and I don't
believe that lilo can either.  This is because of these tools not
understanding the RAID-5 disk layout, not any other limitation.

> Q2: The four disks have identical manufacturer, model and size, but two
> are brand new while two have had NTFS filesystems on them. Accordingly,
> if I run hdparm -g on them, two disks have these specs:
>
> geometry = 2498/255/63, sectors = 40132503, start = 0
>
> while the other two disks have these specs:
>
> geometry = 39813/16/63, sectors = 40132503, start = 0
>
> Is this an issue with software RAID?

No. Software RAID, unlike hardware raid, cares only that you have enough
blocks on the disk. You don't need to match *any* other characteristic
of the disks, and not even that if you don't mind wasting a few bytes.

   Daniel
-- 
The PC is the LSD of the '90s.
        -- Timothy Leary, __Guardian_ (June 1, 1996)

Re: RAID 5 questions

[Jim Paris]

> Q1: Is it true that Linux can't/shouldn't boot off a RAID 5 array?

LILO does not support booting off of RAID 5, and I don't think GRUB
does either.  I usually make a small RAID-1 partition to boot from
(mirrored across all four drives), since LILO does handle that just
fine.

> geometry = 2498/255/63, sectors = 40132503, start = 0
..
> geometry = 39813/16/63, sectors = 40132503, start = 0
> Is this an issue with software RAID? 

No.

-jim

Re: RAID 5 questions

[Alvin Oga]

On Sat, 31 Jul 2004, Jim Paris wrote:

> > geometry = 2498/255/63, sectors = 40132503, start = 0
> ..
> > geometry = 39813/16/63, sectors = 40132503, start = 0
> > Is this an issue with software RAID? 

no but it'd be better/faster/easier to put 1 disk per ide cable
	- its $20 or less to get a promise ide controller

	- or if you're brave, to disable the onboard raid 
	in the bios and use its connectors as regular ide

1 disk per ide cable if you can ...

c ya
alvin

Re: RAID 5 questions

[Luca Berra]

On Sun, Aug 01, 2004 at 11:27:56AM +0930, Ninti Systems wrote:
>Q1: Is it true that Linux can't/shouldn't boot off a RAID 5 array?
it cannot no boot loader can load the kernel from a striped (or raid
3/4/5/6) device
create a small raid 1 array spanning the 4 disks and use it for /boot

>Q2: The four disks have identical manufacturer, model and size, but two
>are brand new while two have had NTFS filesystems on them. Accordingly,
>if I run hdparm -g on them, two disks have these specs:
>
>geometry = 2498/255/63, sectors = 40132503, start = 0
...
>geometry = 39813/16/63, sectors = 40132503, start = 0
>
>Is this an issue with software RAID? 
no, nevertheless if you wipe the partition table on the to 'used' disks
linux will see them just like the 'new' ones.

-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: RAID 5 questions

[Frank van Maarseveen]

On Sun, Aug 01, 2004 at 11:27:56AM +0930, Ninti Systems wrote:
> 
> Q1: Is it true that Linux can't/shouldn't boot off a RAID 5 array?

From software RAID it can't.

USB storage devices (memory stick) are incredibly useful for this --
I've even booted a webserver from a digital camera ;-). The only catch
may be the latency of the USB code when you don't want to use initrd:
The kernel panics (no root dev) because it takes noticeable time before
the kernel sees an USB storage device. But there are several tiny patches
for this, it's merely a nuisance.

-- 
Frank

Re: RAID 5 questions

[Jarmo Järvenpää]

Umm, I maybe writing on complitely different setup (can't check the 
original post), but our server with software raid 5 (3 disks) is booting 
with lilo just fine.


BR,
Jarmo


Jim Paris wrote:

>>Q1: Is it true that Linux can't/shouldn't boot off a RAID 5 array?
> 
> 
> LILO does not support booting off of RAID 5, and I don't think GRUB
> does either.  I usually make a small RAID-1 partition to boot from
> (mirrored across all four drives), since LILO does handle that just
> fine.
> 
> 
>>geometry = 2498/255/63, sectors = 40132503, start = 0
> 
> ..
> 
>>geometry = 39813/16/63, sectors = 40132503, start = 0
>>Is this an issue with software RAID? 
> 
> 
> No.
> 
> -jim
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RAID 5 questions

[Luca Berra]

On Mon, Aug 02, 2004 at 09:01:14AM +0300, Jarmo Järvenpää wrote:
>Umm, I maybe writing on complitely different setup (can't check the 
>original post), but our server with software raid 5 (3 disks) is booting 
>with lilo just fine.
check your setup, but your kernel is not on a raid 5 device.

L.

-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RAID 5 questions

[Jim Paris]

> Umm, I maybe writing on complitely different setup (can't check the 
> original post), but our server with software raid 5 (3 disks) is booting 
> with lilo just fine.

README.raid from lilo-22.5.9 (the latest version) states:

"Only RAID1 is supported.  LILO may be used to boot a system
containing other RAID level partitions, but it may not be installed
on any RAID partition other than RAID level 1."

so I don't believe you.  You may be confusing "booting off RAID-5"
with "having a RAID-5 root".

-jim

Re: RAID 5 questions

[Jarmo Järvenpää]

Hmm, we have 3 SW raid5 devices (md0, md1 and md3), with kernel on md0. 
Lilo.conf has root and boot=/dev/md0. Lilo v22.3.1.




Luca Berra wrote:

> On Mon, Aug 02, 2004 at 09:01:14AM +0300, Jarmo Järvenpää wrote:
> 
>> Umm, I maybe writing on complitely different setup (can't check the 
>> original post), but our server with software raid 5 (3 disks) is 
>> booting with lilo just fine.
> 
> check your setup, but your kernel is not on a raid 5 device.
> 
> L.
> 

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RAID 5 questions

[Jarmo Järvenpää]

Jim Paris wrote:

> README.raid from lilo-22.5.9 (the latest version) states:
> 
> "Only RAID1 is supported.  LILO may be used to boot a system
> containing other RAID level partitions, but it may not be installed
> on any RAID partition other than RAID level 1."
> 
> so I don't believe you.  You may be confusing "booting off RAID-5"
> with "having a RAID-5 root".

Hmm, I'm not sure how to convince you :-)

The only possibility I can think of is that the lilos.conf's 
raid-extra-boot= -line affects the booting, thus it's silently booting 
from /dev/hdX and not from /dev/md0. I'm not sure how to test this though.


BR,
Jarmo

Re: RAID 5 questions

[misty-]

On Sun, Aug 01, 2004 at 12:24:30PM +0200, Frank van Maarseveen wrote:
> On Sun, Aug 01, 2004 at 11:27:56AM +0930, Ninti Systems wrote:
> > 
> > Q1: Is it true that Linux can't/shouldn't boot off a RAID 5 array?
> 
> >From software RAID it can't.
> 
> USB storage devices (memory stick) are incredibly useful for this --
> I've even booted a webserver from a digital camera ;-). The only catch
> may be the latency of the USB code when you don't want to use initrd:
> The kernel panics (no root dev) because it takes noticeable time before
> the kernel sees an USB storage device. But there are several tiny patches
> for this, it's merely a nuisance.

Personally I use sw raid5 as my /, and as I messed up and now can't fix it
to work with a /boot, I instead use a lilo'ed 3.5 inch bootdisk, which loads
the kernel and mounts root. *shrugs* it works fairly well, I can't complain
much.

Tim McGrath

Re: RAID 5 questions

[Frank van Maarseveen]

On Mon, Aug 02, 2004 at 10:27:02AM +0300, Jarmo Järvenpää wrote:
> Jim Paris wrote:
> 
> >README.raid from lilo-22.5.9 (the latest version) states:
> >
> >"Only RAID1 is supported.  LILO may be used to boot a system

yep. and even that may go wrong.

> Hmm, I'm not sure how to convince you :-)

try running lilo with -v -v -v

Then you might see the BIOS drive number, geometry it assumes etc. You
will probably see only _one_ disk and in that case your system boots
by sheer luck.

With -v -v -v I discovered why my system refused to boot from raid 1.
It appears to actually boot from the first disk in the raid set, e.g:

	md1 : active raid1 hdg1[1] hda1[0]
				   ^^^^^^^

This is mounted on /boot and lilo will boot from hda1 (bios disk 0x80).
I had to reassemble the raid set in opposite order to get this because
hdg is unbootable (lilo assumed bios drive 0x81). But maybe my lilo is
old (21.4-4).

-- 
Frank
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RAID 5 questions

[Tim Small]

Frank van Maarseveen wrote:

>try running lilo with -v -v -v
>
>Then you might see the BIOS drive number, geometry it assumes etc. You
>will probably see only _one_ disk and in that case your system boots
>by sheer luck.
>
>With -v -v -v I discovered why my system refused to boot from raid 1.
>It appears to actually boot from the first disk in the raid set, e.g:
>
>	md1 : active raid1 hdg1[1] hda1[0]
>				   ^^^^^^^
>
>This is mounted on /boot and lilo will boot from hda1 (bios disk 0x80).
>I had to reassemble the raid set in opposite order to get this because
>hdg is unbootable (lilo assumed bios drive 0x81). But maybe my lilo is
>old (21.4-4).
>
>  
>
You can do this sort of thing to override lilo's idea of which linux 
device maps to which BIOS disk number e.g. I use this on one box:

disk=/dev/hda
        bios=0x80
boot=/dev/hda

#disk=/dev/hdf
#       bios=0x80
#boot=/dev/hdf

# Specifies the device that should be mounted as root. (`/')
#
root=/dev/md3


I run lilo twice (once with the hda lines uncommented, and once with the 
hdf lines uncommented) - so that the system will still boot with hda 
pulled from the machine - with hda broken, the machine still won't boot, 
but at least you can disconnect hda at that point, or disable it in the 
BIOS (so that hdf becomes 0x80)...

You can do a similar thing with grub, but with the advantage that you 
don't have to play around with the config file every time you upgrade 
the kernel (you could use two different lilo.conf's I suppose)...

I'm not sure if this has already been stated (Frank just implied it), 
but a simple workaround to the original problem is to make a small RAID1 
/boot partition on a couple of the disks, and have the rest of the 
drives carved up as RAID5, and I think that if you're using lilo to load 
the kernel off RAID5, then you're being lucky (AFAIK).

e.g. I sometimes do this:

drive1   drive2  drive3   drive4
[ boot(raid1) ]  [ swap(raid1) ]
[     more-swap (raid5)        ]
[        /root  (raid5)        ]
[        /other (raid5)        ]


Tim.

Re: RAID 5 questions

[H. Peter Anvin]

Followup to:  <20040801102430.GC28625@janus>
By author:    Frank van Maarseveen <frankvm@xs4all.nl>
In newsgroup: linux.dev.raid
>
> On Sun, Aug 01, 2004 at 11:27:56AM +0930, Ninti Systems wrote:
> > 
> > Q1: Is it true that Linux can't/shouldn't boot off a RAID 5 array?
> 
> From software RAID it can't.
> 

Actually, it should be able to if the firmware can see all the drives,
AND you're not running it in degraded mode.  Obviously not a good
solution.  Use RAID-1 for /boot.

	-hpa