RE: Booting from a raid1 device ?

RE: Booting from a raid1 device ?

[Cress, Andrew R]

Don,

Yes, it certainly can be done.  I do it all the time.

It's not hard with lilo, but grub doesn't support raid/md boot devices
yet.  This means that the grub boot record doesn't automatically get
written to the second disk.  In short, choose lilo for root mirroring if
possible.

The procedure is simplest if you use the distro installer to configure
it, but an existing installation can also be converted to root mirror
(raid1).  
See http://www.tldp.org/HOWTO/Software-RAID-HOWTO-4.html  (sections 4.11
thru 4.14)
or http://scsirastools.sourceforge.net/docs/UserGuide (section 4.0)

The md module MUST be staticly linked, not a module
(CONFIG_BLK_DEV_MD=y).  The other pieces (raid1.o, etc.) can be modules
if included in the initrd image.  The partitions should be marked as
raid autodetect type (0xfd).  

To be fully redundant, the '/boot', '/' and swap partitions should each
be mirrored.

Andy

-----Original Message-----
From: linux-raid-owner@vger.kernel.org
[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Paul Clements
Sent: Monday, October 27, 2003 11:01 AM
To: donj@asaca.com
Cc: linux-raid@vger.kernel.org
Subject: Re: Booting from a raid1 device ?


Don Jessup wrote:
> 
>   Is it possible to boot from a raid1 device with the md.o built as a
>  2.4.22 kernel module?

Yes, but it's tricky. You'd have to load md.o and raid1.o in your initrd
(or other early boot time initialization) and then use some, e.g.,
statically-linked mdadm tool on that initrd to start the array. You
cannot, however, use md's RAID autodetect to autostart arrays that have
their component partitions marked as RAID autodetect (0xfd), if md is a
module.

--
Paul


> =====
> Don Jessup
> Asaca/Shibasoku Corp. of America
> 400 Corporate Circle, Unit G
> Golden, CO  80401
> 303-278-1111 X232
> donj@asaca.com
> http://www.asaca.com
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid"
in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Booting from a raid1 device ?

[Michael Tokarev]

Cress, Andrew R wrote:
> Don,
> 
> Yes, it certainly can be done.  I do it all the time.
> 
> It's not hard with lilo, but grub doesn't support raid/md boot devices
> yet.  This means that the grub boot record doesn't automatically get
> written to the second disk.  In short, choose lilo for root mirroring if
> possible.

Ugh-blah.  Excuse me folks, but.. maybe someone will be able to
answer this one:

  why the hell a boot loader should "mirror" anything to the second disk? :)

That is.  Place standard MBR into the boot sector.  An MBR that will boot
from active partition.  Be it e.g. /dev/hda and /dev/hdb.  Create raid1
device from /dev/hda1 and /dev/hdb1.  Mark both partitions as active.
And install whatever boot loader you want into /dev/md1 - it will be
mirrored automatically to /dev/hda1 and /dev/hdb1 (and to whatever else
disks you'll use).

Should your first disk fail, the system will either not boot because
of bios limitations, or bios may "remap" /dev/hdb into /dev/hda (to
be 0x80), and in this case boot procedure will work.

Well, ok, a boot loader should be able to determine real offsets from
the start of the disk, not from the start of raid device.  I don't
know whenever grub can do that - lilo apparently does, as all our
systems are set up exactly this way...

/mjt

Re: Booting from a raid1 device ?

[Lars Marowsky-Bree]

On 2003-10-30T03:54:48,
   Michael Tokarev <mjt@tls.msk.ru> said:

> Ugh-blah.  Excuse me folks, but.. maybe someone will be able to
> answer this one:
> 
>  why the hell a boot loader should "mirror" anything to the second disk? :)

It shouldn't, but it should eventually be smart enough to figure out
which of the two mirrors has the 'most recent' data.

And it should try to load a kernel via multiple drives until it finds a
kernel+initrd combination with a good checksum. 

This would also help raid1 + multipath boot scenarios, as long as the
boot loader itself got started "somehow".


Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>

-- 
High Availability & Clustering	      \ ever tried. ever failed. no matter.
SUSE Labs			      | try again. fail again. fail better.
Research & Development, SUSE LINUX AG \ 	-- Samuel Beckett

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Booting from a raid1 device ?

[Luca Berra]

On Thu, Oct 30, 2003 at 03:54:48AM +0300, Michael Tokarev wrote:
>  why the hell a boot loader should "mirror" anything to the second disk? :)
>
>That is.  Place standard MBR into the boot sector.  An MBR that will boot
>from active partition.  Be it e.g. /dev/hda and /dev/hdb.  Create raid1
>device from /dev/hda1 and /dev/hdb1.  Mark both partitions as active.
>And install whatever boot loader you want into /dev/md1 - it will be
>mirrored automatically to /dev/hda1 and /dev/hdb1 (and to whatever else
>disks you'll use).
1) are you really mirroring on the same ide controller?
2) whoever told you that the first 512 bytes of a partition are fair
game? lvm uses those, and IIRC xfs does too.

L.
Btw i recall reading somewhere that grub actually modifies the disk it
is booting from if luser wants to change the kernel command line.
I dunno if it is true or not, and i am too lazy to read the source. Can
anyone confirm or deny that?

L.

-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: Booting from a raid1 device ?

[Bill Rugolsky Jr.]

On Fri, Oct 31, 2003 at 09:28:42AM +0100, Luca Berra wrote:
> Btw i recall reading somewhere that grub actually modifies the disk it
> is booting from if luser wants to change the kernel command line.
> I dunno if it is true or not, and i am too lazy to read the source. Can
> anyone confirm or deny that?
 
That was me.  It may not be true, because I haven't had time and a
a spare machine to verify.

The symptom that we see is that /boot/grub/stage2 sometimes becomes
desynchronized between the RAID1 partitions after a reboot.

Every night we compare file MD5 sums temporally (day-over-day) and
spatially (between "identically" configured machines).  In the
event that a file block is desynchronized, we will typically see
the MD5 oscillate between two values, depending on which drive MD
chose to populate the cache.  An MD resync of /boot cures the problem.

Since this only seems to occur with /boot/grub/stage2, I've ruled out
most other theories.  We reboot a machine so infrequently that I've only
seen this three times in more than a year, so I haven't been motivated
to track it down.

Regards,

	Bill Rugolsky

RE: Booting from a raid1 device ?

[Cress, Andrew R]

Michael,

I think we agree, but do remember that the 'lifetime of the disk' is the
real question here, and the whole reason we are doing mirroring anyway.
When the disk fails and is swapped out with a replacement, the MBR also
has to be rewritten then, and we want that to be as automatic as
possible.

Andy

-----Original Message-----
From: Michael Tokarev [mailto:mjt@tls.msk.ru] 
Sent: Thursday, October 30, 2003 9:27 AM
To: Cress, Andrew R
Cc: donj@asaca.com; linux-raid@vger.kernel.org
Subject: Re: Booting from a raid1 device ?


[...snip...]

> You have to also have a working MBR on the redundant disk as well, or
> you'll never get to the loader if the first disk really fails.
Usually
> it is the loader's responsibility to set up (or at least verify) the
> MBR.  

Well.. not really.  Sure, working MBR should be on all disks involved.
But it is necessary to set it up only once on every disk, and forgot
about it.  "Standard" MBR that comes with MS-DOS, or any other similar
MBR code will work, and there's no reason to touch it during the whole
lifetime of the disk.

[...snip...]

RE: Booting from a raid1 device ?

[Cress, Andrew R]

Michael,

If you only have an MBR on the first disk, and it fails, your system
will not boot, since BIOS won't be able to read an MBR from the
redundant disk (the only working one).  You may be thinking of a use
case in which the first disk has a corrupted partition, but its MBR is
intact.

You have to also have a working MBR on the redundant disk as well, or
you'll never get to the loader if the first disk really fails.  Usually
it is the loader's responsibility to set up (or at least verify) the
MBR.  

Andy

-----Original Message-----
From: Michael Tokarev [mailto:mjt@tls.msk.ru] 
Sent: Wednesday, October 29, 2003 7:55 PM
To: Cress, Andrew R
Cc: donj@asaca.com; linux-raid@vger.kernel.org
Subject: Re: Booting from a raid1 device ?


Cress, Andrew R wrote:
> Don,
> 
> Yes, it certainly can be done.  I do it all the time.
> 
> It's not hard with lilo, but grub doesn't support raid/md boot devices
> yet.  This means that the grub boot record doesn't automatically get
> written to the second disk.  In short, choose lilo for root mirroring
if
> possible.

Ugh-blah.  Excuse me folks, but.. maybe someone will be able to
answer this one:

  why the hell a boot loader should "mirror" anything to the second
disk? :)

That is.  Place standard MBR into the boot sector.  An MBR that will
boot
from active partition.  Be it e.g. /dev/hda and /dev/hdb.  Create raid1
device from /dev/hda1 and /dev/hdb1.  Mark both partitions as active.
And install whatever boot loader you want into /dev/md1 - it will be
mirrored automatically to /dev/hda1 and /dev/hdb1 (and to whatever else
disks you'll use).

Should your first disk fail, the system will either not boot because
of bios limitations, or bios may "remap" /dev/hdb into /dev/hda (to
be 0x80), and in this case boot procedure will work.

Well, ok, a boot loader should be able to determine real offsets from
the start of the disk, not from the start of raid device.  I don't
know whenever grub can do that - lilo apparently does, as all our
systems are set up exactly this way...

/mjt

Re: Booting from a raid1 device ?

[Michael Tokarev]

Cress, Andrew R wrote:
> Michael,
> 
> If you only have an MBR on the first disk, and it fails, your system
> will not boot, since BIOS won't be able to read an MBR from the
> redundant disk (the only working one).  You may be thinking of a use
> case in which the first disk has a corrupted partition, but its MBR is
> intact.

Corrupted boot partition - yes.  If it contains corrupted MBR, it
will not boot either, because BIOS will not try 2nd disk anyway
(if 1st disk "half-works").  None of current bootloaders support
"fallback" (or redundrand) boot (e.g. try first disk, if read
fails or bad checksum, try another disk etc).  That to say: with
half-working disk, nothing will work.  Note that it is the loader
itself plus the kernel image that (and MBR) that should have
failed - if all this (pretty small) stuff is ok (but there's
a bad block or many bad blocks but elsewhere on the disk), system
will boot just fine.

> You have to also have a working MBR on the redundant disk as well, or
> you'll never get to the loader if the first disk really fails.  Usually
> it is the loader's responsibility to set up (or at least verify) the
> MBR.  

Well.. not really.  Sure, working MBR should be on all disks involved.
But it is necessary to set it up only once on every disk, and forgot
about it.  "Standard" MBR that comes with MS-DOS, or any other similar
MBR code will work, and there's no reason to touch it during the whole
lifetime of the disk.

That's exactly what I said in my previous email: set up standard MBR on
each disk once, and install lilo/grup/whatever on the raid partition,
so that kernel raid code will replicate every e.g. lilo update to all
disks.

The only possible problem with that is - disks with different geometry
and/or partition placement (e.g. raid1 made from /dev/hda1 and /dev/hdb2
- 1st partition on first disk and *second* partition on second, so that
sector addresses of loader data will be different on the two).  For this
case, some support from the bootloader IS needed, but it's rather bizzare
case.

/mjt

RE: Booting from a raid1 device ?

[Cress, Andrew R]

Sandro,

From within grub, you can do
> install (hd0,0)/grub/stage1 (hd1) (hd1,0)/grub/stage2
(hd0,0)/grub/menu.lst
to write the boot sector from hd0 to hd1 without a floppy.  

However, you will need to do this again to any replacement disks added
to the raid-1, and if the primary disk's boot info gets corrupted, grub
won't automatically switch to the mirror/secondary, you'll have to
physically remove the corrupted disk.  

Does anybody know if/when the grub project will add raid support?  

Andy

-----Original Message-----
From: linux-raid-owner@vger.kernel.org
[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Sandro Dentella
Sent: Tuesday, October 28, 2003 3:31 PM
To: linux-raid@vger.kernel.org
Subject: Re: Booting from a raid1 device ?


On Tue, Oct 28, 2003 at 11:56:44AM +0100, Lars Marowsky-Bree wrote:
> On 2003-10-27T23:11:19,
>    Hermann Himmelbauer <dusty@strike.wu-wien.ac.at> said:
> 
>>BTW, assuming I have md.o not as module but statically in the kernel,
which
>>bootloader would support RAID1? I heard LlLO does this job but does
also grub
>>- which I prefer - support Linux SW-RAID1?
> 
> grub doesn't yet afaik.

What does this exactly mean. I always use GRUB for Raid1 setup with boot
partition an md device, and I'm beginning to think I'm using it the
wrong
way!

What I do is I 'setup' the partitios from a floppy:

title Install GRUB into (hd0,0) the first disk
root    (hd0,0)
setup   (hd0)
 
title Install GRUB into (hd1,0) the second disk
root    (hd1,0)
setup   (hd1)


Both partitions (hd0,0) and (hd1,0) are part of /dev/md/0

At that point I'm able to start from both devices... BTW I've never been
able to do that w/o passing from a floppy. If I do that from the CD I
use to
install everithing seems OK but it doesn't boot.


-- 
Sandro Dentella  *:-)
e-mail: sandro.dentella@tin.it 
http://www.tksql.org                    TkSQL Home page - My GPL work
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Booting from a raid1 device ?

[Eric Wood]

Cress, Andrew R wrote:
> Does anybody know if/when the grub project will add raid support?

I personally ask Yoshinori K. Okuji (grub maintainer) and he hinted at
nothing - so don't hold your breath.  Personally, I think it's going to have
to take some assistance (push?) from the RH brains to help them out (IMHO).

-Eric Wood

Booting from a raid1 device ?

[Don Jessup]

  Is it possible to boot from a raid1 device with the md.o built as a 
 2.4.22 kernel module?  




=====
Don Jessup
Asaca/Shibasoku Corp. of America
400 Corporate Circle, Unit G
Golden, CO  80401
303-278-1111 X232
donj@asaca.com
http://www.asaca.com

Re: Booting from a raid1 device ?

[Gordon Henderson]

On Mon, 27 Oct 2003, Don Jessup wrote:

>   Is it possible to boot from a raid1 device with the md.o built as a
>  2.4.22 kernel module?

According to:

  http://www.tldp.org/HOWTO/Software-RAID-HOWTO-4.html#ss4.11

No.

Gordon

Re: Booting from a raid1 device ?

[Paul Clements]

Don Jessup wrote:
> 
>   Is it possible to boot from a raid1 device with the md.o built as a
>  2.4.22 kernel module?

Yes, but it's tricky. You'd have to load md.o and raid1.o in your initrd
(or other early boot time initialization) and then use some, e.g.,
statically-linked mdadm tool on that initrd to start the array. You
cannot, however, use md's RAID autodetect to autostart arrays that have
their component partitions marked as RAID autodetect (0xfd), if md is a
module.

--
Paul


> =====
> Don Jessup
> Asaca/Shibasoku Corp. of America
> 400 Corporate Circle, Unit G
> Golden, CO  80401
> 303-278-1111 X232
> donj@asaca.com
> http://www.asaca.com
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Booting from a raid1 device ?

[Hermann Himmelbauer]

On Monday 27 October 2003 17:00, Paul Clements wrote:
> Don Jessup wrote:
> >   Is it possible to boot from a raid1 device with the md.o built as a
> >  2.4.22 kernel module?
>
> Yes, but it's tricky. You'd have to load md.o and raid1.o in your initrd
> (or other early boot time initialization) and then use some, e.g.,
> statically-linked mdadm tool on that initrd to start the array. You
> cannot, however, use md's RAID autodetect to autostart arrays that have
> their component partitions marked as RAID autodetect (0xfd), if md is a
> module.

BTW, assuming I have md.o not as module but statically in the kernel, which 
bootloader would support RAID1? I heard LlLO does this job but does also grub 
- which I prefer - support Linux SW-RAID1?

What about RAID5? Is there any bootloader that supports it?

		Best Regards,
		Hermann
-- 
x1@aon.at
GPG key ID: 299893C7 (on keyservers)
FP: 0124 2584 8809 EF2A DBF9  4902 64B4 D16B 2998 93C7

Re: Booting from a raid1 device ?

[maarten van den Berg]

On Monday 27 October 2003 23:11, Hermann Himmelbauer wrote:
> On Monday 27 October 2003 17:00, Paul Clements wrote:
> > Don Jessup wrote:


> BTW, assuming I have md.o not as module but statically in the kernel, which
> bootloader would support RAID1? I heard LlLO does this job but does also
> grub - which I prefer - support Linux SW-RAID1?
>
> What about RAID5? Is there any bootloader that supports it?

It has been a while since I read it, but doesn't the Boot+Root+Raid+Lilo howto 
[google] go into those setups ?  It helped me a lot at the time...

A related question:  There are a lot of old and deprecated Raid howto's 
floating around the net, even about the antique version 0.4.  
Is there no mechanism that gets these (very confusing for a newbie!) old 
versions removed or upgraded ?  And is this Boot+Root+Raid+Lilo howto I 
mentioned deprecated, or not ? Shouldn't it be integrated into the 'official' 
Raid howto ? (Or at least, there could be links to each other in the howtos) 

Greetings,
Maarten


> 		Best Regards,
> 		Hermann

-- 
Yes of course I'm sure it's the red cable. I guarante[^%!/+)F#0c|'NO CARRIER

Re: Booting from a raid1 device ?

[Lars Marowsky-Bree]

On 2003-10-27T23:11:19,
   Hermann Himmelbauer <dusty@strike.wu-wien.ac.at> said:

> BTW, assuming I have md.o not as module but statically in the kernel, which 
> bootloader would support RAID1? I heard LlLO does this job but does also grub 
> - which I prefer - support Linux SW-RAID1?

grub doesn't yet afaik.

> What about RAID5? Is there any bootloader that supports it?

No.


Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>

-- 
High Availability & Clustering	      \ ever tried. ever failed. no matter.
SUSE Labs			      | try again. fail again. fail better.
Research & Development, SUSE LINUX AG \ 	-- Samuel Beckett

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Booting from a raid1 device ?

[Michael]

> On 2003-10-27T23:11:19,
>    Hermann Himmelbauer <dusty@strike.wu-wien.ac.at> said:
> 
> > BTW, assuming I have md.o not as module but statically in the kernel, which 
> > bootloader would support RAID1? I heard LlLO does this job but does also grub 
> > - which I prefer - support Linux SW-RAID1?
> 
> grub doesn't yet afaik.
> 
> > What about RAID5? Is there any bootloader that supports it?
> 
> No.
> 

The appropriate solution is to put an 8-16 meg partition as the first 
partition on the first two drives (all the drives) in the system. Set 
those up as raid one and mount them on /boot. Use these as the boot 
boot partitions in a mostly raid 5 system. Likewise, segment swap 
partitions into pairs on the raid5 disk set and create raid 1 arrays 
for swap

Michael

Re: Booting from a raid1 device ?

[Sandro Dentella]

On Tue, Oct 28, 2003 at 11:56:44AM +0100, Lars Marowsky-Bree wrote:
> On 2003-10-27T23:11:19,
>    Hermann Himmelbauer <dusty@strike.wu-wien.ac.at> said:
> 
>>BTW, assuming I have md.o not as module but statically in the kernel, which
>>bootloader would support RAID1? I heard LlLO does this job but does also grub
>>- which I prefer - support Linux SW-RAID1?
> 
> grub doesn't yet afaik.

What does this exactly mean. I always use GRUB for Raid1 setup with boot
partition an md device, and I'm beginning to think I'm using it the wrong
way!

What I do is I 'setup' the partitios from a floppy:

title Install GRUB into (hd0,0) the first disk
root    (hd0,0)
setup   (hd0)
 
title Install GRUB into (hd1,0) the second disk
root    (hd1,0)
setup   (hd1)


Both partitions (hd0,0) and (hd1,0) are part of /dev/md/0

At that point I'm able to start from both devices... BTW I've never been
able to do that w/o passing from a floppy. If I do that from the CD I use to
install everithing seems OK but it doesn't boot.


-- 
Sandro Dentella  *:-)
e-mail: sandro.dentella@tin.it 
http://www.tksql.org                    TkSQL Home page - My GPL work