RAID 1 and grub

RAID 1 and grub

[Richard Scobie]

I have a machine with Fedora core 8 installed onto RAID1 on IDE drives - 
hda and hdc.

hda has failed and after spending some time with a rescue disk mounting 
hdc's /boot partition (hdc1) and changing the grub.conf device 
parameters, I have no success in booting off it.

I then set them back to the original (hd0,0) and moved hdc into hda's 
position.

Booting from there brings up the message: GRUB hard disk error

Can any grub guru offer any assisentence as to what may be missing here?

Regards,

Richard

Re: RAID 1 and grub

[David Rees]

On Jan 30, 2008 2:06 PM, Richard Scobie <richard@sauce.co.nz> wrote:
> hda has failed and after spending some time with a rescue disk mounting
> hdc's /boot partition (hdc1) and changing the grub.conf device
> parameters, I have no success in booting off it.
>
> I then set them back to the original (hd0,0) and moved hdc into hda's
> position.
>
> Booting from there brings up the message: GRUB hard disk error

Have you tried re-running grub-install after booting from a rescue disk?

-Dave

Re: RAID 1 and grub

[Richard Scobie]

David Rees wrote:

> Have you tried re-running grub-install after booting from a rescue disk?
> 
> -Dave

Hi David,

I have but although I can advance further it seems that the BIOS is 
doing some strange things as well, switching drive ordering around.

With a new hda installed and partitioned, ready to be rebuilt, the good 
drive, hdc installed, the grub.conf modified to address (hd2,0) - I have 
  an hdb installed also, and grub installed on hdc, booting with the 
BIOS set to start on hdc hangs with the message "grub stage2" then drops 
to a grub prompt.

I then enter "kernel (hd0,0)/vmlinuz" and it finds the kernel. I would 
have expected this to be on (hd2,0).

Next, "boot root=/dev/md2", "boot root=/dev/hdc3" or "boot 
root=/dev/hda3" all result in the kernel booting then panicing with a 
"cannot open root device".

I suspect you are correct that the Fedora installer, having built and 
installed to RAID1, does not finish the job by installing grub on the 
second drive.

While it is not a problem with this particular box to do a reinstall, it 
does not inspire confidence for a number of others that I have.

This is the first time I have lost the primary member of a RAID1, having 
replaced secondary members a number of times without issue.

Regards,

Richard

Re: RAID 1 and grub

[Richard Scobie]

A followup for the archives:

I found this document very useful:

http://lists.us.dell.com/pipermail/linux-poweredge/2003-July/008898.html

After modifying my grub.conf to refer to (hd0,0), reinstalling grub on 
hdc with:

grub> device (hd0) /dev/hdc

grub> root (hd0,0)

grub> (hd0)

and rebooting with the bios set to boot off hdc, everything burst back 
into life.

I shall now be checking all my Fedora/Centos RAID1 installs for grub 
installed on both drives.

Regards,

Richard

Re: RAID 1 and grub

[David Rees]

On Jan 30, 2008 6:33 PM, Richard Scobie <richard@sauce.co.nz> wrote:
> I found this document very useful:
> http://lists.us.dell.com/pipermail/linux-poweredge/2003-July/008898.html
>
> After modifying my grub.conf to refer to (hd0,0), reinstalling grub on
> hdc with:
>
> grub> device (hd0) /dev/hdc
> grub> root (hd0,0)
> grub> (hd0)
>
> and rebooting with the bios set to boot off hdc, everything burst back
> into life.

FWIW, this step is clearly marked in the Software-RAID HOWTO under
"Booting on RAID":
http://tldp.org/HOWTO/Software-RAID-HOWTO-7.html#ss7.3

If it appears that Fedora isn't doing this when installing on a
Software RAID 1 boot device, I suggest you open a bug.

BTW, I suspect you are missing the command "setup" from your 3rd
command above, it should be:

# grub
grub> device (hd0) /dev/hdc
grub> root (hd0,0)
grub> setup (hd0)

> I shall now be checking all my Fedora/Centos RAID1 installs for grub
> installed on both drives.

Good idea. Whenever setting up a RAID1 device to boot from, I perform
the above 3 steps. I also suggest using labels to identify partitions
and testing the two failure modes and that you are able to boot with
either drive disconnected.

-Dave

Re: RAID 1 and grub

[Richard Scobie]

David Rees wrote:

> FWIW, this step is clearly marked in the Software-RAID HOWTO under
> "Booting on RAID":
> http://tldp.org/HOWTO/Software-RAID-HOWTO-7.html#ss7.3

The one place I didn't look...


> BTW, I suspect you are missing the command "setup" from your 3rd
> command above, it should be:
> 
> # grub
> grub> device (hd0) /dev/hdc
> grub> root (hd0,0)
> grub> setup (hd0)

That is correct.

Regards,

Richard

Re: RAID 1 and grub

[David Greaves]

Richard Scobie wrote:
> David Rees wrote:
> 
>> FWIW, this step is clearly marked in the Software-RAID HOWTO under
>> "Booting on RAID":
>> http://tldp.org/HOWTO/Software-RAID-HOWTO-7.html#ss7.3
> 
> The one place I didn't look...

Good - I hope you'll both look here instead:

http://linux-raid.osdl.org/index.php/Tweaking%2C_tuning_and_troubleshooting#Booting_on_RAID

David

Re: RAID 1 and grub

[Keld Jørn Simonsen]

On Wed, Jan 30, 2008 at 06:47:19PM -0800, David Rees wrote:
> On Jan 30, 2008 6:33 PM, Richard Scobie <richard@sauce.co.nz> wrote:
> 
> FWIW, this step is clearly marked in the Software-RAID HOWTO under
> "Booting on RAID":
> http://tldp.org/HOWTO/Software-RAID-HOWTO-7.html#ss7.3

A good an extesive reference, but somewhat outdated.

> BTW, I suspect you are missing the command "setup" from your 3rd
> command above, it should be:
> 
> # grub
> grub> device (hd0) /dev/hdc
> grub> root (hd0,0)
> grub> setup (hd0)

I do not grasp this. How and where is it said that two disks are
involved? hda and hdc should both be involved.

Best regards
keld

Re: RAID 1 and grub

[Richard Scobie]

Keld Jørn Simonsen wrote:

>> # grub
>> grub> device (hd0) /dev/hdc
>> grub> root (hd0,0)
>> grub> setup (hd0)
> 
> I do not grasp this. How and where is it said that two disks are
> involved? hda and hdc should both be involved.

There are not two disks involved in this instance.

This is used in the scenario where the primary disk in the RAID1 
(/dev/hda), already has grub installed in the MBR and you wish to 
install it on the secondary drive (/dev/hdc).

This then allows for a failed primary drive to be removed and the 
machine to boot from the secondary - (may need BIOS to be set to boot 
from secondary drive).

As an aside, after last weeks discovery that the Fedora 8 install had 
not installed grub on the secondary drive, as part of a RAID 1 install, 
some cursory Googling and searching Redhat's Knowledge base leads me to 
believe that this may well be normal for all Redhat (RHEL/Fedora) RAID1 
installs.

One has nothing to lose by installing grub on the second drive in this 
case and it may save some delay in recovery on losing the primary, 
although as has been pointed out, it is best practice to test missing 
drives as part of initial install testing.

Regards,

Richard
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RAID 1 and grub

[Bill Davidsen]

Richard Scobie wrote:
> A followup for the archives:
>
> I found this document very useful:
>
> http://lists.us.dell.com/pipermail/linux-poweredge/2003-July/008898.html
>
> After modifying my grub.conf to refer to (hd0,0), reinstalling grub on 
> hdc with:
>
> grub> device (hd0) /dev/hdc
>
> grub> root (hd0,0)
>
> grub> (hd0)
>
> and rebooting with the bios set to boot off hdc, everything burst back 
> into life.
>
> I shall now be checking all my Fedora/Centos RAID1 installs for grub 
> installed on both drives.

Have you actually tested this by removing the first hd and booting? 
Depending on the BIOS I believe that the fallback drive will be called 
hdc by the BIOS but will be hdd in the system. That was with RHEL3, but 
worth testing.

-- 
Bill Davidsen <davidsen@tmr.com>
  "Woe unto the statesman who makes war without a reason that will still
  be valid when the war is over..." Otto von Bismark 

Re: RAID 1 and grub

[Peter Rabbitson]

Bill Davidsen wrote:
> Richard Scobie wrote:
>> A followup for the archives:
>>
>> I found this document very useful:
>>
>> http://lists.us.dell.com/pipermail/linux-poweredge/2003-July/008898.html
>>
>> After modifying my grub.conf to refer to (hd0,0), reinstalling grub on 
>> hdc with:
>>
>> grub> device (hd0) /dev/hdc
>>
>> grub> root (hd0,0)
>>
>> grub> (hd0)
>>
>> and rebooting with the bios set to boot off hdc, everything burst back 
>> into life.
>>
>> I shall now be checking all my Fedora/Centos RAID1 installs for grub 
>> installed on both drives.
> 
> Have you actually tested this by removing the first hd and booting? 
> Depending on the BIOS I believe that the fallback drive will be called 
> hdc by the BIOS but will be hdd in the system. That was with RHEL3, but 
> worth testing.
> 

The line:

grub> device (hd0) /dev/hdc

simply means "treat /dev/hdc as the first _bios_ hard disk in the system". 
This way when grub writes to the MBR of hd0, it will be in fact writing to 
/dev/hdc. The reason the drive must be referenced as hd0 (and not hd2) is 
because grub enuerates drives according to the bios, and therefore the drive 
from which the bios is currently booting is _always_ hd0.

Re: RAID 1 and grub

[Richard Scobie]

Bill Davidsen wrote:

> Have you actually tested this by removing the first hd and booting? 
> Depending on the BIOS I believe that the fallback drive will be called 
> hdc by the BIOS but will be hdd in the system. That was with RHEL3, but 
> worth testing.
> 

Hi Bill,

I did not try this particular combination. I shut the box down, removed 
the failed drive (hda) and installed it's replacement and proceeded from 
there.

Once I had discovered that hdc had no grub installed, after running:

grub> device (hd0) /dev/hdc

grub> root (hd0,0)

grub> setup (hd0)

I set the BIOS to boot from hdc and it all worked.

Regards,

Richard