A failed-disk-how-to anywhere?

A failed-disk-how-to anywhere?

[Martin Stender]

Hi there!

I have two identical disks sitting on a Promise dual channel IDE  
controller. I guess both disks are primary's then.

One of the disks have failed, so I bought a new disk, took out the  
failed disk, and put in the new one.
That might seem a little naive, and apparently it was, since the  
system won't boot up now.
It boots fine, when only the old, healthy disk is connected.

By the way, all three partitions are raided - including /boot ...

Anyway, I have removed the old disk from the Raid with:
#mdadm /dev/mdo --remove /dev/hde1
#mdadm /dev/md1 --remove /dev/hde2
#mdadm /dev/md2 --remove /dev/hde3

- but the the problem persists.

I can't seem to find a decent 'How-To' - so how it this supposed to  
be done?

Thanks in advance!
Martin

Re: A failed-disk-how-to anywhere?

[Brad Campbell]

Martin Stender wrote:
> Hi there!
> 
> I have two identical disks sitting on a Promise dual channel IDE 
> controller. I guess both disks are primary's then.
> 
> One of the disks have failed, so I bought a new disk, took out the 
> failed disk, and put in the new one.
> That might seem a little naive, and apparently it was, since the system 
> won't boot up now.
> It boots fine, when only the old, healthy disk is connected.
> 
> By the way, all three partitions are raided - including /boot ...
> 
> Anyway, I have removed the old disk from the Raid with:
> #mdadm /dev/mdo --remove /dev/hde1
> #mdadm /dev/md1 --remove /dev/hde2
> #mdadm /dev/md2 --remove /dev/hde3
> 
> - but the the problem persists.
> 
> I can't seem to find a decent 'How-To' - so how it this supposed to be 
> done?

A little more info would be helpful. How does the machine boot? How are your other disks configured?
Are you booting off the Promise board or on-board controller (making assumptions given your promise 
appears to contain hde, I'm assuming hd[abcd] are on board somewhere..)

I'm going to take a wild stab in the dark now..

My initial thought would be you have hde and hdg in a raid-1 and nothing on the on-board 
controllers. hde has failed and when you removed it your controller tried the 1st disk it could find 
(hdg) to boot of.. Bingo.. away we go.
You plug a new shiny disk into hde and now the controller tries to boot off that, except it's blank 
and therefore a no-go.

I'd either try and force the controller to boot off hdg (which might be a controller bios option) or 
swap hde & hdg.. then it might boot and let you create your partitions on hdg and then add it back 
into the mirror.

How close did I get ?


Brad
-- 
"Human beings, who are almost unique in having the ability
to learn from the experience of others, are also remarkable
for their apparent disinclination to do so." -- Douglas Adams

Re: A failed-disk-how-to anywhere?

[Mike Hardy]

Brad Campbell wrote:
> Martin Stender wrote:
> 
>> Hi there!
>>
>> I have two identical disks sitting on a Promise dual channel IDE
>> controller. I guess both disks are primary's then.
>>
>> One of the disks have failed, so I bought a new disk, took out the
>> failed disk, and put in the new one.
>> That might seem a little naive, and apparently it was, since the
>> system won't boot up now.
>> It boots fine, when only the old, healthy disk is connected.


> My initial thought would be you have hde and hdg in a raid-1 and nothing
> on the on-board controllers. hde has failed and when you removed it your
> controller tried the 1st disk it could find (hdg) to boot of.. Bingo..
> away we go.
> You plug a new shiny disk into hde and now the controller tries to boot
> off that, except it's blank and therefore a no-go.
> 
> I'd either try and force the controller to boot off hdg (which might be
> a controller bios option) or swap hde & hdg.. then it might boot and let
> you create your partitions on hdg and then add it back into the mirror.


I'd add another stab in the dark and guess that you didn't install your
boot loader on both drives.

Not that I've ever done that before (ok, a few times, most recently two
days ago, sigh)

Typically the BIOS will try all hard drives and so it should have rolled
to one that worked, but if only the "failed" drive had the boot loader
then you are of course not bootable.

I solved this by booting rescue mode, starting up the raid arrays,
mounting them, and manually grub installing. Here's a good page for the
grub incantations:
http://gentoo-wiki.com/HOWTO_Gentoo_Install_on_Software_RAID_mirror_and_LVM2_on_top_of_RAID#Bootloader_installation_and_configuration

-Mike

Re: A failed-disk-how-to anywhere?

[Martin Stender]

I'm sorry - I replied to Brad's email a few hours ago, but I was a  
bit too fast, and didn't notice I was replying to his personal account.

Here's my reply - as you can see, all is fine now!  Thank you both!

(doing a '#cat /proc/mdstat', show that everything went well)

Martin

<reply>

Hello Brad

You got very close ... :-)

Thank you!  I swapped the IDE-cables, and the thing booted nicely!   
So from within webmin (I know ... but it's easy to use) I created  
three new linux raid partitions on the new drive, and - also from  
webmin - added those to the array.
Running a #top - I can see some re-syncing is going on - so I guess  
everything is fine!

Big thanks!

Martin

</reply>

On 09/04/2006, at 20.12, Mike Hardy wrote:

>
> Brad Campbell wrote:
>> Martin Stender wrote:
>>
>>> Hi there!
>>>
>>> I have two identical disks sitting on a Promise dual channel IDE
>>> controller. I guess both disks are primary's then.
>>>
>>> One of the disks have failed, so I bought a new disk, took out the
>>> failed disk, and put in the new one.
>>> That might seem a little naive, and apparently it was, since the
>>> system won't boot up now.
>>> It boots fine, when only the old, healthy disk is connected.
>
>
>> My initial thought would be you have hde and hdg in a raid-1 and  
>> nothing
>> on the on-board controllers. hde has failed and when you removed  
>> it your
>> controller tried the 1st disk it could find (hdg) to boot of..  
>> Bingo..
>> away we go.
>> You plug a new shiny disk into hde and now the controller tries to  
>> boot
>> off that, except it's blank and therefore a no-go.
>>
>> I'd either try and force the controller to boot off hdg (which  
>> might be
>> a controller bios option) or swap hde & hdg.. then it might boot  
>> and let
>> you create your partitions on hdg and then add it back into the  
>> mirror.
>
>
> I'd add another stab in the dark and guess that you didn't install  
> your
> boot loader on both drives.
>
> Not that I've ever done that before (ok, a few times, most recently  
> two
> days ago, sigh)
>
> Typically the BIOS will try all hard drives and so it should have  
> rolled
> to one that worked, but if only the "failed" drive had the boot loader
> then you are of course not bootable.
>
> I solved this by booting rescue mode, starting up the raid arrays,
> mounting them, and manually grub installing. Here's a good page for  
> the
> grub incantations:
> http://gentoo-wiki.com/ 
> HOWTO_Gentoo_Install_on_Software_RAID_mirror_and_LVM2_on_top_of_RAID#B 
> ootloader_installation_and_configuration
>
> -Mike
>