converting to raid - Error 2

converting to raid - Error 2

[Timothy D. Lenz]

This is something I've been trying to do for some time and there are 2 
older threads, but the last post got no responce, so starting a new 
thread. The old ones are:

Converting system to raid
Removing drives

The computer with 2 sata drives switched over fine a long time ago, but 
I have yet to get the one with 3 sata drives to boot with raid. Raid 
support is built into the kernel, partitions are type fd.

fdisk -l shows:
-----------------------------------------------------------------------
Disk /dev/hda: 40.0 GB, 40020664320 bytes
255 heads, 63 sectors/track, 4865 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x6b381dfe

    Device Boot      Start         End      Blocks   Id  System
/dev/hda1   *           1        4660    37431418+  83  Linux
/dev/hda2            4661        4865     1646662+   5  Extended
/dev/hda5            4661        4865     1646631   82  Linux swap / Solaris

Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x94140963

    Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1        3040    24418768+  fd  Linux raid 
autodetect
/dev/sda2            3041        3649     4891792+  fd  Linux raid 
autodetect
/dev/sda3            3650       60801   459073440   fd  Linux raid 
autodetect

Disk /dev/sdb: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xf1814421

    Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1        3040    24418768+  fd  Linux raid 
autodetect
/dev/sdb2            3041        3649     4891792+  fd  Linux raid 
autodetect
/dev/sdb3            3650       60801   459073440   fd  Linux raid 
autodetect

Disk /dev/sdc: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x371b6063

    Device Boot      Start         End      Blocks   Id  System
/dev/sdc3            3650       60801   459073440   fd  Linux raid 
autodetect

Disk /dev/md0: 25.0 GB, 25004736512 bytes
2 heads, 4 sectors/track, 6104672 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Disk identifier: 0x00000000

Disk /dev/md0 doesn't contain a valid partition table

Disk /dev/md2: 940.1 GB, 940182208512 bytes
2 heads, 4 sectors/track, 229536672 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Disk identifier: 0x00000000

Disk /dev/md2 doesn't contain a valid partition table

Disk /dev/md1: 5009 MB, 5009113088 bytes
2 heads, 4 sectors/track, 1222928 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Disk identifier: 0x00000000

Disk /dev/md1 doesn't contain a valid partition table
-----------------------------------------------------------------------

Orginaly I had made both md0 and md1 2-way mirrors with a spare. But 
when it wouldn't boot, I was told it might be because of the spare and 
to grow them in. That also didn't work. I change md0 and md1 to 2-way 
mirrors, and removed the boot flag for sdc1 and zeroed superblock And I 
corrected the mdadm.conf file to reflect the changes to md0/1. Still 
would not boot from raid. I removed the two partitions sdc1 and sdc2 
leaving just sdc3 which is part of md2 raid5. Also re-ran grub on md0 
using:

  sudo grub
  grub>device (hd0) /dev/sda
  grub>root (hd0,0)
  grub>setup (hd0)

  grub>device (hd0) /dev/sdb
  grub>root (hd0,0)
  grub>setup (hd0)

To copy the current boot drive hda1 over to md0, I use a GRML boot disk, 
mount /dev/md0 to /mnt/md0 and /dev/sdd1 (/dev/hda1) to /mnt/sdd1, then do:

rsync -caHh --progress --delete /mnt/sdd1/ /mnt/md0.

Then I reboot normally. I have 2 copies of /boot/grub/device.map, 
/boot/grub/menu.lst and /etc/fstab. one for normal boot and one for 
raid. On /mnt/md0 I swap the normal boot files for the raid files by 
adding .old to the names of normal boot and removing .raid from the raid 
versions. Then reboot again and in cmos move the pata drive to the 
bottom of the list so cmos tries the sata drives first. That worked for 
the 32bit linux 2 drive system. But on this 64bit 3 drive I get:

Grub loading stage 1.5
Grub loading, please wait
Error 2

I can't remove the 3rd drive because it has part of md2 and I shouldn't 
need to anyway. Copies of the above mentioned files I put at: 
http://24.255.17.209:2400/vdr/local/raid/

Re: converting to raid - Error 2

[Doug Ledford]

[-- Attachment #1: Type: text/plain, Size: 6015 bytes --]

On 04/30/2010 04:41 PM, Timothy D. Lenz wrote:
> This is something I've been trying to do for some time and there are 2
> older threads, but the last post got no responce, so starting a new
> thread. The old ones are:
> 
> Converting system to raid
> Removing drives
> 
> The computer with 2 sata drives switched over fine a long time ago, but
> I have yet to get the one with 3 sata drives to boot with raid. Raid
> support is built into the kernel, partitions are type fd.
> 
> fdisk -l shows:
> -----------------------------------------------------------------------
> Disk /dev/hda: 40.0 GB, 40020664320 bytes
> 255 heads, 63 sectors/track, 4865 cylinders
> Units = cylinders of 16065 * 512 = 8225280 bytes
> Disk identifier: 0x6b381dfe
> 
>    Device Boot      Start         End      Blocks   Id  System
> /dev/hda1   *           1        4660    37431418+  83  Linux
> /dev/hda2            4661        4865     1646662+   5  Extended
> /dev/hda5            4661        4865     1646631   82  Linux swap /
> Solaris
> 
> Disk /dev/sda: 500.1 GB, 500107862016 bytes
> 255 heads, 63 sectors/track, 60801 cylinders
> Units = cylinders of 16065 * 512 = 8225280 bytes
> Disk identifier: 0x94140963
> 
>    Device Boot      Start         End      Blocks   Id  System
> /dev/sda1   *           1        3040    24418768+  fd  Linux raid
> autodetect
> /dev/sda2            3041        3649     4891792+  fd  Linux raid
> autodetect
> /dev/sda3            3650       60801   459073440   fd  Linux raid
> autodetect
> 
> Disk /dev/sdb: 500.1 GB, 500107862016 bytes
> 255 heads, 63 sectors/track, 60801 cylinders
> Units = cylinders of 16065 * 512 = 8225280 bytes
> Disk identifier: 0xf1814421
> 
>    Device Boot      Start         End      Blocks   Id  System
> /dev/sdb1   *           1        3040    24418768+  fd  Linux raid
> autodetect
> /dev/sdb2            3041        3649     4891792+  fd  Linux raid
> autodetect
> /dev/sdb3            3650       60801   459073440   fd  Linux raid
> autodetect
> 
> Disk /dev/sdc: 500.1 GB, 500107862016 bytes
> 255 heads, 63 sectors/track, 60801 cylinders
> Units = cylinders of 16065 * 512 = 8225280 bytes
> Disk identifier: 0x371b6063
> 
>    Device Boot      Start         End      Blocks   Id  System
> /dev/sdc3            3650       60801   459073440   fd  Linux raid
> autodetect
> 
> Disk /dev/md0: 25.0 GB, 25004736512 bytes
> 2 heads, 4 sectors/track, 6104672 cylinders
> Units = cylinders of 8 * 512 = 4096 bytes
> Disk identifier: 0x00000000
> 
> Disk /dev/md0 doesn't contain a valid partition table
> 
> Disk /dev/md2: 940.1 GB, 940182208512 bytes
> 2 heads, 4 sectors/track, 229536672 cylinders
> Units = cylinders of 8 * 512 = 4096 bytes
> Disk identifier: 0x00000000
> 
> Disk /dev/md2 doesn't contain a valid partition table
> 
> Disk /dev/md1: 5009 MB, 5009113088 bytes
> 2 heads, 4 sectors/track, 1222928 cylinders
> Units = cylinders of 8 * 512 = 4096 bytes
> Disk identifier: 0x00000000
> 
> Disk /dev/md1 doesn't contain a valid partition table
> -----------------------------------------------------------------------
> 
> Orginaly I had made both md0 and md1 2-way mirrors with a spare. But
> when it wouldn't boot, I was told it might be because of the spare and
> to grow them in. That also didn't work. I change md0 and md1 to 2-way
> mirrors, and removed the boot flag for sdc1 and zeroed superblock And I
> corrected the mdadm.conf file to reflect the changes to md0/1. Still
> would not boot from raid. I removed the two partitions sdc1 and sdc2
> leaving just sdc3 which is part of md2 raid5. Also re-ran grub on md0
> using:
> 
>  sudo grub
>  grub>device (hd0) /dev/sda
>  grub>root (hd0,0)
>  grub>setup (hd0)

You can't use the setup macro (can't remember why, I just know it
generally doesn't work) when putting grub on your raid1 array.  You need:

install --stage2=/grub/stage2 /grub/stage1 (hd0) /grub/e2fs_stage1_5
/grub/stage2 /grub/grub.conf

and if that doesn't work in your particular configuration, you can add
the d option after stage1 and before (hd0), but if you use it, then your
boot disk must always be BIOS device 0x80, which means setting your BIOS
to boot off of some disk other than the first disk found usually won't
work and instead you just have to make whatever disk you want to boot
off of the first disk found by the BIOS.  You can switch (hd0) to
(hd0,0) if you want and if you have a normal DOS master boot record.

>  grub>device (hd0) /dev/sdb
>  grub>root (hd0,0)
>  grub>setup (hd0)
> 
> To copy the current boot drive hda1 over to md0, I use a GRML boot disk,
> mount /dev/md0 to /mnt/md0 and /dev/sdd1 (/dev/hda1) to /mnt/sdd1, then do:
> 
> rsync -caHh --progress --delete /mnt/sdd1/ /mnt/md0.
> 
> Then I reboot normally. I have 2 copies of /boot/grub/device.map,
> /boot/grub/menu.lst and /etc/fstab. one for normal boot and one for
> raid. On /mnt/md0 I swap the normal boot files for the raid files by
> adding .old to the names of normal boot and removing .raid from the raid
> versions. Then reboot again and in cmos move the pata drive to the
> bottom of the list so cmos tries the sata drives first. That worked for
> the 32bit linux 2 drive system. But on this 64bit 3 drive I get:
> 
> Grub loading stage 1.5
> Grub loading, please wait
> Error 2
> 
> I can't remove the 3rd drive because it has part of md2 and I shouldn't
> need to anyway. Copies of the above mentioned files I put at:
> http://24.255.17.209:2400/vdr/local/raid/
> -- 
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


-- 
Doug Ledford <dledford@redhat.com>
              GPG KeyID: CFBFF194
	      http://people.redhat.com/dledford

Infiniband specific RPMs available at
	      http://people.redhat.com/dledford/Infiniband


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: converting to raid - Error 2

[Timothy D. Lenz]

On 4/30/2010 1:53 PM, Doug Ledford wrote:
> On 04/30/2010 04:41 PM, Timothy D. Lenz wrote:
>> This is something I've been trying to do for some time and there are 2
>> older threads, but the last post got no responce, so starting a new
>> thread. The old ones are:
>>
>> Converting system to raid
>> Removing drives
>>
>> The computer with 2 sata drives switched over fine a long time ago, but
>> I have yet to get the one with 3 sata drives to boot with raid. Raid
>> support is built into the kernel, partitions are type fd.
>>
>> fdisk -l shows:
>> -----------------------------------------------------------------------
>> Disk /dev/hda: 40.0 GB, 40020664320 bytes
>> 255 heads, 63 sectors/track, 4865 cylinders
>> Units = cylinders of 16065 * 512 = 8225280 bytes
>> Disk identifier: 0x6b381dfe
>>
>>     Device Boot      Start         End      Blocks   Id  System
>> /dev/hda1   *           1        4660    37431418+  83  Linux
>> /dev/hda2            4661        4865     1646662+   5  Extended
>> /dev/hda5            4661        4865     1646631   82  Linux swap /
>> Solaris
>>
>> Disk /dev/sda: 500.1 GB, 500107862016 bytes
>> 255 heads, 63 sectors/track, 60801 cylinders
>> Units = cylinders of 16065 * 512 = 8225280 bytes
>> Disk identifier: 0x94140963
>>
>>     Device Boot      Start         End      Blocks   Id  System
>> /dev/sda1   *           1        3040    24418768+  fd  Linux raid
>> autodetect
>> /dev/sda2            3041        3649     4891792+  fd  Linux raid
>> autodetect
>> /dev/sda3            3650       60801   459073440   fd  Linux raid
>> autodetect
>>
>> Disk /dev/sdb: 500.1 GB, 500107862016 bytes
>> 255 heads, 63 sectors/track, 60801 cylinders
>> Units = cylinders of 16065 * 512 = 8225280 bytes
>> Disk identifier: 0xf1814421
>>
>>     Device Boot      Start         End      Blocks   Id  System
>> /dev/sdb1   *           1        3040    24418768+  fd  Linux raid
>> autodetect
>> /dev/sdb2            3041        3649     4891792+  fd  Linux raid
>> autodetect
>> /dev/sdb3            3650       60801   459073440   fd  Linux raid
>> autodetect
>>
>> Disk /dev/sdc: 500.1 GB, 500107862016 bytes
>> 255 heads, 63 sectors/track, 60801 cylinders
>> Units = cylinders of 16065 * 512 = 8225280 bytes
>> Disk identifier: 0x371b6063
>>
>>     Device Boot      Start         End      Blocks   Id  System
>> /dev/sdc3            3650       60801   459073440   fd  Linux raid
>> autodetect
>>
>> Disk /dev/md0: 25.0 GB, 25004736512 bytes
>> 2 heads, 4 sectors/track, 6104672 cylinders
>> Units = cylinders of 8 * 512 = 4096 bytes
>> Disk identifier: 0x00000000
>>
>> Disk /dev/md0 doesn't contain a valid partition table
>>
>> Disk /dev/md2: 940.1 GB, 940182208512 bytes
>> 2 heads, 4 sectors/track, 229536672 cylinders
>> Units = cylinders of 8 * 512 = 4096 bytes
>> Disk identifier: 0x00000000
>>
>> Disk /dev/md2 doesn't contain a valid partition table
>>
>> Disk /dev/md1: 5009 MB, 5009113088 bytes
>> 2 heads, 4 sectors/track, 1222928 cylinders
>> Units = cylinders of 8 * 512 = 4096 bytes
>> Disk identifier: 0x00000000
>>
>> Disk /dev/md1 doesn't contain a valid partition table
>> -----------------------------------------------------------------------
>>
>> Orginaly I had made both md0 and md1 2-way mirrors with a spare. But
>> when it wouldn't boot, I was told it might be because of the spare and
>> to grow them in. That also didn't work. I change md0 and md1 to 2-way
>> mirrors, and removed the boot flag for sdc1 and zeroed superblock And I
>> corrected the mdadm.conf file to reflect the changes to md0/1. Still
>> would not boot from raid. I removed the two partitions sdc1 and sdc2
>> leaving just sdc3 which is part of md2 raid5. Also re-ran grub on md0
>> using:
>>
>>   sudo grub
>>   grub>device (hd0) /dev/sda
>>   grub>root (hd0,0)
>>   grub>setup (hd0)
>
> You can't use the setup macro (can't remember why, I just know it
> generally doesn't work) when putting grub on your raid1 array.  You need:
>
> install --stage2=/grub/stage2 /grub/stage1 (hd0) /grub/e2fs_stage1_5
> /grub/stage2 /grub/grub.conf
>
> and if that doesn't work in your particular configuration, you can add
> the d option after stage1 and before (hd0), but if you use it, then your
> boot disk must always be BIOS device 0x80, which means setting your BIOS
> to boot off of some disk other than the first disk found usually won't
> work and instead you just have to make whatever disk you want to boot
> off of the first disk found by the BIOS.  You can switch (hd0) to
> (hd0,0) if you want and if you have a normal DOS master boot record.
>
>>   grub>device (hd0) /dev/sdb
>>   grub>root (hd0,0)
>>   grub>setup (hd0)
>>
>> To copy the current boot drive hda1 over to md0, I use a GRML boot disk,
>> mount /dev/md0 to /mnt/md0 and /dev/sdd1 (/dev/hda1) to /mnt/sdd1, then do:
>>
>> rsync -caHh --progress --delete /mnt/sdd1/ /mnt/md0.
>>
>> Then I reboot normally. I have 2 copies of /boot/grub/device.map,
>> /boot/grub/menu.lst and /etc/fstab. one for normal boot and one for
>> raid. On /mnt/md0 I swap the normal boot files for the raid files by
>> adding .old to the names of normal boot and removing .raid from the raid
>> versions. Then reboot again and in cmos move the pata drive to the
>> bottom of the list so cmos tries the sata drives first. That worked for
>> the 32bit linux 2 drive system. But on this 64bit 3 drive I get:
>>
>> Grub loading stage 1.5
>> Grub loading, please wait
>> Error 2
>>
>> I can't remove the 3rd drive because it has part of md2 and I shouldn't
>> need to anyway. Copies of the above mentioned files I put at:
>> http://24.255.17.209:2400/vdr/local/raid/

grrr, used reply instead of reply all again, sorry:

I don't understand why using those commands at the grub> prompt won't 
work. It worked on the other computer. This is grub 1, not grub 2. I 
haven't updated it yet. It is using lenny. I did start the upgrade to 
grub2 on the other computer a short time ago, but haven't yet done the 
final command to finish the change. Trying to solve one thing at a time 
and right now this is the oldest. I don't recall ever doing any install 
--stage2=/grub/st.... on the other system.

Here is what I get when I do this for sda:
---------------------------------------
grub> device (hd0) /dev/sda

grub> root (hd0,0)
  Filesystem type is ext2fs, partition type 0xfd

grub> setup (hd0)
  Checking if "/boot/grub/stage1" exists... yes
  Checking if "/boot/grub/stage2" exists... yes
  Checking if "/boot/grub/e2fs_stage1_5" exists... yes
  Running "embed /boot/grub/e2fs_stage1_5 (hd0)"...  15 sectors are 
embedded.
succeeded
  Running "install /boot/grub/stage1 (hd0) (hd0)1+15 p 
(hd0,0)/boot/grub/stage2 /boot/grub/menu.l
st"... succeeded
Done.
---------------------------------------
And I just noticed something.I used sda, not sda1. I can't remember 
which I did before with the other, but my notes show sda. Would that matter?

Re: converting to raid - Error 2

[Doug Ledford]

[-- Attachment #1: Type: text/plain, Size: 4697 bytes --]

On 04/30/2010 05:58 PM, Timothy D. Lenz wrote:
> 
> 
> On 4/30/2010 1:53 PM, Doug Ledford wrote:

>> You can't use the setup macro (can't remember why, I just know it
>> generally doesn't work) when putting grub on your raid1 array.  You need:
>>
>> install --stage2=/grub/stage2 /grub/stage1 (hd0) /grub/e2fs_stage1_5
>> /grub/stage2 /grub/grub.conf
>>
>> and if that doesn't work in your particular configuration, you can add
>> the d option after stage1 and before (hd0), but if you use it, then your
>> boot disk must always be BIOS device 0x80, which means setting your BIOS
>> to boot off of some disk other than the first disk found usually won't
>> work and instead you just have to make whatever disk you want to boot
>> off of the first disk found by the BIOS.  You can switch (hd0) to
>> (hd0,0) if you want and if you have a normal DOS master boot record.
>>
>>>   grub>device (hd0) /dev/sdb
>>>   grub>root (hd0,0)
>>>   grub>setup (hd0)
>>>
>>> To copy the current boot drive hda1 over to md0, I use a GRML boot disk,
>>> mount /dev/md0 to /mnt/md0 and /dev/sdd1 (/dev/hda1) to /mnt/sdd1,
>>> then do:
>>>
>>> rsync -caHh --progress --delete /mnt/sdd1/ /mnt/md0.
>>>
>>> Then I reboot normally. I have 2 copies of /boot/grub/device.map,
>>> /boot/grub/menu.lst and /etc/fstab. one for normal boot and one for
>>> raid. On /mnt/md0 I swap the normal boot files for the raid files by
>>> adding .old to the names of normal boot and removing .raid from the raid
>>> versions. Then reboot again and in cmos move the pata drive to the
>>> bottom of the list so cmos tries the sata drives first. That worked for
>>> the 32bit linux 2 drive system. But on this 64bit 3 drive I get:
>>>
>>> Grub loading stage 1.5
>>> Grub loading, please wait
>>> Error 2
>>>
>>> I can't remove the 3rd drive because it has part of md2 and I shouldn't
>>> need to anyway. Copies of the above mentioned files I put at:
>>> http://24.255.17.209:2400/vdr/local/raid/
> 
> grrr, used reply instead of reply all again, sorry:
> 
> I don't understand why using those commands at the grub> prompt won't
> work. It worked on the other computer.

Can't answer that for you.  I didn't bother to work out *why* it didn't
work, just that it's known to fail in some situations (the setup
command).  Maybe it's because it pulls information from the device map
and gets the device line wrong (and yes, the setup macro redoes whatever
device line you put in based upon the information in the device map
file).  Maybe it's something else.  Regardless, it's known to fail.

> This is grub 1, not grub 2.

Yes, my instructions were for grub1.

> I
> haven't updated it yet. It is using lenny. I did start the upgrade to
> grub2 on the other computer a short time ago, but haven't yet done the
> final command to finish the change. Trying to solve one thing at a time
> and right now this is the oldest. I don't recall ever doing any install
> --stage2=/grub/st.... on the other system.
> 
> Here is what I get when I do this for sda:
> ---------------------------------------
> grub> device (hd0) /dev/sda
> 
> grub> root (hd0,0)
>  Filesystem type is ext2fs, partition type 0xfd
> 
> grub> setup (hd0)
>  Checking if "/boot/grub/stage1" exists... yes
>  Checking if "/boot/grub/stage2" exists... yes
>  Checking if "/boot/grub/e2fs_stage1_5" exists... yes
>  Running "embed /boot/grub/e2fs_stage1_5 (hd0)"...  15 sectors are
> embedded.
> succeeded
>  Running "install /boot/grub/stage1 (hd0) (hd0)1+15 p
> (hd0,0)/boot/grub/stage2 /boot/grub/menu.l
> st"... succeeded
> Done.
> ---------------------------------------

I know, it looks fine.  But it screws it up anyway.  It isn't reliable.
 You have to use the bare install command like I pointed out above.  You
also need to make sure that your boot partitions on the two drives are
*identical* in terms of placement on the drives (aka, the partitions
need to occupy the same logical sector numbers on both drives, which is
normal when they are both the first partition on each drive and the
drives use the same sector/head counts per track which is typical now a
days).

> And I just noticed something.I used sda, not sda1. I can't remember
> which I did before with the other, but my notes show sda. Would that
> matter?

Normally sda would be the one I would use, but sda1 would work as long
as you have a normal DOS master boot record on the drive.

-- 
Doug Ledford <dledford@redhat.com>
              GPG KeyID: CFBFF194
	      http://people.redhat.com/dledford

Infiniband specific RPMs available at
	      http://people.redhat.com/dledford/Infiniband


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: converting to raid - Error 2

[Timothy D. Lenz]

On 5/5/2010 9:25 AM, Doug Ledford wrote:
> On 04/30/2010 05:58 PM, Timothy D. Lenz wrote:
>>
>>
>> On 4/30/2010 1:53 PM, Doug Ledford wrote:
>
>>> You can't use the setup macro (can't remember why, I just know it
>>> generally doesn't work) when putting grub on your raid1 array.  You need:
>>>
>>> install --stage2=/grub/stage2 /grub/stage1 (hd0) /grub/e2fs_stage1_5
>>> /grub/stage2 /grub/grub.conf
>>>
>>> and if that doesn't work in your particular configuration, you can add
>>> the d option after stage1 and before (hd0), but if you use it, then your
>>> boot disk must always be BIOS device 0x80, which means setting your BIOS
>>> to boot off of some disk other than the first disk found usually won't
>>> work and instead you just have to make whatever disk you want to boot
>>> off of the first disk found by the BIOS.  You can switch (hd0) to
>>> (hd0,0) if you want and if you have a normal DOS master boot record.
>>>
>>>>    grub>device (hd0) /dev/sdb
>>>>    grub>root (hd0,0)
>>>>    grub>setup (hd0)
>>>>
>>>> To copy the current boot drive hda1 over to md0, I use a GRML boot disk,
>>>> mount /dev/md0 to /mnt/md0 and /dev/sdd1 (/dev/hda1) to /mnt/sdd1,
>>>> then do:
>>>>
>>>> rsync -caHh --progress --delete /mnt/sdd1/ /mnt/md0.
>>>>
>>>> Then I reboot normally. I have 2 copies of /boot/grub/device.map,
>>>> /boot/grub/menu.lst and /etc/fstab. one for normal boot and one for
>>>> raid. On /mnt/md0 I swap the normal boot files for the raid files by
>>>> adding .old to the names of normal boot and removing .raid from the raid
>>>> versions. Then reboot again and in cmos move the pata drive to the
>>>> bottom of the list so cmos tries the sata drives first. That worked for
>>>> the 32bit linux 2 drive system. But on this 64bit 3 drive I get:
>>>>
>>>> Grub loading stage 1.5
>>>> Grub loading, please wait
>>>> Error 2
>>>>
>>>> I can't remove the 3rd drive because it has part of md2 and I shouldn't
>>>> need to anyway. Copies of the above mentioned files I put at:
>>>> http://24.255.17.209:2400/vdr/local/raid/
>>
>> grrr, used reply instead of reply all again, sorry:
>>
>> I don't understand why using those commands at the grub>  prompt won't
>> work. It worked on the other computer.
>
> Can't answer that for you.  I didn't bother to work out *why* it didn't
> work, just that it's known to fail in some situations (the setup
> command).  Maybe it's because it pulls information from the device map
> and gets the device line wrong (and yes, the setup macro redoes whatever
> device line you put in based upon the information in the device map
> file).  Maybe it's something else.  Regardless, it's known to fail.
>
>> This is grub 1, not grub 2.
>
> Yes, my instructions were for grub1.
>
>> I
>> haven't updated it yet. It is using lenny. I did start the upgrade to
>> grub2 on the other computer a short time ago, but haven't yet done the
>> final command to finish the change. Trying to solve one thing at a time
>> and right now this is the oldest. I don't recall ever doing any install
>> --stage2=/grub/st.... on the other system.
>>
>> Here is what I get when I do this for sda:
>> ---------------------------------------
>> grub>  device (hd0) /dev/sda
>>
>> grub>  root (hd0,0)
>>   Filesystem type is ext2fs, partition type 0xfd
>>
>> grub>  setup (hd0)
>>   Checking if "/boot/grub/stage1" exists... yes
>>   Checking if "/boot/grub/stage2" exists... yes
>>   Checking if "/boot/grub/e2fs_stage1_5" exists... yes
>>   Running "embed /boot/grub/e2fs_stage1_5 (hd0)"...  15 sectors are
>> embedded.
>> succeeded
>>   Running "install /boot/grub/stage1 (hd0) (hd0)1+15 p
>> (hd0,0)/boot/grub/stage2 /boot/grub/menu.l
>> st"... succeeded
>> Done.
>> ---------------------------------------
>
> I know, it looks fine.  But it screws it up anyway.  It isn't reliable.
>   You have to use the bare install command like I pointed out above.  You
> also need to make sure that your boot partitions on the two drives are
> *identical* in terms of placement on the drives (aka, the partitions
> need to occupy the same logical sector numbers on both drives, which is
> normal when they are both the first partition on each drive and the
> drives use the same sector/head counts per track which is typical now a
> days).
>
>> And I just noticed something.I used sda, not sda1. I can't remember
>> which I did before with the other, but my notes show sda. Would that
>> matter?
>
> Normally sda would be the one I would use, but sda1 would work as long
> as you have a normal DOS master boot record on the drive.
>

At this point I don't know if they are "normal DOS master boot record". 
I set the drives up a year ago or so. Here are the steps from my notes:
---------------------------------------
sudo fdisk /dev/sda
	Command (m for help): <-- n
	Command action
	   e   extended
	   p   primary partition (1-4)
	<-- p
	Partition number (1-4): <-- 1
	First cylinder (1-60801, default 1): <-- <ENTER>
	Using default value 1
	Last cylinder or +size or +sizeM or +sizeK (1-60801, default 60801): 
<-- +25000M
	Command (m for help): <-- t
	Selected partition 1
	Hex code (type L to list codes): <-- fd
	Changed system type of partition 1 to fd (Linux raid auto)
-------
	Command (m for help): <-- n
	Command action
	   e   extended
	   p   primary partition (1-4)
	<-- p
	Partition number (1-4): <-- 2
	First cylinder (3041-60801, default 3041): <-- <ENTER>
	Using default value 3041
	Last cylinder or +size or +sizeM or +sizeK (3041-60801, default 60801): 
<-- +5000M
	Command (m for help): <-- t
	Hex code (type L to list codes): <-- fd
	Changed system type of partition 1 to fd (Linux raid auto)
-------
	Command (m for help): <-- n
	Command action
	   e   extended
	   p   primary partition (1-4)
	<-- p
	Partition number (1-4): <-- 3
	First cylinder (3650-60801, default 3650): <-- <ENTER>
	Using default value 3041
	Last cylinder or +size or +sizeM or +sizeK (3650-60801, default 60801): 
<-- <ENTER>
	Command (m for help): <-- t
	Hex code (type L to list codes): <-- fd
	Changed system type of partition 1 to fd (Linux raid auto)
-------
Make partition 1 bootable
	Command (m for help): <-- a
-------
save and write to disk
	Command (m for help): <-- w
---------------------------------------

I was thinking making one more try, but ether using su instead of sudo 
or just logging in as root as some stuff doesn't seem to work with sudo. 
Need to find the time when I'm home long enough with no interruptions to 
try it:).

Before I try the manual install, do you know what the "1+15 p" is in the 
install line the auto system did?

Re: converting to raid - Error 2

[Doug Ledford]

[-- Attachment #1: Type: text/plain, Size: 2697 bytes --]

On 05/05/2010 02:10 PM, Timothy D. Lenz wrote:

>>> grub>  setup (hd0)
>>>   Checking if "/boot/grub/stage1" exists... yes
>>>   Checking if "/boot/grub/stage2" exists... yes
>>>   Checking if "/boot/grub/e2fs_stage1_5" exists... yes
>>>   Running "embed /boot/grub/e2fs_stage1_5 (hd0)"...  15 sectors are
>>> embedded.
>>> succeeded
>>>   Running "install /boot/grub/stage1 (hd0) (hd0)1+15 p
>>> (hd0,0)/boot/grub/stage2 /boot/grub/menu.l
>>> st"... succeeded
>>> Done.
>>> ---------------------------------------

>> Normally sda would be the one I would use, but sda1 would work as long
>> as you have a normal DOS master boot record on the drive.
>>

A normal DOS master boot record is one that resides in the master boot
record of the drive and that knows to chain load the next boot sector
from the first sector of the partition that's marked active.  So, if you
install a boot loader on sda1, you have to have the normal DOS master
boot record in the master boot record of the drive.  Once you install
grub on the bare drive (aka, on sda instead of sda1), you wipe out any
other boot loader in the master boot record and replace it with a grub
master boot record.  So, at this point you probably don't have a normal
DOS master boot record (unless the grub one suffices but I don't think
it does) and will need to make sure to install whatever new boot loader
you install to sda versus sda1 or else it won't overwrite the previous
grub master boot record.

> I was thinking making one more try, but ether using su instead of sudo
> or just logging in as root as some stuff doesn't seem to work with sudo.
> Need to find the time when I'm home long enough with no interruptions to
> try it:).

Sudo versus su versus logging in as root shouldn't matter as long as
whatever command you are running doesn't come back with some sort of
error message.

> Before I try the manual install, do you know what the "1+15 p" is in the
> install line the auto system did?

Prior to the install line that the setup macro does, it embedded the
e2fs_stage1_5 on the disk between the partition table and the start of
the first partition.  The (hd0)1+15 means to load sectors 1 through 15
of (hd0) as the next stage in the boot loader.  The p is separate and
merely means patch the block list for the next part of the boot loader
into the previous part.  It can be omitted as grub does this anyway
based upon the number of stages you specify.

-- 
Doug Ledford <dledford@redhat.com>
              GPG KeyID: CFBFF194
	      http://people.redhat.com/dledford

Infiniband specific RPMs available at
	      http://people.redhat.com/dledford/Infiniband


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: converting to raid - Error 2

[Timothy D. Lenz]

On 5/5/2010 4:57 PM, Doug Ledford wrote:
> On 05/05/2010 02:10 PM, Timothy D. Lenz wrote:
>
>>>> grub>   setup (hd0)
>>>>    Checking if "/boot/grub/stage1" exists... yes
>>>>    Checking if "/boot/grub/stage2" exists... yes
>>>>    Checking if "/boot/grub/e2fs_stage1_5" exists... yes
>>>>    Running "embed /boot/grub/e2fs_stage1_5 (hd0)"...  15 sectors are
>>>> embedded.
>>>> succeeded
>>>>    Running "install /boot/grub/stage1 (hd0) (hd0)1+15 p
>>>> (hd0,0)/boot/grub/stage2 /boot/grub/menu.l
>>>> st"... succeeded
>>>> Done.
>>>> ---------------------------------------
>
>>> Normally sda would be the one I would use, but sda1 would work as long
>>> as you have a normal DOS master boot record on the drive.
>>>
>
> A normal DOS master boot record is one that resides in the master boot
> record of the drive and that knows to chain load the next boot sector
> from the first sector of the partition that's marked active.  So, if you
> install a boot loader on sda1, you have to have the normal DOS master
> boot record in the master boot record of the drive.  Once you install
> grub on the bare drive (aka, on sda instead of sda1), you wipe out any
> other boot loader in the master boot record and replace it with a grub
> master boot record.  So, at this point you probably don't have a normal
> DOS master boot record (unless the grub one suffices but I don't think
> it does) and will need to make sure to install whatever new boot loader
> you install to sda versus sda1 or else it won't overwrite the previous
> grub master boot record.
>
>> I was thinking making one more try, but ether using su instead of sudo
>> or just logging in as root as some stuff doesn't seem to work with sudo.
>> Need to find the time when I'm home long enough with no interruptions to
>> try it:).
>
> Sudo versus su versus logging in as root shouldn't matter as long as
> whatever command you are running doesn't come back with some sort of
> error message.
>
>> Before I try the manual install, do you know what the "1+15 p" is in the
>> install line the auto system did?
>
> Prior to the install line that the setup macro does, it embedded the
> e2fs_stage1_5 on the disk between the partition table and the start of
> the first partition.  The (hd0)1+15 means to load sectors 1 through 15
> of (hd0) as the next stage in the boot loader.  The p is separate and
> merely means patch the block list for the next part of the boot loader
> into the previous part.  It can be omitted as grub does this anyway
> based upon the number of stages you specify.
>

Thank you for the reply and I should be able to follow it, but I think I 
get confused trying to follow it. Looks like you are saying I messed up 
the MBR.

First a bit of backing up a few post, you mentioned adding the "d" option:

-------------------------------------------------------------------
"install --stage2=/grub/stage2 /grub/stage1 (hd0) /grub/e2fs_stage1_5
/grub/stage2 /grub/grub.conf

and if that doesn't work in your particular configuration, you can add
the d option after stage1 and before (hd0), but if you use it, then your
boot disk must always be BIOS device 0x80, which means setting your BIOS
to boot off of some disk other than the first disk found usually won't
work and instead you just have to make whatever disk you want to boot
off of the first disk found by the BIOS."
-------------------------------------------------------------------

I want to avoid that because it would defeat one of the advantages of 
mirroring. Today the 32bit system, the one that I got converted seems to 
have started loosing sda after a shutdown restart, but it came up fine 
because it rolled over and booted on sdb. More about that down near the 
bottom and new possible complications with that...

-------------------------------------------------------------------
"You can switch (hd0) to (hd0,0) if you want and if you have a normal 
DOS master boot record."
-------------------------------------------------------------------

 From what I understand that would be the same as using sda1 instead of sda?

To use your install line, the instructions I had did the following:

  grub>device (hd0) /dev/sda
  grub>root (hd0,0)

first so that grub would treat the target drive as (hd0) for setting up 
boot information, maybe something to do with information in menu.lst 
which only points (hd0,0) as the boot device:


title		Debian GNU/Linux, kernel 2.6.25.9.20081002.1
root		(hd0,0)
kernel		/boot/vmlinuz-2.6.25.9.20081002.1 root=/dev/md0 ro quiet

title		Debian GNU/Linux, kernel 2.6.25.9.20081002.1 (single-user mode)
root		(hd0,0)
kernel		/boot/vmlinuz-2.6.25.9.20081002.1 root=/dev/md0 ro quiet single

I should still do that with your install command? I don't want to mess 
the current working boot drive or the system will no longer be bootable. 
Want to make sure it acts on the correct drive.

I got lost in the problems with the MBR. I used fdisk to set the first 
partition of each drive bootable. Do I need to mark them unbootable and 
then back again to fix the MBR or will the above install line fix it..

-------------------------------------------------------------------
The (hd0)1+15 means to load sectors 1 through 15
of (hd0) as the next stage in the boot loader.  The p is separate and
merely means patch the block list for the next part of the boot loader
into the previous part.  It can be omitted as grub does this anyway
based upon the number of stages you specify.
-------------------------------------------------------------------

To be clear, the "1+15" can also be omitted correct?
===============================================================

Now, 1 step forward, 4 back :(.  While waiting for a reply to my last 
post, I had a bit of time so thought I would just go ahead and try using 
su instead of sudo... Turned on the monitor and set the kvm to that 
computer and found:

   end_request: I/O error, dev fd0, sector 0

repeated 5 times on the screen. No idea why it started trying to read 
the floppy. Last reboot was Apr 28 19:16 and in the logs:

Apr 28 19:16:29 x64VDR kernel: EXT3-fs: mounted filesystem with ordered 
data mode.
Apr 28 19:16:29 x64VDR kernel: NET: Registered protocol family 10
Apr 28 19:16:34 x64VDR kernel: warning: `ntpd' uses 32-bit capabilities 
(legacy support in use)
Apr 28 19:16:35 x64VDR kernel: Clocksource tsc unstable (delta = 
-77401545 ns)
Apr 28 19:16:39 x64VDR kernel: eth0: no IPv6 routers present
Apr 30 14:53:38 x64VDR kernel: end_request: I/O error, dev fd0, sector 0
Apr 30 15:13:58 x64VDR kernel: end_request: I/O error, dev fd0, sector 0
Apr 30 16:30:30 x64VDR kernel: end_request: I/O error, dev fd0, sector 0
Apr 30 16:31:43 x64VDR kernel: end_request: I/O error, dev fd0, sector 0
Apr 30 17:02:30 x64VDR kernel: end_request: I/O error, dev fd0, sector 0
May  2 00:57:01 x64VDR kernel: md: data-check of RAID array md0

Good news, the array check on the 3 arrays seemed ok.

But I also found several entries of this in the user.log

   Apr 28 19:16:47 x64VDR usbmount[1388]: cannot read from /dev/sdd

Only time I know sdd is referenced is when I boot with the cd to copy 
sync from the working drive hda1 (which becomes sdd1) to the boot array 
md0, or the array I've been trying to get bootable. And it's talking 
about a usb device. Now I did plug a usb drive in some time back to copy 
data over, but that was unmounted and removed after the copy and the 
computer has been rebooted since then. yet I also find inthe current 
dmesg log:

scsi 6:0:0:0: Direct-Access     Generic  STORAGE DEVICE   9325 PQ: 0 ANSI: 0
sd 6:0:0:0: [sdd] Attached SCSI removable disk
usb-storage: device scan complete

The floppy drive is one of those combos with built in card readers which 
is connected to usb, but I never got that to work right. only time it 
would reconize when a card was plugged in or removed was when you also 
added ore removed a usb memory stick or drive... Strange and anoying.

Soooo, instead of try to make the array bootable again, I decided to 
check email... and have to nice little emails from the 32bit system that 
I was able to get booting on raid. Both said the same thing...

-------------------------------------------------------------------
From: mdadm monitoring <root>
subject: Fail event on /dev/md0:LLLx64-32

This is an automatically generated mail message from mdadm
running on LLLx64-32

A Fail event had been detected on md device /dev/md0.

It could be related to component device /dev/sda1.

Faithfully yours, etc.

P.S. The /proc/mdstat file currently contains the following:

Personalities : [raid1] [raid6] [raid5] [raid4] [multipath]
md1 : active raid1 sdb2[1] sda2[0]
       4891712 blocks [2/2] [UU]

md2 : active raid1 sdb3[1] sda3[2](F)
       459073344 blocks [2/1] [_U]

md0 : active raid1 sdb1[1] sda1[2](F)
       24418688 blocks [2/1] [_U]

unused devices: <none>
-------------------------------------------------------------------

I had shut it down to put a second ATSC tuner card in trying to track 
down another problem I've been fighting since I set it up. After it was 
up I noticed a high pitched wine, a little louder then is normal for WD 
drives, then it went away. have noticed it before from time to time, but 
not often. Guess this confirms a possible from another thread I had:

"possible bus loading problem during resync"

So now i get to figure out setting up a replacement drive and going 
through grub on that one again. Only, I started the update to grub 2 on 
that system. It's at the 1/2 way point of the change over. I was trying 
to find out exactly what to check for to be sure it would work if I 
finished the conversion and /boot/grub is full of .mod files. This 
should complicate getting that new drive setup.. :(