Problems setting up RAID1

Problems setting up RAID1

[George Huber]

all,

I am having problems setting up RAID 1 using Fedora core 3.  Here are
the steps that I have taken to date.

1) On the first SCSI drive, I installed a clean version of Linux, the
    hard drive is partitioned as follows:
    /dev/sda1              /boot              512 Mbytes
    /dev/sda2              /cvsroot           20 GBytes
    /dev/sda3              /archives          20 GBytes
    /dev/sda4     (extended partition)
    /dev/sda5              /home              10 GBytes
    /dev/sda6              swap                2 GBytes
    /dev/sda7              /                  14 GBytes

2) Downloaded the souce code for 2.6.11-8 recompiled the kernel with
    RAID support included in the kernel, not as modules.

3) Build the partions on the second SCSI drive to match the first.

4) Set the partition types of sd[ab]2, sd[ab]3, sd[ab]5, sd[ab]6 and
    sd[ab]7 to 0xFD (Linux Software Raid Autodetect)

5) rebooted the host, on reboot I had three mirroed partitions:
    /dev/md0 : /dev/sda2 and /dev/sdb2
    /dev/md1 : /dev/sda3 and /dev/sdb3
    /dev/md2 : /dev/sda5 and /dev/sdb5

    Question 1 - why did swap and root fail to become part of the array?

6) using mdadm with the examine flag, I found that neither /dev/sd[ab]6
or
    /dev/sd[ab]7 had raid superblocks on them.  Attempting to build
arrays
    using:
    mdadm --create /dev/md3 --level=1 --raid-devices=2 /dev/sd[ab]6
    failed (sort of expected since swap was mounted).

7) booted with a rescue disk, and did not mount any of my partitions,
    and then executed the following commands:

    mdadm --create /dev/md3 --level=1 --raid-devices=2 /dev/sd[ab]6
    mdadm --create /dev/md4 --level=2 --raid-devices=2 /dev/sd[ab]7

    both of these commands succeeded, and cat-ing /proc/mdstat showed
    to raid arrays that were in the process of synching.

8) I let the two arrays finish synching, and then rebooted the computer

    normally.  I received a `ton' of messages about not being able to
    touch various files in the root file system because the root file
    system is read-only.  After a while I finally did get a shell, and
    I was able cat /proc/mdstat - I saw the expected five arrays,
however
    the array for the root file system only had one drive in it:

    ....
    md4 : active raid sdb7[1]
        11759424 blocks [2/1] [_U]
    ....

    All the other raid arrays had two entries (as expected) as were
readible
    as well as writable.

    On examining dmesg, I fould the following:

    md: Autodetecting RAID arrays
    md: could not bd_claim sda7
    md: autorun...
    md: considering sdb7...
    md:  adding sdb7...

    so it seems that the RAID subsysem is not able to access / control
    my root partition.

    Question 2 - What do I need to do to be able to RAID my root
partition?
    Question 3 - What do I need to do to fix my `read-only' file system
                 problems?
      Questions 2 and 3 are probably related.

    Note: I was able to execute the command:
    mount / -o remount,rw

    to make my root filesystem read-write.

9) Finally, I rebooted into rescue mode without mounting any of my
drives,
    manually mounted /dev/sda7 and copied /etc/mdadm.conf to the floppy
    and then unmounted /dev/sda7.  Next I ran:
    mdadm --assemble --scan --config=/mnt/floppy/mdadmconf
    and all five of my RAID arrays came up, with two members each and
the
    array holding my root file system was writable.

Any suggestiong on what I should try next.

Re: Problems setting up RAID1

[Andre Noll]

On Tue, 17 May 2005 13:48:21 -0400 you wrote in local.lists.linux-raid:
> I am having problems setting up RAID 1 using Fedora core 3.  Here are
> the steps that I have taken to date.

[...]


>     Note: I was able to execute the command:
>     mount / -o remount,rw

That command should be the first thing to to after fscking /. Usually it
is run from a bootup script.

> Any suggestiong on what I should try next.

Did you try root=/dev/md7 in your lilo.conf (or at the lilo prompt)?

And please check you have the correct rootfs entry in your /etc/fstab.
-- 
Andre Noll, http://www.mathematik.tu-darmstadt.de/~noll

Re: Problems setting up RAID1

[George Huber]

Andre Noll wrote:
> On Tue, 17 May 2005 13:48:21 -0400 you wrote in local.lists.linux-raid:
> 
>>I am having problems setting up RAID 1 using Fedora core 3.  Here are
>>the steps that I have taken to date.
> 
> 
> [...]
> 
> 
> 
>>    Note: I was able to execute the command:
>>    mount / -o remount,rw
> 
> 
> That command should be the first thing to to after fscking /. Usually it
> is run from a bootup script.
> 

Agreed, however it appears that the system is not doing this from a 
start up script due to the large numbers of error messages regarding a 
read-only file system.  I tried this from a command prompt in the hope
that it would give me an error message that I could use in 
trouble-shooting but it worked normally.
> 
>>Any suggestiong on what I should try next.
> 
> 
> Did you try root=/dev/md7 in your lilo.conf (or at the lilo prompt)?
> 
Fedora core 3 used GRUB as the boot loader (part of my problem may be 
that all of the how-to's that I have found still deal with lilo).  The
only options for a boot loader with Fedora core 3 are 'GRUB' and not to
install a boot loader.  Any tips on using GRUB with RAID

> And please check you have the correct rootfs entry in your /etc/fstab.

Verified.

Thanks,
George

Re: Problems setting up RAID1

[Andre Noll]

On Wed, 18 May 2005 16:32:54 -0400 you wrote in local.lists.linux-raid:

>> Did you try root=/dev/md7 in your lilo.conf (or at the lilo prompt)?
>> 
> Fedora core 3 used GRUB as the boot loader (part of my problem may be 
> that all of the how-to's that I have found still deal with lilo).  The
> only options for a boot loader with Fedora core 3 are 'GRUB' and not to
> install a boot loader.  Any tips on using GRUB with RAID

Nope, I've never used grub. Be aware that strange things may happen if
the boot loader tells the kernel to use /dev/sda7 or /dev/sdb7 instead
of /dev/md7 as the root device. However, this is only a problem with
linux 2.4, see the thread on raid1-diseaster on reboot.

Re: Problems setting up RAID1

[Gil]

George Huber wrote:
> all,
>
> 
> I am having problems setting up RAID 1 using Fedora core 3.  Here are
> the steps that I have taken to date.
> 
> 1) On the first SCSI drive, I installed a clean version of Linux, the
>    hard drive is partitioned as follows:
>    /dev/sda1              /boot              512 Mbytes
>    /dev/sda2              /cvsroot           20 GBytes
>    /dev/sda3              /archives          20 GBytes
>    /dev/sda4     (extended partition)
>    /dev/sda5              /home              10 GBytes
>    /dev/sda6              swap                2 GBytes
>    /dev/sda7              /                  14 GBytes

If I understand you correctly, I think you went wrong by installing
first and *then* trying to go to RAID.  In the general case you need to
create raid devices and then filesystems on top of those devices, not
the other way around.

Changing a filesystem in-place to be hosted on an MD device is possible
but I couldn't tell you how to do it (it involves shrinking the FS just so).

If you're willing to start again, the FC3 installer's parititioning app
is pretty good and should be able create all your RAID devices and
filesystems.  You shouldn't have to do anything manually unless you want
to stack RAID devices or something like that.

If you don't like Disk Druid, at the point where it prompts you to start
partitioning, you can always drop out to the root prompt with
Ctrl+Alt+F3 (or F2, I don't remember) and issue the fdisk commands
yourself.  The sequence you might use is:

1. fdisk /dev/sda, all partitions except sda4 get type 0xfd.

2. copy partitions from sda to sdb, e.g.
sfdisk -d /dev/sda | sfdisk /dev/sdb

3. create raid partitions, e.g.
mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sd[ab]1
mdadm --create /dev/md3 --level=1 --raid-devices=2 /dev/sd[ab]6
...

4. create filesystems and swap
mke2fs -j /dev/md0 # other options or filesystems probably apply
mkswap /dev/md3
...

5. return to installation via Ctrl-Alt-F7, and select manual
partitioning.  This should reflect your partitioning choices -- all you
should need to do is nominate where the partitions get mounted.

6. proceed with installation as normal.


If you had wanted to make /boot a RAID1 as well, you might need to take
an extra step because the FC3 installer may not get that right,
regardless of your choice to use Disk Druid to lay out the partitions or
some other manual process.  That step, in your case, would be:

7. At the point where the installer prompts you to reboot for the first
time after copying, drop out to the shell again and run grub by hand.
Fedora's grub doesn't understand MD devices so you may need to install
boot sectors by hand.  See
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=114690 for discussion.

Supposedly the installer is smarter than just calling grub-install, but
I don't trust it and install the bootsectors myself.

# /mnt/sysimage/sbin/grub # can't remember if chroot is required
device (hd0) /dev/sda
root (hd0,0)
setup (hd0)
device (hd0) /dev/sdb
root (hd0,0)
setup (hd0)

This is a bit of a cargo-cult invocation because I don't understand grub
well enough to really know what's really going on.  I had read elsewhere
(and since forgotten the link), that the device lines make sure that
grub treats both disks as a boot disk even though sdb is not sda.

The explanation is probably rubbish, but that sequence does leave me
with two bootable disks (I can boot from either in the BIOS).


> 2) Downloaded the souce code for 2.6.11-8 recompiled the kernel with
>    RAID support included in the kernel, not as modules.

Fedora's mkinitrd process takes care of loading the necessary modules so
don't do this unless you need to.  (I had to because I bought SATA disks
and want to monitor them with SMART).

> 3) Build the partions on the second SCSI drive to match the first.
> 
> 4) Set the partition types of sd[ab]2, sd[ab]3, sd[ab]5, sd[ab]6 and
>    sd[ab]7 to 0xFD (Linux Software Raid Autodetect)

I think the rest of this may be follow on damage from the fact that you
created md devices over existing filesystems.

> 
> 5) rebooted the host, on reboot I had three mirroed partitions:
>    /dev/md0 : /dev/sda2 and /dev/sdb2
>    /dev/md1 : /dev/sda3 and /dev/sdb3
>    /dev/md2 : /dev/sda5 and /dev/sdb5
> 
>    Question 1 - why did swap and root fail to become part of the array?
>
> [snip]

--Gil

Re: Problems setting up RAID1

[George Huber]

Gil wrote:
> George Huber wrote:
> 
>>all,

[...snip...]

Thanks Gil, your instructions proved invaluable and I have the
RAID setup.

George

Re: Problems setting up RAID1

[George Huber]

Gil wrote:
> George Huber wrote:
> 
[...snip...}
Thanks,

Your instructions worked perfectly.  I have my RAID setup.

George

Re: Problems setting up RAID1

[Tyler]

[-- Attachment #1: Type: text/plain, Size: 101 bytes --]

I posted this response about 2 weeks ago, for someone trying to do the 
same thing.

Regards,
Tyler.

[-- Attachment #2: Re: raid 1 with static kernel. --]
[-- Type: message/rfc822, Size: 3849 bytes --]

From: Tyler <pml@dtbb.net>
To: Glatzel Tino <tino.glatzel@badenIT.de>, linux-raid@vger.kernel.org
Subject: Re: raid 1 with static kernel.
Date: Wed, 04 May 2005 08:56:21 -0700
Message-ID: <4278F0A5.60406@dtbb.net>

Try this howto:

http://www.linuxsa.org.au/mailing-list/2003-07/1270.html

Regards,
Tyler.

Glatzel Tino wrote:

>hello list,
>
>
>my system is a Debian GNU Linux Sarge. The kernelversion is 2.6.10 without
>modulsupport.
>The partion id is  fd.
>In the first step, the complete system is installed on /dev/hda3.
>In the second step, i have created a raid 1 device:
>mdadm --create /dev/md0 --level 1 --raid-device=2 /dev/hda1 /dev/hdc1
>mkfs -t ext3 /dev/md0
>mount /dev/md0 /mnt
>cp -pr /boot /mnt
>umount /mnt
>
>I want to mount /dev/md0 in /boot
>change in the /etc/fstab
>/dev/md0       /boot   ext3    defaults    0      2
>
>
>
>after a reboot, i see the mounted devices:
>
>/dev/hda3 on / type ext3 (rw,errors=remount-ro)
>/dev/md0 on /boot type ext3 (rw)
>
>so, i think i will never need the files in the old /boot directory.
>
>umount /boot
>cd /boot
>ls
>System.map-2.6.8-1-386
>config-2.6.8-1-386      vmlinuz-2.6.10
>grub                    vmlinuz-2.6.10-raid-grsec
>initrd.img-2.6.8-1-386  vmlinuz-2.6.10-raid-grsec-hotplug
>lost+found              vmlinuz-2.6.8-1-386
>System.map-2.6.10
>
>
>
>rm -r *
>cd ..
>mount /boot
>ls /boot
>System.map-2.6.8-1-386
>config-2.6.8-1-386      vmlinuz-2.6.10
>grub                    vmlinuz-2.6.10-raid-grsec
>initrd.img-2.6.8-1-386  vmlinuz-2.6.10-raid-grsec-hotplug
>lost+found              vmlinuz-2.6.8-1-386
>System.map-2.6.10
>
>
>after a reboot, i see the following message from Grub:
>
>GRUB Loading Stage1.5
>
>GRUB loading, please wait ...
>Error 15
>
>
>Ok, i think GRUB can`t access to /dev/md0
>what can i do?
>
>
>Tino
>-
>To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>
>  
>

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html