Software RAID-1: step-by-step help needed...

Software RAID-1: step-by-step help needed...

[Boniforti Flavio]

Hello all!
I'm a pure newbie in facts of RAID on Linux, therefore I'd like to get 
some help and some explanations about RAID-1 (Software-RAID) onto a 
Linux server I have to configure.
How should I proceed to get my RAID-1 set up and working?
Are there known issues about the booting process on RAID?
Do I have to "mirror" each single partition or is it possible to 
"mirror" the whole disk?

Thank you in advance.
-- 
-----------------------------------
Boniforti Flavio
Provincia del Verbano-Cusio-Ossola
Ufficio Informatica

Tecnoparco del Lago Maggiore
Via dell'Industria, 25
28924 Verbania
-----------------------------------

Re: Software RAID-1: step-by-step help needed...

[Gordon Henderson]

On Tue, 10 Aug 2004, Boniforti Flavio wrote:

> Hello all!
> I'm a pure newbie in facts of RAID on Linux, therefore I'd like to get
> some help and some explanations about RAID-1 (Software-RAID) onto a
> Linux server I have to configure.
> How should I proceed to get my RAID-1 set up and working?
> Are there known issues about the booting process on RAID?
> Do I have to "mirror" each single partition or is it possible to
> "mirror" the whole disk?

You can do either. Personally, I create identical partitions on each drive
and then mirror them.

Probably the easiest way to get started is to pick a distribution that
lets you build the disks as a mirror from install time - Eg. Red Hat (or
whatever it's called these days) If you have an existing server with one
disk and want to add a 2nd as a mirror, thats not too difficult. you need
to read the stuff in the Software RADI HowTo which has the information on
how to do it all.

Booting RAID1 with Lilo is easy, I presume you can do it too with grub,
but I don't use grub.

Gordon

Re: Software RAID-1: step-by-step help needed...

[Boniforti Flavio]

Gordon Henderson wrote:

> You can do either. Personally, I create identical partitions on each drive
> and then mirror them.

OK. What would be the difference (in terms of performance and/or ease of 
configuration/maintenance) between the two methods?

> Probably the easiest way to get started is to pick a distribution that
> lets you build the disks as a mirror from install time - Eg. Red Hat (or
> whatever it's called these days) If you have an existing server with one
> disk and want to add a 2nd as a mirror, thats not too difficult. you need
> to read the stuff in the Software RADI HowTo which has the information on
> how to do it all.

I see, but I *have to* install Debian on it.
I already installed it (as a test) following the instructions found at:
http://www.inittab.de/manuals/debootstrap.html
I guess the hardest task will be configuring RAID, as the system works 
with the above procedure...

> Booting RAID1 with Lilo is easy, I presume you can do it too with grub,
> but I don't use grub.

I'll stick with LiLo because I'm used to use it on other systems.

Regards,
-- 
-----------------------------------
Boniforti Flavio
Provincia del Verbano-Cusio-Ossola
Ufficio Informatica

Tecnoparco del Lago Maggiore
Via dell'Industria, 25
28924 Verbania
-----------------------------------

Re: Software RAID-1: step-by-step help needed...

[Gordon Henderson]

On Tue, 10 Aug 2004, Boniforti Flavio wrote:

> Gordon Henderson wrote:
>
> > You can do either. Personally, I create identical partitions on each drive
> > and then mirror them.
>
> OK. What would be the difference (in terms of performance and/or ease of
> configuration/maintenance) between the two methods?

OK - this is a very intersting and highly debatable subject!

Personally, I like partitions - lots of them, but thats becasue I'm a
bolring old fart who's been involved with Unix systems of one sort or
another for over 20 years now, and if paritions were good enough for me 20
years ago then they'll work now... (No, thats probably not a good
argument, but hey)

These days I have 2 basic schemes - firstly

  /		Small partition, usually 256MB or so.
  swap		Double RAM just for old-times sake.
  /usr		2GB. Maybe 1GB if it's not running X and space is tight.
  /var		Rest of disk and arrange /home to be a symlink to
			/var/home

This isn't ideal, if someone fills up /var then you can have problems
logging in for example, but just about every other scenario has issues
too.  On a bigger capacity server then I might do the following:

  /		Big partition of maybe 4GB. Includes /var to hold
		mail spool, etc.
  swap		Double RAM
  /usr		1 or 2GB as before
  /mounts/local			Half of rest of disk
  /mounts/local.yesterday	Remaining space

I'll then nightly archive the local partition to the yesterday partition
then re-mount it read-only and dump it to tape. (problems with dump &
ext2/3 on live filesystems)

These aren't ideal (what is?) but with reasonably well behaved users it
works very well and I've been using setups like this for quite some years
now with very results.

Some Linux distros will create a smaller /boot partition. I understand
this was required in the bad old days when the BIOIS could only address
1024 cylinders, so you had to make sure the kernel image was inside that
limit. These days with a modern BIOS it's not required (although others
will have other probably good reasons for using it anyway)

Traditionally, Windows will just put one huge partition on the whole disk
(or maybe 2 once disk got over 4GB and FAT couldn't handle it) and some of
the other Linux distributions do just them themselves.  Technically you
get more disk space avalable to you that way, but these days disk space is
cheap, and with multiple partitions the chance of a head crash taking out
the entire disk is rare and hopefully only limited to one partition (thats
one theory for multiple partitions anyway) (Do heads still "crash"?
Probably unlikely with modern drives, but who knows...)

> > Probably the easiest way to get started is to pick a distribution that
> > lets you build the disks as a mirror from install time - Eg. Red Hat (or
> > whatever it's called these days) If you have an existing server with one
> > disk and want to add a 2nd as a mirror, thats not too difficult. you need
> > to read the stuff in the Software RADI HowTo which has the information on
> > how to do it all.
>
> I see, but I *have to* install Debian on it.

OK. Debian is good - it's what I use :)

But Woddy doesn't have an installer that'll let you install directly onto
RAID )-: I head Sarge will, but thats not going to be releases for a wee
while yet...

> I already installed it (as a test) following the instructions found at:
> http://www.inittab.de/manuals/debootstrap.html

Er - thats a rather "intersting" way to install Debian - I use the
standard Woody CD set..

> I guess the hardest task will be configuring RAID, as the system works
> with the above procedure...

OK. So with Debian, I basic, but minimal system on the first drive, not
touching the 2nd drive at all during this process... I'll usually compile
up a custome kernel at this point to make sure the IDE controllers have
DMA, if nothing else.

Then with the 2nd drive, partition it to be identical to the first disk,
then follow through the runes on the HowTo for mirroring the disks.

You can do it 2 ways - either create a mirror set with 1 disk, the other
being degraded immediately, then raidhotadd it, or create them step by
step, temporarily relocating each partition as you go. (Eg into the swap
partition, after you turn swap off!) I use the latter method.

First, move root to another partition:

Eg. mkfs /dev/hda2 (swap partition), mount /dev/hda2 /mnt ; cd / ;
find . -xdev | cpio -pm /mnt

This is the hard part - cd /mnt, edit /mnt/etc/fstab and /mnt/etc/lilo to
reflect the fact that root is now on /dev/hda2, (make sure swap isn't
going to be mounted!) run lilo -r /mnt and reboot. Hopefully the system
will now boot with /dev/hda2 being root.

I don't know madm, so (because I've been doing it this way for the past 6
years) I then edit an /etc/raidtab to create a raid1 with /dev/hda1 and
/dev/hdc1, mkraid it, mkfs it, mount it under /mnt and copy root back into
it as before, edit /mnt/etc/lilo.conf and /mnt/etc/fstab to make it
mounted under /dev/md0. The key lines in lilo.conf are then:

  boot=/dev/md0
  root=/dev/md0
  raid-extra-boot=/dev/hda,/dev/hdc

and run lilo -r /mnt

(of-course, substitute /dev/sdX for hdX if you are using SCSI)

finally, use cfdisk to set the partition type to fd for /dev/hda1 and
/dev/hdc1 and reboot...

If this works you'll then have root under /dev/md0 and therefore under
RAID1... Then you just repeat this for the other partitions - copy them
into a spare partition (eg the swap one), create an entry in /etc/raditab
for them, mkraid/mkfs/mount/copy/ edit /etc/fstab, cfdisk the partition
types and off you go. Finally make the swap partition RAID1, run mkswap on
it, and re-enable it in the fstab.

It's tedious, but it doesn't really take that long if you just install a
bare minimal system to start with, then raid the lot, then use dselect or
tasksel to install all the other applications you need.

One other tip if you are using ext3 is to tune2fs -i0 -c0 /dev/hdaX after
the mkfs -t ext3 ... to make sure it's not force checked after so many
days/mounts.

Enjoy...

Gordon

Re: Software RAID-1: step-by-step help needed...

[Boniforti Flavio]

Gordon Henderson wrote:

> OK - this is a very intersting and highly debatable subject!

[cut]

Well, I didn't mention at all what my server will have to do: it's 
simply a squid-proxy and will have apache installed (with PHP and 
mySQL), but just for small "intranet" applications (more "testing" like)...
This is why I was about to apply following partition scheme:

/dev/sda1 - 20 GB (Primary, ext3)
/dev/sda2 -  1 GB (Primary, SWAP)
/dev/sda5 - 52 GB (Logical, ext3)

I will install / onto sda1 and /var onto sda5, what do you think?

[snip]

> Some Linux distros will create a smaller /boot partition. I understand
> this was required in the bad old days when the BIOIS could only address
> 1024 cylinders, so you had to make sure the kernel image was inside that
> limit. These days with a modern BIOS it's not required (although others
> will have other probably good reasons for using it anyway)

I don't know if I should bother with a separate /boot partition, because 
actually every Debian boots off >1024 cyl...

[cut]

>>I already installed it (as a test) following the instructions found at:
>>http://www.inittab.de/manuals/debootstrap.html
> 
> 
> Er - thats a rather "intersting" way to install Debian - I use the
> standard Woody CD set..

Yes, but I have to install it that way, because the original Woody discs 
  don't contain a working driver for my SCSI Controller. :(

>>I guess the hardest task will be configuring RAID, as the system works
>>with the above procedure...
> 
> 
> OK. So with Debian, I basic, but minimal system on the first drive, not
> touching the 2nd drive at all during this process... I'll usually compile
> up a custome kernel at this point to make sure the IDE controllers have
> DMA, if nothing else.

I will install the system with the above mentioned "alien" procedure, 
then I would like to *first* activate RAID-1 (maybe it could need to 
recompile?) and then boot off RAID and see that everything works...

> Then with the 2nd drive, partition it to be identical to the first disk,
> then follow through the runes on the HowTo for mirroring the disks.

I already partitioned both disks the same way (booting with System 
Rescue CD and using cfdisk).

> You can do it 2 ways - either create a mirror set with 1 disk, the other
> being degraded immediately, then raidhotadd it, or create them step by
> step, temporarily relocating each partition as you go. (Eg into the swap
> partition, after you turn swap off!) I use the latter method.
> 
> First, move root to another partition:
> 
> Eg. mkfs /dev/hda2 (swap partition), mount /dev/hda2 /mnt ; cd / ;
> find . -xdev | cpio -pm /mnt
> 
> This is the hard part - cd /mnt, edit /mnt/etc/fstab and /mnt/etc/lilo to
> reflect the fact that root is now on /dev/hda2, (make sure swap isn't
> going to be mounted!) run lilo -r /mnt and reboot. Hopefully the system
> will now boot with /dev/hda2 being root.
> 
> I don't know madm, so (because I've been doing it this way for the past 6
> years) I then edit an /etc/raidtab to create a raid1 with /dev/hda1 and
> /dev/hdc1, mkraid it, mkfs it, mount it under /mnt and copy root back into
> it as before, edit /mnt/etc/lilo.conf and /mnt/etc/fstab to make it
> mounted under /dev/md0. The key lines in lilo.conf are then:
> 
>   boot=/dev/md0
>   root=/dev/md0
>   raid-extra-boot=/dev/hda,/dev/hdc
> 
> and run lilo -r /mnt
> 
> (of-course, substitute /dev/sdX for hdX if you are using SCSI)
> 
> finally, use cfdisk to set the partition type to fd for /dev/hda1 and
> /dev/hdc1 and reboot...
> 
> If this works you'll then have root under /dev/md0 and therefore under
> RAID1... Then you just repeat this for the other partitions - copy them
> into a spare partition (eg the swap one), create an entry in /etc/raditab
> for them, mkraid/mkfs/mount/copy/ edit /etc/fstab, cfdisk the partition
> types and off you go. Finally make the swap partition RAID1, run mkswap on
> it, and re-enable it in the fstab.

I don't actually understand what you are doing hereby... I mean: doesn't 
it suffice to start RAID and it handles the "mirroring" and all the 
stuff? Just following the HOW TO wouldn't be enough?

> It's tedious, but it doesn't really take that long if you just install a
> bare minimal system to start with, then raid the lot, then use dselect or
> tasksel to install all the other applications you need.
> 
> One other tip if you are using ext3 is to tune2fs -i0 -c0 /dev/hdaX after
> the mkfs -t ext3 ... to make sure it's not force checked after so many
> days/mounts.

That's another "black hole" in my knowledge: what is this thing above?!?

> Enjoy...
> 
> Gordon

Thank you very much for your time and your explanations...

-- 
-----------------------------------
Boniforti Flavio
Provincia del Verbano-Cusio-Ossola
Ufficio Informatica

Tecnoparco del Lago Maggiore
Via dell'Industria, 25
28924 Verbania
-----------------------------------

Re: Software RAID-1: step-by-step help needed...

[Daniel Pittman]

On 10 Aug 2004, Boniforti Flavio wrote:
> Gordon Henderson wrote:
>
>> You can do either. Personally, I create identical partitions on each drive
>> and then mirror them.
>
> OK. What would be the difference (in terms of performance and/or ease of
> configuration/maintenance) between the two methods?

Partitions are vastly easier than whole-disk mirrors. If you set the
partition type to 'fd' and compile in the requisite RAID stuff the
kernel will start the RAID devices for you automatically.

Doing whole-disk mirrors, or modular RAID, requires an initrd to be in
place to create the root disk for you. This is harder, although some
distributions automate the process for you.

>> Probably the easiest way to get started is to pick a distribution that
>> lets you build the disks as a mirror from install time - Eg. Red Hat (or
>> whatever it's called these days) If you have an existing server with one
>> disk and want to add a 2nd as a mirror, thats not too difficult. you need
>> to read the stuff in the Software RADI HowTo which has the information on
>> how to do it all.
>
> I see, but I *have to* install Debian on it.
> I already installed it (as a test) following the instructions found at:
> http://www.inittab.de/manuals/debootstrap.html
> I guess the hardest task will be configuring RAID, as the system works 
> with the above procedure...

The easiest way to get a RAID array in Debian is to install the testing
or unstable branch; the installer there supports creating the devices as
part of the boot sequence.

For Debian/stable, the process is also relatively easy but not
automatic:

1. Partition both disks the way you want.
2. Install Debian to the first hard disk, with no RAID.
3. Install mdadm.
4. Create "degraded" RAID devices using only the second disk.
5. Boot to a non-intrusive mode (init=/bin/sh, usually).
6. Use 'tar' or whatever to transfer data from the hard disk to the RAID
   device.
7. Edit fstab, etc, in the RAID partition.
8. Boot with the RAID device as your root device.
9. Hot-add the other partitions to the running RAID devices.

This is a pretty trivial process, but a bit time consuming. Degraded
mirrors make it pretty easy, though, and I have done it many times while
installing systems.

Regards,
        Daniel
-- 
All television is children's television.
        -- Richard P. Adler

Re: Software RAID-1: step-by-step help needed...

[David Greaves]

>>I see, but I *have to* install Debian on it.
>>I already installed it (as a test) following the instructions found at:
>>http://www.inittab.de/manuals/debootstrap.html
>>I guess the hardest task will be configuring RAID, as the system works 
>>with the above procedure...
>>    
>>
>
>The easiest way to get a RAID array in Debian is to install the testing
>or unstable branch; the installer there supports creating the devices as
>part of the boot sequence.
>  
>
Just to reinforce that http://www.debian.org/devel/debian-installer/
is just a couple of days old (7 Aug) and you may not have noticed this.

Since you've selected Debian (good) it'd be nice to try the new 
installer and see if it offers the features you need.

let us know...

David

Re: Software RAID-1: step-by-step help needed...

[Gordon Henderson]

On Tue, 10 Aug 2004, Boniforti Flavio wrote:

> > OK - this is a very intersting and highly debatable subject!
>
> [cut]
>
> Well, I didn't mention at all what my server will have to do: it's
> simply a squid-proxy and will have apache installed (with PHP and
> mySQL), but just for small "intranet" applications (more "testing" like)...
> This is why I was about to apply following partition scheme:
>
> /dev/sda1 - 20 GB (Primary, ext3)
> /dev/sda2 -  1 GB (Primary, SWAP)
> /dev/sda5 - 52 GB (Logical, ext3)
>
> I will install / onto sda1 and /var onto sda5, what do you think?

I think you are probably going to waste a lot of space under /. 4GB will
be more than enough for / and /usr. 6GB if you are really paranoid. This
assumes that all the squid & apache logs go under /var, as well as the
squid cache and MySql databases. (I don't use the supplied Debian version,
I always compile from source, so you'll have to check here)

> >>I already installed it (as a test) following the instructions found at:
> >>http://www.inittab.de/manuals/debootstrap.html
> >
> > Er - thats a rather "intersting" way to install Debian - I use the
> > standard Woody CD set..
>
> Yes, but I have to install it that way, because the original Woody discs
>   don't contain a working driver for my SCSI Controller. :(

Thats getting to be more of a problem with Woody these days - I think
their supplied 2.4 kernel is 2.4.18... I've had intersting times with
promise IDE controllers and various on-board Ethernet controllers...

> >>I guess the hardest task will be configuring RAID, as the system works
> >>with the above procedure...
> >
> > OK. So with Debian, I basic, but minimal system on the first drive, not
> > touching the 2nd drive at all during this process... I'll usually compile
> > up a custome kernel at this point to make sure the IDE controllers have
> > DMA, if nothing else.
>
> I will install the system with the above mentioned "alien" procedure,
> then I would like to *first* activate RAID-1 (maybe it could need to
> recompile?) and then boot off RAID and see that everything works...
>
> > Then with the 2nd drive, partition it to be identical to the first disk,
> > then follow through the runes on the HowTo for mirroring the disks.
>
> I already partitioned both disks the same way (booting with System
> Rescue CD and using cfdisk).

OK.

> I don't actually understand what you are doing hereby... I mean: doesn't
> it suffice to start RAID and it handles the "mirroring" and all the
> stuff? Just following the HOW TO wouldn't be enough?

My method came from the HowTo... However, I've not re-read it for some
time... My way is Method 1 in the How-To, You might want to look at Method
2 in the How To. You might run out of disk space using my method trying to
copy / (which contains /usr) to a 1GB swap partition. You ought to be OK
with /var though - unless you've already loaded it up with stuff.

> > One other tip if you are using ext3 is to tune2fs -i0 -c0 /dev/hdaX after
> > the mkfs -t ext3 ... to make sure it's not force checked after so many
> > days/mounts.
>
> That's another "black hole" in my knowledge: what is this thing above?!?

When you create an ext2 (or ext3) filesystem, the system keeps track of
how many times it's been mounted and how many days have passed since it's
had an fsck run on it. If more than N days or more than M mounts, then it
will force an fsck at boot time... With ext3, in-theory the filesystem
will be in-sync all the time, so there should never be a need. The tune2fs
command alters the settings of number of days and mounts for that
particular filesystem.

Gordon

Re: Software RAID-1: step-by-step help needed...

[Lucas Albers]

I wrote this:
http://rootraiddoc.alioth.debian.org

Directions on converting to raid1 on debian.
Been used by a couple hundred people, so let me know if you somehow find
some errors.

-- 
Luke Computer Science System Administrator
Security Administrator,College of Engineering
Montana State University-Bozeman,Montana