Q: RAID-1 w/2x160GB, ReiserFS, Debian "woody", homebrew 2.4.25 kernel

Q: RAID-1 w/2x160GB, ReiserFS, Debian "woody", homebrew 2.4.25 kernel

[Jens Benecke]

Hi everybody,

I'm planning to set up a new RAID mirrored system with the above specs. Both
disks are master (hda and hdc). I'm currently trying to decide between LVM2
(is it in 2.4 already?), MD, and a "manual" nightly rsync onto the second
disk.

I'd like a few pointers (RTFwhatever welcome) to the following questions:


How about RAIDing the root partition? If one drive fails will the other be
able to boot via LILO? How about GRUB? Which do you prefer?

How about (/var)/tmp? I (suppose I'll) need it on both disks, does it make
sense to mirror it as well?

Can I mirror the whole disk? Or do I need to mirror each partition
seperately?

Does MD or LVM2 do hot sync, i.e. if one drive fails will I be able to stick
in a replacement, and stop worrying? Or do I need to repartition the new
disk exactly as the old one, before being able to sync?

How does LVM2/MD deal with failing harddisks, which is why I do the mirror
at all? I've heard about MD not detecting read errors because the "other"
disk was reading fine, and crashing completely when one disk was finally
replaced because the data on the other disk was also corrupt. Is that still
the current case?


The goal is to have as "stress free" a system as possible - i.e. with as
little manual configuration, and in event of emergencies, as little work to
do, as possible.


Thank you for your help!


-- 
Jens Benecke
http://www.hitchhikers.de - Europas kostenlose Mitfahrzentrale seit 1998
http://www.rb-hosting.de - Webhosting mit Extras - PHP ab €9 - SSH ab €19
http://www.spamfreemail.de - 100% saubere Postfächer, garantiert!

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Q: RAID-1 w/2x160GB, ReiserFS, Debian "woody", homebrew 2.4.25 kernel

[Mikael Abrahamsson]

On Wed, 4 Aug 2004, Jens Benecke wrote:

> The goal is to have as "stress free" a system as possible - i.e. with as
> little manual configuration, and in event of emergencies, as little work to
> do, as possible.

You probably should go for hardware RAID1 instead, this will present the
drive as a single drive to the OS and will handle all failure scenarios, 
you will not be hindered by fixing LILO to boot etc. Yes, it'll cost a bit 
more but if your goal is simplicity and carefree running, this is probably 
the best way.

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

Re: Q: RAID-1 w/2x160GB, ReiserFS, Debian "woody", homebrew 2.4.25 kernel

[Jens Benecke]

Am Mittwoch, 4. August 2004 11:00 schrieb Mikael Abrahamsson:

> On Wed, 4 Aug 2004, Jens Benecke wrote:
> > The goal is to have as "stress free" a system as possible - i.e. with
> > as little manual configuration, and in event of emergencies, as little
> > work to do, as possible.
>
> You probably should go for hardware RAID1 instead, this will present the
> drive as a single drive to the OS and will handle all failure scenarios,
> you will not be hindered by fixing LILO to boot etc. Yes, it'll cost a
> bit more but if your goal is simplicity and carefree running, this is
> probably the best way.

Hi Mikael,

thanks for your answer. What hardware RAID1 options would I have here that 
are somewhat affordable? I see 3ware 2-drive RAID controllers for €120.- 
here, and Promise UDMA for €60.-

I hear Promise only offers binary drivers and is only "software RAID", is 
the 3ware solution worth the extra €60 (less hassle and more 
compatbility)?


Thanks!

-- 
Dipl.-Ing. Jens Benecke
http://www.hitchhikers.de - Europas kostenlose Mitfahrzentrale seit 1998
http://www.rb-hosting.de - Webhosting mit Extras - PHP ab €9 - SSH ab €19
http://www.spamfreemail.de - 100% saubere Postfächer, garantiert!
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Q: RAID-1 w/2x160GB, ReiserFS, Debian "woody", homebrew 2.4.25 kernel

[Mikael Abrahamsson]

On Wed, 4 Aug 2004, Jens Benecke wrote:

> thanks for your answer. What hardware RAID1 options would I have here that 
> are somewhat affordable? I see 3ware 2-drive RAID controllers for ¤120.- 
> here, and Promise UDMA for ¤60.-
> 
> I hear Promise only offers binary drivers and is only "software RAID", is 
> the 3ware solution worth the extra ¤60 (less hassle and more 
> compatbility)?

I have never tried to boot off of a 3ware raid, but to Linux it looks like 
a scsi drive so I would imagine that it should work well. Someone else who 
has actually done this for real is probably better suited to answer.

I never use software raid for the partitions that need to be present at 
boot, since I have had total nightmares with root mirroring on other OSes 
(Solaris and DEC OSF), and I feel that backups of those partitions are 
better than the problems one might run into with software root mirroring.

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Q: RAID-1 w/2x160GB, ReiserFS, Debian "woody", homebrew 2.4.25 kernel

[Tim Small]

Mikael Abrahamsson wrote:

> I have never tried to boot off of a 3ware raid, but to Linux it looks 
> like
>
>a scsi drive so I would imagine that it should work well. Someone else who 
>has actually done this for real is probably better suited to answer.
>  
>
Yes, this works.

>I never use software raid for the partitions that need to be present at 
>boot, since I have had total nightmares with root mirroring on other OSes 
>(Solaris and DEC OSF), and I feel that backups of those partitions are 
>better than the problems one might run into with software root mirroring.
>  
>
I've had success with booting Linux from software RAID (including with 
drive failures), and if for some reason the boot loader gets in a knot, 
you can always use a grub floppy (since Linux software mirror partitions 
look like normal Linux partitions to the boot loader)...

Tim.

Re: Q: RAID-1 w/2x160GB, ReiserFS, Debian 'woody', homebrew 2.4.25 kernel

[Robin Bowes]

On Wed, August 4, 2004 9:42, Jens Benecke said:
> I'm planning to set up a new RAID mirrored system with the above specs. Both
> disks are master (hda and hdc). I'm currently trying to decide between LVM2 (is it in 2.4
> already?), MD, and a "manual" nightly rsync onto the second disk.

I use md and mdadm on Fedora Core 2. What distro are you contemplating?

> 
> How about RAIDing the root partition? If one drive fails will the other be
> able to boot via LILO? How about GRUB? Which do you prefer?

I have my root partition on a RAID1 mirror. I use grub and have "installed" grub to both mirrored drives so I can boot off either, e.g. if one fails. That reminds me, I must test this.
 
> How about (/var)/tmp? I (suppose I'll) need it on both disks, does it make
> sense to mirror it as well?

You *could* put /var/tmp or /tmp on separate partitions either mirrored or not, but if you want to keep things "stress free" I would keep /var/tmp and /tmp on the root partition.

> Can I mirror the whole disk? Or do I need to mirror each partition
> seperately?

You can do either. You mirror the disk using md and can then either create a filesystem on the whole disk, or use lvm to create logical volumes within the md device.

> Does MD or LVM2 do hot sync, i.e. if one drive fails will I be able to stick
> in a replacement, and stop worrying? Or do I need to repartition the new disk exactly as
> the old one, before being able to sync?

I'm not sure about this. My understanding is that you will need to shutdown the system to replace the bad disk and partition the new disk manually before md will resync, but this could be wrong.

I have six 250GB SATA disks, all partitioned identically with two partitions of 1.5GB and 248.5 GB. I have them configured as RAID devices using md follows:

md0   sda1 + sdd1    RAID1    1.5GB   root filesystem
md2   sdb1 + sde1    RAID1    1.5GB   swap 1
md3   sdc1 + sdf1    RAID1    1.5GB   currently not used
md5   sd[abcdef]2    RAID5    994GB   lvm2 volume group (4+1+spare)

Then within the lvm2 volume group I have the following logical volumes:

dude_usr   10GB     /usr
dude_var   5GB      /var
dude_home  979GB    /home

I did my initial install just to the mirrored root parition . Once I had done the initial install I set up the lvm2 array and volumes and migrated /usr and /var over.

It's working great so far.

> How does LVM2/MD deal with failing harddisks, which is why I do the mirror
> at all? I've heard about MD not detecting read errors because the "other" disk was
> reading fine, and crashing completely when one disk was finally replaced because the
> data on the other disk was also corrupt. Is that still the current case?

Again, I don't know about this.
 
> The goal is to have as "stress free" a system as possible - i.e. with as
> little manual configuration, and in event of emergencies, as little work to do, as
> possible.

If you want stress free, buy a Netapps storage appliance ;o)

R.
-- 
http://robinbowes.com

Re: Q: RAID-1 w/2x160GB, ReiserFS, Debian 'woody', homebrew 2.4.25 kernel

[Mark Watts]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


> I have six 250GB SATA disks, all partitioned identically with two
> partitions of 1.5GB and 248.5 GB. I have them configured as RAID devices
> using md follows:

Talking of partitioning several disks the same, is there an easy way to 
automate this?

Mark.

- -- 
Mark Watts
Senior Systems Engineer
QinetiQ Trusted Information Management
Trusted Solutions and Services group
GPG Public Key ID: 455420ED

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFBELqYBn4EFUVUIO0RAo8YAJsH5JhUMGtuT0FBrahFYIE7MiNj5gCfR3D6
wrGHUOZr/UsKY6xd4OAr8Z0=
=E7sA
-----END PGP SIGNATURE-----

Re: Q: RAID-1 w/2x160GB, ReiserFS, Debian 'woody', homebrew 2.4.25 kernel

[Tim Small]

Mark Watts wrote:

>Talking of partitioning several disks the same, is there an easy way to 
>automate this?
>  
>
Hmm, I gave a way to do this in my last email, but I've realised that I 
missed out the "count" argument to dd...  I said:

> dd if=/dev/hda of=/dev/hdc ; echo w | fdisk /dev/hdc 

This will work, but will take a while ;o)  it should really have been:

dd if=/dev/hda of=/dev/hdc count=1 && echo w | fdisk /dev/hdc



Tim.

Re: Q: RAID-1 w/2x160GB, ReiserFS, Debian 'woody', homebrew 2.4.25 kernel

[Robin Bowes]

On Wed, August 4, 2004 11:29, Mark Watts said:
 
> Talking of partitioning several disks the same, is there an easy way to
> automate this?
> 

Yes. Something like:

# sfdisk -d /dev/sda > partition.save
# sfdisk /dev/sdb < partition.save

Or all at once:

# sfdisk -d /dev/sda | sfdisk /dev/sdb

I partition all my drives the same so if one fails it is easy to re-partion the new drive by copying the structure from an existing drive.

R.
-- 
http://robinbowes.com

Re: Q: RAID-1 w/2x160GB, ReiserFS, Debian 'woody', homebrew 2.4.25 kernel

[Alvin Oga]

On Wed, 4 Aug 2004, Mark Watts wrote:

> Talking of partitioning several disks the same, is there an easy way to 
> automate this?

# imagine the sequence of the characters you type to fdisk
echo n
echo p
echo 1
echo 1
echo +256M
.....

and run that script for each new unpartitioned/virgin /dev/hda disk ..

	fdisk.automated.sh | fdisk /dev/hda

sample partition script:
	http://linux-consulting.com/Boot/Linux-1U/clone.suse-8.2.fdisk.sh1

sample cloning script:
	... tar ... onto target disk

	http://linux-consulting.com/Boot/Linux-1U/clone.suse-8.2.sh

works good when you're doing 10's, 100's of identical disks ... or 
same partition scheme ...

c ya
alvin

Re: Q: RAID-1 w/2x160GB, ReiserFS, Debian 'woody', homebrew 2.4.25 kernel

[Mark Hahn]

> 	fdisk.automated.sh | fdisk /dev/hda

no offense, but yuck.  I recently cloned the partitions from one sata onto the other 7:

sfdisk -d /dev/sda > parts
for d in /dev/sd[b-h]; do sfdisk $d < parts; done

regards, mark hahn.

Re: Q: RAID-1 w/2x160GB, ReiserFS, Debian 'woody', homebrew 2.4.25 kernel

[Jens Benecke]

Robin Bowes wrote:

> On Wed, August 4, 2004 9:42, Jens Benecke said:
>> I'm planning to set up a new RAID mirrored system with the above specs.
>> Both disks are master (hda and hdc). I'm currently trying to decide
>> between LVM2 (is it in 2.4 already?), MD, and a "manual" nightly rsync
>> onto the second disk.
> 
> I use md and mdadm on Fedora Core 2. What distro are you contemplating?

Hi,

see subject (Debian). I've looked at SuSE though, it seems with their setup
RAID and ReiserFS is much easier to handle. I'm running SuSE at home though
and the sheer number of security updates that come in kind of make me a bit
uneasy for an internet server.
 
>> How about RAIDing the root partition? If one drive fails will the other
>> be able to boot via LILO? How about GRUB? Which do you prefer?
> 
> I have my root partition on a RAID1 mirror. I use grub and have
> "installed" grub to both mirrored drives so I can boot off either, e.g. if
> one fails. That reminds me, I must test this.

That's exactly what I want.
  
>> How about (/var)/tmp? I (suppose I'll) need it on both disks, does it
>> make sense to mirror it as well?
> 
> You *could* put /var/tmp or /tmp on separate partitions either mirrored or
> not, but if you want to keep things "stress free" I would keep /var/tmp
> and /tmp on the root partition.

No. I don't want a rogue script to fill up my root partition.
 
>> Can I mirror the whole disk? Or do I need to mirror each partition
>> seperately?
> 
> You can do either. You mirror the disk using md and can then either create
> a filesystem on the whole disk, or use lvm to create logical volumes
> within the md device.

Hm... I thought LVM did mirroring and striping as well? I didn't think you
can or should use them together. Wouldn't that degrade performance as well?
 
>> Does MD or LVM2 do hot sync, i.e. if one drive fails will I be able to
>> stick in a replacement, and stop worrying? Or do I need to repartition
>> the new disk exactly as the old one, before being able to sync?
> 
> I'm not sure about this. My understanding is that you will need to
> shutdown the system to replace the bad disk and partition the new disk
> manually before md will resync, but this could be wrong.

What I mean is, will md resync automatically or would I have to initiate
this manually?
 
> I have six 250GB SATA disks, all partitioned identically with two
> partitions of 1.5GB and 248.5 GB. I have them configured as RAID devices
> using md follows:

I think you mean 1.5TB. :-)
 
> md0   sda1 + sdd1    RAID1    1.5GB   root filesystem
> md2   sdb1 + sde1    RAID1    1.5GB   swap 1
> md3   sdc1 + sdf1    RAID1    1.5GB   currently not used
> md5   sd[abcdef]2    RAID5    994GB   lvm2 volume group (4+1+spare)
> 
> Then within the lvm2 volume group I have the following logical volumes:
> 
> dude_usr   10GB     /usr
> dude_var   5GB      /var
> dude_home  979GB    /home
> 
> I did my initial install just to the mirrored root parition . Once I had
> done the initial install I set up the lvm2 array and volumes and migrated
> /usr and /var over.
> 
> It's working great so far.

This setup seems quite complicated. Did you test the setup, ie. removed one
of the MD disks and looked what happened?
 
>> The goal is to have as "stress free" a system as possible - i.e. with as
>> little manual configuration, and in event of emergencies, as little work
>> to do, as possible.
> 
> If you want stress free, buy a Netapps storage appliance ;o)

If they do all the rest that I need (smtp, web, file server, ftp server,
mysql, spamassassin, etc etc etc) fine. If not, I'd need to set up another
box anyway, so I wouldn't see the advantage.


-- 
Jens Benecke
http://www.hitchhikers.de - Europas kostenlose Mitfahrzentrale seit 1998
http://www.rb-hosting.de - Webhosting mit Extras - PHP ab €9 - SSH ab €19
http://www.spamfreemail.de - 100% saubere Postfächer, garantiert!

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Q: RAID-1 w/2x160GB, ReiserFS, Debian 'woody', homebrew 2.4.25 kernel

[Daniel Pittman]

On 5 Aug 2004, Jens Benecke wrote:
> Robin Bowes wrote:
>
>> On Wed, August 4, 2004 9:42, Jens Benecke said:

[...]

> see subject (Debian). I've looked at SuSE though, it seems with their setup
> RAID and ReiserFS is much easier to handle. 

The upcoming release of Debian will feature MD and LVM device creation
at install time, but is not yet stable; the current stable release does
not support either of them.

I usually do a minimal install on one "missing" disk in a RAID set, then
copy that to the RAID, reboot and add back the missing disk. Painful,
but possible.

[...]

> Hm... I thought LVM did mirroring and striping as well? I didn't think you
> can or should use them together. Wouldn't that degrade performance as
> well?

It can, if you want it to, but it doesn't have to.  I usually just use
MD devices created over partitions, because that is supported by
in-kernel detection, and because it avoids the LVM ... instability of
the last few years.

I am told that LVM is much more stable these days, though. :)

>>> Does MD or LVM2 do hot sync, i.e. if one drive fails will I be able to
>>> stick in a replacement, and stop worrying? Or do I need to repartition
>>> the new disk exactly as the old one, before being able to sync?
>>
>> I'm not sure about this. My understanding is that you will need to
>> shutdown the system to replace the bad disk and partition the new disk
>> manually before md will resync, but this could be wrong.
>
> What I mean is, will md resync automatically or would I have to initiate
> this manually?

MD will resync if there is a hot-spare available, but will not
automatically suck a replacement disk into place. You need to partition
(if you use them), then manually add the new disk to the RAID set.

With mirroring, of course, I can't see the point in using a hot-spare
over having additional devices in the mirror set, except for PCI and
disk bus bandwidth.

For most servers they cost less than the risk that using hot-spare disks
implies. :)

[...]

>>> The goal is to have as "stress free" a system as possible - i.e. with as
>>> little manual configuration, and in event of emergencies, as little work
>>> to do, as possible.
>>
>> If you want stress free, buy a Netapps storage appliance ;o)
>
> If they do all the rest that I need (smtp, web, file server, ftp server,
> mysql, spamassassin, etc etc etc) fine. If not, I'd need to set up another
> box anyway, so I wouldn't see the advantage.

If you can afford them then they are one of the most stress free ways to
get a lot of disk space. They are not a substitute for standard storage,
though, and they do cost a bit...

        Daniel
-- 
To lie about a far country is easy.
        -- Amharic proverb

Re: Q: RAID-1 w/2x160GB, ReiserFS, Debian 'woody', homebrew 2.4.25 kernel

[Tim Small]

Daniel Pittman wrote:

>>see subject (Debian). I've looked at SuSE though, it seems with their setup RAID and ReiserFS is much easier to handle. 
>>    
>>
>The upcoming release of Debian will feature MD and LVM device creation
>at install time, but is not yet stable; the current stable release does
>not support either of them.
>  
>
Sarge is pretty good at the moment (at least I am running production 
systems on it, and have been for the last few months) - the disadvantage 
of installing a new Woody system now is that security updates for it 
will probably stop before it's reached the end of it's useful life.  Of 
course, as it's Debian, you should be able to upgrade to Sarge 
relatively painlessly, but I'd probably rather avoid the hassle, and go 
straight to Sarge now.  I think Debian Installer currently includes 
support for installing Sarge directly onto md mirrors (including 
handling the boot-loader stuff), but I haven't tried it yet...  Might be 
worth trying one of the daily builds from:

http://www.debian.org/devel/debian-installer/

The disadvantage of installing Sarge yet is that it won't go through the 
official security updates until it is released, but in the mean time, 
you can pull security-fixes in from unstable (using apt-pinning if you 
like) - sounds nasty but it seems to work quite well in practice.  Sorry 
for getting distribution-centric on the list!

Tim.

Re: Q: RAID-1 w/2x160GB, ReiserFS, Debian 'woody', homebrew 2.4.25 kernel

[Jens Benecke]

Tim Small wrote:

> Daniel Pittman wrote:
 
>>>see subject (Debian). I've looked at SuSE though, it seems with their
>>>setup RAID and ReiserFS is much easier to handle.

>>The upcoming release of Debian will feature MD and LVM device creation
>>at install time, but is not yet stable; the current stable release does
>>not support either of them.
  
> Sarge is pretty good at the moment (at least I am running production
> systems on it, and have been for the last few months) - the disadvantage
> of installing a new Woody system now is that security updates for it
> will probably stop before it's reached the end of it's useful life.  Of

Hi Tim,

Are you sure? I thought woody will be supported for quite some time. I've
been running an updated Debian with a couple self-compiled packages since
1999, upgraded a couple times but woody has been quite pain-less so far.
Only the current set up has been messed up a bit so far.

I'm planning to use SysCP (www.syscp.de) as a webhosting management
interface, which requires woody - for now. It'll probably switch to Sarge
when it becomes stable, which is why I don't plan on doing this right now.

I was faovouring SuSE because they have more up to date packages (Postfix
2.1, Apache 2.0.49, PHP 4.3.4, MySQL 4.0), and because I wouldn't have to
bake my own kernel (I need a couple patches that Debian doesn't supply,
most importantly the GRsecurity patch set, and ReiserFS quota and
data-logging).
But I found that I can get most of the packages via backports.org and
dotdeb.net, and that my homebrew 2.4.25 kernel with those patches just runs
far too painlessly to give up right now.

> course, as it's Debian, you should be able to upgrade to Sarge
> relatively painlessly, but I'd probably rather avoid the hassle, and go
> straight to Sarge now.  I think Debian Installer currently includes
> support for installing Sarge directly onto md mirrors (including
> handling the boot-loader stuff), but I haven't tried it yet...  Might be
> worth trying one of the daily builds from:
> http://www.debian.org/devel/debian-installer/

The problem here is that I won't ever see the box. It's a rented dedicated
server. And the company, they let you choose hardware and distribution and
partitioning but then just install the default with enabled "ssh" and let
you do the rest.
 
> The disadvantage of installing Sarge yet is that it won't go through the
> official security updates until it is released, but in the mean time,
> you can pull security-fixes in from unstable (using apt-pinning if you
> like) - sounds nasty but it seems to work quite well in practice.  Sorry
> for getting distribution-centric on the list!

This sounds like too much manual intervention to me. I just want the damn
thing to run. I'm past the experimenting stage where I compiled half the
system myself. :-)


-- 
Jens Benecke
http://www.hitchhikers.de - Europas kostenlose Mitfahrzentrale seit 1998
http://www.rb-hosting.de - Webhosting mit Extras - PHP ab €9 - SSH ab €19
http://www.spamfreemail.de - 100% saubere Postfächer, garantiert!

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Q: RAID-1 w/2x160GB, ReiserFS, Debian 'woody', homebrew 2.4.25 kernel

[robin-lists]

> -----Original Message-----
> From: linux-raid-owner@vger.kernel.org 
> [mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Jens Benecke
> Sent: 05 August 2004 06:17
> To: linux-raid@vger.kernel.org
> Subject: Re: Q: RAID-1 w/2x160GB, ReiserFS, Debian 'woody', 
> homebrew 2.4.25 kernel
> 
> Robin Bowes wrote:
> 
> > On Wed, August 4, 2004 9:42, Jens Benecke said:

> > 
> > I use md and mdadm on Fedora Core 2. What distro are you 
> contemplating?
> 
> Hi,
> 
> see subject (Debian). I've looked at SuSE though, it seems 
> with their setup RAID and ReiserFS is much easier to handle. 
> I'm running SuSE at home though and the sheer number of 
> security updates that come in kind of make me a bit uneasy 
> for an internet server.

Ooops./ I didn't read the subject.

> >> How about RAIDing the root partition? If one drive fails will the 
> >> other be able to boot via LILO? How about GRUB? Which do 
> you prefer?
> > 
> > I have my root partition on a RAID1 mirror. I use grub and have 
> > "installed" grub to both mirrored drives so I can boot off either, 
> > e.g. if one fails. That reminds me, I must test this.
> 
> That's exactly what I want.

I've got some rough notes I made. I may tidy them up and publish them on my
web site.
   
> >> How about (/var)/tmp? I (suppose I'll) need it on both 
> disks, does it 
> >> make sense to mirror it as well?
> > 
> > You *could* put /var/tmp or /tmp on separate partitions either 
> > mirrored or not, but if you want to keep things "stress 
> free" I would 
> > keep /var/tmp and /tmp on the root partition.
> 
> No. I don't want a rogue script to fill up my root partition.

That, of course, is the reason you would want to keep /var/tmp and/or /tmp
on separate partitions (or on the same partition - just symlink so /var/tmp
and /tmp are the same.)

>  
> >> Can I mirror the whole disk? Or do I need to mirror each partition 
> >> seperately?
> > 
> > You can do either. You mirror the disk using md and can then either 
> > create a filesystem on the whole disk, or use lvm to create logical 
> > volumes within the md device.
> 
> Hm... I thought LVM did mirroring and striping as well? I 
> didn't think you can or should use them together. Wouldn't 
> that degrade performance as well?
>  
> >> Does MD or LVM2 do hot sync, i.e. if one drive fails will 
> I be able 
> >> to stick in a replacement, and stop worrying? Or do I need to 
> >> repartition the new disk exactly as the old one, before 
> being able to sync?
> > 
> > I'm not sure about this. My understanding is that you will need to 
> > shutdown the system to replace the bad disk and partition 
> the new disk 
> > manually before md will resync, but this could be wrong.
> 
> What I mean is, will md resync automatically or would I have 
> to initiate this manually?

Md should re-sync automatically.

>  
> > I have six 250GB SATA disks, all partitioned identically with two 
> > partitions of 1.5GB and 248.5 GB. I have them configured as RAID 
> > devices using md follows:
> 
> I think you mean 1.5TB. :-)

Nope, 1.5GB. See below.

>  
> > md0   sda1 + sdd1    RAID1    1.5GB   root filesystem
> > md2   sdb1 + sde1    RAID1    1.5GB   swap 1
> > md3   sdc1 + sdf1    RAID1    1.5GB   currently not used
> > md5   sd[abcdef]2    RAID5    994GB   lvm2 volume group (4+1+spare)
> > 
> > Then within the lvm2 volume group I have the following 
> logical volumes:
> > 
> > dude_usr   10GB     /usr
> > dude_var   5GB      /var
> > dude_home  979GB    /home
> > 
> > I did my initial install just to the mirrored root parition 
> . Once I 
> > had done the initial install I set up the lvm2 array and 
> volumes and 
> > migrated /usr and /var over.
> > 
> > It's working great so far.
> 
> This setup seems quite complicated. Did you test the setup, 
> ie. removed one of the MD disks and looked what happened?

I deliberated long and hard about how to implement my new system and this
was about the least complicated route!

> >> The goal is to have as "stress free" a system as possible 
> - i.e. with 
> >> as little manual configuration, and in event of emergencies, as 
> >> little work to do, as possible.
> > 
> > If you want stress free, buy a Netapps storage appliance ;o)
> 
> If they do all the rest that I need (smtp, web, file server, 
> ftp server, mysql, spamassassin, etc etc etc) fine. If not, 
> I'd need to set up another box anyway, so I wouldn't see the 
> advantage.

I was being slightly slippant - Netapps just do big storage arrays with very
high availability. For example, if they detect that a drive is failing they
send an email to Netapps and an engineer comes round with a replacement disk
all without any user intervention! I guess they're the Rolls Royce of
storage solutions. Expensive though.

R.
--
http://robinbowes.com  

RE: Q: RAID-1 w/2x160GB, ReiserFS, Debian 'woody', homebrew 2.4.25 kernel

[Jens Benecke]

robin-lists@robinbowes.com wrote:

>> >> How about RAIDing the root partition? If one drive fails will the
>> >> other be able to boot via LILO? How about GRUB? Which do
>> > 
>> > I have my root partition on a RAID1 mirror. I use grub and have
>> > "installed" grub to both mirrored drives so I can boot off either,
>> > e.g. if one fails. That reminds me, I must test this.
>> 
>> That's exactly what I want.
> 
> I've got some rough notes I made. I may tidy them up and publish them on
> my web site.

Hi Robin,

I'm still a bit torn between buying a 3ware hardware RAID for €140.- and
just using 'md'. The box is an Athlon64 3GHz, so it'll have enough CPU
power to do the RAID in software, but with partitioning etc, I think the
hardware raid will be easier to handle because you can just ignore it. As
long as the 3ware BIOS will handle failing drives, read/write errors, etc
etc. properly.

Right?
    
>> > keep /var/tmp and /tmp on the root partition.
>> 
>> No. I don't want a rogue script to fill up my root partition.
> 
> That, of course, is the reason you would want to keep /var/tmp and/or /tmp
> on separate partitions (or on the same partition - just symlink so
> /var/tmp and /tmp are the same.)

I would probably not symlink but I'd put them both on the same partition. 
  
>> What I mean is, will md resync automatically or would I have
>> to initiate this manually?
> 
> Md should re-sync automatically.

... even onto a new blank disk? Or would you have to tell md manually "this
is the new spare, please sync"? Is this easy to do? Something like

        - copy partition table
        - for FOO in 1 2 3 4 5 ; do tell md "sync hda$FOO to hdc$FOO"

or is it more complicated?
 
>> This setup seems quite complicated. Did you test the setup,
>> ie. removed one of the MD disks and looked what happened?
> I deliberated long and hard about how to implement my new system and this
> was about the least complicated route!

Oh. ;)
 

>> >> The goal is to have as "stress free" a system as possible
>> >> as little manual configuration, and in event of emergencies, as
>> >> little work to do, as possible.
>> > If you want stress free, buy a Netapps storage appliance ;o)
>> If they do all the rest that I need (smtp, web, file server,
> I was being slightly slippant - Netapps just do big storage arrays with
> very high availability. For example, if they detect that a drive is
> failing they send an email to Netapps and an engineer comes round with a
> replacement disk all without any user intervention! I guess they're the
> Rolls Royce of storage solutions. Expensive though.

Ah. Well I don't think this'll fit in our budget. :)


-- 
Jens Benecke
http://www.hitchhikers.de - Europas kostenlose Mitfahrzentrale seit 1998
http://www.rb-hosting.de - Webhosting mit Extras - PHP ab €9 - SSH ab €19
http://www.spamfreemail.de - 100% saubere Postfächer, garantiert!

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Q: RAID-1 w/2x160GB, ReiserFS, Debian 'woody', homebrew 2.4.25 kernel

[Mark Watts]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


> I'm still a bit torn between buying a 3ware hardware RAID for €140.- and
> just using 'md'. The box is an Athlon64 3GHz, so it'll have enough CPU
> power to do the RAID in software, but with partitioning etc, I think the
> hardware raid will be easier to handle because you can just ignore it. As
> long as the 3ware BIOS will handle failing drives, read/write errors, etc
> etc. properly.

3ware (8506-4LP) is performing very badly here on Dual Opteron so I wouldn't 
recomend it.
Even a software raid-5 with the 3ware in jbod mode wasn't great.

Mark.

- -- 
Mark Watts
Senior Systems Engineer
QinetiQ Trusted Information Management
Trusted Solutions and Services group
GPG Public Key ID: 455420ED

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFBEioqBn4EFUVUIO0RApf/AKC1A+9grRDHL2Yg6xU1hiE7LICZCQCfRkbH
x4oIjT2aqo0pazXQ5eDXwzY=
=pd4i
-----END PGP SIGNATURE-----
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Q: RAID-1 w/2x160GB, ReiserFS, Debian 'woody', homebrew 2.4.25 kernel

[Katarina WONG]

On Thu, 5 Aug 2004 13:38:02 +0100, Mark Watts <m.watts@eris.qinetiq.com>  
wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
>> I'm still a bit torn between buying a 3ware hardware RAID for €140.- and
>> just using 'md'. The box is an Athlon64 3GHz, so it'll have enough CPU
>> power to do the RAID in software, but with partitioning etc, I think the
>> hardware raid will be easier to handle because you can just ignore it.  
>> As
>> long as the 3ware BIOS will handle failing drives, read/write errors,  
>> etc
>> etc. properly.
>
> 3ware (8506-4LP) is performing very badly here on Dual Opteron so I  
> wouldn't
> recomend it.
> Even a software raid-5 with the 3ware in jbod mode wasn't great.
>

I recon that Jens was talking from a functional standpoint (hotspare,  
auto-rebuild, etc...). I too would be interested in hearing your  
experience on these. Also, can you expand on your "bad performance"  
statement ?

Thx

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Q: RAID-1 w/2x160GB, ReiserFS, Debian 'woody', homebrew 2.4.25 kernel

[robin-lists]

> -----Original Message-----
> From: linux-raid-owner@vger.kernel.org 
> [mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Jens Benecke
> Sent: 05 August 2004 12:08
> To: linux-raid@vger.kernel.org
> Subject: RE: Q: RAID-1 w/2x160GB, ReiserFS, Debian 'woody', 
> homebrew 2.4.25 kernel
> 
> robin-lists@robinbowes.com wrote:
> 
> >> >> How about RAIDing the root partition? If one drive 
> fails will the 
> >> >> other be able to boot via LILO? How about GRUB? Which do
> >> > 
> >> > I have my root partition on a RAID1 mirror. I use grub and have 
> >> > "installed" grub to both mirrored drives so I can boot 
> off either, 
> >> > e.g. if one fails. That reminds me, I must test this.
> >> 
> >> That's exactly what I want.
> > 
> > I've got some rough notes I made. I may tidy them up and 
> publish them 
> > on my web site.
> 
> Hi Robin,
> 
> I'm still a bit torn between buying a 3ware hardware RAID for 
> ?140.- and just using 'md'. The box is an Athlon64 3GHz, so 
> it'll have enough CPU power to do the RAID in software, but 
> with partitioning etc, I think the hardware raid will be 
> easier to handle because you can just ignore it. As long as 
> the 3ware BIOS will handle failing drives, read/write errors, 
> etc etc. properly.
> 
> Right?

Not sure what your question is. 3ware cards are generally held in high
regard, although I have seen some traffic about performance problems,
although these seem to be contradicted by an equal number of "it works Ok
for me" traffic.

I needed at least 6 x 250GB drives so the 3Ware option was a little pricey,
hence my decision to go with md.

>     
> >> > keep /var/tmp and /tmp on the root partition.
> >> 
> >> No. I don't want a rogue script to fill up my root partition.
> > 
> > That, of course, is the reason you would want to keep 
> /var/tmp and/or 
> > /tmp on separate partitions (or on the same partition - 
> just symlink 
> > so /var/tmp and /tmp are the same.)
> 
> I would probably not symlink but I'd put them both on the 
> same partition. 

I have a separate /var partition so I will just symlink /tmp to /var/tmp.

>   
> >> What I mean is, will md resync automatically or would I have to 
> >> initiate this manually?
> > 
> > Md should re-sync automatically.
> 
> ... even onto a new blank disk? Or would you have to tell md 
> manually "this is the new spare, please sync"? Is this easy 
> to do? Something like
> 
>         - copy partition table
>         - for FOO in 1 2 3 4 5 ; do tell md "sync hda$FOO to hdc$FOO"
> 
> or is it more complicated?

Now that you've made me think about it I'm not entirely sure. I would do
this:

(assume /dev/sdc has failed).

Shutdown server.
Pull dead drive
Insert new drive
Boot up server

# sfdisk -d /dev/sda | sfdisk /dev/sdc

Now, it might be necessary to explicitly "remove" the failed device from the
arrays (before shutting down?) and to add it back in after replacing the
disk but I don't know.

I'm sure there are others on this list who can keep us right!

For example, would this work?:

# mdadm /dev/md5 -f /dev/sdc2 -r /dev/sdc2 -a /dev/sdc2

According to my understanding, this does the following:

1. Marks /dev/sdc2 as faulty in /dev/md5 (if the drive has failed it should
already be marked faulty??)
2. removes /dev/sdc2 from /dev/md5
3. Adds /dev/sdc2 to /dev/md5

Can anyone else clarify this?

> >> This setup seems quite complicated. Did you test the setup, ie. 
> >> removed one of the MD disks and looked what happened?

I really ought to do this before I put the machine into "production"!

> > I deliberated long and hard about how to implement my new 
> system and 
> > this was about the least complicated route!
> 
> Oh. ;)
>  
> 
> >> >> The goal is to have as "stress free" a system as possible as 
> >> >> little manual configuration, and in event of emergencies, as 
> >> >> little work to do, as possible.
> >> > If you want stress free, buy a Netapps storage appliance ;o)
> >> If they do all the rest that I need (smtp, web, file server,
> > I was being slightly slippant - Netapps just do big storage arrays 
> > with very high availability. For example, if they detect 
> that a drive 
> > is failing they send an email to Netapps and an engineer 
> comes round 
> > with a replacement disk all without any user intervention! I guess 
> > they're the Rolls Royce of storage solutions. Expensive though.
> 
> Ah. Well I don't think this'll fit in our budget. :)

That should of course have read "flippant". But I think you got my meaning.
:)
 
R.
--
http://robinbowes.com  

Re: Q: RAID-1 w/2x160GB, ReiserFS, Debian "woody", homebrew 2.4.25 kernel

[Tim Small]

I'll just deal with the MD solution, since it's a while since I played 
with LVM...

Jens Benecke wrote:

>How about RAIDing the root partition? If one drive fails will the other be
>able to boot via LILO? How about GRUB? Which do you prefer?
>
>  
>
Yes you can do this - see last post, I prefer Grub, see the thread with 
the subject "GRUB + RAID howto" in the page linked below - I suggest 
trying both methods stated there, and testing which one works for you...

http://www.linuxsa.org.au/mailing-list/2003-07/thread.html

>How about (/var)/tmp? I (suppose I'll) need it on both disks, does it make
>sense to mirror it as well?
>  
>
I would think so..

>Can I mirror the whole disk? Or do I need to mirror each partition
>seperately?
>  
>
Each partition.

>Does MD or LVM2 do hot sync, i.e. if one drive fails will I be able to stick
>in a replacement, and stop worrying? Or do I need to repartition the new
>disk exactly as the old one, before being able to sync?
>
>  
>
You will need to partition it, and use mdadm to add the new partitions 
to the array.  But you can simply do something like this:

dd if=/dev/hda of=/dev/hdc ; echo w | fdisk /dev/hdc

>How does LVM2/MD deal with failing harddisks, which is why I do the mirror
>at all? I've heard about MD not detecting read errors because the "other"
>disk was reading fine, and crashing completely when one disk was finally
>replaced because the data on the other disk was also corrupt. Is that still
>the current case?
>  
>
You need to arrange to read check the drive (you probably want to do 
this on single drive systems as well) I would advise using 
smartmontools, (e.g. from Sarge), with these lines like these in 
/etc/smartd.conf (also comment out the DEVICESCAN line):

/dev/hda -a -s L/../../6/01 -m root
/dev/hdc -a -s L/../../6/02 -m root

This way you stand a better chance of catching blocks which are going 
bad before they become unreadable - it will ask the drives to carry out 
an extended self test (e.g. surface scan etc.) at 1am/2am on Saturday.  
Note that this will work with SCSI disks, PATA disks, and SATA disks 
which use the old IDE driver, but not yet with SATA disks using libata 
(you could make do with a "dd if=/dev/sda of=/dev/null" in a cron job 
instead).

>The goal is to have as "stress free" a system as possible - i.e. with as
>little manual configuration, and in event of emergencies, as little work to
>do, as possible.
>
>  
>
On Debian, you should install the raidtools2 package, this will install 
a cron job which will email you on degraded array events (the smartd 
lines will email root on SMART errors as well) - but you should also 
install mdadm to use the new "mdadm" management tools.  I would advise 
using Sarge, instead of Woody, unless you have a good reason not too...

Tim.