recommended sata controllers?

recommended sata controllers?

[Mark Hahn]

do you have any recommendations for multiport sata controllers?
my bias is in the direction of multiple cheap cards, rather than
expensive ones like 3ware.

ideally, I'd like 8-12 ports in a box, preferably on PCI-X.
(I'll be using a dual-Opteron/Xeon server board with 1 CPU,
and don't mind dumb controllers at all)

has anyone had good luck with either the Promise FastTrack s150/tx4
(4pt, and seems to be well-supported but is PCI-X compatible?)
or the highpoint rr1820 (6 ports, claims 64x133 PCI-X).

thanks, mark hahn.

Re: recommended sata controllers?

[Andrew Hogue]

Hi,

I tried the Adaptec 1210-SA, however it's linux drivers are only for stock
redhat 7.2 and redhat 8 kernels... and its closed source.  So I wouldn't
recommend it until they open it up.

I'm currently using the onboard controller which is a CMD Technology 3112
device, seems to work pretty well with the current driver in 2.4.22.

-- Andrew


On Tue, 18 Nov 2003, Mark Hahn wrote:

> do you have any recommendations for multiport sata controllers?
> my bias is in the direction of multiple cheap cards, rather than
> expensive ones like 3ware.
>
> ideally, I'd like 8-12 ports in a box, preferably on PCI-X.
> (I'll be using a dual-Opteron/Xeon server board with 1 CPU,
> and don't mind dumb controllers at all)
>
> has anyone had good luck with either the Promise FastTrack s150/tx4
> (4pt, and seems to be well-supported but is PCI-X compatible?)
> or the highpoint rr1820 (6 ports, claims 64x133 PCI-X).
>
> thanks, mark hahn.
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

HELP.. how to remove a spare drive i accidently added

[James R Bamford]

Help..

as in i came home early to test and then to rebuild the array that just
takes ages..

I followed the advice in the FAQ.. took out one drive, booted up.. it listed
it as failed.. I powered down.. reconnected drive.. booted up.. it still
only used 1 drive... I thought i'd try using mdadm so i tried to add it..
its gone and added it as spare...

I'd tried assemble but that only ever used the 1 drive.. not both..

now i try to remove the spare

[root@backup root]# mdadm /dev/md0 -r /dev/hdg1
mdadm: hot remove failed for /dev/hdg1: Device or resource busy

and just cannot

I stop the array and i can't remove it as array isn't running..

Ahh stupid.. I've just looked at /proc/mdstat

Personalities : [raid1]
read_ahead 1024 sectors
md0 : active raid1 hde1[0] hdg1[2]
      16064896 blocks [2/1] [U_]
      [====>................]  recovery = 22.5% (3625536/16064896)
finish=20.2min speed=10225K/sec
unused devices: <none>

Its already recovering... phew.. i didn't find this clear..

mdadm -D revealed this

/dev/md0:
        Version : 00.90.00
  Creation Time : Mon Nov 17 22:16:09 2003
     Raid Level : raid1
     Array Size : 16064896 (15.32 GiB 16.45 GB)
    Device Size : 16064896 (15.32 GiB 16.45 GB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Tue Nov 18 20:03:00 2003
          State : dirty, no-errors
 Active Devices : 1
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 1


    Number   Major   Minor   RaidDevice State
       0      33        1        0      active sync   /dev/hde1
       1       0        0        1      faulty removed
       2      34        1        2      spare   /dev/hdg1
           UUID : f3be955c:8902d2ea:fb455a5e:8ea38b70
         Events : 0.24

It didn't really shout out to me that the spare drive was being used to
restore.. I did in panic go to the FAQ and follow their advice of
"raidhotadd /dev/mdX /dev/sdX" but this failed as it was already in use as
spare..

So could someone point out what to do when my harddrive really does fail...
take out the faulty one.. put a new one in, partition it.. and then add it
with

mdadm -a /dev/md0 /dev/<harddisk>

this will add it with spare and is equivelant to raidhotadd so will start
syncing the drive in!?

If its that easy then thats fine :)

Great even...

Thanks... sorry.. I'm nearly up to speed with this.. slowly..

Jim

Is my raid setup speed ok?!

[James R Bamford]

Just resyncing my full partitions now hopefully ready for use now that i've
got a grasp of how its working.. i've posted before but here is the current
progress on it sync-ing

[root@backup root]# cat /proc/mdstat
Personalities : [raid1]
read_ahead 1024 sectors
md0 : active raid1 hdg1[1] hde1[0]
      156288256 blocks [2/2] [UU]
      [====>................]  resync = 24.8% (38889996/156288256)
finish=195.6min speed=10001K/sec
unused devices: <none>

the speed varies but its around 10000k/sec

This is driven off a Promise Ultra 100 ATA controller.. a p3 450mhz ~450meg
ram (a weak link possibly), and

2 x 160gig Samsung SP1614N drives with 8meg cache

just ran hdparm to check dma was on

[root@backup root]# hdparm /dev/hde

/dev/hde:
 multcount    = 16 (on)
 IO_support   =  0 (default 16-bit)
 unmaskirq    =  0 (off)
 using_dma    =  1 (on)
 keepsettings =  0 (off)
 readonly     =  0 (off)
 readahead    =  8 (on)
 geometry     = 19457/255/63, sectors = 312581808, start = 0

[root@backup root]# hdparm /dev/hdg

/dev/hdg:
 multcount    = 16 (on)
 IO_support   =  0 (default 16-bit)
 unmaskirq    =  0 (off)
 using_dma    =  1 (on)
 keepsettings =  0 (off)
 readonly     =  0 (off)
 readahead    =  8 (on)
 geometry     = 19457/255/63, sectors = 312581808, start = 0

Is this speed normal.. someone told me drives can get up to 20-30meg/s ..
I'd of expected my drives to be near to that.. I get very good performance
with these drives in my other machines.. just wondering if its quite low...
if there is anything i can do to change this!? what kind of speeds have
other people had with IDE drives?

Thanks

Jim

Re: HELP.. how to remove a spare drive i accidently added

[Neil Brown]

On Tuesday November 18, jim@jimtreats.com wrote:
> 
> mdadm -D revealed this
> 
.....
> 
>     Number   Major   Minor   RaidDevice State
>        0      33        1        0      active sync   /dev/hde1
>        1       0        0        1      faulty removed
>        2      34        1        2      spare   /dev/hdg1
>            UUID : f3be955c:8902d2ea:fb455a5e:8ea38b70
>          Events : 0.24
> 
> It didn't really shout out to me that the spare drive was being used to
> restore.. 

Good point.  I will try to put is something like "spare-rebuilding",
though it is not actually possible to tell form user-space which
spare is being rebuilt.

> I did in panic go to the FAQ and follow their advice of
> "raidhotadd /dev/mdX /dev/sdX" but this failed as it was already in use as
> spare..
> 
> So could someone point out what to do when my harddrive really does fail...
> take out the faulty one.. put a new one in, partition it.. and then add it
> with
> 
> mdadm -a /dev/md0 /dev/<harddisk>
> 
> this will add it with spare and is equivelant to raidhotadd so will start
> syncing the drive in!?

Yes, this is completely correct.

NeilBrown

How safe is software RAID compared to how safe hardware RAID is!?

[James R Bamford]

Hi folks..

hopefully everything will be working soon and i can get on with trying to
get some backup scripts working :)

I am going to stick with this software RAID for the near future if it
carries on working correctly.. I was just wondering what peoples views were
of software vs hardware raid... anything and everything really.. I'm not
that up on all the technology... I know that 3ware have a good rep in
hardware... I also imagine that for more exotic RAID configurations its
obviously a help to not stress the CPU with the RAID tasks.. for me tho with
simple mirroring the CPU costs are minimal.. the linux core is a sturdy base
to build upon so is software raid in this way a perfectly acceptible
reliable RAID solution..

Just curious..

Thanks for everyones help

Jim

Re: How safe is software RAID compared to how safe hardware RAID is!?

[Neil Brown]

On Tuesday November 18, jim@jimtreats.com wrote:
> Hi folks..
> 
> hopefully everything will be working soon and i can get on with trying to
> get some backup scripts working :)
> 
> I am going to stick with this software RAID for the near future if it
> carries on working correctly.. I was just wondering what peoples views were
> of software vs hardware raid... anything and everything really.. I'm not
> that up on all the technology... I know that 3ware have a good rep in
> hardware... I also imagine that for more exotic RAID configurations its
> obviously a help to not stress the CPU with the RAID tasks.. for me tho with
> simple mirroring the CPU costs are minimal.. the linux core is a sturdy base
> to build upon so is software raid in this way a perfectly acceptible
> reliable RAID solution..
> 
> Just curious..

Not an easy question to answer.
RAID is all about surviving failures, and there are alots of possible
failure modes, some more predictable than others.

The obvious failure modes are handled perfectly well by all solutions.

The less obvious failure modes are probably handled as best as
possible by all solutions, but with these, the question is usually
"how can I recover", and the answer there depends on how much support
you can get, and how much you can work out yourself.

For support: linux-raid is an adequate support community, but there is
no guarantee of instant support.  I don't know what support there
might be around various hardware raid solution.  Again, someone on
linux-raid might be able to help, or they might not.

For "how much you can work out yourself", that depends partly on you.
Your hardware raid might come with good doco, or it might not.
Software RAID somes with assorted bits of doco, and complete source.

You are part of your system, and system reliability depends in-part of
how much you know.  With software raid you can learn as much as you
want to, but it might require more work that you care to put it.

Software raid can be affected by lots of parts of the system.
Reliability of hardware raid is more tightly connected to the
controller and drives.

As examples of "less obvious failure modes":

 I had a mirrored pair that kept getting data corruption.  It turned
 out that one of the drives had a bad bit in an internal buffer, and
 would occasionally return 1 for the 12th bit (or similar) of each
 sector even when it should be zero.   Neither software raid or
 hardware raid would cope with that.

 Some people find that their IDE controller works fine until they try
 to use RAID1.  RAID1 hits multiple discs concurrently a lot, and some
 (few, specific) ide controllers don't appear to cope.  You probably
 would not get that with hardware raid.  You can with software raid
 because it is a "whole-system" thing.


I prefer software raid because I like the fact that I can see inside
and understand it completely.  Some people like hardware raid because
"it just works".
Both are valid perspectives and as you, the sys-admin, are part of
the system, your prespective and opinion is a significant part of what
makes the system "reliable".

NeilBrown

Re: How safe is software RAID compared to how safe hardware RAID is!?

[Kourosh]

On Tue, 2003-11-18 at 15:12, James R Bamford wrote:
> Hi folks..
> 
> hopefully everything will be working soon and i can get on with trying to
> get some backup scripts working :)
> 
> I am going to stick with this software RAID for the near future if it
> carries on working correctly.. I was just wondering what peoples views were
> of software vs hardware raid... anything and everything really.. I'm not
> that up on all the technology... I know that 3ware have a good rep in
> hardware... I also imagine that for more exotic RAID configurations its
> obviously a help to not stress the CPU with the RAID tasks.. for me tho with
> simple mirroring the CPU costs are minimal.. the linux core is a sturdy base
> to build upon so is software raid in this way a perfectly acceptible
> reliable RAID solution..
> 
> Just curious..
> 
> Thanks for everyones help
> 
> Jim

Really depends on what you're doing and what you want out of it.  For
higher end systems with lots of I/O and very large storage needs it may
be necessary to go with a high quality hardware RAID card with a large
battery backed cache.  For smaller systems Linux does software RAID very
well, especially in these days of fast, inexpensive processors and
inexpensive memory (compared to several years ago).

One thing to consider is what happens a few years down the line and your
hardware RAID card dies.  Can you replace it?  What happens to your
data?  At least with software RAID you can replace the controller and
everything can be brought back up.

I've been using soft RAID on my own and several client servers for a few
years with no real problems.

Hope this helps.

-- 
Kourosh <linux-raid@mindwaresystems.com>

Re: How safe is software RAID compared to how safe hardware RAID is!?

[Frank Smith]

--On Tuesday, November 18, 2003 23:12:43 +0000 James R Bamford <jim@jimtreats.com> wrote:

> Hi folks..
> 
> hopefully everything will be working soon and i can get on with trying to
> get some backup scripts working :)
> 
> I am going to stick with this software RAID for the near future if it
> carries on working correctly.. I was just wondering what peoples views were
> of software vs hardware raid... anything and everything really.. I'm not
> that up on all the technology... I know that 3ware have a good rep in
> hardware... I also imagine that for more exotic RAID configurations its
> obviously a help to not stress the CPU with the RAID tasks.. for me tho with
> simple mirroring the CPU costs are minimal.. the linux core is a sturdy base
> to build upon so is software raid in this way a perfectly acceptible
> reliable RAID solution..
> 
> Just curious..
> 
> Thanks for everyones help
> 
> Jim
> 

They may be equally reliable, but it seems to me that there are a
lot more opportunities to accidently destroy a software raid than
a hardware raid.

Frank

-- 
Frank Smith                                      fsmith@hoovers.com
Systems Administrator                           Voice: 512-374-4673
Hoover's Online                                   Fax: 512-374-4501

Re: How safe is software RAID compared to how safe hardware RAID

[Michael]

> Hi folks..
> 
> hopefully everything will be working soon and i can get on with
> trying to get some backup scripts working :)
> 
> I am going to stick with this software RAID for the near future if
> it carries on working correctly.. I was just wondering what peoples
> views were of software vs hardware raid... anything and everything
> really.. I'm not that up on all the technology... I know that 3ware
> have a good rep in hardware... I also imagine that for more exotic
> RAID configurations its obviously a help to not stress the CPU with
> the RAID tasks.. for me tho with simple mirroring the CPU costs are
> minimal.. the linux core is a sturdy base to build upon so is
> software raid in this way a perfectly acceptible reliable RAID
> solution..
> 

Your choice is to use "hardware" raid which is really just software 
raid implemented with a little bitty processor and some firmware, or 
"software" raid which is open source and runs on your honken' 
big-fast (relatively speaking) main processor. For i86 this means 
that the raid parity calculations are carried out by the mmi unit 
which sits idle most of the time anyway. It would be tough to 
actually find the missing cpu cycles. I've not done the benchmarks, 
but others on the list claim that software raid is faster than any of 
the hardware implimentations. My experience with several raid cards 
leads me to believe that software raid is far more robust.

Michael
running linux software raid since 1996

Re: How safe is software RAID compared to how safe hardware RAID is!?

[Franc Carter]

Excellent, answer - I'm going to 'lift' your concept of 'you are part
of the sytem', it describes the situation vary nicely.

cheers

On Wed, 19 Nov 2003 10:34 am, Neil Brown wrote:
> On Tuesday November 18, jim@jimtreats.com wrote:
> > Hi folks..
> > 
> > hopefully everything will be working soon and i can get on with trying to
> > get some backup scripts working :)
> > 
> > I am going to stick with this software RAID for the near future if it
> > carries on working correctly.. I was just wondering what peoples views 
were
> > of software vs hardware raid... anything and everything really.. I'm not
> > that up on all the technology... I know that 3ware have a good rep in
> > hardware... I also imagine that for more exotic RAID configurations its
> > obviously a help to not stress the CPU with the RAID tasks.. for me tho 
with
> > simple mirroring the CPU costs are minimal.. the linux core is a sturdy 
base
> > to build upon so is software raid in this way a perfectly acceptible
> > reliable RAID solution..
> > 
> > Just curious..
> 
> Not an easy question to answer.
> RAID is all about surviving failures, and there are alots of possible
> failure modes, some more predictable than others.
> 
> The obvious failure modes are handled perfectly well by all solutions.
> 
> The less obvious failure modes are probably handled as best as
> possible by all solutions, but with these, the question is usually
> "how can I recover", and the answer there depends on how much support
> you can get, and how much you can work out yourself.
> 
> For support: linux-raid is an adequate support community, but there is
> no guarantee of instant support.  I don't know what support there
> might be around various hardware raid solution.  Again, someone on
> linux-raid might be able to help, or they might not.
> 
> For "how much you can work out yourself", that depends partly on you.
> Your hardware raid might come with good doco, or it might not.
> Software RAID somes with assorted bits of doco, and complete source.
> 
> You are part of your system, and system reliability depends in-part of
> how much you know.  With software raid you can learn as much as you
> want to, but it might require more work that you care to put it.
> 
> Software raid can be affected by lots of parts of the system.
> Reliability of hardware raid is more tightly connected to the
> controller and drives.
> 
> As examples of "less obvious failure modes":
> 
>  I had a mirrored pair that kept getting data corruption.  It turned
>  out that one of the drives had a bad bit in an internal buffer, and
>  would occasionally return 1 for the 12th bit (or similar) of each
>  sector even when it should be zero.   Neither software raid or
>  hardware raid would cope with that.
> 
>  Some people find that their IDE controller works fine until they try
>  to use RAID1.  RAID1 hits multiple discs concurrently a lot, and some
>  (few, specific) ide controllers don't appear to cope.  You probably
>  would not get that with hardware raid.  You can with software raid
>  because it is a "whole-system" thing.
> 
> 
> I prefer software raid because I like the fact that I can see inside
> and understand it completely.  Some people like hardware raid because
> "it just works".
> Both are valid perspectives and as you, the sys-admin, are part of
> the system, your prespective and opinion is a significant part of what
> makes the system "reliable".
> 
> NeilBrown
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

-- 
Franc Carter     Ph:61-2-8374-5071      Fax: 61-2-8374-5070
Systems Manager, SIRCA Ltd              http://www.sirca.org.au/

DISCLAIMER: The contents of this email, inclusive of attachments, may
be legally privileged and confidential. Any unauthorised use of the
contents is expressly prohibited. If you have received this message in
error or are not the intended recipient, you should destroy the email
message along with any attachment(s).  Unintended recipients of this
email are prohibited from retaining, disclosing, distributing or using
any information contained herein.  This email is also subject to
copyright. No part of it should be reproduced, adapted or transmitted
without the written consent of the copyright owner.

RE: How safe is software RAID compared to how safe hardware RAID is!?

[James R Bamford]

Thanks for everyones replies.. that's given me a real feel for how its being
used by you guys..

I've gone through a week of discovery and should the need arise i will
certainly look into it even more.. hopefully i've gone far enough that for
the near future *touch wood* i'll be at that "it just works" stage, and the
next linux learning step can be mine to frustrate me :)

Having had problems with the device i will be testing it a lot over the next
few days/weeks.. i'm comforted by the array staying up in the event of a
drive failure.. I'm a little worried that I am not going to spot a drive
fail.. I guess thats what the mdadm monitor stuff is for!?!? if i run this
once will it work forever or just until i power down?!

Its also reassuring that during my wrestling with the corruption problems i
was having i discovered that whilst not ideal you can mount the constituent
mirror raid drives seperately and deal with them... this perhaps gives me
something that i can fall back on although the normal array should be online
anyways so its kind of redundant.. if need be i can make a backup of my
backup of those ULTRA important files before attempting to rebuild the
array...

Thanks again... The only thing that i was kind of justifying paying £100 and
above on for the 3ware cards was that it claimed to be able to get SMART
info off my hdds.. i don't know if the promise ultra 100 ATA controller does
this.. and i'm not sure how you get SMART/temperature information out of
drives in linux anyways.. I will go searching for that... with a new case
ordered though my backup machine should be sufficiently cooled

Cheers

Jim

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How safe is software RAID compared to how safe hardware RAID is!?

[3tcdgwg3]

>
>  Some people find that their IDE controller works fine until they try
>  to use RAID1.  RAID1 hits multiple discs concurrently a lot, and some
>  (few, specific) ide controllers don't appear to cope.  You probably
>  would not get that with hardware raid.  You can with software raid
>  because it is a "whole-system" thing.
>
>

Did I do anything wrong?

I didn't see RAID 1 hits multiple drives concurrently, specially for
reading.

I set up RAID 5, RAID 1 and single disk partitions on x86 machine,
kernel 2.4-18.

For 800 MB breading, RAID 5 take 2/3 of time used by single disk,
but RAID 1 take 1.1 of time used by single disk.

Thanks.

RE: Is my raid setup speed ok?!

[Guy]

It may be too late for this re-sync, but in the future do this:
echo 100000 > /proc/sys/dev/raid/speed_limit_max

The default speed limit is 10000, that's what you are getting.
So you have reached a software imposed limit, not a hardware limit.


-----Original Message-----
From: linux-raid-owner@vger.kernel.org
[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of James R Bamford
Sent: Tuesday, November 18, 2003 5:05 PM
To: linux-raid@vger.kernel.org
Subject: Is my raid setup speed ok?!

Just resyncing my full partitions now hopefully ready for use now that i've
got a grasp of how its working.. i've posted before but here is the current
progress on it sync-ing

[root@backup root]# cat /proc/mdstat
Personalities : [raid1]
read_ahead 1024 sectors
md0 : active raid1 hdg1[1] hde1[0]
      156288256 blocks [2/2] [UU]
      [====>................]  resync = 24.8% (38889996/156288256)
finish=195.6min speed=10001K/sec
unused devices: <none>

the speed varies but its around 10000k/sec

This is driven off a Promise Ultra 100 ATA controller.. a p3 450mhz ~450meg
ram (a weak link possibly), and

2 x 160gig Samsung SP1614N drives with 8meg cache

just ran hdparm to check dma was on

[root@backup root]# hdparm /dev/hde

/dev/hde:
 multcount    = 16 (on)
 IO_support   =  0 (default 16-bit)
 unmaskirq    =  0 (off)
 using_dma    =  1 (on)
 keepsettings =  0 (off)
 readonly     =  0 (off)
 readahead    =  8 (on)
 geometry     = 19457/255/63, sectors = 312581808, start = 0

[root@backup root]# hdparm /dev/hdg

/dev/hdg:
 multcount    = 16 (on)
 IO_support   =  0 (default 16-bit)
 unmaskirq    =  0 (off)
 using_dma    =  1 (on)
 keepsettings =  0 (off)
 readonly     =  0 (off)
 readahead    =  8 (on)
 geometry     = 19457/255/63, sectors = 312581808, start = 0

Is this speed normal.. someone told me drives can get up to 20-30meg/s ..
I'd of expected my drives to be near to that.. I get very good performance
with these drives in my other machines.. just wondering if its quite low...
if there is anything i can do to change this!? what kind of speeds have
other people had with IDE drives?

Thanks

Jim

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How safe is software RAID compared to how safe hardware RAID is!?

[Luca Berra]

On Tue, Nov 18, 2003 at 11:12:43PM -0000, James R Bamford wrote:
>carries on working correctly.. I was just wondering what peoples views were
>of software vs hardware raid... anything and everything really.. I'm not
i believe the real difference is:
raid with a battery
raid without a battery

hw raid with a battery can improve reliability in case of system crash
or power failure, without it cannot.

L.

-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

RE: Is my raid setup speed ok?!

[James R Bamford]

> -----Original Message-----
> From: Guy [mailto:bugzilla@watkins-home.com]
> Sent: 19 November 2003 04:10
> To: 'James R Bamford'; linux-raid@vger.kernel.org
> Subject: RE: Is my raid setup speed ok?!
>
>
> It may be too late for this re-sync, but in the future do this:
> echo 100000 > /proc/sys/dev/raid/speed_limit_max
>
> The default speed limit is 10000, that's what you are getting.
> So you have reached a software imposed limit, not a hardware limit.
>
>

Thanks.. I wondered about this.. it could really do with being added into
the HOWTO..

Is this fix a do once.. fix everytime.. or something that i have to do each
time i plan to rebuild an array.. (isn't proc deleted every boot!? so
someone has asked me...)

Thanks tho

Jim

RE: Is my raid setup speed ok?!

[Guy]

Each time you re-boot.

It's not a fix.  It's just changing a default.  If your system ran too slow
during a re-build, you would want to lower the number.

As I understand it, this limits the data rate to each disk.  Since you had 2
disks, you were getting 20000K per second of disk i/o.

It would be nice if /proc/mdstat would display the current min and max speed
limits.  It would also be nice if there was a place you could set them just
once.  Maybe /etc/raidtab.  Maybe a new file /etc/md.conf.  I would prefer
to have an overall throughput speed limit, and a per array group speed
limit.  The overall throughput speed limit would not be per disk, but total
disk i/o.

Guy

-----Original Message-----
From: James R Bamford [mailto:jim@jimtreats.com] 
Sent: Wednesday, November 19, 2003 8:57 AM
To: Guy; 'James R Bamford'; linux-raid@vger.kernel.org
Subject: RE: Is my raid setup speed ok?!

> -----Original Message-----
> From: Guy [mailto:bugzilla@watkins-home.com]
> Sent: 19 November 2003 04:10
> To: 'James R Bamford'; linux-raid@vger.kernel.org
> Subject: RE: Is my raid setup speed ok?!
>
>
> It may be too late for this re-sync, but in the future do this:
> echo 100000 > /proc/sys/dev/raid/speed_limit_max
>
> The default speed limit is 10000, that's what you are getting.
> So you have reached a software imposed limit, not a hardware limit.
>
>

Thanks.. I wondered about this.. it could really do with being added into
the HOWTO..

Is this fix a do once.. fix everytime.. or something that i have to do each
time i plan to rebuild an array.. (isn't proc deleted every boot!? so
someone has asked me...)

Thanks tho

Jim

Re: How safe is software RAID compared to how safe hardware RAID is!?

[Bill Rugolsky Jr.]

On Tue, Nov 18, 2003 at 11:12:43PM -0000, James R Bamford wrote:
> I am going to stick with this software RAID for the near future if it
> carries on working correctly.. I was just wondering what peoples views were
> of software vs hardware raid... anything and everything really.. I'm not
> that up on all the technology... I know that 3ware have a good rep in
> hardware... I also imagine that for more exotic RAID configurations its
> obviously a help to not stress the CPU with the RAID tasks.. for me tho with
> simple mirroring the CPU costs are minimal.. the linux core is a sturdy base
> to build upon so is software raid in this way a perfectly acceptible
> reliable RAID solution..
 
I have had a 3ware 7800 RAID controller for a while, and I'm pleased with
it, but I use it in JBOD mode, with Linux MD on top.  Why?

1. The version of the firmware that I have on the card requires one
   to reboot into the BIOS to change the configuration.  My understanding
   is that later firmware revisions allow it to be reconfigured from
   the command line.

2. The 3dm monitoring package was closed source.  That may have changed.

3. The RAID meta-data was undocumented last time I checked.  With Linux MD,
   I can plug the drives into any IDE controller and figure out what is
   going on.  Not so long ago, I had a two drive failure in the same day;
   I was able to partially resurrect the second drive, and recovered the
   data from it.

The benefit of hardware RAID1, in particular, is to not send the same
data twice over the bus.  Were I trying to eek every last bit of
performance, I'd use the hardware solution.

I played with the 3ware hardware RAID back at 2.4.early when there were
concerns about the interaction between ext3, LVM, and MD.  But once that
settled down, I switched to soft-RAID.

Ideally, hardware RAID vendors would publish complete meta-data specs,
and provide a simple userland tool to extract and parse when the drive
is plugged into a non-RAID controller.  Perhaps some do.
If meta-data were documented, it might be relatively easy to extract data
with just a script, or perhaps a small Device Mapper module.

Regards,

	Bill Rugolsky

RE: Is my raid setup speed ok?!

[Matthew Mitchell]

On Wed, 2003-11-19 at 10:49, Guy wrote:

> It would be nice if /proc/mdstat would display the current min and max speed
> limits.  It would also be nice if there was a place you could set them just
> once.  Maybe /etc/raidtab.  Maybe a new file /etc/md.conf.  I would prefer
> to have an overall throughput speed limit, and a per array group speed
> limit.  The overall throughput speed limit would not be per disk, but total
> disk i/o.

Setting it at boot time is a job for sysctl.  See /etc/sysctl.conf
and/or man sysctl.

-m

RE: Is my raid setup speed ok?!

[Guy]

Sysctl...That's new to me, thanks!

Guy

-----Original Message-----
From: linux-raid-owner@vger.kernel.org
[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Matthew Mitchell
Sent: Wednesday, November 19, 2003 12:07 PM
To: Guy
Cc: 'James R Bamford'; linux-raid@vger.kernel.org
Subject: RE: Is my raid setup speed ok?!

On Wed, 2003-11-19 at 10:49, Guy wrote:

> It would be nice if /proc/mdstat would display the current min and max
speed
> limits.  It would also be nice if there was a place you could set them
just
> once.  Maybe /etc/raidtab.  Maybe a new file /etc/md.conf.  I would prefer
> to have an overall throughput speed limit, and a per array group speed
> limit.  The overall throughput speed limit would not be per disk, but
total
> disk i/o.

Setting it at boot time is a job for sysctl.  See /etc/sysctl.conf
and/or man sysctl.

-m


-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How safe is software RAID compared to how safe hardware RAID

[Kanoa Withington]

Jim,

It's a good question. I think most people will agree that having the
source code and the user support gives one a good feeling about Linux
software RAID. I would recommend it over hardware RAID for every
application except RAID 5.

The majority of the I/O errors I see on modern disks are read errors
rather than catastrophic disk failures. A software RAID 5 array goes
into degraded mode if it encounters a read error. Aside from the
immediate loss of redundancy, if there are any latent unreadable
blocks on the remaining disks, even an unreadable block which is not
in use by the file system, the array simply cannot be repaired. In a
very large array this is not uncommon. If the unreadable block was in
use by the file system, then there will be loss of data. Some hardware
controllers attempt to repair an unreadable block _before_ degrading
the array, an operation which in most cases is sucessfull.

This is a failure mode where I would consider a hardware controller to
be more reliable.

In a software RAID 1 array, each of the sub-volumes can be used just
like a normal partition+file system outside of the array and the whole
domain of software RAID, which is very nice for fault recovery. With a
hardware controller this usually cannot be done, the data on the disks
is useless without the controller.

This is a failure mode where I would consider a software array to be
more reliable.

-Kanoa

On Tue, 18 Nov 2003, James R Bamford wrote:

> Hi folks..
>
> hopefully everything will be working soon and i can get on with trying to
> get some backup scripts working :)
>
> I am going to stick with this software RAID for the near future if it
> carries on working correctly.. I was just wondering what peoples views were
> of software vs hardware raid... anything and everything really.. I'm not
> that up on all the technology... I know that 3ware have a good rep in
> hardware... I also imagine that for more exotic RAID configurations its
> obviously a help to not stress the CPU with the RAID tasks.. for me tho with
> simple mirroring the CPU costs are minimal.. the linux core is a sturdy base
> to build upon so is software raid in this way a perfectly acceptible
> reliable RAID solution..
>
> Just curious..
>
> Thanks for everyones help
>
> Jim
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: How safe is software RAID compared to how safe hardware RAID

[Gordon Henderson]

On Wed, 19 Nov 2003, Kanoa Withington wrote:

> In a software RAID 1 array, each of the sub-volumes can be used just
> like a normal partition+file system outside of the array and the whole
> domain of software RAID, which is very nice for fault recovery. With a
> hardware controller this usually cannot be done, the data on the disks
> is useless without the controller.

Just as an aside, I've recently had to build several near identical
machines with a mirrored disk set (ie. s/w RAID1) and I used this feature
to "clone" the other boxes... ie. build one with 2 disks, make sure the
mirrors are syncd up, then take one drive out and use it to build a 2nd,
and a 3rd, etc.  each time putting in a blank 2nd disk and partitioning it
as before then letting the mirror re-sync...

I'm sure there may be easier ways, but this worked for me!

Gordon

Re: How safe is software RAID compared to how safe hardware RAID is!?

[Tomi Orava]

>  I had a mirrored pair that kept getting data corruption.  It turned
>  out that one of the drives had a bad bit in an internal buffer, and
>  would occasionally return 1 for the 12th bit (or similar) of each
>  sector even when it should be zero.   Neither software raid or
>  hardware raid would cope with that.

How did you manage to figure out what was the reason for this
corruption ? I mean, does there exist a some sort of "generic"
way how to solve such errors, or was it just hard work in comparing
what got written to the different disks ...

>  Some people find that their IDE controller works fine until they try
>  to use RAID1.  RAID1 hits multiple discs concurrently a lot, and some
>  (few, specific) ide controllers don't appear to cope.  You probably
>  would not get that with hardware raid.  You can with software raid
>  because it is a "whole-system" thing.

Can you name the controllers which have had problems in this area ?
I'm having big problems in Linux 2.4 kernels IDE-stability with
heavy I/O (all disks are RAID1), perhaps either the Sil680 or
HPT374 are just problematic controllers ... ? (it would be just great
to solve all of these problems by buying new IDE-cards :)

Regards,
Tomi Orava

Re: How safe is software RAID compared to how safe hardware RAID is!?

[Neil Brown]

On Thursday November 20, tomimo+linux-raid@ncircle.nullnet.fi wrote:
> 
> >  I had a mirrored pair that kept getting data corruption.  It turned
> >  out that one of the drives had a bad bit in an internal buffer, and
> >  would occasionally return 1 for the 12th bit (or similar) of each
> >  sector even when it should be zero.   Neither software raid or
> >  hardware raid would cope with that.
> 
> How did you manage to figure out what was the reason for this
> corruption ? I mean, does there exist a some sort of "generic"
> way how to solve such errors, or was it just hard work in comparing
> what got written to the different disks ...

I think the generic solution is "Tear your hair out and wander around
muttering to everyone how this couldn't possibly be happening and
seeking sympathy".

Like lots of problem solving, it is a case of gathering as much
information as possible, staring at it for a while, and then
explaining to someone who has absolutely no expertise in the field
exactly why this collection of evidence is internally inconsistent and
it just isn't possible.  Usually that brings the answer out very
quickly.  (Some people say a brick wall will do, but I find a person
works much better).

One interesting aspect was that the drive with the problems was the
one that it would preferentially rebuild the raid from.  That tended
to make transient errors more permanent as they would get written onto
the second driver.  I'm not sure if that helpped or hindered though.

> 
> >  Some people find that their IDE controller works fine until they try
> >  to use RAID1.  RAID1 hits multiple discs concurrently a lot, and some
> >  (few, specific) ide controllers don't appear to cope.  You probably
> >  would not get that with hardware raid.  You can with software raid
> >  because it is a "whole-system" thing.
> 
> Can you name the controllers which have had problems in this area ?
> I'm having big problems in Linux 2.4 kernels IDE-stability with
> heavy I/O (all disks are RAID1), perhaps either the Sil680 or
> HPT374 are just problematic controllers ... ? (it would be just great
> to solve all of these problems by buying new IDE-cards :)

I'm afraid I haven't collected names.  I avoid IDE raid like the
plague, and just stick with SCSI - too many bad experiences.
NOTE: This is not advice on my part to not use IDE.  My experience is
not sufficient to base a recommendation on.  But for me, my budget
allows SCSI, and it just makes me feel a lot safer.

NeilBrown

Re: How safe is software RAID compared to how safe hardware RAID is!?

[Rechenberg, Andrew]

We've been using software RAID for about 10 months now on our large
production database server and I have nothing but good things to say
about SW RAID.  

I believe we have one of the larger Linux software RAID arrays and it's
been great.  We've had some other hardware issues unrelated to the array
and the array has faired well.  The only thing you have to remember is
that if your machine goes down hard, your arrays will rebuild upon
reboot because they weren't marked as clean.

We also are getting MUCH better performance from SW RAID that our Dell
PERC3/QC hardware RAID cards.  Here are some numbers:

PERC3/QC single channel RAID0, 64KB chunk, ext3 ordered:
- 90MB/s sequential read

Adaptec 39160 single channel Linux SW RAID0, 64kb chunk, ext3 ord:
- 138MB/s

We are now running a 56 disk SW RAID10 array (52 disks + 4 spares) and
performance is great!

Let me know if you have any questions, but I highly recommend Linux
software RAID.

On Tue, 2003-11-18 at 18:12, James R Bamford wrote:
> Hi folks..
> 
> hopefully everything will be working soon and i can get on with trying to
> get some backup scripts working :)
> 
> I am going to stick with this software RAID for the near future if it
> carries on working correctly.. I was just wondering what peoples views were
> of software vs hardware raid... anything and everything really.. I'm not
> that up on all the technology... I know that 3ware have a good rep in
> hardware... I also imagine that for more exotic RAID configurations its
> obviously a help to not stress the CPU with the RAID tasks.. for me tho with
> simple mirroring the CPU costs are minimal.. the linux core is a sturdy base
> to build upon so is software raid in this way a perfectly acceptible
> reliable RAID solution..
> 
> Just curious..
> 
> Thanks for everyones help
> 
> Jim
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
-- 

Regards,
Andrew Rechenberg
Infrastructure Team, Sherman Financial Group

Re: How safe is software RAID compared to how safe hardware RAID is!?

[Lars Marowsky-Bree]

On 2003-11-21T09:02:47,
   "Rechenberg, Andrew" <ARechenberg@shermanfinancialgroup.com> said:

> We also are getting MUCH better performance from SW RAID that our Dell
> PERC3/QC hardware RAID cards.  Here are some numbers:

But, frankly, the perc3's are some of the worst performance RAID
controllers I have ever seen ;-)


Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>

-- 
High Availability & Clustering	      \ ever tried. ever failed. no matter.
SUSE Labs			      | try again. fail again. fail better.
Research & Development, SUSE LINUX AG \ 	-- Samuel Beckett

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How safe is software RAID compared to how safe hardware RAID is!?

[Eric Wood]

3tcdgwg3 wrote:
> Did I do anything wrong?
>
> I didn't see RAID 1 hits multiple drives concurrently, specially for
> reading.
>
> I set up RAID 5, RAID 1 and single disk partitions on x86 machine,
> kernel 2.4-18.
>
> For 800 MB breading, RAID 5 take 2/3 of time used by single disk,
> but RAID 1 take 1.1 of time used by single disk.

Doing anything (reading or writing) with RAID 1 set will always be slower
than with a single drive.  Because even with reading, you're still updating
the atime on both drives.  As a test, mount that partition with noatime and
see if RAID-1 reading test nears to 1.

Here's a nice message from while back:

http://www.cs.helsinki.fi/linux/linux-kernel/2002-17/0313.html
<snip> Jaime Medrano wrote:
> I have taken a look at the read balancing code at raid1.c and I have found
> that when a sequential read happens no balancing is done, and so all the
> reading is done from only one of the mirrors while the others are iddle
<snip>

So RAID-1 sets don't stripe read.  RAID 5 is forced to stripe seek because
of its very nature of having XORed data across the drives.

-eric wood

HighPoint 1820 (8pt sata pcix, $170)

[Mark Hahn]

do you have any comment on the HighPoint 1820 card?  it's an 8-port SATA
controller with PCI-X, and appears to list for about $US 170 (pricewatch).
there's a driver (apparently binary-only, therefore of unknown quality)
available from the vendor, but I suspect no real (kerne.org or libata)
driver support...

comments on HighPoint in general?  I've seen a certain amount of traffic from
people having trouble with their previous cards, but that's a very biased 
sample, and who knows - maybe they've improved ;)

thanks, mark hahn.
(I don't really care about how well it does HW raid; I'd use MD anyway...)

Re: HighPoint 1820 (8pt sata pcix, $170)

[jlewis]

On Fri, 28 Nov 2003, Mark Hahn wrote:

> 
> do you have any comment on the HighPoint 1820 card?  it's an 8-port SATA
> controller with PCI-X, and appears to list for about $US 170 (pricewatch).
> there's a driver (apparently binary-only, therefore of unknown quality)
> available from the vendor, but I suspect no real (kerne.org or libata)
> driver support...
> 
> comments on HighPoint in general?  I've seen a certain amount of traffic from
> people having trouble with their previous cards, but that's a very biased 
> sample, and who knows - maybe they've improved ;)

I'd be very hesitant.  I recently bought one of their cheaper cards 
(Rocket100) thinking "there's a driver in the kernel, it looks supported" 
only to find the current kernel driver does not work and neither do their 
binary drivers (driver disks) for Red Hat 8.0 or 9.

I ended up filing a bugzilla report with Red Hat and ordering a Promise 
Ultra100TX2 to replace the HPT.
  
----------------------------------------------------------------------
 Jon Lewis *jlewis@lewis.org*|  I route
 Senior Network Engineer     |  therefore you are
 Atlantic Net                |  
_________ http://www.lewis.org/~jlewis/pgp for PGP public key_________

How to change RAID1 to NON_RAID (lilo problem)

[Mehrdad Ziaei]

Hi,

I have Linux Redhat 7.3 configured with software raid1.
I removed second disk from all mirror raids.
Now, lilo quits without showing any error messaes.
If I add second disk to raid for /boot, then lilo will start working fine
again.

My questions are:
1. How can I change my raid1 configuration to have only 1 disk and working
without any problem?
2. How can I change raid1 to non-raid?

Thank you for your help
Mehrdad.

==========================
[root@linux ]# strace lilo -t
execve("/sbin/lilo", ["lilo", "-t"], [/* 27 vars */]) = 0
uname({sys="Linux", node="linux", ...}) = 0
brk(0)                                  = 0x8058b04
....
....
open("/dev/md0", 0x4)                   = 4
fstat64(4, {st_mode=S_IFBLK|0660, st_rdev=makedev(9, 0), ...}) = 0
ioctl(4, 0x80480911, 0xbfffea40)        = 0
ioctl(4, 0x80140912, 0xbfffea20)        = 0
_exit(0)                                = ?
=====================
[root@linux ]# cat /proc/mdstat
Personalities : [raid1]
read_ahead 1024 sectors
md0 : active raid1 sda1[1]
      513984 blocks [2/1] [_U]

md1 : active raid1 sda2[1]
      34828800 blocks [2/1] [_U]

md2 : active raid1 sda3[1]
      570240 blocks [2/1] [_U]
=====================
[root@linux ]# cat /etc/raidtab
raiddev             /dev/md1
raid-level                  1
nr-raid-disks               1
chunk-size                  64k
persistent-superblock       1
nr-spare-disks              0
    device          /dev/sda2
    raid-disk     0
raiddev             /dev/md0
raid-level                  1
nr-raid-disks               1
chunk-size                  64k
persistent-superblock       1
nr-spare-disks              0
    device          /dev/sda1
    raid-disk     0
raiddev             /dev/md2
raid-level                  0
nr-raid-disks               1
chunk-size                  64k
persistent-superblock       1
nr-spare-disks              0
    device          /dev/sda3
    raid-disk     0
====================

Re: HighPoint 1820 (8pt sata pcix, $170)

[Jeff Garzik]

On Fri, Nov 28, 2003 at 01:04:24AM -0500, Mark Hahn wrote:
> 
> do you have any comment on the HighPoint 1820 card?  it's an 8-port SATA
> controller with PCI-X, and appears to list for about $US 170 (pricewatch).
> there's a driver (apparently binary-only, therefore of unknown quality)
> available from the vendor, but I suspect no real (kerne.org or libata)
> driver support...

HighPoint is just about the only manufacturer that hasn't contacted me
about getting a SATA driver for their hardware into the kernel :)

I would go for Promise or ServerWorks 4-port and 8-port SATA cards, if
you don't wanna spend the extra bucks for a real RAID card (and I
personally trust software RAID over hardware RAID any day).

	Jeff