Ok to go ahead with this setup?

Ok to go ahead with this setup?

[Christian Pernegger]

Hi list!

Having experienced firsthand the pain that hardware RAID controllers
can be -- my 3ware 7500-8 died and it took me a week to find even a
7508-8 -- I would like to switch to kernel software RAID.

Here's a tentative setup:

Intel SE7230NH1-E mainboard
Pentium D 930
2x1GB Crucial 533 DDR2 ECC
Intel SC5295-E enclosure

Promise Ultra133 TX2 (2ch PATA)
   - 2x Maxtor 6B300R0 (300GB, DiamondMax 10) in RAID1

Onboard Intel ICH7R (4ch SATA)
   - 4x Western Digital WD5000YS (500GB, Caviar RE2) in RAID5

* Does this hardware work flawlessly with Linux?

* Is it advisable to boot from the mirror?
  Would the box still boot with only one of the disks?

* Can I use EVMS as a frontend?
  Does it even use md or is EVMS's RAID something else entirely?

* Should I use the 300s as a single mirror, or span multiple ones over
the two disks?

* Am I even correct in assuming that I could stick an array in another
box and have it work?

Comments welcome

Thanks,

Chris

Re: Ok to go ahead with this setup?

[Molle Bestefich]

Christian Pernegger wrote:
> Intel SE7230NH1-E mainboard
> Pentium D 930

HPA recently said that x86_64 CPUs have better RAID5 performance.

> Promise Ultra133 TX2 (2ch PATA)
>    - 2x Maxtor 6B300R0 (300GB, DiamondMax 10) in RAID1
>
> Onboard Intel ICH7R (4ch SATA)
>    - 4x Western Digital WD5000YS (500GB, Caviar RE2) in RAID5

Is it a NAS kind of device?

In that case, drop the 2x 300GB disks and get 6x 500GB instead.  You
can partition those so that you have a RAID1 spanning the first 10GB
of all 6 drives for use as the system partition, and use the rest in a
RAID5.

> * Does this hardware work flawlessly with Linux?

No clue.

> * Is it advisable to boot from the mirror?

Should work.

>   Would the box still boot with only one of the disks?

If you configure things correctly - better test it.

> * Can I use EVMS as a frontend?

Yes.

>   Does it even use md or is EVMS's RAID something else entirely?

EVMS uses a lot of underlying software, MD being one component.

> * Should I use the 300s as a single mirror, or span multiple ones over
> the two disks?

What would the purpose be?

> * Am I even correct in assuming that I could stick an array in another
> box and have it work?

Work for what?

> Comments welcome

Get gigabit nics, in case you want to fiddle with iSCSI? :-).

Re: Ok to go ahead with this setup?

[H. Peter Anvin]

Molle Bestefich wrote:
> Christian Pernegger wrote:
>> Intel SE7230NH1-E mainboard
>> Pentium D 930
> 
> HPA recently said that x86_64 CPUs have better RAID5 performance.

Actually, anything with SSE2 should be OK.

	-hpa

Re: Ok to go ahead with this setup?

[Christian Pernegger]

> > Pentium D 930
>
> HPA recently said that x86_64 CPUs have better RAID5 performance.

Good to know. I did intend to use Debian-amd64 anyway.

> Is it a NAS kind of device?

Yes, mostly. It also runs a caching NNTP proxy and drives our
networked audio players :)
Personal file server describes it best, I think.

> In that case, drop the 2x 300GB disks and get 6x 500GB instead.  You
> can partition those so that you have a RAID1 spanning the first 10GB
> of all 6 drives for use as the system partition, and use the rest in a
> RAID5.

Good idea, it's just that I already have the listed disks, and I need
at least 150 GB effective capacity on the mirror for important work
data, not just the OS. Anything specific wrong with the Maxtors?

> > * Should I use the 300s as a single mirror, or span multiple ones over
> > the two disks?
>
> What would the purpose be?

I read somewhere that this could reduce rebuild time when a "disk"
(partition in this case) is kicked offline because of a timeout or
somesuch. Sounds a bit fishy, which is why I'm asking.

> > * Am I even correct in assuming that I could stick an array in another
> > box and have it work?
>
> Work for what?

Well, access to the data. The point of the whole exercise is that I
don't want to be cut off from my data, just because a part of the host
(not the disks) died.

> Get gigabit nics, in case you want to fiddle with iSCSI? :-)

The board has one or two Intel Gb NICs, they usually work fine ...

And iSCSI sounds way too expensive. :)

Thanks,

C.

Re: Ok to go ahead with this setup?

[Molle Bestefich]

Christian Pernegger wrote:
> Anything specific wrong with the Maxtors?

No.  I've used Maxtor for a long time and I'm generally happy with them.

They break now and then, but their online warranty system is great.
I've also been treated kindly by their help desk - talked to a cute
gal from Maxtor in Ireland over the phone just yesterday ;-).

Then again, they've just been acquired by Seagate, or so, so things
may change for the worse, who knows.

I'd watch out regarding the Western Digital disks, apparently they
have a bad habit of turning themselves off when used in RAID mode, for
some reason:
http://thread.gmane.org/gmane.linux.kernel.device-mapper.devel/1980/

> I read somewhere that this could reduce rebuild time when a "disk"
> (partition in this case) is kicked offline because of a timeout or
> somesuch. Sounds a bit fishy, which is why I'm asking.

The bitmap code in MD is for fast rebuilding, if you need that.

> The point of the whole exercise is that I don't want to be cut off
> from my data, just because a part of the host (not the disks) died.

I don't see any large risks of that happening with the setup you've outlined.

> And iSCSI sounds way too expensive. :)

I think a host adapter is around $100.

Re: Ok to go ahead with this setup?

[H. Peter Anvin]

Molle Bestefich wrote:
> Christian Pernegger wrote:
>> Anything specific wrong with the Maxtors?
> 
> No.  I've used Maxtor for a long time and I'm generally happy with them.
> 
> They break now and then, but their online warranty system is great.
> I've also been treated kindly by their help desk - talked to a cute
> gal from Maxtor in Ireland over the phone just yesterday ;-).
> 
> Then again, they've just been acquired by Seagate, or so, so things
> may change for the worse, who knows.
> 
> I'd watch out regarding the Western Digital disks, apparently they
> have a bad habit of turning themselves off when used in RAID mode, for
> some reason:
> http://thread.gmane.org/gmane.linux.kernel.device-mapper.devel/1980/
> 

I have exactly the opposite experience.  More than 50% of Maxtor drives 
fail inside 18 months; WDs seem to be really solid.

	-hpa

Re: Ok to go ahead with this setup?

[Bill Davidsen]

Christian Pernegger wrote:

> Hi list!
>
> Having experienced firsthand the pain that hardware RAID controllers
> can be -- my 3ware 7500-8 died and it took me a week to find even a
> 7508-8 -- I would like to switch to kernel software RAID.
>
> Here's a tentative setup:
>
> Intel SE7230NH1-E mainboard
> Pentium D 930
> 2x1GB Crucial 533 DDR2 ECC
> Intel SC5295-E enclosure
>
> Promise Ultra133 TX2 (2ch PATA)
>   - 2x Maxtor 6B300R0 (300GB, DiamondMax 10) in RAID1
>
> Onboard Intel ICH7R (4ch SATA)
>   - 4x Western Digital WD5000YS (500GB, Caviar RE2) in RAID5
>
> * Does this hardware work flawlessly with Linux?
>
> * Is it advisable to boot from the mirror?
>  Would the box still boot with only one of the disks?


Let me say this about firmware mirror: while virtually every BIOS will 
boot the "next" disk if the first fails, some will not fail over if the 
first drive is returning a parity but still returning data. Take that 
data any way you want, drive failure at power cycle is somewhat more 
likely than failure while running.

>
> * Can I use EVMS as a frontend?
>  Does it even use md or is EVMS's RAID something else entirely?
>
> * Should I use the 300s as a single mirror, or span multiple ones over
> the two disks?
>
> * Am I even correct in assuming that I could stick an array in another
> box and have it work?
>
> Comments welcome
>
> Thanks,
>
> Chris
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>


-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: Ok to go ahead with this setup?

[Bill Davidsen]

Molle Bestefich wrote:

> Christian Pernegger wrote:
>
>> Anything specific wrong with the Maxtors?
>
>
> No.  I've used Maxtor for a long time and I'm generally happy with them.
>
> They break now and then, but their online warranty system is great.
> I've also been treated kindly by their help desk - talked to a cute
> gal from Maxtor in Ireland over the phone just yesterday ;-).
>
> Then again, they've just been acquired by Seagate, or so, so things
> may change for the worse, who knows.
>
> I'd watch out regarding the Western Digital disks, apparently they
> have a bad habit of turning themselves off when used in RAID mode, for
> some reason:
> http://thread.gmane.org/gmane.linux.kernel.device-mapper.devel/1980/ 


Based on three trials in five years, I'm happy with WD and Seagate. WD 
didn't ask when I bought it, just the serial for manufacturing date.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: Ok to go ahead with this setup?

[Ricky Beam]

On Fri, 23 Jun 2006, Molle Bestefich wrote:
>I'd watch out regarding the Western Digital disks, apparently they
>have a bad habit of turning themselves off when used in RAID mode, for
>some reason:
>http://thread.gmane.org/gmane.linux.kernel.device-mapper.devel/1980/

Where "for some reason" == HEAT.  I've seen Maxtor, Seagate, AND Western
Digital drives all shutdown when they get too hot -- so hot you cannot
touch them.  I know this all too well because Dell is stupid or lazy
to design their cases with proper ventilation over the drives; one drive
simply gets hot, two drives get hot enough to discolor their plastic
drive sleds.

Unless you're talking about little laptop drives, hard drives need active
cooling.  A few CFM is usually enough.  A LOT of people underestimate
the cooling needs of their drives. (and sadly that includes far too many
manufacturers of IDE/SATA drive cages.)

--Ricky

Re: Ok to go ahead with this setup?

[Mike Dresser]

On Fri, 23 Jun 2006, Molle Bestefich wrote:

> Christian Pernegger wrote:
>> Anything specific wrong with the Maxtors?
>
> I'd watch out regarding the Western Digital disks, apparently they
> have a bad habit of turning themselves off when used in RAID mode, for
> some reason:
> http://thread.gmane.org/gmane.linux.kernel.device-mapper.devel/1980/

The MaxLine III's (7V300F0) with VA111630/670 firmware currently timeout 
on a weekly or less basis.. I'm still testing VA111680 on a 15x300 gig 
array

Mike

Re: Ok to go ahead with this setup?

[bart]

Mike Dresser wrote:
> 
> On Fri, 23 Jun 2006, Molle Bestefich wrote:
> 
> > Christian Pernegger wrote:
> >> Anything specific wrong with the Maxtors?
> >
> > I'd watch out regarding the Western Digital disks, apparently they
> > have a bad habit of turning themselves off when used in RAID mode, for
> > some reason:
> > http://thread.gmane.org/gmane.linux.kernel.device-mapper.devel/1980/
> 
> The MaxLine III's (7V300F0) with VA111630/670 firmware currently timeout
> on a weekly or less basis.. I'm still testing VA111680 on a 15x300 gig
> array
> 
We also see similar problem on Maxtor 6V250F0 drives: they 'crash' randomly at
a weeks timescale. Only way to get them back is by power cycling. Tried both
SuperMicro SATA card (Marvell chip) and Promise Fastrak, firmware updates from
Maxtor did not fix it yet. We were already forced to exchange all drives at
a customer because he does not want to use Maxtor's anymore. Neither do we :(

	Bart

Re: Ok to go ahead with this setup?

[Brad Campbell]

bart@ardistech.com wrote:
> Mike Dresser wrote:
>> On Fri, 23 Jun 2006, Molle Bestefich wrote:
>>
>>> Christian Pernegger wrote:
>>>> Anything specific wrong with the Maxtors?
>>> I'd watch out regarding the Western Digital disks, apparently they
>>> have a bad habit of turning themselves off when used in RAID mode, for
>>> some reason:
>>> http://thread.gmane.org/gmane.linux.kernel.device-mapper.devel/1980/
>> The MaxLine III's (7V300F0) with VA111630/670 firmware currently timeout
>> on a weekly or less basis.. I'm still testing VA111680 on a 15x300 gig
>> array
>>
> We also see similar problem on Maxtor 6V250F0 drives: they 'crash' randomly at
> a weeks timescale. Only way to get them back is by power cycling. Tried both
> SuperMicro SATA card (Marvell chip) and Promise Fastrak, firmware updates from
> Maxtor did not fix it yet. We were already forced to exchange all drives at
> a customer because he does not want to use Maxtor's anymore. Neither do we :(


Whereas I have 28 7Y250M0 drives sitting in a couple of arrays here that have behaved perfectly 
(aside from some grown defects) for over 18000 hours so far. They are *all* sitting on Promise 
SATA150TX4 cards on 2.6 kernels.

I'm looking at another server and another 15 drives at the moment, and it's Maxtors I'm looking at.

Everyone has different experience. I would not touch Seagate with a 10 foot pole (blew up way too 
many logic boards when I was using them), and I got bitten *badly* by the WD firmware issue with 
RAID (firmware upgrade fixed that, but can't replace the data I lost when 3 of them failed at the 
same time and the array got corrupted).

Having said that, it was MaxLineIII 300G drives I was looking at, so perhaps I'll wait a little 
longer and hear some more stories before I drop $$ on 15 of them.

Brad
-- 
"Human beings, who are almost unique in having the ability
to learn from the experience of others, are also remarkable
for their apparent disinclination to do so." -- Douglas Adams

Re: Ok to go ahead with this setup?

[Christian Pernegger]

> The MaxLine III's (7V300F0) with VA111630/670 firmware currently timeout
> on a weekly or less basis..

I have just one 7V300F0, so no idea how it behaves is a RAID. It's
been fine apart from the fact that my VIA southbridge SATA controllers
doesn't even detect it ... :(
(Anyone else notice compatibility problems are through the roof lately?)

The 8x 6B300R0 (PATA) have been excellent on my PATA 3ware.
Untypically for Maxtor, not one has died yet (over a year) :)

The 8x 6Y120L0 (PATA) died at a rate of about two a year.

I mainly use Maxtor due to the fact that the RMA process is automated,
hassle-free and fast. They will exchange a drive when the first bad
sector errors start to show up and not insist on a low level format to
"fix" the problem.
The Atlas SCSI line they inherited from Quantum isn't half-bad either.

What happens now that Seagate has bought them nobody knows.

C.

Re: Ok to go ahead with this setup?

[bart]

Christian Pernegger wrote:
> 
> > The MaxLine III's (7V300F0) with VA111630/670 firmware currently timeout
> > on a weekly or less basis..
> 
> I have just one 7V300F0, so no idea how it behaves is a RAID. It's
> been fine apart from the fact that my VIA southbridge SATA controllers
> doesn't even detect it ... :(
>
You'll need a drive firmware update for this..

> (Anyone else notice compatibility problems are through the roof lately?)
> 
Yes, the manufacturers are busy with SATAII while SATAI still not being stable....

> The 8x 6B300R0 (PATA) have been excellent on my PATA 3ware.
> Untypically for Maxtor, not one has died yet (over a year) :)
> 
> The 8x 6Y120L0 (PATA) died at a rate of about two a year.
> 
> I mainly use Maxtor due to the fact that the RMA process is automated,
> hassle-free and fast. They will exchange a drive when the first bad
> sector errors start to show up and not insist on a low level format to
> "fix" the problem.
> 
They did not offer to exchage our +30 drives that are having timeouts....

Drive issues in RAID vs. not-RAID ..

[Gordon Henderson]

I've seen a few comments to the effect that some disks have problems when
used in a RAID setup and I'm a bit preplexed as to why this might be..

What's the difference between a drive in a RAID set (either s/w or h/w)
and a drive on it's own, assuming the load, etc. is roughly the same in
each setup?

Is it just "bad feeling" or is there any scientific reasons for it?

Cheers,

Gordon

Re: Ok to go ahead with this setup?

[Justin Piszcz]

On Wed, 28 Jun 2006, bart@ardistech.com wrote:

> Mike Dresser wrote:
>>
>> On Fri, 23 Jun 2006, Molle Bestefich wrote:
>>
>>> Christian Pernegger wrote:
>>>> Anything specific wrong with the Maxtors?
>>>
>>> I'd watch out regarding the Western Digital disks, apparently they
>>> have a bad habit of turning themselves off when used in RAID mode, for
>>> some reason:
>>> http://thread.gmane.org/gmane.linux.kernel.device-mapper.devel/1980/
>>
>> The MaxLine III's (7V300F0) with VA111630/670 firmware currently timeout
>> on a weekly or less basis.. I'm still testing VA111680 on a 15x300 gig
>> array
>>
> We also see similar problem on Maxtor 6V250F0 drives: they 'crash' randomly at
> a weeks timescale. Only way to get them back is by power cycling. Tried both
> SuperMicro SATA card (Marvell chip) and Promise Fastrak, firmware updates from
> Maxtor did not fix it yet. We were already forced to exchange all drives at
> a customer because he does not want to use Maxtor's anymore. Neither do we :(
>
> 	Bart
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

> The MaxLine III's (7V300F0) with VA111630/670 firmware currently timeout
> on a weekly or less basis.. I'm still testing VA111680 on a 15x300 gig
> array

How do you have the 15 drives attached? Did you buy a SATA raid card?  Do 
you have multiple (cheap JBOD SATA cards)?  If so, which did you use? I 
cannot seem to find any PCI-e cards with >= 4-8 slots that support JBOD 
under $700-$900.

Re: Ok to go ahead with this setup?

[Christian Pernegger]

> Here's a tentative setup:
>
> Intel SE7230NH1-E mainboard
> Pentium D 930
> 2x1GB Crucial 533 DDR2 ECC
> Intel SC5295-E enclosure

The above components have finally arrived ... and I was shocked to see
that the case's drive bays do not have their own fan, nor can I think
of anywhere to put one. Apparently Intel thinks the case airflow is
enough to cool 6 rather densly packed drives.

I could get one of their hotswap backplanes (which include a fan) but
I don't need hotswap, really.

Does anyone have enough experience with this or similar Intel cases to comment?

Sorry for being OT,

C.

Re: Ok to go ahead with this setup?

[Mike Dresser]

On Wed, 28 Jun 2006, Justin Piszcz wrote:

> How do you have the 15 drives attached? Did you buy a SATA raid card?  Do you 
> have multiple (cheap JBOD SATA cards)?  If so, which did you use? I cannot 
> seem to find any PCI-e cards with >= 4-8 slots that support JBOD under 
> $700-$900.

We're using a 3ware 9550SX-16ML, which is a 133mhz pci-x card.  They also 
have a 9590SE that does the same with PCI-E, though I don't know if the 
stock 2.6.x kernel supports these yet.

Mike

Re: Drive issues in RAID vs. not-RAID ..

[Neil Brown]

On Wednesday June 28, gordon@drogon.net wrote:
> 
> I've seen a few comments to the effect that some disks have problems when
> used in a RAID setup and I'm a bit preplexed as to why this might be..
> 
> What's the difference between a drive in a RAID set (either s/w or h/w)
> and a drive on it's own, assuming the load, etc. is roughly the same in
> each setup?
> 
> Is it just "bad feeling" or is there any scientific reasons for it?

I don't think that 'disks' have problems being in a raid, but I
believe some controllers do (though I don't know whether it is the
controller or the driver that is at fault).  RAID make concurrent
requests much more likely and so is likely to push hard at any locking
issues.

NeilBrown