Offtopic: hardware advice for SAS RAID6

Offtopic: hardware advice for SAS RAID6

[Richard Michael]

On the heels of last week's post asking about hardware recommendations,
I'd like to ask a few questions too. :)

I'm considering my first SAS purchase.  I'm planning to build a software
RAID6 array using a SAS JBOD attached to a linux box.  I haven't decided
on any of the hardware specifics.

I'm leaning toward this PCI express LSI 3801e controller:
http://www.lsi.com/storage_home/products_home/host_bus_adapters/sas_hbas/lsisas3801e/index.html

Although, Adaptec has a similar PCI-X model.

I'd probably purchase a cheap Dell rackmount 1U server (e.g. PowerEdge
860) for the controller.  It has dual Gb ethernet, which I'd channel
bond for decent network I/O performance.

The JBOD would ideally be 1U or 2U holding 8 or 10 disks.  If I
understand SAS correctly, I'd probably have a unit with 2 SFF-8088
miniSAS connectors (although, I believe these connectors only support 4
devices, so if the JBOD is >8 disks, I don't know what would happen).

I'm completely undecided on the JBOD itself; recommendations here would
be greatly appreciated.  It's a bit of a shot in the dark.

I'd appreciate feedback and suggestions on the hardware above, or a
discussion of the performance.  (E.g. a discussion of SAS
ports/bandwith, PCI express lanes/bandwidth, disks and network to
determine the through put of this setup.)

Cheers,
Richard

RE: Offtopic: hardware advice for SAS RAID6

[Daniel Korstad]

I would be very interested to hear how that card works or other suggestions discussions too.
 
I have a 10 disk RAID 6 all inside a large case, using on board MB SATA and a couple 4 port PCI SATA cards.  For one, my PCI bus is saturated and a bottle neck and it is a bit of a pain to replace drives by having to open the case each time.
 
I have been running this for years without issues, but would like to upgrade the system at some point and would like to use mdadm software raid, with an external enclosure allowing easy drive swapping.
 
The most I can contribute to this thread are SATA Multilane Enclosures I have been eyeing.
http://www.pc-pitstop.com/sata_enclosures/
 
But like you, I need a card that is Linux friendly and provides hard drive raid control to mdadm.
 
I'll follow this one and would like to hear about your experiences with the SAS card you choose.

----- Original Message -----
From: Richard Michael <rmichael-raid@edgeofthenet.org>
Sent: Tue, 11/20/2007 10:08am
To: linux-raid@vger.kernel.org
Subject: Offtopic: hardware advice for SAS RAID6

On the heels of last week's post asking about hardware recommendations,
I'd like to ask a few questions too. :)

I'm considering my first SAS purchase.  I'm planning to build a software
RAID6 array using a SAS JBOD attached to a linux box.  I haven't decided
on any of the hardware specifics.

I'm leaning toward this PCI express LSI 3801e controller:
http://www.lsi.com/storage_home/products_home/host_bus_adapters/sas_hbas/lsisas3801e/index.html

Although, Adaptec has a similar PCI-X model.

I'd probably purchase a cheap Dell rackmount 1U server (e.g. PowerEdge
860) for the controller.  It has dual Gb ethernet, which I'd channel
bond for decent network I/O performance.

The JBOD would ideally be 1U or 2U holding 8 or 10 disks.  If I
understand SAS correctly, I'd probably have a unit with 2 SFF-8088
miniSAS connectors (although, I believe these connectors only support 4
devices, so if the JBOD is >8 disks, I don't know what would happen).

I'm completely undecided on the JBOD itself; recommendations here would
be greatly appreciated.  It's a bit of a shot in the dark.

I'd appreciate feedback and suggestions on the hardware above, or a
discussion of the performance.  (E.g. a discussion of SAS
ports/bandwith, PCI express lanes/bandwidth, disks and network to
determine the through put of this setup.)

Cheers,
Richard
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Offtopic: hardware advice for SAS RAID6

[Sven Rudolph]

Richard Michael <rmichael-raid@edgeofthenet.org> writes:

> I'm considering my first SAS purchase.  I'm planning to build a software
> RAID6 array using a SAS JBOD attached to a linux box.  I haven't decided
> on any of the hardware specifics.
>
> I'm leaning toward this PCI express LSI 3801e controller:
> http://www.lsi.com/storage_home/products_home/host_bus_adapters/sas_hbas/lsisas3801e/index.html

I use a very similiar controllers, the LSI 3442X-R and 3442E-R. They
have only one external connector, and this is a different connector
type. It looks a bit more stable (mechanically), but this might be a
matter of taste and I have never used the other connector type.  (I
had to update the controllers using the latest firmware available on
the LSI web site.)

> The JBOD would ideally be 1U or 2U holding 8 or 10 disks.  If I
> understand SAS correctly, I'd probably have a unit with 2 SFF-8088
> miniSAS connectors (although, I believe these connectors only support 4
> devices, so if the JBOD is >8 disks, I don't know what would happen).

The JBODs that I am aware of use internal SAS port expanders (see
<http://en.wikipedia.org/wiki/Serial_Attached_SCSI#SAS_Expanders>). The
connection to the host has four SAS lanes (the concept is similiar to
the up to 16 lanes on a PCI-Express card). Each disk is connected via
one SAS lane, and all these lanes are switched by the port
expander. So you do not need 8 lanes on the host computer in order to
connect 8 disks. 

I use DELLs MD1000 JBOD (some with SATA, some with SAS), and it works
fine. I have up to three JBODs connected to one SAS host port.

> I'd appreciate feedback and suggestions on the hardware above, or a
> discussion of the performance.  (E.g. a discussion of SAS
> ports/bandwith, PCI express lanes/bandwidth, disks and network to
> determine the through put of this setup.)

Currently I don't see limits that must be considered. Simple math with
bandwidths. Nowadays I prefer PCI-Express over PCI-X. AFAIR the LSI
cards have 8 PCI-Express lanes, so you need a matching port on the
mainboard.

	Sven