OT: sas controllers/expanders and "desktop" drives?

OT: sas controllers/expanders and "desktop" drives?

[Matt Garman]

Hello,

For building a Linux md system with many disks, I've been looking at
the LSI SAS 3081E-R controller card[1] with the SuperMicro
846TQ-R900B chassis[2].  The SAS3018E card looks like it has a
mature open-source Linux driver.  The chassis has a SAS expander;
I'm not sure which one (but I have a question in to SuperMicro).

Anyway, my question is this: has anyone used this setup, or one
like it (sas controller + sas expander) with "desktop" class drives?
The web research I've done suggests that most SAS controllers and
expanders are targeted at the enterprise market, and therefore,
aren't guaranteed to work with the (significantly cheaper) desktop
drives.

For example, from the FAQ for that SuperMicro chassis[3]:

    Question: We want to use the new Seagate 1.5TB ST31500341AS
    drives in combination with a storage solution based on your
    846TQ chassis, is this compatible?

    Answer: This ST31500341AS hard drive is designed to be a Desktop
    type hard drive, not an Enterprise hard drive which is needed in
    this setup.  We don't recommend this combination, so please use
    the Seagate 1TB ST31000340NS drive instead. This drive is an
    Enterprise drive and validated for our 846TQ chassis.

In short, we want to build a server with 24 drives.  The desktop
drives are significantly cheaper than the enterprise drives.  But
getting 24 ports is cheapest with these SAS controllers + expanders.

Thanks,
Matt


[1] http://www.lsi.com/storage_home/products_home/host_bus_adapters/sas_hbas/lsisas3081er/index.html

[2] http://www.supermicro.com/products/chassis/4U/846/SC846TQ-R900.cfm

[3] http://www.supermicro.com/support/faqs/faq.cfm?faq=8230

Re: OT: sas controllers/expanders and "desktop" drives?

[Bill Davidsen]

Matt Garman wrote:
> Hello,
>
> For building a Linux md system with many disks, I've been looking at
> the LSI SAS 3081E-R controller card[1] with the SuperMicro
> 846TQ-R900B chassis[2].  The SAS3018E card looks like it has a
> mature open-source Linux driver.  The chassis has a SAS expander;
> I'm not sure which one (but I have a question in to SuperMicro).
>
> Anyway, my question is this: has anyone used this setup, or one
> like it (sas controller + sas expander) with "desktop" class drives?
> The web research I've done suggests that most SAS controllers and
> expanders are targeted at the enterprise market, and therefore,
> aren't guaranteed to work with the (significantly cheaper) desktop
> drives.
>
> For example, from the FAQ for that SuperMicro chassis[3]:
>
>     Question: We want to use the new Seagate 1.5TB ST31500341AS
>     drives in combination with a storage solution based on your
>     846TQ chassis, is this compatible?
>
>     Answer: This ST31500341AS hard drive is designed to be a Desktop
>     type hard drive, not an Enterprise hard drive which is needed in
>     this setup.  We don't recommend this combination, so please use
>     the Seagate 1TB ST31000340NS drive instead. This drive is an
>     Enterprise drive and validated for our 846TQ chassis.
>
> In short, we want to build a server with 24 drives.  The desktop
> drives are significantly cheaper than the enterprise drives.  But
> getting 24 ports is cheapest with these SAS controllers + expanders.
>
>
>   
I have the feeling that what they meant is that the drives have firmware 
intended for desktop use and will not work well in RAID usage. This has 
been discussed here before, the desktop drives have firmware which tries 
repeatedly to recover data on a failed read, and which therefore may 
hang for 20-30 sec before reporting an error or finally returning the 
data without an error indication.

Until/unless Neil changes his idea that timeout should be in the driver 
or the driver, these drives work poorly with RAID.

Note, this is just the way I read the reply, and there may be a way to 
tell the firmware to fast fail, depending on tools and firmware.

> [1] http://www.lsi.com/storage_home/products_home/host_bus_adapters/sas_hbas/lsisas3081er/index.html
>
> [2] http://www.supermicro.com/products/chassis/4U/846/SC846TQ-R900.cfm
>
> [3] http://www.supermicro.com/support/faqs/faq.cfm?faq=8230
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>   


-- 
Bill Davidsen <davidsen@tmr.com>
  "We can't solve today's problems by using the same thinking we
   used in creating them." - Einstein

Re: OT: sas controllers/expanders and "desktop" drives?

[Matt Garman]

On Sat, Mar 20, 2010 at 11:34:19AM -0400, Bill Davidsen wrote:
> Matt Garman wrote:
> >For example, from the FAQ for that SuperMicro chassis[3]:
> >
> >    Question: We want to use the new Seagate 1.5TB ST31500341AS
> >    drives in combination with a storage solution based on your
> >    846TQ chassis, is this compatible?
> >
> >    Answer: This ST31500341AS hard drive is designed to be a
> >    Desktop type hard drive, not an Enterprise hard drive which
> >    is needed in this setup.  We don't recommend this
> >    combination, so please use the Seagate 1TB ST31000340NS drive
> >    instead. This drive is an Enterprise drive and validated for
> >    our 846TQ chassis.
> >
> >In short, we want to build a server with 24 drives.  The desktop
> >drives are significantly cheaper than the enterprise drives.  But
> >getting 24 ports is cheapest with these SAS controllers +
> >expanders.
>  
> I have the feeling that what they meant is that the drives have
> firmware intended for desktop use and will not work well in RAID
> usage. This has been discussed here before, the desktop drives
> have firmware which tries repeatedly to recover data on a failed
> read, and which therefore may hang for 20-30 sec before reporting
> an error or finally returning the data without an error
> indication.

That makes sense.  For what it's worth, I found out that the
expander chip on the SuperMicro 846 case is the LSI SASx36[1].
(Note that there are several versions of this case; the E1 and E2
are the ones that have the expander.  The TQ version doesn't have
the expander chip.)

> Until/unless Neil changes his idea that timeout should be in the
> driver or the driver, these drives work poorly with RAID.

For this particular application, my requirements are a little
different---I actually just want a bunch of individual drives; I
won't be using RAID.  (But there seems to be a lot of folks on this
list with a wide range of hardware experience, and I was hoping to
catch someone with experience with this particular configuration.)

Anyway, we went ahead and ordered the hardware---LSI SAS3081E-R
card, case with the LSI SASX36 expander, and both WD Green and
Samsung EcoGreen drives (both desktop class).  I'll be able to
follow up shortly and report how well this configuration works (or
doesn't) for standalone usage.

As for RAID usage (md or hardware): just to be clear, my
understanding is that the biggest problem is that, even if the
desktop drives work fine for non-RAID use, they'll still be
"difficult" in an array.  That is, they take too long to report
errors, and thus get kicked out of the array.

From my perspective, this isn't a true hardware incompatibility;
it's a software (or firmware) "mismatch".  If this is correct, then
the "real" answer to the FAQ I copied above is more nuanced.  The
current answer should be qualified with "for use in a RAID", and
then go on to suggest that the drives should be ok for standalone
use.

-Matt

[1] http://www.lsi.com/storage_home/products_home/standard_product_ics/sas_expander_ics/lsisasx36/index.html

Re: OT: sas controllers/expanders and "desktop" drives?

[Bill Davidsen]

Matt Garman wrote:
> On Sat, Mar 20, 2010 at 11:34:19AM -0400, Bill Davidsen wrote:
>   
>> Matt Garman wrote:
>>     
>>> For example, from the FAQ for that SuperMicro chassis[3]:
>>>
>>>    Question: We want to use the new Seagate 1.5TB ST31500341AS
>>>    drives in combination with a storage solution based on your
>>>    846TQ chassis, is this compatible?
>>>
>>>    Answer: This ST31500341AS hard drive is designed to be a
>>>    Desktop type hard drive, not an Enterprise hard drive which
>>>    is needed in this setup.  We don't recommend this
>>>    combination, so please use the Seagate 1TB ST31000340NS drive
>>>    instead. This drive is an Enterprise drive and validated for
>>>    our 846TQ chassis.
>>>
>>> In short, we want to build a server with 24 drives.  The desktop
>>> drives are significantly cheaper than the enterprise drives.  But
>>> getting 24 ports is cheapest with these SAS controllers +
>>> expanders.
>>>       
>>  
>> I have the feeling that what they meant is that the drives have
>> firmware intended for desktop use and will not work well in RAID
>> usage. This has been discussed here before, the desktop drives
>> have firmware which tries repeatedly to recover data on a failed
>> read, and which therefore may hang for 20-30 sec before reporting
>> an error or finally returning the data without an error
>> indication.
>>     
>
> That makes sense.  For what it's worth, I found out that the
> expander chip on the SuperMicro 846 case is the LSI SASx36[1].
> (Note that there are several versions of this case; the E1 and E2
> are the ones that have the expander.  The TQ version doesn't have
> the expander chip.)
>
>   
>> Until/unless Neil changes his idea that timeout should be in the
>> driver or the driver, these drives work poorly with RAID.
>>     
>
> For this particular application, my requirements are a little
> different---I actually just want a bunch of individual drives; I
> won't be using RAID.  (But there seems to be a lot of folks on this
> list with a wide range of hardware experience, and I was hoping to
> catch someone with experience with this particular configuration.)
>
> Anyway, we went ahead and ordered the hardware---LSI SAS3081E-R
> card, case with the LSI SASX36 expander, and both WD Green and
> Samsung EcoGreen drives (both desktop class).  I'll be able to
> follow up shortly and report how well this configuration works (or
> doesn't) for standalone usage.
>
> As for RAID usage (md or hardware): just to be clear, my
> understanding is that the biggest problem is that, even if the
> desktop drives work fine for non-RAID use, they'll still be
> "difficult" in an array.  That is, they take too long to report
> errors, and thus get kicked out of the array.
>
> >From my perspective, this isn't a true hardware incompatibility;
> it's a software (or firmware) "mismatch".  If this is correct, then
> the "real" answer to the FAQ I copied above is more nuanced.  The
> current answer should be qualified with "for use in a RAID", and
> then go on to suggest that the drives should be ok for standalone
> use.
>   

See the information provided by Randy Terbush on this, you may be able 
to just change the ERT if you have the right drives.

-- 
Bill Davidsen <davidsen@tmr.com>
  "We can't solve today's problems by using the same thinking we
   used in creating them." - Einstein

Re: OT: sas controllers/expanders and "desktop" drives?

[Richard Scobie]

Matt Garman wrote:

> As for RAID usage (md or hardware): just to be clear, my
> understanding is that the biggest problem is that, even if the
> desktop drives work fine for non-RAID use, they'll still be
> "difficult" in an array.  That is, they take too long to report
> errors, and thus get kicked out of the array.

Another important, (from a performance perspective), feature of the 
enterprise drives which will be factor in your setup, is that they are 
optimised for use in a high vibration environment (RAFF in Western 
Digital parlance), unlike desktop drives.

The following illustrates the problem in a somewhat humourous way:

http://www.youtube.com/watch?v=tDacjrSCeq4

While the desktop drives will work, throughput and seek times may be 
compromised due to vibration and resonance effects from many drives in 
one enclosure.

Regards,

Richard

Re: OT: sas controllers/expanders and "desktop" drives?

[Simon Matthews]

A couple of years ago, I was managing 2 systems. Both were built with
Supermicro motherboards and chassis and high-end 3ware controllers
(9550sx if I recall correctly). One system was built with
enterprise-class SATA  drives, while the other was built with
desktop-class SATA drives. They were both configured for RAID5.

The system with desktop-class drives occasionally kicked a drive out
of the array. The controller would report that the drive had failed
and then that it was back again a few seconds later, but this was
enough for the controller to kick it out of the array and require a
rebuild.

Caution: one anecdote is not hard data!

Simon

On Wed, Mar 17, 2010 at 7:36 AM, Matt Garman <matthew.garman@gmail.com> wrote:
>
> Hello,
>
> For building a Linux md system with many disks, I've been looking at
> the LSI SAS 3081E-R controller card[1] with the SuperMicro
> 846TQ-R900B chassis[2].  The SAS3018E card looks like it has a
> mature open-source Linux driver.  The chassis has a SAS expander;
> I'm not sure which one (but I have a question in to SuperMicro).
>
> Anyway, my question is this: has anyone used this setup, or one
> like it (sas controller + sas expander) with "desktop" class drives?
> The web research I've done suggests that most SAS controllers and
> expanders are targeted at the enterprise market, and therefore,
> aren't guaranteed to work with the (significantly cheaper) desktop
> drives.
>
> For example, from the FAQ for that SuperMicro chassis[3]:
>
>    Question: We want to use the new Seagate 1.5TB ST31500341AS
>    drives in combination with a storage solution based on your
>    846TQ chassis, is this compatible?
>
>    Answer: This ST31500341AS hard drive is designed to be a Desktop
>    type hard drive, not an Enterprise hard drive which is needed in
>    this setup.  We don't recommend this combination, so please use
>    the Seagate 1TB ST31000340NS drive instead. This drive is an
>    Enterprise drive and validated for our 846TQ chassis.
>
> In short, we want to build a server with 24 drives.  The desktop
> drives are significantly cheaper than the enterprise drives.  But
> getting 24 ports is cheapest with these SAS controllers + expanders.
>
> Thanks,
> Matt
>
>
> [1] http://www.lsi.com/storage_home/products_home/host_bus_adapters/sas_hbas/lsisas3081er/index.html
>
> [2] http://www.supermicro.com/products/chassis/4U/846/SC846TQ-R900.cfm
>
> [3] http://www.supermicro.com/support/faqs/faq.cfm?faq=8230
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: OT: sas controllers/expanders and "desktop" drives?

[Thomas Fjellstrom]

On March 17, 2010, Matt Garman wrote:
> Hello,
> 
> For building a Linux md system with many disks, I've been looking at
> the LSI SAS 3081E-R controller card[1] with the SuperMicro
> 846TQ-R900B chassis[2].  The SAS3018E card looks like it has a
> mature open-source Linux driver.  The chassis has a SAS expander;
> I'm not sure which one (but I have a question in to SuperMicro).
> 
> Anyway, my question is this: has anyone used this setup, or one
> like it (sas controller + sas expander) with "desktop" class drives?
> The web research I've done suggests that most SAS controllers and
> expanders are targeted at the enterprise market, and therefore,
> aren't guaranteed to work with the (significantly cheaper) desktop
> drives.
> 
> For example, from the FAQ for that SuperMicro chassis[3]:
> 
>     Question: We want to use the new Seagate 1.5TB ST31500341AS
>     drives in combination with a storage solution based on your
>     846TQ chassis, is this compatible?
> 
>     Answer: This ST31500341AS hard drive is designed to be a Desktop
>     type hard drive, not an Enterprise hard drive which is needed in
>     this setup.  We don't recommend this combination, so please use
>     the Seagate 1TB ST31000340NS drive instead. This drive is an
>     Enterprise drive and validated for our 846TQ chassis.
> 
> In short, we want to build a server with 24 drives.  The desktop
> drives are significantly cheaper than the enterprise drives.  But
> getting 24 ports is cheapest with these SAS controllers + expanders.

If it helps at all, I'm using an 8 port SAS card to run my 5 disk md RAID 5 
array with plain old 1TB Seagate Baracuda's. Works fine. Its been running 
for a good 6 months now. I can't say how well it'll work in the future, but 
it works well enough for now.

> Thanks,
> Matt
> 
> 
> [1]
> http://www.lsi.com/storage_home/products_home/host_bus_adapters/sas_hbas
> /lsisas3081er/index.html
> 
> [2] http://www.supermicro.com/products/chassis/4U/846/SC846TQ-R900.cfm
> 
> [3] http://www.supermicro.com/support/faqs/faq.cfm?faq=8230
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


-- 
Thomas Fjellstrom
tfjellstrom@shaw.ca