From: Stan Hoeppner <stan@hardwarefreak.com>
To: Brian Candler <B.Candler@pobox.com>
Cc: Christoph Hellwig <hch@infradead.org>, xfs@oss.sgi.com
Subject: Re: Performance problem - reads slower than writes
Date: Fri, 03 Feb 2012 13:42:54 -0600 [thread overview]
Message-ID: <4F2C38BE.2010002@hardwarefreak.com> (raw)
In-Reply-To: <20120203115434.GA649@nsrc.org>
On 2/3/2012 5:54 AM, Brian Candler wrote:
> and the drives are Hitachi Deskstar 5K3000 HDS5C3030ALA630:
3TB, 32MB cache, 5940 RPM
> The files have an average size of 0.65MB
You stated you're writing 100,000 of these files across 1,000
directories, with Bonnie, then reading them back with dd in your custom
script. You state this is similar to your production workload.
You've hit the peak read rate of these Hitachi drives. As others
pointed out, if you need more read performance than the dozen of these
you plan to RAID stripe, then you'll need to swap them for units with a
faster spindle:
7.2k 1.21x
10k 1.68x
15k 2.53x
or with SSDs, which will yield an order of magnitude increase. Your
stated need is 20M 500-800KB files, or 20GB if my math is correct. Four
of these enterprise Intel SLC SSDs in a layered mdRAID0 over mdRAID1
will give you ~375 file reads/sec at 800KB per file, again if my math is
correct:
http://www.newegg.com/Product/Product.aspx?Item=N82E16820167062
for $480 USD total investment. You might get by with a mirrored pair
depending on your actual space needs, but performance would be half.
You're probably wondering why I didn't recommend an mdRAID10 instead, or
a 3 SSD RAID5. All of the mdRAID striped RAID codes serialize on a
single master thread, except for RAID0 and the linear concatenation
(--linear). With storage devices capable of 35K IOPS each, that single
thread, even running on a 3+GHz core, has trouble keeping up.
The LSI 9201-16 you have is based on the SAS2116 chip, which isn't fast
enough in RAID10 mode to keep up with the SSDs. In straight HBA mode it
is.
--
Stan
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
next prev parent reply other threads:[~2012-02-03 19:42 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-01-30 22:00 Performance problem - reads slower than writes Brian Candler
2012-01-31 2:05 ` Dave Chinner
2012-01-31 10:31 ` Brian Candler
2012-01-31 14:16 ` Brian Candler
2012-01-31 20:25 ` Dave Chinner
2012-02-01 7:29 ` Stan Hoeppner
2012-02-03 18:47 ` Brian Candler
2012-02-03 19:03 ` Christoph Hellwig
2012-02-03 21:01 ` Brian Candler
2012-02-03 21:17 ` Brian Candler
2012-02-05 22:50 ` Dave Chinner
2012-02-05 22:43 ` Dave Chinner
2012-01-31 14:52 ` Christoph Hellwig
2012-01-31 21:52 ` Brian Candler
2012-02-01 0:50 ` Raghavendra D Prabhu
2012-02-01 3:59 ` Dave Chinner
2012-02-03 11:54 ` Brian Candler
2012-02-03 19:42 ` Stan Hoeppner [this message]
2012-02-03 22:10 ` Brian Candler
2012-02-04 9:59 ` Stan Hoeppner
2012-02-04 11:24 ` Brian Candler
2012-02-04 12:49 ` Stan Hoeppner
2012-02-04 20:04 ` Brian Candler
2012-02-04 20:44 ` Joe Landman
2012-02-06 10:40 ` Brian Candler
2012-02-07 17:30 ` Brian Candler
2012-02-05 5:16 ` Stan Hoeppner
2012-02-05 9:05 ` Brian Candler
2012-01-31 20:06 ` Dave Chinner
2012-01-31 21:35 ` Brian Candler
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4F2C38BE.2010002@hardwarefreak.com \
--to=stan@hardwarefreak.com \
--cc=B.Candler@pobox.com \
--cc=hch@infradead.org \
--cc=xfs@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox