Re: XFS: Abysmal write performance because of excessive seeking (allocation groups to blame?)

[Stan Hoeppner <stan@hardwarefreak.com>]

On 4/9/2012 6:52 AM, Stefan Ring wrote:

> Whatever the problem with the controller may be, it behaves quite
> nicely usually. It seems clear though, that, regardless of the storage
> technology, it cannot be a good idea to schedule tiny blocks in the
> order that XFS schedules them in my case.
> 
> This:
> AG0 *   *   *
> AG1  *   *   *
> AG2   *   *   *
> AG3    *   *   *
> 
> cannot be better than this:
> 
> AG0 ***
> AG1    ***
> AG2       ***
> AG3          ***

With 4 AGs this must represent the RAID6 or RAID10 case.  Those don't
seem to show any overlapping concurrency.  Maybe I'm missing something,
but it should look more like this, at least in the concat case:

AG0 ***
AG1 ***
AG2 ***

> Yes, in theory, a good cache controller should be able to sort this
> out. But at least this particular controller is not able to do so and
> could use a little help. 

Is the cache in write-through or write-back mode?  The latter should
allow for aggressive reordering.  The former none, or very little.  And
is all of it dedicated to writes, or is it split?  If split, dedicate it
all to writes.  Linux is going to cache block reads anyway, so it makes
little sense to cache them in the controller as well.

> Also, a single consumer-grade drive is
> certainly not helped by this write ordering.

Are you referring to the Mushkin SSD I mentioned?  The SandForce 2281
onboard the Enhanced Chronos Deluxe is capable of a *sustained* 20,000
4KB random write IOPs, 60,000 peak.  Mushkin states 90,000, which may be
due to their use of Toggle Mode NAND instead ONFi, and/or they're simply
fudging.  Regardless, 20K real write IOPS is enough to make
scheduling/ordering mostly irrelevant I'd think.  Just format with 8 AGs
to be on the safe side for DLP (directory level parallelism), and you're
off to the races.  The features of the SF2000 series make MLC SSDs based
on it much more like 'enterprise' SLC SSDs in most respects.  The lines
between "consumer" and "enterprise" SSDs have already been blurred as
many vendors have already been selling "enterprise" MLC SSDs for a while
now, including Intel, Kingston, OCZ, PNY, and Seagate.  All are based on
the same SandForce 2281 as in this Mushkin, or the 2282, which is
required for devices over 512GB.

-- 
Stan

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs