Re: mkfs.xfs error creating large agcount an raid

[Stan Hoeppner <stan@hardwarefreak.com>]

On 6/26/2011 6:59 PM, Dave Chinner wrote:
> On Sun, Jun 26, 2011 at 02:53:43AM -0300, Marcus Pereira wrote:

>> There is already insane head seeking at this server, hundreds of
>> simultaneous users reading their mailboxes.

I missed this part of the reply probably due to the reply formatting
style used.

> Perhaps you should just usethe defaults first and only consider
> changes if there is an obvious problem,?
> 
>> In fact I was trying to
>> reduce the head seeking with larger agcounts.
> 
> AGs are not for reducing seeking - they are for increasing
> allocation parallelism and scaling freespace indexes to extremely
> large filesystem sizes.
> 
> In fact, trying to use more than a few hundred AGs will hit internal
> AG indexing scalability limitations, especially as you start to fill
> up AGs and have to scan for AGs with free space in them.
> 
> IOWs, using large numbers of AGs are inadvisable for many reasons.

Marcus, if you are seeing excessive head seeking with this maildir
workload, the problem isn't the number of AGs.  The problem is that
you're using striping for a small file high random IOPS workload, and
you don't have enough spindles, thus not enough seek bandwidth.

I recommend 3 changes, one of which I previously mentioned:

1.  Use 8 mirror pairs instead of 4
2.  Don't use striping.  Make an mdraid --linear device of the 8 mirrors
3.  Format with '-d agcount=32' which will give you 4 AGs per spindle

With striping, each file IO will generate multiple head seeks per
spindle across all spindles.  With a linear array each file IO will
typically only generate seeks on a single spindle, as the metadata and
file data should be within the same AG.

Test this configuration and post your results.

-- 
Stan

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs