Re: 1B files, slow file creation, only AG0 used

[Dave Chinner <david@fromorbit.com>]

On Fri, Mar 09, 2012 at 06:13:27PM -0800, Michael Spiegle wrote:
> We're seeing some very strange behavior with XFS on the default kernel
> for CentOS 5.6 (note, I have also 3.2.9 and witnessed the same issue).
>  The dataset on this server is about 1B small files (anywhere from 1KB
> to 50KB).  We first noticed it when creating files in a directory.  A
> simple 'touch' would take over 300ms on a completely idle system.  If
> I simply create a different directory, touching files is 1ms or
> faster.  Example:
> 
> # time touch 0
> real    0m0.323s
> user    0m0.000s
> sys     0m0.323s
> 
> # mkdir tmp2
> # time touch tmp2/0
> real    0m0.001s
> user    0m0.000s
> sys     0m0.000s

Entirely normal. some operations require Io to complete (e.g.
reading directory blocks to find where to insert the new entry),
while adding the first file to a directory generally requires zero
IO. You're seeing the difference between cold cache and hot cache
performance.

> We've done quite a bit of testing and debugging, and while we don't
> have an answer yet, we've noticed that our filesystem was created with
> the default of 32 AGs.  When using xfs_db, we notice that all
> allocations appear to be in AG0 only.

Go look up what the inode32 and inode64 mount options do. The
default is inode32....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs