Re: bad performance on touch/cp file on XFS system

[Dave Chinner <david@fromorbit.com>]

On Mon, Aug 25, 2014 at 06:31:10PM +0800, Zhang Qiang wrote:
> 2014-08-25 17:08 GMT+08:00 Dave Chinner <david@fromorbit.com>:
> 
> > On Mon, Aug 25, 2014 at 04:47:39PM +0800, Zhang Qiang wrote:
> > > I have checked icount and ifree, but I found there are about 11.8 percent
> > > free, so the free inode should not be too few.
> > >
> > > Here's the detail log, any new clue?
> > >
> > > # mount /dev/sda4 /data1/
> > > # xfs_info /data1/
> > > meta-data=/dev/sda4              isize=256    agcount=4, agsize=142272384
> >
> > 4 AGs
> >
> Yes.
> 
> >
> > > icount = 220619904
> > > ifree = 26202919
> >
> > And 220 million inodes. There's your problem - that's an average
> > of 55 million inodes per AGI btree assuming you are using inode64.
> > If you are using inode32, then the inodes will be in 2 btrees, or
> > maybe even only one.
> >
> 
> You are right, all inodes stay on one AG.
> 
> BTW, why i allocate 4 AGs, and all inodes stay in one AG for inode32?,

Because the top addresses in the 2nd AG go over 32 bits, hence only
AG 0 can be used for inodes. Changing to inode64 will give you some
relief, but any time allocation occurs in AG0 is will be slow. i.e.
you'll be trading always slow for "unpredictably slow".

> > With that many inodes, I'd be considering moving to 32 or 64 AGs to
> > keep the btree size down to a more manageable size. The free inode
> 
> btree would also help, but, really, 220M inodes in a 2TB filesystem
> > is really pushing the boundaries of sanity.....
> >
> 
> So the better inodes size in one AG is about 5M,

Not necessarily. But for your storage it's almost certainly going to
minimise the problem you are seeing.

> is there any documents
> about these options I can learn more?

http://xfs.org/index.php/XFS_Papers_and_Documentation

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs