Re: ENOSPC at 90% with plenty of inodes

From: Dave Chinner <david@fromorbit.com>
To: Stan Hoeppner <stan@hardwarefreak.com>
Cc: xfs@oss.sgi.com
Subject: Re: ENOSPC at 90% with plenty of inodes
Date: Tue, 12 Oct 2010 21:29:54 +1100	[thread overview]
Message-ID: <20101012102954.GQ4681@dastard> (raw)
In-Reply-To: <4CB3B964.40109@hardwarefreak.com>

On Mon, Oct 11, 2010 at 08:27:00PM -0500, Stan Hoeppner wrote:
> Dave Chinner put forth on 10/11/2010 5:35 PM:
> > On Mon, Oct 11, 2010 at 03:03:28PM +0100, James Braid wrote:
> >> On Fri, Oct 8, 2010 at 23:51, Dave Chinner <david@fromorbit.com> wrote:
> >>> Sounds like fragmented free space. What is the output of:
> >>>
> >>> # xfs_db -r -c "freesp -s" <device>
> >>
> >> # xfs_db -r -c "freesp -s" /dev/sdb
> >>    from      to extents  blocks    pct
> >>       1       1 2298052 2298052  40.52
> >>       2       3 1568338 3337017  58.84
> >>       4       7    8432   35716   0.63
> >>       8      15      50     423   0.01
> >> total free extents 3874872
> >> total free blocks 5671208
> >> average free extent size 1.46359
> >>
> >> Which seems to say there are a few tiny pieces of free space
> >> available? The files that were failing to be written were a few
> >> hundred bytes in size.
> > 
> > The error has nothing to do with the size of the files, but
> > everything to do with being able to allocate more inodes. Inode
> > allocation requires 4 contiguous blocks (for 256 byte inodes, more
> > for larger inodes) with alignment constraints. That means when you
> > run out of 8 block or larger free extents, inode allocation will
> > start failing and you'll get ENOSPC being reported.
> > 
> >> We haven't seen any errors so far today, but xfs_fsr ran over the
> >> weekend, so perhaps I guess it's reorganized the filesystem.
> > 
> > Only a little. xfs_fsr will not improve fragmented free space
> > conditions (indeed, it normally fragments free space more). The only
> > way to reduce the fragmentation of free space is to remove a
> > significant amount of data and inodes from the filesystem...
> 
> Hay Dave, would a "backup/reformat/restore" help with free space
> fragmentation in this case?

Of course. But that's the last resort....

> If so, could/should the OP specify anything
> during the mkfs.xfs reformat that may help alleviate or mitigate his
> problem in the future?

No. These problems usually appear in filesystems that have run at
greater than 85-90% full for extended periods of time without being
emptied at all. Once you start to free up space, it naturally
defragments itself, but if you never free up any significant amount
of space in the filesytesm, this cannot occur and so fragmentation
just keeps getting worse....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs