Re: df bigger than ls?

Dave Chinner
2012-03-08T10:03:17Z

On Thu, Mar 08, 2012 at 09:04:12AM +0100, Arkadiusz Miśkiewicz wrote:
> On Wednesday 07 of March 2012, Eric Sandeen wrote:
>
> > XFS speculatively preallocates space off the end of a file. The amount of
> > space allocated depends on the present size of the file, and the amount of
> > available free space. This can be overridden
> > with mount -o allocsize=64k (or other size for example)
>
> What was the default before speculative preallocation was added (or changed
> somewhere in 2.6.3x line afaik) ?

It's changed many times over the life of XFS.

Originally it was quite a large number and could happen anywhere in
the file - I can't remember exactly what it was but some time around
1997 on Irix it got brought back to 16 blocks per written region
because of issues with Irix NFS servers keeping tens of thousands of
files open (and hence never having speculative preallocation
removed) with multiple preallocations all through sparse files and
hence running filesystems and user quotas out of space.

This 16 block size was the default when ported to linux when porting
to Linux, but would also round up to stripe unit size if the file
was larger than 512kb. In 2004 the speculative preallocation was
reduced to EOF preallocation only instead of anywhere in a hole in a
file. Somewhere along the line the default size got changed to
PAGE_SIZE rather than 16 blocks. Then the allocsize mount option was
introduced in 2005 to enable it to be increased for those that
needed large preallocation (typically PVR users), and the maximum
increased to 1GB.

It got broken by changes in 2008, fixed in 2009, and then made
dynamic in 2.6.38 and now it behaves more like it did in 1995 for
files being extended.

So there's a long history of changing the speculative preallocation
behaviour of XFS to adapt to changing storage characteristics,
workloads and capacities.....

Cheers,

Dave.
--
Dave Chinner
david@fromorbit.com