Re: df bigger than ls?

Dave Chinner
2012-03-09T00:17:15Z

On Thu, Mar 08, 2012 at 10:23:48AM -0600, Ben Myers wrote:
> On Thu, Mar 08, 2012 at 01:10:54PM +1100, Dave Chinner wrote:
> > On Wed, Mar 07, 2012 at 12:04:26PM -0600, Eric Sandeen wrote:
> > > and repeat the above test 3 times, the 3rd run gets ENOSPC, and
> > > the last file written comes up short, while the first one retains
> > > all it's extra preallocated space:
> > >
> > > # du -hc bigfile* 2.0G bigfile1 1.1G bigfile2 907M
> > > bigfile3
> > >
> > > Dave, is this working as intended?
> >
> > Yes. Your problem is that you have a very small filesystem, which is
> > not the case that we optimise XFS for. :/
>
> I have seen a similar problem on some very large filesystems too.

So report the problems to the list, don't keep them to yourself. If
we don't hear about problems, then as far as we are concerned they
don't exist.

> This
> is not just dependent upon the size of the filesystem, but also the
> workload. I think it is also a big problem for folks using quotas.

Nobody has reported significant quota problems, either....

> Alex and I discussed this problem briefly awhile ago. What is the best
> way to lose when you hit ENOSPC (project quotas) or EDQUOT in
> xfs_iomap_write_delay? You want to be fair; one user hitting his quota
> shouldn't be able to steal some other user's block reservations unless
> you really are near ENOSPC for the entire filesystem.
>
> I suggested something like... track inodes with preallocated block
> reservations in LRU order and by dquot, so that the poor fella who is at
> EDQUOT will first clean up the preallocations that resulted in quota
> being enforced, try again, and then work on preallocations of other
> users only if it can help in his situation. IIRC Alex shut me down when
> he heard LRU. ;)

And I agree with Alex. Nothing additional needs to be tracked on top
of inodes with speculative prealloc. Just the search filter would
need to be different (i.e. only select inodes with a specific dquot
attached).

> Now that block reservations count toward quotas the symptom will
> probably be a little different.

Block reservations have always counted towards quotas, it's just
that they were never reported.

Cheers,

Dave.
--
Dave Chinner
david@fromorbit.com