Re: Weird behaviour with project quotas

["Darrick J. Wong" <djwong@kernel.org>]

On Fri, Dec 13, 2024 at 04:42:51PM +0100, Emmanuel Florac wrote:
> Le Thu, 12 Dec 2024 12:25:47 -0800
> "Darrick J. Wong" <djwong@kernel.org> écrivait:
> 
> 
> > Does this recreate the symptoms?
> > 
> <snip>
> > # df /mnt /mnt/dir
> > Filesystem      Size  Used Avail Use% Mounted on
> > /dev/sda         20G  420M   20G   3% /mnt
> > /dev/sda        2.0G     0  2.0G   0% /mnt
> > # fallocate -l 19g /mnt/a
> > # df /mnt /mnt/dir
> > Filesystem      Size  Used Avail Use% Mounted on
> > /dev/sda         20G   20G  345M  99% /mnt
> > /dev/sda        2.0G     0  2.0G   0% /mnt
> > 
> > Clearly, df should be reporting 345M available for both cases, since
> > we haven't actually used any of project 55's blocks.
> > 
> > # xfs_io -f -c 'pwrite -S 0x59 0 1m' -c fsync -c 'stat -vvvv'
> > /mnt/dir/fork wrote 1048576/1048576 bytes at offset 0
> > 1 MiB, 256 ops; 0.0008 sec (1.121 GiB/sec and 293915.0402 ops/sec)
> > fd.path = "/mnt/dir/fork"
> > fd.flags = non-sync,non-direct,read-write
> > stat.ino = 134
> > stat.type = regular file
> > stat.size = 1048576
> > stat.blocks = 2048
> > stat.atime = Thu Dec 12 12:11:06 2024
> > stat.mtime = Thu Dec 12 12:11:06 2024
> > stat.ctime = Thu Dec 12 12:11:06 2024
> > fsxattr.xflags = 0x0 []
> > fsxattr.projid = 55
> > fsxattr.extsize = 0
> > fsxattr.cowextsize = 0
> > fsxattr.nextents = 1
> > fsxattr.naextents = 0
> > dioattr.mem = 0x200
> > dioattr.miniosz = 512
> > dioattr.maxiosz = 2147483136
> > # df /mnt /mnt/dir
> > Filesystem      Size  Used Avail Use% Mounted on
> > /dev/sda         20G   20G  344M  99% /mnt
> > /dev/sda        2.0G  1.0M  2.0G   1% /mnt
> > 
> > I think this behavior comes from xfs_fill_statvfs_from_dquot, which
> > does this:
> > 
> > 	limit = blkres->softlimit ?
> > 		blkres->softlimit :
> > 		blkres->hardlimit;
> > 	if (limit && statp->f_blocks > limit) {
> > 		statp->f_blocks = limit;
> > 		statp->f_bfree = statp->f_bavail =
> > 			(statp->f_blocks > blkres->reserved) ?
> > 			 (statp->f_blocks - blkres->reserved) : 0;
> > 	}
> > 
> > I think the f_bfree/f_bavail assignment is wrong because it doesn't
> > handle the case where f_bfree was less than (limit - reserved).
> > 
> > 	if (limit) {
> > 		uint64_t	remaining = 0;
> > 
> > 		if (statp->f_blocks > limit)
> > 			statp->f_blocks = limit;
> > 		if (limit > blkres->reserved)
> > 			remaining = limit - blkres->reserved;
> > 		statp->f_bfree = min(statp->f_bfree, remaining);
> > 		statp->f_bavail = min(statp->f_bavail, remaining);
> > 	}
> > 
> > This fixes the df output a bit:
> > # df /mnt /mnt/dir
> > Filesystem      Size  Used Avail Use% Mounted on
> > /dev/sda         20G   20G  344M  99% /mnt
> > /dev/sda        2.0G  1.7G  344M  84% /mnt
> > 
> > Though the "used" column is nonsense now.  But I guess that's why
> > statfs only defines total blocks and free/available blocks.
> 
> Yep, that looks exactly like the problem we've met. Does the fact that
> not all folders have project quota change something in that case ?

No, I don't think that changes anything.  If you can build your own
kernel, can you try this out?

--D

xfs: don't over-report free space or inodes in statvfs

Emmanual Florac reports a strange occurrence when project quota limits
are enabled, free space is lower than the remaining quota, and someone
runs statvfs:

  # mkfs.xfs -f /dev/sda
  # mount /dev/sda /mnt -o prjquota
  # xfs_quota  -x -c 'limit -p bhard=2G 55' /mnt
  # mkdir /mnt/dir
  # xfs_io -c 'chproj 55' -c 'chattr +P' -c 'stat -vvvv' /mnt/dir
  # fallocate -l 19g /mnt/a
  # df /mnt /mnt/dir
  Filesystem      Size  Used Avail Use% Mounted on
  /dev/sda         20G   20G  345M  99% /mnt
  /dev/sda        2.0G     0  2.0G   0% /mnt

I think the bug here is that xfs_fill_statvfs_from_dquot unconditionally
assigns to f_bfree without checking that the filesystem has enough free
space to fill the remaining project quota.  However, this is a
longstanding behavior of xfs so it's unclear what to do here.

Cc: <stable@vger.kernel.org> # v2.6.18
Fixes: 932f2c323196c2 ("[XFS] statvfs component of directory/project quota support, code originally by Glen.")
Reported-by: Emmanuel Florac <eflorac@intellique.com>
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 fs/xfs/xfs_qm_bhv.c |   27 +++++++++++++++++----------
 1 file changed, 17 insertions(+), 10 deletions(-)

diff --git a/fs/xfs/xfs_qm_bhv.c b/fs/xfs/xfs_qm_bhv.c
index 847ba29630e9d8..db5b8afd9d1b97 100644
--- a/fs/xfs/xfs_qm_bhv.c
+++ b/fs/xfs/xfs_qm_bhv.c
@@ -32,21 +32,28 @@ xfs_fill_statvfs_from_dquot(
 	limit = blkres->softlimit ?
 		blkres->softlimit :
 		blkres->hardlimit;
-	if (limit && statp->f_blocks > limit) {
-		statp->f_blocks = limit;
-		statp->f_bfree = statp->f_bavail =
-			(statp->f_blocks > blkres->reserved) ?
-			 (statp->f_blocks - blkres->reserved) : 0;
+	if (limit) {
+		uint64_t	remaining = 0;
+
+		if (limit > blkres->reserved)
+			remaining = limit - blkres->reserved;
+
+		statp->f_blocks = min(statp->f_blocks, limit);
+		statp->f_bfree = min(statp->f_bfree, remaining);
+		statp->f_bavail = min(statp->f_bavail, remaining);
 	}
 
 	limit = dqp->q_ino.softlimit ?
 		dqp->q_ino.softlimit :
 		dqp->q_ino.hardlimit;
-	if (limit && statp->f_files > limit) {
-		statp->f_files = limit;
-		statp->f_ffree =
-			(statp->f_files > dqp->q_ino.reserved) ?
-			 (statp->f_files - dqp->q_ino.reserved) : 0;
+	if (limit) {
+		uint64_t	remaining = 0;
+
+		if (limit > dqp->q_ino.reserved)
+			remaining = limit - dqp->q_ino.reserved;
+
+		statp->f_files = min(statp->f_files, limit);
+		statp->f_ffree = min(statp->f_ffree, remaining);
 	}
 }