XFS group quota circumvention via NFS and chgrp

["Dr. Thomas Orgis" <thomas.orgis@uni-hamburg.de>]

Dear Linux XFS folks,

I noticed that xfs group quotas can be circumvented via NFS* on vanilla
Kernel 6.6.x (slightly differing versions on client and server) as
follows:

1. A user has a primary group and an auxilliary group.
2. There are group quotas for both, possibly very restrictive.
3. User can create files with the primary group (within quota) and then
4. `chgrp $auxgroup file` to get rid of the quota and shift it to the
   other group.

The kicker is: The quota for the auxilliary group is ignored in this
case. So I can store unlimited amounts of data by chgrp'ing it away to
the other group in pieces.

This is not intentional behaviour, or is it?

This is _not_ the case with ext4 being served via NFS. Long form
comparison follows.

A1. ext4 locally

A fresh ext4 fs populated with some test files and quota set:

user@server:/srv/test/userx$ quota -g -f /srv/test/ grep -v :\ none
Disk quotas for group userx (gid 1005): 
     Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
/dev/mapper/xyz-quotatest
                 307216*   1024    1024    none       4*      1       1    none
Disk quotas for group auxgroupxx (gid 187100007): 
     Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
/dev/mapper/xyz-quotatest
                      0       1       1               1*      1       1        

user@server:/srv/test/userx$ ls -l
total 307212
-rw-r--r-- 1 userx userx      104857600 Jun 15 14:52 bar
-rw-r--r-- 1 userx userx      209715206 Jun 15 15:01 bla
-rw-r--r-- 1 userx userx              6 Jun 15 14:51 blar
-rw-r--r-- 1 userx auxgroupxx         0 Jun 15 15:38 foo

user@server:/srv/test/userx$ chgrp auxgroupxx bla
chgrp: changing group of 'bla': Disk quota exceeded

Or, in other words:

user@server:/srv/test/userx$ strace chgrp auxgroupxx bla 2>&1 | grep chown
fchownat(AT_FDCWD, "bla", -1, 187100007, 0) = -1 EDQUOT (Disk quota exceeded)

This is all well as it should be. If I am root locally, though, it does
not matter:

user@server:/srv/test/userx# strace chgrp auxgroupxx bla 2>&1 | grep chown
fchownat(AT_FDCWD, "bla", -1, 187100007, 0) = 0

The data is moved to the other group even if the quota does not allow
it.

# repquota -g /srv/test/
*** Report for group quotas on device /dev/mapper/xyz-quotatest
Block grace time: 00:00; Inode grace time: 00:00
                        Block limits                File limits
Group           used    soft    hard  grace    used  soft  hard  grace
----------------------------------------------------------------------
root      --      20       0       0              2     0     0       
userx     ++  102408    1024    1024   none       3     1     1   none
auxgroupxx ++  204808       1       1   none       2     1     1   none

But that is fine, as root is allowed to do anything, I presume.

A2. ext4 via NFS

There is no difference on NFS for non-root. The user is not allowed to
circumvent the group quota of auxgroupxx:

userx@client:/mnt/test/userx$ ls -l
total 307212
-rw-r--r-- 1 userx userx      104857600 2026-06-15 14:52 bar
-rw-r--r-- 1 userx userx      209715210 2026-06-15 21:14 bla
-rw-r--r-- 1 userx userx              6 2026-06-15 14:51 blar
-rw-r--r-- 1 userx auxgroupxx         0 2026-06-15 15:38 foo

userx@client:/mnt/test/userx$ strace chgrp auxgroupxx bla 2>&1 | grep chown
fchownat(AT_FDCWD, "bla", -1, 187100007, 0) = -1 EDQUOT (Disk quota exceeded)

Since root over NFS, at least with root squashing, is a bit less root,
it is reassuring that for the superuser, it also fails on the NFS client:

userx@client:/mnt/test/userx# strace chgrp auxgroupxx bla 2>&1 | grep chown
fchownat(AT_FDCWD, "bla", -1, 187100007, 0) = -1 EPERM (Operation not permitted)

B1. xfs locally

To be fair, I also created a new volume with

	mkfs.xfs /dev/mapper/xyz-quotatest2
	mount -o usrquota,grpquota  /dev/mapper/xyz-quotatest2 /srv/test2/
	xfs_quota -x -c 'timer -g 60 -d' /srv/test2
	xfs_quota -x -c 'limit -g bsoft=100m bhard=101m userx' /srv/test2
	xfs_quota -x -c 'limit -g bsoft=4k bhard=4k auxgroupxx' /srv/test2

and confirmed the findings I had with the existing older fs.

root@server# xfs_quota -x -c 'report -g' /srv/test2/
Group quota on /srv/test2 (/dev/mapper/xyz-quotatest2)
                               Blocks
Group ID         Used       Soft       Hard    Warn/Grace
---------- --------------------------------------------------
root                0          0          0     00  [0 days]
userx          103424     102400     103424     00 [00:00:11]
auxgroupxx          0          4          4     00 [--------]

root@server# repquota -g /srv/test2/
*** Report for group quotas on device /dev/mapper/xyz-quotatest2
Block grace time: 00:01; Inode grace time: 00:01
                        Block limits                File limits
Group           used    soft    hard  grace    used  soft  hard  grace
----------------------------------------------------------------------
root      --       0       0       0              3     0     0
userx     +-  103424  102400  103424   none       3     0     0
auxgroupxx --       0       4       4              1     0     0

userx@server:/srv/test2/userx$ ls -l
total 103424
-rw-r--r-- 1 userx userx      104857600 Jun 16 10:06 testzero
-rw-r--r-- 1 userx userx        1048576 Jun 16 10:25 testzero2
-rw-r--r-- 1 userx auxgroupxx         0 Jun 16 10:25 wedge

The userx should not be able to create files in auxgroupxx, as its
quota of 4K is already exhausted. And indeed, it works that way locally.

userx@server:/srv/test2/userx$ strace chgrp auxgroupxx testzero 2>&1 | grep chown
fchownat(AT_FDCWD, "testzero", -1, 187100007, 0) = -1 EDQUOT (Disk quota exceeded)

Root can do it, as expected from the experience with ext4:

root@server:/srv/test2/userx/t# strace chgrp auxgroupxx testzero 2>&1 | grep chown
fchownat(AT_FDCWD, "testzero", -1, 187100007, 0) = 0

root@server# repquota -g /srv/test2/
*** Report for group quotas on device /dev/mapper/xyz-quotatest2
Block grace time: 00:01; Inode grace time: 00:01
                        Block limits                File limits
Group           used    soft    hard  grace    used  soft  hard  grace
----------------------------------------------------------------------
root      --       0       0       0              3     0     0
userx     --    1024  102400  103424              2     0     0
auxgroupxx +-  102400       4       4   none       2     0     0


Though, the block quota is actually also enforced for root here:

root@server# dd if=/dev/zero of=wedge bs=1 count=4096
dd: error writing 'wedge': Disk quota exceeded
1+0 records in
0+0 records out
0 bytes copied, 9.6329e-05 s, 0.0 kB/s

Is that really intentional, btw.? Enforcing quota for root this way,
but not for chgrp/chown?


B2. xfs via NFS

Switching to a client node that has test2 mounted via NFS, after moving
testzero back to the userx group.

userx@client:/mnt/test2/userx$ quota -g -f /mnt/test2 | grep -v ': no'
Disk quotas for group userx (gid 1005):
     Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
  server:/test2  103424* 102400  103424    none       3       0       0
Disk quotas for group auxgroupxx (gid 187100007):
     Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
  server:/test2       0       4       4               1       0       0

userx@client:/mnt/test2/userx$s trace chgrp auxgroupxx testzero 2>&1 | grep chown
fchownat(AT_FDCWD, "testzero", -1, 187100007, 0) = 0

userx@client:/mnt/test2/userx$ quota -g -f /mnt/test2 | grep -v ': no'
Disk quotas for group userx (gid 1005):
     Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
  server:/test2    1024  102400  103424               2       0       0
Disk quotas for group auxgroupxx (gid 187100007):
     Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
  server:/test2  102400*      4       4   00:01       2       0       0

The user happily moved 100M of data over to auxgroupxx and has quota
freed to start to comsume more data. The grace period should not
matter, as the soft limit is clearly hit, right? And it's only a minute
… so waiting a bit and preparing the next chunk:

userx@client:/mnt/test2/userx$ dd if=/dev/zero of=testzero3 bs=1M count=100
100+0 records in
100+0 records out
104857600 bytes (105 MB, 100 MiB) copied, 0.128547 s, 816 MB/s

userx@client:/mnt/test2/userx$ quota -g -f /mnt/test2 | grep -v ': no'
Disk quotas for group userx (gid 1005):
     Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
  server:/test2  103424* 102400  103424    none       3       0       0
Disk quotas for group auxgroupxx (gid 187100007):
     Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
  server:/test2  102400*      4       4    none       2       0       0

Quota filled again, any grace period passed. Let's give us some space!

userx@client:/mnt/test2/userx$s trace chgrp auxgroupxx testzero3 2>&1 | grep chown
fchownat(AT_FDCWD, "testzero3", -1, 187100007, 0) = 0

userx@client:/mnt/test2/userx$ quota -g -f /mnt/test2 | grep -v ': no'
Disk quotas for group userx (gid 1005):
     Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
  server:/test2    1024  102400  103424               2       0       0
Disk quotas for group auxgroupxx (gid 187100007):
     Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
  server:/test2  204800*      4       4    none       3       0       0

The added data is now in auxgroupxx's overdrawn quota, too. Repeat. The
user has no effective quota.

For completeness:

root@client# strace chgrp auxgroupxx testzero2  2>&1 | grep chown
fchownat(AT_FDCWD, "testzero2", -1, 187100007, 0) = -1 EPERM (Operation not permitted)

Root is squashed and not able to directly modify the ownership, as expected.

Now, is this non-enforcement for the group quota is a bug in XFS, or
rather in the translation to the quota data as NFS sees it? A bug in
NFS in enforcement? But only when serving XFS? I did not check ZFS over
NFS yet, but with ext4, there is at least one example where it works as
expected. I also checked with BeeGFS (on top of ZFS) that it enforces
group quotas as I expect. I noted some confusion with grace periods …
and face more confusion myself on reading the xfs_quota(8) section on
the timer command.** Anyway, I waited until after any grace period in
the last example to avoid that complication.

Or is this possibly something fixed in a very recent kernel, by any
chance? Or a regression in 6.6? I am observing this in a production
setup where I cannot just freely swap out things.


Alrighty then,

Thomas


* Kernel NFSv4 over RDMA with sec=sys, if that matters.

** „Allows  the  quota  enforcement  timeout  (i.e. the amount of time
allowed to pass before the soft limits are enforced as the hard
limits)” vs. „When  setting  any other individual timer by id or name,
the value is the number of seconds from now, at which time the hard
limits will be enforced.  This allows extending the grace time of an
individual user who has exceeded soft limits.” — The hard limits being
enforced after grace does mean the soft limits becoming hard, right?
The hard limits are always enforced, without grace, are they not?

PS: We use group quotas also for individual quotas precisely to be able
to mix personal quotas and working group quotas in a meaningful manner.

PPS: I learned before that project quotas are also no solution if you
value enforcement, as people can move data into unrestricted project
IDs at will. I hoped that plain user/group quotas are enforced, also
over NFS.

-- 
Dr. Thomas Orgis
HPC @ Universität Hamburg