* (Un)known issue with XFS group quotas on LVM? @ 2014-06-11 15:43 John Desantis 2014-06-11 23:07 ` Dave Chinner 0 siblings, 1 reply; 9+ messages in thread From: John Desantis @ 2014-06-11 15:43 UTC (permalink / raw) To: xfs Hello list! A big "thanks!" to all XFS developers for a great product. Now, to the point! We're using an XFS file system created on an LVM with a standard kernel (2.6.32-431.3.1). We're using quotas without an issue, with the exception of the group quota. It seems that group quotas are ignored at times, at least from our configuration (we use remote user and group look-ups and authentication). I've only been able to get root running sudo -g dd.. to create a group file which reflects changes in the group quota limit. We've mounted the file system in question with both user and group quotas enabled; I didn't see anywhere in the man page that said this isn't possible, although I did see where group and project quotas cannot be used simultaneously on the same file system. Has anyone experienced this behavior before? I feel that I did some pretty good searching in the archives, Google, and the BugZilla page, but there were no "real" results which seemed to describe the same issue we're facing. Strace didn't provide any clues either. I don't think it's a sync issue because I've allowed a few days to pass in between "test" runs. I've tested the quota by: 1.) Using sudo -u user -g group dd.... on both the exported file system and its mount path on the server itself. 2.) Using sudo -u user -g group dd.. on the client and the file system's mount path. 3.) Using sudo -g group dd... on both the exported file system and its mount path on the server itself; this method produces expected results. 4.) Using sudo -g group dd.. on the client and the file system's mount path; this method also produces expected results. Items #1 and #2 produce the error. I just wanted to ping the list before creating a local testing group to see if it's an issue with remote group look-ups. Thank you, John DeSantis _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: (Un)known issue with XFS group quotas on LVM? 2014-06-11 15:43 (Un)known issue with XFS group quotas on LVM? John Desantis @ 2014-06-11 23:07 ` Dave Chinner 2014-06-12 14:04 ` John Desantis 0 siblings, 1 reply; 9+ messages in thread From: Dave Chinner @ 2014-06-11 23:07 UTC (permalink / raw) To: John Desantis; +Cc: xfs On Wed, Jun 11, 2014 at 11:43:09AM -0400, John Desantis wrote: > Hello list! > > A big "thanks!" to all XFS developers for a great product. > > Now, to the point! > > We're using an XFS file system created on an LVM with a standard > kernel (2.6.32-431.3.1). RHEL kernel? If so, you probably want to report it to your support contact to see if it's a known issue with a fix/workaround..... > We're using quotas without an issue, with > the exception of the group quota. It seems that group quotas are > ignored at times, at least from our configuration (we use remote user > and group look-ups and authentication). I've only been able to get > root running sudo -g dd.. to create a group file which reflects > changes in the group quota limit. That doesn't sound like a filesystem problem. Sounds more like the remote user/group configuration is getting squashed. Do the systems have the same user/group configurations, or are the clients sending group IDs the server doesn't recognise? As it is, describing the problem in text doesn't tell me anything about the problem. You need to cut-n-paste the CLI and the output showing the incorrect behaviour so we can attempt to reproduce the same behaviour you are seeing. > We've mounted the file system in question with both user and group > quotas enabled; I didn't see anywhere in the man page that said this > isn't possible, although I did see where group and project quotas > cannot be used simultaneously on the same file system. > > Has anyone experienced this behavior before? I feel that I did some > pretty good searching in the archives, Google, and the BugZilla page, > but there were no "real" results which seemed to describe the same > issue we're facing. Strace didn't provide any clues either. I don't > think it's a sync issue because I've allowed a few days to pass in > between "test" runs. > > I've tested the quota by: > > 1.) Using sudo -u user -g group dd.... on both the exported file > system and its mount path on the server itself. > 2.) Using sudo -u user -g group dd.. on the client and the file > system's mount path. > 3.) Using sudo -g group dd... on both the exported file system and > its mount path on the server itself; this method produces expected > results. > 4.) Using sudo -g group dd.. on the client and the file system's > mount path; this method also produces expected results. > > Items #1 and #2 produce the error. I can't see how that is related to the filesystem - the filesystem will either account for group quota usage or it won't. It doesn't care what primary gid the process uses, it will simply account usage to that... > I just wanted to ping the list before creating a local testing group > to see if it's an issue with remote group look-ups. Quite possibly that's a problem when generating quota reports, but again that's a userspace problem, not a filesystem quota accounting issue... Cheers, Dave. -- Dave Chinner david@fromorbit.com _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: (Un)known issue with XFS group quotas on LVM? 2014-06-11 23:07 ` Dave Chinner @ 2014-06-12 14:04 ` John Desantis 2014-06-13 0:01 ` Dave Chinner 0 siblings, 1 reply; 9+ messages in thread From: John Desantis @ 2014-06-12 14:04 UTC (permalink / raw) To: Dave Chinner; +Cc: xfs Dave, Thanks for your reply. Let me address your questions and replies. > That doesn't sound like a filesystem problem. Sounds more like the > remote user/group configuration is getting squashed. Do the systems > have the same user/group configurations, or are the clients sending > group IDs the server doesn't recognise? Stracing shows that the group and user ID's are not being squashed and they are resolving each time to their respective UID/GID's. > As it is, describing the problem in text doesn't tell me anything > about the problem. You need to cut-n-paste the CLI and the output > showing the incorrect behaviour so we can attempt to reproduce the > same behaviour you are seeing. Busted. I figured before sending the usual gamut of console "action", I'd "save" time by simply presenting the symptoms. > I can't see how that is related to the filesystem - the filesystem > will either account for group quota usage or it won't. It doesn't > care what primary gid the process uses, it will simply account usage > to that... You're correct (but you already knew this!). I can tell you that running the quota report does show the group block count changing. It looks like this "issue" is my misunderstanding of how XFS is checking the quota. Perhaps you could shed some light? When you're using active user and group quotas on the same file system and the user quota is lower than the group quota, does XFS first check against the user quota and then the group quota? Is the order of operations user, group, and then project or does it depend on the mounting order of quotas? In the case I presented yesterday, the user in question had a quota of 25 GB, but the group had an unlimited quota. Each time that I used sudo -u user -g group or logged into the user's account, the quota was always exceeded. Now that I have removed the user's quota, the group quota is now being respected properly. Again, thank you! John DeSantis 2014-06-11 19:07 GMT-04:00 Dave Chinner <david@fromorbit.com>: > On Wed, Jun 11, 2014 at 11:43:09AM -0400, John Desantis wrote: >> Hello list! >> >> A big "thanks!" to all XFS developers for a great product. >> >> Now, to the point! >> >> We're using an XFS file system created on an LVM with a standard >> kernel (2.6.32-431.3.1). > > RHEL kernel? If so, you probably want to report it to your support > contact to see if it's a known issue with a fix/workaround..... > >> We're using quotas without an issue, with >> the exception of the group quota. It seems that group quotas are >> ignored at times, at least from our configuration (we use remote user >> and group look-ups and authentication). I've only been able to get >> root running sudo -g dd.. to create a group file which reflects >> changes in the group quota limit. > > That doesn't sound like a filesystem problem. Sounds more like the > remote user/group configuration is getting squashed. Do the systems > have the same user/group configurations, or are the clients sending > group IDs the server doesn't recognise? > > As it is, describing the problem in text doesn't tell me anything > about the problem. You need to cut-n-paste the CLI and the output > showing the incorrect behaviour so we can attempt to reproduce the > same behaviour you are seeing. > >> We've mounted the file system in question with both user and group >> quotas enabled; I didn't see anywhere in the man page that said this >> isn't possible, although I did see where group and project quotas >> cannot be used simultaneously on the same file system. >> >> Has anyone experienced this behavior before? I feel that I did some >> pretty good searching in the archives, Google, and the BugZilla page, >> but there were no "real" results which seemed to describe the same >> issue we're facing. Strace didn't provide any clues either. I don't >> think it's a sync issue because I've allowed a few days to pass in >> between "test" runs. >> >> I've tested the quota by: >> >> 1.) Using sudo -u user -g group dd.... on both the exported file >> system and its mount path on the server itself. >> 2.) Using sudo -u user -g group dd.. on the client and the file >> system's mount path. >> 3.) Using sudo -g group dd... on both the exported file system and >> its mount path on the server itself; this method produces expected >> results. >> 4.) Using sudo -g group dd.. on the client and the file system's >> mount path; this method also produces expected results. >> >> Items #1 and #2 produce the error. > > I can't see how that is related to the filesystem - the filesystem > will either account for group quota usage or it won't. It doesn't > care what primary gid the process uses, it will simply account usage > to that... > >> I just wanted to ping the list before creating a local testing group >> to see if it's an issue with remote group look-ups. > > Quite possibly that's a problem when generating quota reports, but > again that's a userspace problem, not a filesystem quota accounting > issue... > > Cheers, > > Dave. > -- > Dave Chinner > david@fromorbit.com _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: (Un)known issue with XFS group quotas on LVM? 2014-06-12 14:04 ` John Desantis @ 2014-06-13 0:01 ` Dave Chinner 2014-06-13 14:50 ` John Desantis 0 siblings, 1 reply; 9+ messages in thread From: Dave Chinner @ 2014-06-13 0:01 UTC (permalink / raw) To: John Desantis; +Cc: xfs On Thu, Jun 12, 2014 at 10:04:29AM -0400, John Desantis wrote: > Dave, > > Thanks for your reply. Let me address your questions and replies. > > > That doesn't sound like a filesystem problem. Sounds more like the > > remote user/group configuration is getting squashed. Do the systems > > have the same user/group configurations, or are the clients sending > > group IDs the server doesn't recognise? > > Stracing shows that the group and user ID's are not being squashed and > they are resolving each time to their respective UID/GID's. > > > As it is, describing the problem in text doesn't tell me anything > > about the problem. You need to cut-n-paste the CLI and the output > > showing the incorrect behaviour so we can attempt to reproduce the > > same behaviour you are seeing. > > Busted. I figured before sending the usual gamut of console "action", > I'd "save" time by simply presenting the symptoms. > > > I can't see how that is related to the filesystem - the filesystem > > will either account for group quota usage or it won't. It doesn't > > care what primary gid the process uses, it will simply account usage > > to that... > > You're correct (but you already knew this!). I can tell you that > running the quota report does show the group block count changing. > > It looks like this "issue" is my misunderstanding of how XFS is > checking the quota. Perhaps you could shed some light? > > When you're using active user and group quotas on the same file system > and the user quota is lower than the group quota, does XFS first check > against the user quota and then the group quota? Is the order of > operations user, group, and then project or does it depend on the > mounting order of quotas? Order doesn't matter because the filesystem is supposed to check against all enforced quotas, not just the first one. > In the case I presented yesterday, the user in question had a quota of > 25 GB, but the group had an unlimited quota. Each time that I used > sudo -u user -g group or logged into the user's account, the quota was > always exceeded. Now that I have removed the user's quota, the group > quota is now being respected properly. Again, you need to post the commands and output that demonstrate the problem you are describing. I'm having trouble understanding what your problem is because I can't see the raw output that you are seeing... Cheers, Dave. -- Dave Chinner david@fromorbit.com _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: (Un)known issue with XFS group quotas on LVM? 2014-06-13 0:01 ` Dave Chinner @ 2014-06-13 14:50 ` John Desantis 2014-06-13 16:22 ` Brian Foster 0 siblings, 1 reply; 9+ messages in thread From: John Desantis @ 2014-06-13 14:50 UTC (permalink / raw) To: Dave Chinner; +Cc: xfs Dave, > Order doesn't matter because the filesystem is supposed to check > against all enforced quotas, not just the first one. Thanks for confirming. > Again, you need to post the commands and output that demonstrate the > problem you are describing. I'm having trouble understanding what > your problem is because I can't see the raw output that you are > seeing... Ok, here are the default user and group quotas of my test account: xfs_quota> quota -uh desantis.admin Disk quotas for User desantis.admin (663800172) Filesystem Blocks Quota Limit Warn/Time Mounted on /dev/mapper/vg_drbd-shares 256K 0 0 00 [------] /export/shares xfs_quota> quota -gh rcadm Disk quotas for Group rcadm (1034) Filesystem Blocks Quota Limit Warn/Time Mounted on /dev/mapper/vg_drbd-shares 43.5G 0 0 00 [------] /export/shares Now, we'll limit them and then re-display the quotas: xfs_quota> limit -g bsoft=50G bhard=50G rcadm xfs_quota> limit -u bsoft=10G bhard=10G desantis.admin xfs_quota> quota -uh desantis.admin Disk quotas for User desantis.admin (663800172) Filesystem Blocks Quota Limit Warn/Time Mounted on /dev/mapper/vg_drbd-shares 256K 10G 10G 00 [------] /export/shares xfs_quota> quota -gh rcadm Disk quotas for Group rcadm (1034) Filesystem Blocks Quota Limit Warn/Time Mounted on /dev/mapper/vg_drbd-shares 43.5G 50G 50G 00 [------] /export/shares Here come the tests: # sudo -u desantis.admin -g rcadm dd if=/dev/zero of=/export/shares/rcadm/testfile_1 bs=1024M count=11 dd: writing `/export/shares/rcadm/testfile_1': Disk quota exceeded 7+0 records in 6+0 records out 6933651456 bytes (6.9 GB) copied, 19.7811 s, 351 MB/s xfs_quota> quota -uh desantis.admin Disk quotas for User desantis.admin (663800172) Filesystem Blocks Quota Limit Warn/Time Mounted on /dev/mapper/vg_drbd-shares 6.5G 10G 10G 00 [------] /export/shares xfs_quota> limit -u bsoft=0 bhard=0 desantis.admin xfs_quota> quota -uh desantis.admin Disk quotas for User desantis.admin (663800172) Filesystem Blocks Quota Limit Warn/Time Mounted on /dev/mapper/vg_drbd-shares 6.5G 0 0 00 [------] /export/shares # sudo -u desantis.admin -g rcadm dd if=/dev/zero of=/export/shares/rcadm/testfile_1 bs=1024M count=5 5+0 records in 5+0 records out 5368709120 bytes (5.4 GB) copied, 10.8275 s, 496 MB/s # sudo -u desantis.admin -g rcadm dd if=/dev/zero of=/export/shares/rcadm/testfile_2 bs=1024M count=5 dd: writing `/export/shares/rcadm/testfile_2': Disk quota exceeded 1+0 records in 0+0 records out 1028136960 bytes (1.0 GB) copied, 4.60859 s, 223 MB/s xfs_quota> quota -gh rcadm Disk quotas for Group rcadm (1034) Filesystem Blocks Quota Limit Warn/Time Mounted on /dev/mapper/vg_drbd-shares 50.0G 50G 50G 00 [------] /export/shares As you can see with the output, my user quota of 10 GB was enforced over the higher group quota of 50 GB. My original thought process was that since the file being created was specified for the group with the larger quota, that XFS would enforce that limit over the user quota. After reading over several pieces of quota documentation (RTFM!) from different O.S. distributions, it looks like you can either have a user or group quota getting enforced, but not both in the sense I was originally thinking. That being said, I really do appreciate the responses and extra set of eyes and apologize if I've wasted too much time. John DeSantis 2014-06-12 20:01 GMT-04:00 Dave Chinner <david@fromorbit.com>: > On Thu, Jun 12, 2014 at 10:04:29AM -0400, John Desantis wrote: >> Dave, >> >> Thanks for your reply. Let me address your questions and replies. >> >> > That doesn't sound like a filesystem problem. Sounds more like the >> > remote user/group configuration is getting squashed. Do the systems >> > have the same user/group configurations, or are the clients sending >> > group IDs the server doesn't recognise? >> >> Stracing shows that the group and user ID's are not being squashed and >> they are resolving each time to their respective UID/GID's. >> >> > As it is, describing the problem in text doesn't tell me anything >> > about the problem. You need to cut-n-paste the CLI and the output >> > showing the incorrect behaviour so we can attempt to reproduce the >> > same behaviour you are seeing. >> >> Busted. I figured before sending the usual gamut of console "action", >> I'd "save" time by simply presenting the symptoms. >> >> > I can't see how that is related to the filesystem - the filesystem >> > will either account for group quota usage or it won't. It doesn't >> > care what primary gid the process uses, it will simply account usage >> > to that... >> >> You're correct (but you already knew this!). I can tell you that >> running the quota report does show the group block count changing. >> >> It looks like this "issue" is my misunderstanding of how XFS is >> checking the quota. Perhaps you could shed some light? >> >> When you're using active user and group quotas on the same file system >> and the user quota is lower than the group quota, does XFS first check >> against the user quota and then the group quota? Is the order of >> operations user, group, and then project or does it depend on the >> mounting order of quotas? > > Order doesn't matter because the filesystem is supposed to check > against all enforced quotas, not just the first one. > >> In the case I presented yesterday, the user in question had a quota of >> 25 GB, but the group had an unlimited quota. Each time that I used >> sudo -u user -g group or logged into the user's account, the quota was >> always exceeded. Now that I have removed the user's quota, the group >> quota is now being respected properly. > > Again, you need to post the commands and output that demonstrate the > problem you are describing. I'm having trouble understanding what > your problem is because I can't see the raw output that you are > seeing... > > Cheers, > > Dave. > -- > Dave Chinner > david@fromorbit.com _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: (Un)known issue with XFS group quotas on LVM? 2014-06-13 14:50 ` John Desantis @ 2014-06-13 16:22 ` Brian Foster 2014-06-13 17:00 ` John Desantis 0 siblings, 1 reply; 9+ messages in thread From: Brian Foster @ 2014-06-13 16:22 UTC (permalink / raw) To: John Desantis; +Cc: xfs On Fri, Jun 13, 2014 at 10:50:19AM -0400, John Desantis wrote: > Dave, > > > Order doesn't matter because the filesystem is supposed to check > > against all enforced quotas, not just the first one. > > Thanks for confirming. > > > Again, you need to post the commands and output that demonstrate the > > problem you are describing. I'm having trouble understanding what > > your problem is because I can't see the raw output that you are > > seeing... > > Ok, here are the default user and group quotas of my test account: > > xfs_quota> quota -uh desantis.admin > Disk quotas for User desantis.admin (663800172) > Filesystem Blocks Quota Limit Warn/Time Mounted on > /dev/mapper/vg_drbd-shares > 256K 0 0 00 [------] /export/shares > > xfs_quota> quota -gh rcadm > Disk quotas for Group rcadm (1034) > Filesystem Blocks Quota Limit Warn/Time Mounted on > /dev/mapper/vg_drbd-shares > 43.5G 0 0 00 [------] /export/shares > > Now, we'll limit them and then re-display the quotas: > > xfs_quota> limit -g bsoft=50G bhard=50G rcadm > xfs_quota> limit -u bsoft=10G bhard=10G desantis.admin > > xfs_quota> quota -uh desantis.admin > Disk quotas for User desantis.admin (663800172) > Filesystem Blocks Quota Limit Warn/Time Mounted on > /dev/mapper/vg_drbd-shares > 256K 10G 10G 00 [------] /export/shares > > xfs_quota> quota -gh rcadm > Disk quotas for Group rcadm (1034) > Filesystem Blocks Quota Limit Warn/Time Mounted on > /dev/mapper/vg_drbd-shares > 43.5G 50G 50G 00 [------] /export/shares > So you've got 10G quota with about as much available and a 50G quota with ~6.5GB available. > Here come the tests: > > # sudo -u desantis.admin -g rcadm dd if=/dev/zero > of=/export/shares/rcadm/testfile_1 bs=1024M count=11 > dd: writing `/export/shares/rcadm/testfile_1': Disk quota exceeded > 7+0 records in > 6+0 records out > 6933651456 bytes (6.9 GB) copied, 19.7811 s, 351 MB/s > It looks like you've hit the 50G quota. > xfs_quota> quota -uh desantis.admin > Disk quotas for User desantis.admin (663800172) > Filesystem Blocks Quota Limit Warn/Time Mounted on > /dev/mapper/vg_drbd-shares > 6.5G 10G 10G 00 [------] /export/shares > > xfs_quota> limit -u bsoft=0 bhard=0 desantis.admin > Now the 10G quota is gone. > xfs_quota> quota -uh desantis.admin > Disk quotas for User desantis.admin (663800172) > Filesystem Blocks Quota Limit Warn/Time Mounted on > /dev/mapper/vg_drbd-shares > 6.5G 0 0 00 [------] /export/shares > > # sudo -u desantis.admin -g rcadm dd if=/dev/zero > of=/export/shares/rcadm/testfile_1 bs=1024M count=5 > 5+0 records in > 5+0 records out > 5368709120 bytes (5.4 GB) copied, 10.8275 s, 496 MB/s > You overwrite the previously written 6.5G file to 5.4G (dd will truncate). > # sudo -u desantis.admin -g rcadm dd if=/dev/zero > of=/export/shares/rcadm/testfile_2 bs=1024M count=5 > dd: writing `/export/shares/rcadm/testfile_2': Disk quota exceeded > 1+0 records in > 0+0 records out > 1028136960 bytes (1.0 GB) copied, 4.60859 s, 223 MB/s > Then create a new file and run into the 50G limit again. > xfs_quota> quota -gh rcadm > Disk quotas for Group rcadm (1034) > Filesystem Blocks Quota Limit Warn/Time Mounted on > /dev/mapper/vg_drbd-shares > 50.0G 50G 50G 00 [------] /export/shares > > > As you can see with the output, my user quota of 10 GB was enforced > over the higher group quota of 50 GB. My original thought process was > that since the file being created was specified for the group with the > larger quota, that XFS would enforce that limit over the user quota. > After reading over several pieces of quota documentation (RTFM!) from > different O.S. distributions, it looks like you can either have a user > or group quota getting enforced, but not both in the sense I was > originally thinking. > This all seems like normal behavior to me. What gives you the impression that you are hitting the 10G quota? The fact that the quota is reset and writing continues? Note that dd will truncate a file before it starts writing unless notrunc is specified. Brian > That being said, I really do appreciate the responses and extra set of > eyes and apologize if I've wasted too much time. > > John DeSantis > > 2014-06-12 20:01 GMT-04:00 Dave Chinner <david@fromorbit.com>: > > On Thu, Jun 12, 2014 at 10:04:29AM -0400, John Desantis wrote: > >> Dave, > >> > >> Thanks for your reply. Let me address your questions and replies. > >> > >> > That doesn't sound like a filesystem problem. Sounds more like the > >> > remote user/group configuration is getting squashed. Do the systems > >> > have the same user/group configurations, or are the clients sending > >> > group IDs the server doesn't recognise? > >> > >> Stracing shows that the group and user ID's are not being squashed and > >> they are resolving each time to their respective UID/GID's. > >> > >> > As it is, describing the problem in text doesn't tell me anything > >> > about the problem. You need to cut-n-paste the CLI and the output > >> > showing the incorrect behaviour so we can attempt to reproduce the > >> > same behaviour you are seeing. > >> > >> Busted. I figured before sending the usual gamut of console "action", > >> I'd "save" time by simply presenting the symptoms. > >> > >> > I can't see how that is related to the filesystem - the filesystem > >> > will either account for group quota usage or it won't. It doesn't > >> > care what primary gid the process uses, it will simply account usage > >> > to that... > >> > >> You're correct (but you already knew this!). I can tell you that > >> running the quota report does show the group block count changing. > >> > >> It looks like this "issue" is my misunderstanding of how XFS is > >> checking the quota. Perhaps you could shed some light? > >> > >> When you're using active user and group quotas on the same file system > >> and the user quota is lower than the group quota, does XFS first check > >> against the user quota and then the group quota? Is the order of > >> operations user, group, and then project or does it depend on the > >> mounting order of quotas? > > > > Order doesn't matter because the filesystem is supposed to check > > against all enforced quotas, not just the first one. > > > >> In the case I presented yesterday, the user in question had a quota of > >> 25 GB, but the group had an unlimited quota. Each time that I used > >> sudo -u user -g group or logged into the user's account, the quota was > >> always exceeded. Now that I have removed the user's quota, the group > >> quota is now being respected properly. > > > > Again, you need to post the commands and output that demonstrate the > > problem you are describing. I'm having trouble understanding what > > your problem is because I can't see the raw output that you are > > seeing... > > > > Cheers, > > > > Dave. > > -- > > Dave Chinner > > david@fromorbit.com > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: (Un)known issue with XFS group quotas on LVM? 2014-06-13 16:22 ` Brian Foster @ 2014-06-13 17:00 ` John Desantis 2014-06-13 17:59 ` Brian Foster 0 siblings, 1 reply; 9+ messages in thread From: John Desantis @ 2014-06-13 17:00 UTC (permalink / raw) To: Brian Foster; +Cc: xfs Brian, > This all seems like normal behavior to me. What gives you the impression > that you are hitting the 10G quota? The fact that the quota is reset and > writing continues? Note that dd will truncate a file before it starts > writing unless notrunc is specified. Yes, I am aware of the truncating, but not that particular option - thanks! I had experienced the "issue" when a user quota was set while the group quota was unlimited. See below. xfs_quota> limit -u bsoft=5G bhard=5G desantis.admin xfs_quota> quota -uh desantis.admin Disk quotas for User desantis.admin (663800172) Filesystem Blocks Quota Limit Warn/Time Mounted on /dev/mapper/vg_drbd-shares 256K 5G 5G 00 [------] /export/shares xfs_quota> limit -g bsoft=unlimited bhard=unlimited rcadm xfs_quota> quota -gh rcadm Disk quotas for Group rcadm (1034) Filesystem Blocks Quota Limit Warn/Time Mounted on /dev/mapper/vg_drbd-shares 43.5G 16E 16E 00 [------] /export/shares # sudo -u desantis.admin -g rcadm dd if=/dev/zero of=/export/shares/rcadm/testfile_morethan5G bs=1024M count=10 dd: writing `/export/shares/rcadm/testfile_morethan5G': Disk quota exceeded 5+0 records in 4+0 records out 5368442880 bytes (5.4 GB) copied, 10.5371 s, 509 MB/s xfs_quota> quota -uh desantis.admin Disk quotas for User desantis.admin (663800172) Filesystem Blocks Quota Limit Warn/Time Mounted on /dev/mapper/vg_drbd-shares 5.0G 5G 5G 00 [------] /export/shares xfs_quota> quota -gh rcadm Disk quotas for Group rcadm (1034) Filesystem Blocks Quota Limit Warn/Time Mounted on /dev/mapper/vg_drbd-shares 48.5G 16E 16E 00 [------] /export/shares As you can see, although the group quota did augment 5 GB, the user quota was enforced over the group quota. This is what led to my confusion. I had thought that the group quota would be respected over the user quota because I was specifying the group for the file being created. John DeSantis 2014-06-13 12:22 GMT-04:00 Brian Foster <bfoster@redhat.com>: > On Fri, Jun 13, 2014 at 10:50:19AM -0400, John Desantis wrote: >> Dave, >> >> > Order doesn't matter because the filesystem is supposed to check >> > against all enforced quotas, not just the first one. >> >> Thanks for confirming. >> >> > Again, you need to post the commands and output that demonstrate the >> > problem you are describing. I'm having trouble understanding what >> > your problem is because I can't see the raw output that you are >> > seeing... >> >> Ok, here are the default user and group quotas of my test account: >> >> xfs_quota> quota -uh desantis.admin >> Disk quotas for User desantis.admin (663800172) >> Filesystem Blocks Quota Limit Warn/Time Mounted on >> /dev/mapper/vg_drbd-shares >> 256K 0 0 00 [------] /export/shares >> >> xfs_quota> quota -gh rcadm >> Disk quotas for Group rcadm (1034) >> Filesystem Blocks Quota Limit Warn/Time Mounted on >> /dev/mapper/vg_drbd-shares >> 43.5G 0 0 00 [------] /export/shares >> >> Now, we'll limit them and then re-display the quotas: >> >> xfs_quota> limit -g bsoft=50G bhard=50G rcadm >> xfs_quota> limit -u bsoft=10G bhard=10G desantis.admin >> >> xfs_quota> quota -uh desantis.admin >> Disk quotas for User desantis.admin (663800172) >> Filesystem Blocks Quota Limit Warn/Time Mounted on >> /dev/mapper/vg_drbd-shares >> 256K 10G 10G 00 [------] /export/shares >> >> xfs_quota> quota -gh rcadm >> Disk quotas for Group rcadm (1034) >> Filesystem Blocks Quota Limit Warn/Time Mounted on >> /dev/mapper/vg_drbd-shares >> 43.5G 50G 50G 00 [------] /export/shares >> > > So you've got 10G quota with about as much available and a 50G quota > with ~6.5GB available. > >> Here come the tests: >> >> # sudo -u desantis.admin -g rcadm dd if=/dev/zero >> of=/export/shares/rcadm/testfile_1 bs=1024M count=11 >> dd: writing `/export/shares/rcadm/testfile_1': Disk quota exceeded >> 7+0 records in >> 6+0 records out >> 6933651456 bytes (6.9 GB) copied, 19.7811 s, 351 MB/s >> > > It looks like you've hit the 50G quota. > >> xfs_quota> quota -uh desantis.admin >> Disk quotas for User desantis.admin (663800172) >> Filesystem Blocks Quota Limit Warn/Time Mounted on >> /dev/mapper/vg_drbd-shares >> 6.5G 10G 10G 00 [------] /export/shares >> >> xfs_quota> limit -u bsoft=0 bhard=0 desantis.admin >> > > Now the 10G quota is gone. > >> xfs_quota> quota -uh desantis.admin >> Disk quotas for User desantis.admin (663800172) >> Filesystem Blocks Quota Limit Warn/Time Mounted on >> /dev/mapper/vg_drbd-shares >> 6.5G 0 0 00 [------] /export/shares >> >> # sudo -u desantis.admin -g rcadm dd if=/dev/zero >> of=/export/shares/rcadm/testfile_1 bs=1024M count=5 >> 5+0 records in >> 5+0 records out >> 5368709120 bytes (5.4 GB) copied, 10.8275 s, 496 MB/s >> > > You overwrite the previously written 6.5G file to 5.4G (dd will > truncate). > >> # sudo -u desantis.admin -g rcadm dd if=/dev/zero >> of=/export/shares/rcadm/testfile_2 bs=1024M count=5 >> dd: writing `/export/shares/rcadm/testfile_2': Disk quota exceeded >> 1+0 records in >> 0+0 records out >> 1028136960 bytes (1.0 GB) copied, 4.60859 s, 223 MB/s >> > > Then create a new file and run into the 50G limit again. > >> xfs_quota> quota -gh rcadm >> Disk quotas for Group rcadm (1034) >> Filesystem Blocks Quota Limit Warn/Time Mounted on >> /dev/mapper/vg_drbd-shares >> 50.0G 50G 50G 00 [------] /export/shares >> >> >> As you can see with the output, my user quota of 10 GB was enforced >> over the higher group quota of 50 GB. My original thought process was >> that since the file being created was specified for the group with the >> larger quota, that XFS would enforce that limit over the user quota. >> After reading over several pieces of quota documentation (RTFM!) from >> different O.S. distributions, it looks like you can either have a user >> or group quota getting enforced, but not both in the sense I was >> originally thinking. >> > > This all seems like normal behavior to me. What gives you the impression > that you are hitting the 10G quota? The fact that the quota is reset and > writing continues? Note that dd will truncate a file before it starts > writing unless notrunc is specified. > > Brian > >> That being said, I really do appreciate the responses and extra set of >> eyes and apologize if I've wasted too much time. >> >> John DeSantis >> >> 2014-06-12 20:01 GMT-04:00 Dave Chinner <david@fromorbit.com>: >> > On Thu, Jun 12, 2014 at 10:04:29AM -0400, John Desantis wrote: >> >> Dave, >> >> >> >> Thanks for your reply. Let me address your questions and replies. >> >> >> >> > That doesn't sound like a filesystem problem. Sounds more like the >> >> > remote user/group configuration is getting squashed. Do the systems >> >> > have the same user/group configurations, or are the clients sending >> >> > group IDs the server doesn't recognise? >> >> >> >> Stracing shows that the group and user ID's are not being squashed and >> >> they are resolving each time to their respective UID/GID's. >> >> >> >> > As it is, describing the problem in text doesn't tell me anything >> >> > about the problem. You need to cut-n-paste the CLI and the output >> >> > showing the incorrect behaviour so we can attempt to reproduce the >> >> > same behaviour you are seeing. >> >> >> >> Busted. I figured before sending the usual gamut of console "action", >> >> I'd "save" time by simply presenting the symptoms. >> >> >> >> > I can't see how that is related to the filesystem - the filesystem >> >> > will either account for group quota usage or it won't. It doesn't >> >> > care what primary gid the process uses, it will simply account usage >> >> > to that... >> >> >> >> You're correct (but you already knew this!). I can tell you that >> >> running the quota report does show the group block count changing. >> >> >> >> It looks like this "issue" is my misunderstanding of how XFS is >> >> checking the quota. Perhaps you could shed some light? >> >> >> >> When you're using active user and group quotas on the same file system >> >> and the user quota is lower than the group quota, does XFS first check >> >> against the user quota and then the group quota? Is the order of >> >> operations user, group, and then project or does it depend on the >> >> mounting order of quotas? >> > >> > Order doesn't matter because the filesystem is supposed to check >> > against all enforced quotas, not just the first one. >> > >> >> In the case I presented yesterday, the user in question had a quota of >> >> 25 GB, but the group had an unlimited quota. Each time that I used >> >> sudo -u user -g group or logged into the user's account, the quota was >> >> always exceeded. Now that I have removed the user's quota, the group >> >> quota is now being respected properly. >> > >> > Again, you need to post the commands and output that demonstrate the >> > problem you are describing. I'm having trouble understanding what >> > your problem is because I can't see the raw output that you are >> > seeing... >> > >> > Cheers, >> > >> > Dave. >> > -- >> > Dave Chinner >> > david@fromorbit.com >> >> _______________________________________________ >> xfs mailing list >> xfs@oss.sgi.com >> http://oss.sgi.com/mailman/listinfo/xfs _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: (Un)known issue with XFS group quotas on LVM? 2014-06-13 17:00 ` John Desantis @ 2014-06-13 17:59 ` Brian Foster 2014-06-13 18:21 ` John Desantis 0 siblings, 1 reply; 9+ messages in thread From: Brian Foster @ 2014-06-13 17:59 UTC (permalink / raw) To: John Desantis; +Cc: xfs On Fri, Jun 13, 2014 at 01:00:41PM -0400, John Desantis wrote: > Brian, > > > This all seems like normal behavior to me. What gives you the impression > > that you are hitting the 10G quota? The fact that the quota is reset and > > writing continues? Note that dd will truncate a file before it starts > > writing unless notrunc is specified. > > Yes, I am aware of the truncating, but not that particular option - thanks! > > I had experienced the "issue" when a user quota was set while the > group quota was unlimited. See below. > > xfs_quota> limit -u bsoft=5G bhard=5G desantis.admin > > xfs_quota> quota -uh desantis.admin > Disk quotas for User desantis.admin (663800172) > Filesystem Blocks Quota Limit Warn/Time Mounted on > /dev/mapper/vg_drbd-shares > 256K 5G 5G 00 [------] /export/shares > > xfs_quota> limit -g bsoft=unlimited bhard=unlimited rcadm > > xfs_quota> quota -gh rcadm > Disk quotas for Group rcadm (1034) > Filesystem Blocks Quota Limit Warn/Time Mounted on > /dev/mapper/vg_drbd-shares > 43.5G 16E 16E 00 [------] /export/shares > > Ok, so a 5G user quota and effectively unlimited group quota. > # sudo -u desantis.admin -g rcadm dd if=/dev/zero > of=/export/shares/rcadm/testfile_morethan5G bs=1024M count=10 > dd: writing `/export/shares/rcadm/testfile_morethan5G': Disk quota exceeded > 5+0 records in > 4+0 records out > 5368442880 bytes (5.4 GB) copied, 10.5371 s, 509 MB/s > And you can write to the 5G quota. > xfs_quota> quota -uh desantis.admin > Disk quotas for User desantis.admin (663800172) > Filesystem Blocks Quota Limit Warn/Time Mounted on > /dev/mapper/vg_drbd-shares > 5.0G 5G 5G 00 [------] /export/shares > > > xfs_quota> quota -gh rcadm > Disk quotas for Group rcadm (1034) > Filesystem Blocks Quota Limit Warn/Time Mounted on > /dev/mapper/vg_drbd-shares > 48.5G 16E 16E 00 [------] /export/shares > > As you can see, although the group quota did augment 5 GB, the user > quota was enforced over the group quota. This is what led to my > confusion. I had thought that the group quota would be respected over > the user quota because I was specifying the group for the file being > created. > Those are tracking how many blocks are accounted against each quota. As the output shows, the 5G from this file is being accounted against both. This means that for an allocation on this file to succeed, both quotas must have enough room for the allocation. It sounds like the misunderstanding is viewing a quota as an "allowance" rather than a "limit?" In the scenario above, the group quota effectively becomes a no-op. You'll never hit that limit, but that has no effect on the enforcement of other quotas against which a particular file might also be accounted. For example, if you were to set a new project quota of 1GB on the root directory of this mount, you won't have the ability to allocate any more than 1GB regardless of the user/group of the underlying files. Brian > John DeSantis > > > > 2014-06-13 12:22 GMT-04:00 Brian Foster <bfoster@redhat.com>: > > On Fri, Jun 13, 2014 at 10:50:19AM -0400, John Desantis wrote: > >> Dave, > >> > >> > Order doesn't matter because the filesystem is supposed to check > >> > against all enforced quotas, not just the first one. > >> > >> Thanks for confirming. > >> > >> > Again, you need to post the commands and output that demonstrate the > >> > problem you are describing. I'm having trouble understanding what > >> > your problem is because I can't see the raw output that you are > >> > seeing... > >> > >> Ok, here are the default user and group quotas of my test account: > >> > >> xfs_quota> quota -uh desantis.admin > >> Disk quotas for User desantis.admin (663800172) > >> Filesystem Blocks Quota Limit Warn/Time Mounted on > >> /dev/mapper/vg_drbd-shares > >> 256K 0 0 00 [------] /export/shares > >> > >> xfs_quota> quota -gh rcadm > >> Disk quotas for Group rcadm (1034) > >> Filesystem Blocks Quota Limit Warn/Time Mounted on > >> /dev/mapper/vg_drbd-shares > >> 43.5G 0 0 00 [------] /export/shares > >> > >> Now, we'll limit them and then re-display the quotas: > >> > >> xfs_quota> limit -g bsoft=50G bhard=50G rcadm > >> xfs_quota> limit -u bsoft=10G bhard=10G desantis.admin > >> > >> xfs_quota> quota -uh desantis.admin > >> Disk quotas for User desantis.admin (663800172) > >> Filesystem Blocks Quota Limit Warn/Time Mounted on > >> /dev/mapper/vg_drbd-shares > >> 256K 10G 10G 00 [------] /export/shares > >> > >> xfs_quota> quota -gh rcadm > >> Disk quotas for Group rcadm (1034) > >> Filesystem Blocks Quota Limit Warn/Time Mounted on > >> /dev/mapper/vg_drbd-shares > >> 43.5G 50G 50G 00 [------] /export/shares > >> > > > > So you've got 10G quota with about as much available and a 50G quota > > with ~6.5GB available. > > > >> Here come the tests: > >> > >> # sudo -u desantis.admin -g rcadm dd if=/dev/zero > >> of=/export/shares/rcadm/testfile_1 bs=1024M count=11 > >> dd: writing `/export/shares/rcadm/testfile_1': Disk quota exceeded > >> 7+0 records in > >> 6+0 records out > >> 6933651456 bytes (6.9 GB) copied, 19.7811 s, 351 MB/s > >> > > > > It looks like you've hit the 50G quota. > > > >> xfs_quota> quota -uh desantis.admin > >> Disk quotas for User desantis.admin (663800172) > >> Filesystem Blocks Quota Limit Warn/Time Mounted on > >> /dev/mapper/vg_drbd-shares > >> 6.5G 10G 10G 00 [------] /export/shares > >> > >> xfs_quota> limit -u bsoft=0 bhard=0 desantis.admin > >> > > > > Now the 10G quota is gone. > > > >> xfs_quota> quota -uh desantis.admin > >> Disk quotas for User desantis.admin (663800172) > >> Filesystem Blocks Quota Limit Warn/Time Mounted on > >> /dev/mapper/vg_drbd-shares > >> 6.5G 0 0 00 [------] /export/shares > >> > >> # sudo -u desantis.admin -g rcadm dd if=/dev/zero > >> of=/export/shares/rcadm/testfile_1 bs=1024M count=5 > >> 5+0 records in > >> 5+0 records out > >> 5368709120 bytes (5.4 GB) copied, 10.8275 s, 496 MB/s > >> > > > > You overwrite the previously written 6.5G file to 5.4G (dd will > > truncate). > > > >> # sudo -u desantis.admin -g rcadm dd if=/dev/zero > >> of=/export/shares/rcadm/testfile_2 bs=1024M count=5 > >> dd: writing `/export/shares/rcadm/testfile_2': Disk quota exceeded > >> 1+0 records in > >> 0+0 records out > >> 1028136960 bytes (1.0 GB) copied, 4.60859 s, 223 MB/s > >> > > > > Then create a new file and run into the 50G limit again. > > > >> xfs_quota> quota -gh rcadm > >> Disk quotas for Group rcadm (1034) > >> Filesystem Blocks Quota Limit Warn/Time Mounted on > >> /dev/mapper/vg_drbd-shares > >> 50.0G 50G 50G 00 [------] /export/shares > >> > >> > >> As you can see with the output, my user quota of 10 GB was enforced > >> over the higher group quota of 50 GB. My original thought process was > >> that since the file being created was specified for the group with the > >> larger quota, that XFS would enforce that limit over the user quota. > >> After reading over several pieces of quota documentation (RTFM!) from > >> different O.S. distributions, it looks like you can either have a user > >> or group quota getting enforced, but not both in the sense I was > >> originally thinking. > >> > > > > This all seems like normal behavior to me. What gives you the impression > > that you are hitting the 10G quota? The fact that the quota is reset and > > writing continues? Note that dd will truncate a file before it starts > > writing unless notrunc is specified. > > > > Brian > > > >> That being said, I really do appreciate the responses and extra set of > >> eyes and apologize if I've wasted too much time. > >> > >> John DeSantis > >> > >> 2014-06-12 20:01 GMT-04:00 Dave Chinner <david@fromorbit.com>: > >> > On Thu, Jun 12, 2014 at 10:04:29AM -0400, John Desantis wrote: > >> >> Dave, > >> >> > >> >> Thanks for your reply. Let me address your questions and replies. > >> >> > >> >> > That doesn't sound like a filesystem problem. Sounds more like the > >> >> > remote user/group configuration is getting squashed. Do the systems > >> >> > have the same user/group configurations, or are the clients sending > >> >> > group IDs the server doesn't recognise? > >> >> > >> >> Stracing shows that the group and user ID's are not being squashed and > >> >> they are resolving each time to their respective UID/GID's. > >> >> > >> >> > As it is, describing the problem in text doesn't tell me anything > >> >> > about the problem. You need to cut-n-paste the CLI and the output > >> >> > showing the incorrect behaviour so we can attempt to reproduce the > >> >> > same behaviour you are seeing. > >> >> > >> >> Busted. I figured before sending the usual gamut of console "action", > >> >> I'd "save" time by simply presenting the symptoms. > >> >> > >> >> > I can't see how that is related to the filesystem - the filesystem > >> >> > will either account for group quota usage or it won't. It doesn't > >> >> > care what primary gid the process uses, it will simply account usage > >> >> > to that... > >> >> > >> >> You're correct (but you already knew this!). I can tell you that > >> >> running the quota report does show the group block count changing. > >> >> > >> >> It looks like this "issue" is my misunderstanding of how XFS is > >> >> checking the quota. Perhaps you could shed some light? > >> >> > >> >> When you're using active user and group quotas on the same file system > >> >> and the user quota is lower than the group quota, does XFS first check > >> >> against the user quota and then the group quota? Is the order of > >> >> operations user, group, and then project or does it depend on the > >> >> mounting order of quotas? > >> > > >> > Order doesn't matter because the filesystem is supposed to check > >> > against all enforced quotas, not just the first one. > >> > > >> >> In the case I presented yesterday, the user in question had a quota of > >> >> 25 GB, but the group had an unlimited quota. Each time that I used > >> >> sudo -u user -g group or logged into the user's account, the quota was > >> >> always exceeded. Now that I have removed the user's quota, the group > >> >> quota is now being respected properly. > >> > > >> > Again, you need to post the commands and output that demonstrate the > >> > problem you are describing. I'm having trouble understanding what > >> > your problem is because I can't see the raw output that you are > >> > seeing... > >> > > >> > Cheers, > >> > > >> > Dave. > >> > -- > >> > Dave Chinner > >> > david@fromorbit.com > >> > >> _______________________________________________ > >> xfs mailing list > >> xfs@oss.sgi.com > >> http://oss.sgi.com/mailman/listinfo/xfs > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: (Un)known issue with XFS group quotas on LVM? 2014-06-13 17:59 ` Brian Foster @ 2014-06-13 18:21 ` John Desantis 0 siblings, 0 replies; 9+ messages in thread From: John Desantis @ 2014-06-13 18:21 UTC (permalink / raw) To: Brian Foster; +Cc: xfs Brian, > Those are tracking how many blocks are accounted against each quota. As > the output shows, the 5G from this file is being accounted against both. > This means that for an allocation on this file to succeed, both quotas > must have enough room for the allocation. This is what I had did earlier to satisfy users while the confusion lingered. I basically augmented whichever the smaller quota was. > It sounds like the misunderstanding is viewing a quota as an "allowance" > rather than a "limit?" In the scenario above, the group quota > effectively becomes a no-op. You'll never hit that limit, but that has > no effect on the enforcement of other quotas against which a particular > file might also be accounted. > > For example, if you were to set a new project quota of 1GB on the root > directory of this mount, you won't have the ability to allocate any more > than 1GB regardless of the user/group of the underlying files. I do believe this is from where my confusion was stemming. A big thanks to you and Dave for clearing this up! John DeSantis 2014-06-13 13:59 GMT-04:00 Brian Foster <bfoster@redhat.com>: > On Fri, Jun 13, 2014 at 01:00:41PM -0400, John Desantis wrote: >> Brian, >> >> > This all seems like normal behavior to me. What gives you the impression >> > that you are hitting the 10G quota? The fact that the quota is reset and >> > writing continues? Note that dd will truncate a file before it starts >> > writing unless notrunc is specified. >> >> Yes, I am aware of the truncating, but not that particular option - thanks! >> >> I had experienced the "issue" when a user quota was set while the >> group quota was unlimited. See below. >> >> xfs_quota> limit -u bsoft=5G bhard=5G desantis.admin >> >> xfs_quota> quota -uh desantis.admin >> Disk quotas for User desantis.admin (663800172) >> Filesystem Blocks Quota Limit Warn/Time Mounted on >> /dev/mapper/vg_drbd-shares >> 256K 5G 5G 00 [------] /export/shares >> >> xfs_quota> limit -g bsoft=unlimited bhard=unlimited rcadm >> >> xfs_quota> quota -gh rcadm >> Disk quotas for Group rcadm (1034) >> Filesystem Blocks Quota Limit Warn/Time Mounted on >> /dev/mapper/vg_drbd-shares >> 43.5G 16E 16E 00 [------] /export/shares >> >> > > Ok, so a 5G user quota and effectively unlimited group quota. > >> # sudo -u desantis.admin -g rcadm dd if=/dev/zero >> of=/export/shares/rcadm/testfile_morethan5G bs=1024M count=10 >> dd: writing `/export/shares/rcadm/testfile_morethan5G': Disk quota exceeded >> 5+0 records in >> 4+0 records out >> 5368442880 bytes (5.4 GB) copied, 10.5371 s, 509 MB/s >> > > And you can write to the 5G quota. > >> xfs_quota> quota -uh desantis.admin >> Disk quotas for User desantis.admin (663800172) >> Filesystem Blocks Quota Limit Warn/Time Mounted on >> /dev/mapper/vg_drbd-shares >> 5.0G 5G 5G 00 [------] /export/shares >> >> >> xfs_quota> quota -gh rcadm >> Disk quotas for Group rcadm (1034) >> Filesystem Blocks Quota Limit Warn/Time Mounted on >> /dev/mapper/vg_drbd-shares >> 48.5G 16E 16E 00 [------] /export/shares >> >> As you can see, although the group quota did augment 5 GB, the user >> quota was enforced over the group quota. This is what led to my >> confusion. I had thought that the group quota would be respected over >> the user quota because I was specifying the group for the file being >> created. >> > > Those are tracking how many blocks are accounted against each quota. As > the output shows, the 5G from this file is being accounted against both. > This means that for an allocation on this file to succeed, both quotas > must have enough room for the allocation. > > It sounds like the misunderstanding is viewing a quota as an "allowance" > rather than a "limit?" In the scenario above, the group quota > effectively becomes a no-op. You'll never hit that limit, but that has > no effect on the enforcement of other quotas against which a particular > file might also be accounted. > > For example, if you were to set a new project quota of 1GB on the root > directory of this mount, you won't have the ability to allocate any more > than 1GB regardless of the user/group of the underlying files. > > Brian > >> John DeSantis >> >> >> >> 2014-06-13 12:22 GMT-04:00 Brian Foster <bfoster@redhat.com>: >> > On Fri, Jun 13, 2014 at 10:50:19AM -0400, John Desantis wrote: >> >> Dave, >> >> >> >> > Order doesn't matter because the filesystem is supposed to check >> >> > against all enforced quotas, not just the first one. >> >> >> >> Thanks for confirming. >> >> >> >> > Again, you need to post the commands and output that demonstrate the >> >> > problem you are describing. I'm having trouble understanding what >> >> > your problem is because I can't see the raw output that you are >> >> > seeing... >> >> >> >> Ok, here are the default user and group quotas of my test account: >> >> >> >> xfs_quota> quota -uh desantis.admin >> >> Disk quotas for User desantis.admin (663800172) >> >> Filesystem Blocks Quota Limit Warn/Time Mounted on >> >> /dev/mapper/vg_drbd-shares >> >> 256K 0 0 00 [------] /export/shares >> >> >> >> xfs_quota> quota -gh rcadm >> >> Disk quotas for Group rcadm (1034) >> >> Filesystem Blocks Quota Limit Warn/Time Mounted on >> >> /dev/mapper/vg_drbd-shares >> >> 43.5G 0 0 00 [------] /export/shares >> >> >> >> Now, we'll limit them and then re-display the quotas: >> >> >> >> xfs_quota> limit -g bsoft=50G bhard=50G rcadm >> >> xfs_quota> limit -u bsoft=10G bhard=10G desantis.admin >> >> >> >> xfs_quota> quota -uh desantis.admin >> >> Disk quotas for User desantis.admin (663800172) >> >> Filesystem Blocks Quota Limit Warn/Time Mounted on >> >> /dev/mapper/vg_drbd-shares >> >> 256K 10G 10G 00 [------] /export/shares >> >> >> >> xfs_quota> quota -gh rcadm >> >> Disk quotas for Group rcadm (1034) >> >> Filesystem Blocks Quota Limit Warn/Time Mounted on >> >> /dev/mapper/vg_drbd-shares >> >> 43.5G 50G 50G 00 [------] /export/shares >> >> >> > >> > So you've got 10G quota with about as much available and a 50G quota >> > with ~6.5GB available. >> > >> >> Here come the tests: >> >> >> >> # sudo -u desantis.admin -g rcadm dd if=/dev/zero >> >> of=/export/shares/rcadm/testfile_1 bs=1024M count=11 >> >> dd: writing `/export/shares/rcadm/testfile_1': Disk quota exceeded >> >> 7+0 records in >> >> 6+0 records out >> >> 6933651456 bytes (6.9 GB) copied, 19.7811 s, 351 MB/s >> >> >> > >> > It looks like you've hit the 50G quota. >> > >> >> xfs_quota> quota -uh desantis.admin >> >> Disk quotas for User desantis.admin (663800172) >> >> Filesystem Blocks Quota Limit Warn/Time Mounted on >> >> /dev/mapper/vg_drbd-shares >> >> 6.5G 10G 10G 00 [------] /export/shares >> >> >> >> xfs_quota> limit -u bsoft=0 bhard=0 desantis.admin >> >> >> > >> > Now the 10G quota is gone. >> > >> >> xfs_quota> quota -uh desantis.admin >> >> Disk quotas for User desantis.admin (663800172) >> >> Filesystem Blocks Quota Limit Warn/Time Mounted on >> >> /dev/mapper/vg_drbd-shares >> >> 6.5G 0 0 00 [------] /export/shares >> >> >> >> # sudo -u desantis.admin -g rcadm dd if=/dev/zero >> >> of=/export/shares/rcadm/testfile_1 bs=1024M count=5 >> >> 5+0 records in >> >> 5+0 records out >> >> 5368709120 bytes (5.4 GB) copied, 10.8275 s, 496 MB/s >> >> >> > >> > You overwrite the previously written 6.5G file to 5.4G (dd will >> > truncate). >> > >> >> # sudo -u desantis.admin -g rcadm dd if=/dev/zero >> >> of=/export/shares/rcadm/testfile_2 bs=1024M count=5 >> >> dd: writing `/export/shares/rcadm/testfile_2': Disk quota exceeded >> >> 1+0 records in >> >> 0+0 records out >> >> 1028136960 bytes (1.0 GB) copied, 4.60859 s, 223 MB/s >> >> >> > >> > Then create a new file and run into the 50G limit again. >> > >> >> xfs_quota> quota -gh rcadm >> >> Disk quotas for Group rcadm (1034) >> >> Filesystem Blocks Quota Limit Warn/Time Mounted on >> >> /dev/mapper/vg_drbd-shares >> >> 50.0G 50G 50G 00 [------] /export/shares >> >> >> >> >> >> As you can see with the output, my user quota of 10 GB was enforced >> >> over the higher group quota of 50 GB. My original thought process was >> >> that since the file being created was specified for the group with the >> >> larger quota, that XFS would enforce that limit over the user quota. >> >> After reading over several pieces of quota documentation (RTFM!) from >> >> different O.S. distributions, it looks like you can either have a user >> >> or group quota getting enforced, but not both in the sense I was >> >> originally thinking. >> >> >> > >> > This all seems like normal behavior to me. What gives you the impression >> > that you are hitting the 10G quota? The fact that the quota is reset and >> > writing continues? Note that dd will truncate a file before it starts >> > writing unless notrunc is specified. >> > >> > Brian >> > >> >> That being said, I really do appreciate the responses and extra set of >> >> eyes and apologize if I've wasted too much time. >> >> >> >> John DeSantis >> >> >> >> 2014-06-12 20:01 GMT-04:00 Dave Chinner <david@fromorbit.com>: >> >> > On Thu, Jun 12, 2014 at 10:04:29AM -0400, John Desantis wrote: >> >> >> Dave, >> >> >> >> >> >> Thanks for your reply. Let me address your questions and replies. >> >> >> >> >> >> > That doesn't sound like a filesystem problem. Sounds more like the >> >> >> > remote user/group configuration is getting squashed. Do the systems >> >> >> > have the same user/group configurations, or are the clients sending >> >> >> > group IDs the server doesn't recognise? >> >> >> >> >> >> Stracing shows that the group and user ID's are not being squashed and >> >> >> they are resolving each time to their respective UID/GID's. >> >> >> >> >> >> > As it is, describing the problem in text doesn't tell me anything >> >> >> > about the problem. You need to cut-n-paste the CLI and the output >> >> >> > showing the incorrect behaviour so we can attempt to reproduce the >> >> >> > same behaviour you are seeing. >> >> >> >> >> >> Busted. I figured before sending the usual gamut of console "action", >> >> >> I'd "save" time by simply presenting the symptoms. >> >> >> >> >> >> > I can't see how that is related to the filesystem - the filesystem >> >> >> > will either account for group quota usage or it won't. It doesn't >> >> >> > care what primary gid the process uses, it will simply account usage >> >> >> > to that... >> >> >> >> >> >> You're correct (but you already knew this!). I can tell you that >> >> >> running the quota report does show the group block count changing. >> >> >> >> >> >> It looks like this "issue" is my misunderstanding of how XFS is >> >> >> checking the quota. Perhaps you could shed some light? >> >> >> >> >> >> When you're using active user and group quotas on the same file system >> >> >> and the user quota is lower than the group quota, does XFS first check >> >> >> against the user quota and then the group quota? Is the order of >> >> >> operations user, group, and then project or does it depend on the >> >> >> mounting order of quotas? >> >> > >> >> > Order doesn't matter because the filesystem is supposed to check >> >> > against all enforced quotas, not just the first one. >> >> > >> >> >> In the case I presented yesterday, the user in question had a quota of >> >> >> 25 GB, but the group had an unlimited quota. Each time that I used >> >> >> sudo -u user -g group or logged into the user's account, the quota was >> >> >> always exceeded. Now that I have removed the user's quota, the group >> >> >> quota is now being respected properly. >> >> > >> >> > Again, you need to post the commands and output that demonstrate the >> >> > problem you are describing. I'm having trouble understanding what >> >> > your problem is because I can't see the raw output that you are >> >> > seeing... >> >> > >> >> > Cheers, >> >> > >> >> > Dave. >> >> > -- >> >> > Dave Chinner >> >> > david@fromorbit.com >> >> >> >> _______________________________________________ >> >> xfs mailing list >> >> xfs@oss.sgi.com >> >> http://oss.sgi.com/mailman/listinfo/xfs >> >> _______________________________________________ >> xfs mailing list >> xfs@oss.sgi.com >> http://oss.sgi.com/mailman/listinfo/xfs _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2014-06-13 18:21 UTC | newest] Thread overview: 9+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2014-06-11 15:43 (Un)known issue with XFS group quotas on LVM? John Desantis 2014-06-11 23:07 ` Dave Chinner 2014-06-12 14:04 ` John Desantis 2014-06-13 0:01 ` Dave Chinner 2014-06-13 14:50 ` John Desantis 2014-06-13 16:22 ` Brian Foster 2014-06-13 17:00 ` John Desantis 2014-06-13 17:59 ` Brian Foster 2014-06-13 18:21 ` John Desantis
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox