Rambling noise #2: Learning to use the v8 pquota/uquota patchset

Rambling noise #2: Learning to use the v8 pquota/uquota patchset

[Michael L. Semon]

Hi!  I seem to have no luck in getting v8 of the pquota/uquota patchset 
working and have it pass xfstests with flying colors.  Is v8 of the 
pquota/gquota patchset sufficient to make the new separate pquota/gquota 
bits work?  Or is it an incremental patchset?

The basic functionality works, in my opinion, and I hope nobody wastes 
time with a nice, educated reply.  It would be mostly wasted on me and 
is better saved for somebody else.  A reply of "do this...and 
this...btw, how did this test come out?" would be welcomed, though ;-)

Anyway, some vague observations as I grasp for straws...

1) The xfstests quota group tests seem to fail in different ways than 
the way they did before applying the patches.

2) Nothing has oopsed.

3) In testing using the `xfs_quota -x` command, the patches seem to 
work.  On `mount -t xfs -o gquota` mounts, using the quota command from 
within the xfs_quota shell, the group quotas show but not the projid 
quotas.  On `mount -t xfs -o pquota` mounts, the projid quotas show but 
not the gquota mounts.  This is different than the old behavior, where 
the gquota numbers might be recycled into projid numbers.

4) The results of `xfsquota -c print` are confusing.  Maybe they're 
showing the XFS view when they show things like 
'uqnoenforce,gquota,pquota' for a mount that is gquota only.  They're 
doubly confusing once /etc/projid and /etc/projects have been set up. 
The 'gqnoenforce' and 'pqnoenforce' flags show up at times for reasons 
that are unknown to me.

5) `mount -t xfs -o gquota,pquota` is not possible at this time.

6) The patches applied cleanly to a git Linux 3.10-rc1 kernel + xfs-oss, 
with only whitespace errors reported.

7) I question whether 'bsoft=' has a visible effect on projid quotas, 
whether using your patches or not.  Did it ever work?

8) I had no feel on whether the filesystem had to be mounted once as 
gquota, then once as pquota, for the full dual functionality to work.

9) It looks like xfs_repair doesn't ruin anything, but the `xfsquota -c 
print` output looks a little different on the next mount.

That's about all that could be put together in a coherent manner.  Sleep 
awaits.

The PC is a 32-bit Pentium 4.  In addition to the kernel mentioned in 
(6), there are a few J. Liu and Dave Chinner patches applied as well.

Best of luck!

Michael

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Rambling noise #2: Learning to use the v8 pquota/uquota patchset

[Jeff Liu]

Hi Michael,

Have you tried this patch set with the v2 user space changes?

Looks this is the latest update from Chandra that I can searched from
the mailing list:
http://oss.sgi.com/archives/xfs/2012-07/msg00293.html

Thanks,
-Jeff
On 05/14/2013 04:52 PM, Michael L. Semon wrote:
> Hi!  I seem to have no luck in getting v8 of the pquota/uquota patchset 
> working and have it pass xfstests with flying colors.  Is v8 of the 
> pquota/gquota patchset sufficient to make the new separate pquota/gquota 
> bits work?  Or is it an incremental patchset?
> 
> The basic functionality works, in my opinion, and I hope nobody wastes 
> time with a nice, educated reply.  It would be mostly wasted on me and 
> is better saved for somebody else.  A reply of "do this...and 
> this...btw, how did this test come out?" would be welcomed, though ;-)
> 
> Anyway, some vague observations as I grasp for straws...
> 
> 1) The xfstests quota group tests seem to fail in different ways than 
> the way they did before applying the patches.
> 
> 2) Nothing has oopsed.
> 
> 3) In testing using the `xfs_quota -x` command, the patches seem to 
> work.  On `mount -t xfs -o gquota` mounts, using the quota command from 
> within the xfs_quota shell, the group quotas show but not the projid 
> quotas.  On `mount -t xfs -o pquota` mounts, the projid quotas show but 
> not the gquota mounts.  This is different than the old behavior, where 
> the gquota numbers might be recycled into projid numbers.
> 
> 4) The results of `xfsquota -c print` are confusing.  Maybe they're 
> showing the XFS view when they show things like 
> 'uqnoenforce,gquota,pquota' for a mount that is gquota only.  They're 
> doubly confusing once /etc/projid and /etc/projects have been set up. 
> The 'gqnoenforce' and 'pqnoenforce' flags show up at times for reasons 
> that are unknown to me.
> 
> 5) `mount -t xfs -o gquota,pquota` is not possible at this time.
> 
> 6) The patches applied cleanly to a git Linux 3.10-rc1 kernel + xfs-oss, 
> with only whitespace errors reported.
> 
> 7) I question whether 'bsoft=' has a visible effect on projid quotas, 
> whether using your patches or not.  Did it ever work?
> 
> 8) I had no feel on whether the filesystem had to be mounted once as 
> gquota, then once as pquota, for the full dual functionality to work.
> 
> 9) It looks like xfs_repair doesn't ruin anything, but the `xfsquota -c 
> print` output looks a little different on the next mount.
> 
> That's about all that could be put together in a coherent manner.  Sleep 
> awaits.
> 
> The PC is a 32-bit Pentium 4.  In addition to the kernel mentioned in 
> (6), there are a few J. Liu and Dave Chinner patches applied as well.
> 
> Best of luck!
> 
> Michael
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Rambling noise #2: Learning to use the v8 pquota/uquota patchset

[Michael L. Semon]

No.  I'll try this later on today, then post back to the group.

BTW, I'm terrible at switching between group quotas and projid quotas, 
and am aware that I'm too stupid to report what I see.  However, because 
I have a projid quota to limit the size of Samba shares used by Windows 
7 backup, I'm reporting this despite my stupidity.  I ask for everyone's 
forgiveness in advance.

Thanks for your help!

Michael

On 05/14/2013 09:52 AM, Jeff Liu wrote:
> Hi Michael,
>
> Have you tried this patch set with the v2 user space changes?
>
> Looks this is the latest update from Chandra that I can searched from
> the mailing list:
> http://oss.sgi.com/archives/xfs/2012-07/msg00293.html
>
> Thanks,
> -Jeff
> On 05/14/2013 04:52 PM, Michael L. Semon wrote:
>> Hi!  I seem to have no luck in getting v8 of the pquota/uquota patchset
>> working and have it pass xfstests with flying colors.  Is v8 of the
>> pquota/gquota patchset sufficient to make the new separate pquota/gquota
>> bits work?  Or is it an incremental patchset?
>>
>> The basic functionality works, in my opinion, and I hope nobody wastes
>> time with a nice, educated reply.  It would be mostly wasted on me and
>> is better saved for somebody else.  A reply of "do this...and
>> this...btw, how did this test come out?" would be welcomed, though ;-)
>>
>> Anyway, some vague observations as I grasp for straws...
>>
>> 1) The xfstests quota group tests seem to fail in different ways than
>> the way they did before applying the patches.
>>
>> 2) Nothing has oopsed.
>>
>> 3) In testing using the `xfs_quota -x` command, the patches seem to
>> work.  On `mount -t xfs -o gquota` mounts, using the quota command from
>> within the xfs_quota shell, the group quotas show but not the projid
>> quotas.  On `mount -t xfs -o pquota` mounts, the projid quotas show but
>> not the gquota mounts.  This is different than the old behavior, where
>> the gquota numbers might be recycled into projid numbers.
>>
>> 4) The results of `xfsquota -c print` are confusing.  Maybe they're
>> showing the XFS view when they show things like
>> 'uqnoenforce,gquota,pquota' for a mount that is gquota only.  They're
>> doubly confusing once /etc/projid and /etc/projects have been set up.
>> The 'gqnoenforce' and 'pqnoenforce' flags show up at times for reasons
>> that are unknown to me.
>>
>> 5) `mount -t xfs -o gquota,pquota` is not possible at this time.
>>
>> 6) The patches applied cleanly to a git Linux 3.10-rc1 kernel + xfs-oss,
>> with only whitespace errors reported.
>>
>> 7) I question whether 'bsoft=' has a visible effect on projid quotas,
>> whether using your patches or not.  Did it ever work?
>>
>> 8) I had no feel on whether the filesystem had to be mounted once as
>> gquota, then once as pquota, for the full dual functionality to work.
>>
>> 9) It looks like xfs_repair doesn't ruin anything, but the `xfsquota -c
>> print` output looks a little different on the next mount.
>>
>> That's about all that could be put together in a coherent manner.  Sleep
>> awaits.
>>
>> The PC is a 32-bit Pentium 4.  In addition to the kernel mentioned in
>> (6), there are a few J. Liu and Dave Chinner patches applied as well.
>>
>> Best of luck!
>>
>> Michael
>>
>> _______________________________________________
>> xfs mailing list
>> xfs@oss.sgi.com
>> http://oss.sgi.com/mailman/listinfo/xfs
>>
>

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Rambling noise #2: Learning to use the v8 pquota/uquota patchset

[Michael L. Semon]

On 05/14/2013 09:52 AM, Jeff Liu wrote:
> Hi Michael,
>
> Have you tried this patch set with the v2 user space changes?
>
> Looks this is the latest update from Chandra that I can searched from
> the mailing list:
> http://oss.sgi.com/archives/xfs/2012-07/msg00293.html
>
> Thanks,
> -Jeff

It looks like it's not patching very well in light of Dave's CRC 
userspace changes, and I botched the mkfs part of the merge.

For xfsprogs include/xfs_sb.h, Patch #1 has this:

+#define XFS_SB_VERSION2_NO_OQUOTA      0x00000100      /* sep prj quota 
inode */

However, in the current xfsprogs include/xfs_sb.h, there is this line:

#define XFS_SB_VERSION2_CRCBIT		0x00000100	/* metadata CRCs */

This overloaded meaning of 0x00000100 did not seem right, so after my 
failure with the finished mkfs.xfs, it was time to go back to the 
current git xfsprogs.  I'm sure that the userspace will be reworked 
before the kernel code is final.  I'll show some patience, and Dave's 
upcoming CRC code activation will probably force me to back Chandra's 
patchset out anyway.

Despite my bumbling through it all, the kernel gquota/pquota patchset 
shows promise.

Thanks!

Michael

P.S. - I applied your xfs_growfs_data_private patch, and all seems well 
so far.  In particular, there was no change in the test results from 
`./check -xfs -g growfs`.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Rambling noise #2: Learning to use the v8 pquota/uquota patchset

[Jeff Liu]

On 05/15/2013 03:13 PM, Michael L. Semon wrote:
> On 05/14/2013 09:52 AM, Jeff Liu wrote:
>> Hi Michael,
>>
>> Have you tried this patch set with the v2 user space changes?
>>
>> Looks this is the latest update from Chandra that I can searched from
>> the mailing list:
>> http://oss.sgi.com/archives/xfs/2012-07/msg00293.html
>>
>> Thanks,
>> -Jeff
> 
> It looks like it's not patching very well in light of Dave's CRC 
> userspace changes, and I botched the mkfs part of the merge.
Yes, that is based on the older version of xfsprogs with out CRC changes
applied.

So we have to wait until an updated version is delivered. :)

Thanks,
-Jeff
> 
> For xfsprogs include/xfs_sb.h, Patch #1 has this:
> 
> +#define XFS_SB_VERSION2_NO_OQUOTA      0x00000100      /* sep prj quota 
> inode */
> 
> However, in the current xfsprogs include/xfs_sb.h, there is this line:
> 
> #define XFS_SB_VERSION2_CRCBIT		0x00000100	/* metadata CRCs */
> 
> This overloaded meaning of 0x00000100 did not seem right, so after my 
> failure with the finished mkfs.xfs, it was time to go back to the 
> current git xfsprogs.  I'm sure that the userspace will be reworked 
> before the kernel code is final.  I'll show some patience, and Dave's 
> upcoming CRC code activation will probably force me to back Chandra's 
> patchset out anyway.
> 
> Despite my bumbling through it all, the kernel gquota/pquota patchset 
> shows promise.
> 
> Thanks!
> 
> Michael
> 
> P.S. - I applied your xfs_growfs_data_private patch, and all seems well 
> so far.  In particular, there was no change in the test results from 
> `./check -xfs -g growfs`.
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Rambling noise #2: Learning to use the v8 pquota/uquota patchset

[Chandra Seetharaman]

On Tue, 2013-05-14 at 04:52 -0400, Michael L. Semon wrote:
> Hi!  I seem to have no luck in getting v8 of the pquota/uquota patchset 
> working and have it pass xfstests with flying colors.  Is v8 of the 
> pquota/gquota patchset sufficient to make the new separate pquota/gquota 
> bits work?  Or is it an incremental patchset?

Hi Michael,

I will post the user level changes that accompanies these changes. 

With those changes you can do a new mkfs and use pquota and gquota
together.

Regards,

Chandra

> 
> The basic functionality works, in my opinion, and I hope nobody wastes 
> time with a nice, educated reply.  It would be mostly wasted on me and 
> is better saved for somebody else.  A reply of "do this...and 
> this...btw, how did this test come out?" would be welcomed, though ;-)
> 
> Anyway, some vague observations as I grasp for straws...
> 
> 1) The xfstests quota group tests seem to fail in different ways than 
> the way they did before applying the patches.
> 
> 2) Nothing has oopsed.
> 
> 3) In testing using the `xfs_quota -x` command, the patches seem to 
> work.  On `mount -t xfs -o gquota` mounts, using the quota command from 
> within the xfs_quota shell, the group quotas show but not the projid 
> quotas.  On `mount -t xfs -o pquota` mounts, the projid quotas show but 
> not the gquota mounts.  This is different than the old behavior, where 
> the gquota numbers might be recycled into projid numbers.
> 
> 4) The results of `xfsquota -c print` are confusing.  Maybe they're 
> showing the XFS view when they show things like 
> 'uqnoenforce,gquota,pquota' for a mount that is gquota only.  They're 
> doubly confusing once /etc/projid and /etc/projects have been set up. 
> The 'gqnoenforce' and 'pqnoenforce' flags show up at times for reasons 
> that are unknown to me.
> 
> 5) `mount -t xfs -o gquota,pquota` is not possible at this time.
> 
> 6) The patches applied cleanly to a git Linux 3.10-rc1 kernel + xfs-oss, 
> with only whitespace errors reported.
> 
> 7) I question whether 'bsoft=' has a visible effect on projid quotas, 
> whether using your patches or not.  Did it ever work?
> 
> 8) I had no feel on whether the filesystem had to be mounted once as 
> gquota, then once as pquota, for the full dual functionality to work.
> 
> 9) It looks like xfs_repair doesn't ruin anything, but the `xfsquota -c 
> print` output looks a little different on the next mount.
> 
> That's about all that could be put together in a coherent manner.  Sleep 
> awaits.
> 
> The PC is a 32-bit Pentium 4.  In addition to the kernel mentioned in 
> (6), there are a few J. Liu and Dave Chinner patches applied as well.
> 
> Best of luck!
> 
> Michael
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
> 


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Rambling noise #2: Learning to use the v8 pquota/uquota patchset

[Chandra Seetharaman]

On Tue, 2013-05-14 at 04:52 -0400, Michael L. Semon wrote:
Hi Michael,

Thanks for testing the code and providing feedback.

> Hi!  I seem to have no luck in getting v8 of the pquota/uquota patchset 
> working and have it pass xfstests with flying colors.  Is v8 of the 
> pquota/gquota patchset sufficient to make the new separate pquota/gquota 
> bits work?  Or is it an incremental patchset?
> 
> The basic functionality works, in my opinion, and I hope nobody wastes 
> time with a nice, educated reply.  It would be mostly wasted on me and 
> is better saved for somebody else.  A reply of "do this...and 
> this...btw, how did this test come out?" would be welcomed, though ;-)
> 
> Anyway, some vague observations as I grasp for straws...
> 
> 1) The xfstests quota group tests seem to fail in different ways than 
> the way they did before applying the patches.

I ran the xfstests before and after these changes with the old tool
chain and did not see any differences. Can you please tell me which
tests failed.
 
> 
> 2) Nothing has oopsed.
> 
> 3) In testing using the `xfs_quota -x` command, the patches seem to 
> work.  On `mount -t xfs -o gquota` mounts, using the quota command from 
> within the xfs_quota shell, the group quotas show but not the projid 
> quotas.  On `mount -t xfs -o pquota` mounts, the projid quotas show but 
> not the gquota mounts.  This is different than the old behavior, where 
> the gquota numbers might be recycled into projid numbers.

Since we use different fields in superblock now, this should be
expected.

> 
> 4) The results of `xfsquota -c print` are confusing.  Maybe they're 
> showing the XFS view when they show things like 
> 'uqnoenforce,gquota,pquota' for a mount that is gquota only.  They're 
> doubly confusing once /etc/projid and /etc/projects have been set up. 
> The 'gqnoenforce' and 'pqnoenforce' flags show up at times for reasons 
> that are unknown to me.

Can you please send me the exact commands you used. It will be good for
me to test and add it to xfstests.

> 
> 5) `mount -t xfs -o gquota,pquota` is not possible at this time.

yes. you need new xfstools to make it work.

> 
> 6) The patches applied cleanly to a git Linux 3.10-rc1 kernel + xfs-oss, 
> with only whitespace errors reported.
> 
> 7) I question whether 'bsoft=' has a visible effect on projid quotas, 
> whether using your patches or not.  Did it ever work?

There are some xfstests that do test bsoft. Let me know if you think
they are incorrect, we can revisit them.
> 
> 8) I had no feel on whether the filesystem had to be mounted once as 
> gquota, then once as pquota, for the full dual functionality to work.

No, you will be mounting filesystem as you did in (5) and it is supposed
to work (wait for the new tools).

> 
> 9) It looks like xfs_repair doesn't ruin anything, but the `xfsquota -c 
> print` output looks a little different on the next mount.

Lets retest with tool changes.

> 
> That's about all that could be put together in a coherent manner.  Sleep 
> awaits.
> 
> The PC is a 32-bit Pentium 4.  In addition to the kernel mentioned in 
> (6), there are a few J. Liu and Dave Chinner patches applied as well.
> 
> Best of luck!
> 
> Michael
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
> 


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Rambling noise #2: Learning to use the v8 pquota/uquota patchset

[Dave Chinner]

On Wed, May 15, 2013 at 09:16:20PM +0800, Jeff Liu wrote:
> On 05/15/2013 03:13 PM, Michael L. Semon wrote:
> > On 05/14/2013 09:52 AM, Jeff Liu wrote:
> >> Hi Michael,
> >>
> >> Have you tried this patch set with the v2 user space changes?
> >>
> >> Looks this is the latest update from Chandra that I can searched from
> >> the mailing list:
> >> http://oss.sgi.com/archives/xfs/2012-07/msg00293.html
> >>
> >> Thanks,
> >> -Jeff
> > 
> > It looks like it's not patching very well in light of Dave's CRC 
> > userspace changes, and I botched the mkfs part of the merge.
> Yes, that is based on the older version of xfsprogs with out CRC changes
> applied.
> 
> So we have to wait until an updated version is delivered. :)

There's a tarball that currently applies the CRC changes to the
libxfs code in userspace. I'll post a new series out later today
and you can rebase userspace on top of that.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Rambling noise #2: Learning to use the v8 pquota/uquota patchset

[Michael L. Semon]

On 05/15/2013 04:41 PM, Chandra Seetharaman wrote:
> On Tue, 2013-05-14 at 04:52 -0400, Michael L. Semon wrote:
>> Hi!  I seem to have no luck in getting v8 of the pquota/uquota patchset
>> working and have it pass xfstests with flying colors.  Is v8 of the
>> pquota/gquota patchset sufficient to make the new separate pquota/gquota
>> bits work?  Or is it an incremental patchset?
>
> Hi Michael,
>
> I will post the user level changes that accompanies these changes.
>
> With those changes you can do a new mkfs and use pquota and gquota
> together.
>
> Regards,
>
> Chandra

Excellent!

As for the rest of your issues, I'll have to retest very slowly using 
your new userspace patches, documenting every step in order.  It will 
also require before/after runs of xfstests.

I do not know what your patches do for filesystems that already had 
quota on them, so I ask you to check one thing.  It looked like the 
result of the normal `quota` command was having issues.  On my root 
filesystem that was mounted uquota only, the following is the normal 
behavior (as an ordinary user):

mls:~$ quota
Disk quotas for user mls (uid 3001):
      Filesystem  blocks   quota   limit   grace   files   quota   limit 
   grace
       /dev/sda1  165955  262144  327680            2345    8194   16384

However, before I switched out kernels here, such a command just left a 
blank, as if there was no quota there.  If you would at that to your 
list of spot-check tests, that would be great.  It might work fine for 
you.  [For the same situation, `xfs_quota -c quota /` worked just fine.]

You're leading a blind man in the dark, so I thank you for your patience.

Michael

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Rambling noise #2: Learning to use the v8 pquota/uquota patchset

[Chandra Seetharaman]

On Wed, 2013-05-15 at 17:56 -0400, Michael L. Semon wrote:
Hi Michael,

Irrespective of the user space changes, this patchset should work
properly with the current tools and existing filesystems providing
current behavior.

Let me know of the anomalies as the ones below.

Thanks

Chandra 
> On 05/15/2013 04:41 PM, Chandra Seetharaman wrote:
> > On Tue, 2013-05-14 at 04:52 -0400, Michael L. Semon wrote:
> >> Hi!  I seem to have no luck in getting v8 of the pquota/uquota patchset
> >> working and have it pass xfstests with flying colors.  Is v8 of the
> >> pquota/gquota patchset sufficient to make the new separate pquota/gquota
> >> bits work?  Or is it an incremental patchset?
> >
> > Hi Michael,
> >
> > I will post the user level changes that accompanies these changes.
> >
> > With those changes you can do a new mkfs and use pquota and gquota
> > together.
> >
> > Regards,
> >
> > Chandra
> 
> Excellent!
> 
> As for the rest of your issues, I'll have to retest very slowly using 
> your new userspace patches, documenting every step in order.  It will 
> also require before/after runs of xfstests.
> 
> I do not know what your patches do for filesystems that already had 
> quota on them, so I ask you to check one thing.  It looked like the 
> result of the normal `quota` command was having issues.  On my root 
> filesystem that was mounted uquota only, the following is the normal 
> behavior (as an ordinary user):
> 
> mls:~$ quota
> Disk quotas for user mls (uid 3001):
>       Filesystem  blocks   quota   limit   grace   files   quota   limit 
>    grace
>        /dev/sda1  165955  262144  327680            2345    8194   16384
> 
> However, before I switched out kernels here, such a command just left a 
> blank, as if there was no quota there.  If you would at that to your 
> list of spot-check tests, that would be great.  It might work fine for 
> you.  [For the same situation, `xfs_quota -c quota /` worked just fine.]
> 
> You're leading a blind man in the dark, so I thank you for your patience.
> 
> Michael
> 


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs