Common error in case of running out of the number of ACL entries

Common error in case of running out of the number of ACL entries

[Jeff Liu]

Hi Folks,

While looking into the behaviour of XFS in case of running out of the
maximum number of supported acl entries, I observed that we would
return various different errors in this situation.  As per Christoph's
suggestion, I gathered up them from some common file systems with a
simple tests which were shown as following:

Btrfs: No space left on device (ENOSPC)
Ext3: No space left on device (ENOSPC)
Ext4: No space left oo long (E2BIG)
F2fs: Numerical result out of range (ERANGE)
OCFS2: Argument list too long (E2BIG)
XFS: Invalid argument (EINVAL)

It seems that return either above error would mislead the end user, though
Eric's Sandeen once pointed out that ENOSPC should be used in this case in
terms of the setxattr(2) man page.
<quote>
       If there is insufficient space remaining to store the extended attribute,
       errno is set to either ENOSPC, or EDQUOT if quota enforcement was the cause.

However, the "insufficient space" is obscure, it might be the file system
space or no more space in metadata blocks, etc...

Hence, should we consolidate this scenario to figure out a more clearer
common error?


Thanks,
-Jeff

Test script:
#!/bin/bash

testfile=$1
touch $testfile
i=0
while :
do
        setfacl -m u:$i:rw- $testfile
        let i+=1
done

Re: Common error in case of running out of the number of ACL entries

[Jeff Liu]

On 12/13 2013 22:43 PM, Jeff Liu wrote:
> Hi Folks,
> 
> While looking into the behaviour of XFS in case of running out of the
> maximum number of supported acl entries, I observed that we would
> return various different errors in this situation.  As per Christoph's
> suggestion, I gathered up them from some common file systems with a
> simple tests which were shown as following:
> 
> Btrfs: No space left on device (ENOSPC)
> Ext3: No space left on device (ENOSPC)
> Ext4: No space left oo long (E2BIG)
Sorry, copy-to-paste error. :)

Same to Ext3, Ext4 also return ENOSPC.  Instead, JFS would return E2BIG
as "Argument list too long".  Also, I'm willing to collect further more
info for other file systems resides at fs/ as far as I can if required.

> F2fs: Numerical result out of range (ERANGE)
> OCFS2: Argument list too long (E2BIG)
> XFS: Invalid argument (EINVAL)
> 
> It seems that return either above error would mislead the end user, though
> Eric's Sandeen once pointed out that ENOSPC should be used in this case in
> terms of the setxattr(2) man page.
> <quote>
>        If there is insufficient space remaining to store the extended attribute,
>        errno is set to either ENOSPC, or EDQUOT if quota enforcement was the cause.
> 
> However, the "insufficient space" is obscure, it might be the file system
> space or no more space in metadata blocks, etc...
> 
> Hence, should we consolidate this scenario to figure out a more clearer
> common error?

Thanks,
-Jeff

Re: Common error in case of running out of the number of ACL entries

[Carlos Maiolino]

Hi,

> Hi Folks,
> 
> While looking into the behaviour of XFS in case of running out of the
> maximum number of supported acl entries, I observed that we would
> return various different errors in this situation.  As per Christoph's
> suggestion, I gathered up them from some common file systems with a
> simple tests which were shown as following:
> 
> Btrfs: No space left on device (ENOSPC)
> Ext3: No space left on device (ENOSPC)
> Ext4: No space left oo long (E2BIG)
> F2fs: Numerical result out of range (ERANGE)
> OCFS2: Argument list too long (E2BIG)
> XFS: Invalid argument (EINVAL)
> 
> It seems that return either above error would mislead the end user, though
> Eric's Sandeen once pointed out that ENOSPC should be used in this case in
> terms of the setxattr(2) man page.
> <quote>
>        If there is insufficient space remaining to store the extended attribute,
>        errno is set to either ENOSPC, or EDQUOT if quota enforcement was the cause.
> 
> However, the "insufficient space" is obscure, it might be the file system
> space or no more space in metadata blocks, etc...
> 
> Hence, should we consolidate this scenario to figure out a more clearer
> common error?
> 
> 
Consolidate the erros here would make things much more easier for
users/applications which has a heavy usage of xattrs, for example, glusterfs.
But, I still think that -ENOSPC or -EDQUOT are the best and easier approaches
here. Whether it's FS space or metadata blocks, both of them are lack of space,
which properly describes why a tentative to add a new xattr failed.
At least for me, this looks much easier than add a new error flag and make all
applications using xattrs to change their code to fit new behavior (or maybe
they'll need to change it anyway to fit -ENOSPC :).

-- 
Carlos

Re: Common error in case of running out of the number of ACL entries

[Dave Chinner]

On Fri, Dec 13, 2013 at 02:16:54PM -0200, Carlos Maiolino wrote:
> Hi,
> 
> > Hi Folks,
> > 
> > While looking into the behaviour of XFS in case of running out of the
> > maximum number of supported acl entries, I observed that we would
> > return various different errors in this situation.  As per Christoph's
> > suggestion, I gathered up them from some common file systems with a
> > simple tests which were shown as following:
> > 
> > Btrfs: No space left on device (ENOSPC)
> > Ext3: No space left on device (ENOSPC)
> > Ext4: No space left oo long (E2BIG)
> > F2fs: Numerical result out of range (ERANGE)
> > OCFS2: Argument list too long (E2BIG)
> > XFS: Invalid argument (EINVAL)
> > 
> > It seems that return either above error would mislead the end user, though
> > Eric's Sandeen once pointed out that ENOSPC should be used in this case in
> > terms of the setxattr(2) man page.
> > <quote>
> >        If there is insufficient space remaining to store the extended attribute,
> >        errno is set to either ENOSPC, or EDQUOT if quota enforcement was the cause.
> > 
> > However, the "insufficient space" is obscure, it might be the file system
> > space or no more space in metadata blocks, etc...
> > 
> > Hence, should we consolidate this scenario to figure out a more clearer
> > common error?
> > 
> > 
> Consolidate the erros here would make things much more easier for
> users/applications which has a heavy usage of xattrs, for example, glusterfs.
> But, I still think that -ENOSPC or -EDQUOT are the best and easier approaches
> here. Whether it's FS space or metadata blocks, both of them are lack of space,
> which properly describes why a tentative to add a new xattr failed.
> At least for me, this looks much easier than add a new error flag and make all
> applications using xattrs to change their code to fit new behavior (or maybe
> they'll need to change it anyway to fit -ENOSPC :).

This isn't an "out-of-space" error where there are no more
resources available to create the ACL. Therefore, ENOSPC is not an
appropriate error.

What we are hitting a maximum size limit defined by XATTR
implemenation used to store the ACLs. Hence the error is equivalent
to a user trying to set an xattr of larger than XATTR_SIZE_MAX. In
that case, we have:

	setxattr: E2BIG
	getxattr: E2BIG
	xfs_attrmulti_attr_get: EINVAL
	xfs_attrmulti_attr_set: EINVAL

So, what we see here is that XFS returning EINVAL for too many ACLs
is internally consistent with it's ioctl-based xattr interfaces, but
only JFS and OCFS2 are internally consistent with the VFS xattr
code.

Hence, IMO, the correct thing to do here is make everything
consistent with the VFS [gs]etxattr calls and return E2BIG when the
xattr that holds the ACLs grows too large to fit within the size
limits bounded by the xattr implementation....

Cheers,

Dave.

> 
> -- 
> Carlos
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

-- 
Dave Chinner
david@fromorbit.com

Re: Common error in case of running out of the number of ACL entries

[Jeff Liu]

On 12/14 2013 07:51 AM, Dave Chinner wrote:
> On Fri, Dec 13, 2013 at 02:16:54PM -0200, Carlos Maiolino wrote:
>> Hi,
>>
>>> Hi Folks,
>>>
>>> While looking into the behaviour of XFS in case of running out of the
>>> maximum number of supported acl entries, I observed that we would
>>> return various different errors in this situation.  As per Christoph's
>>> suggestion, I gathered up them from some common file systems with a
>>> simple tests which were shown as following:
>>>
>>> Btrfs: No space left on device (ENOSPC)
>>> Ext3: No space left on device (ENOSPC)
>>> Ext4: No space left oo long (E2BIG)
>>> F2fs: Numerical result out of range (ERANGE)
>>> OCFS2: Argument list too long (E2BIG)
>>> XFS: Invalid argument (EINVAL)
>>>
>>> It seems that return either above error would mislead the end user, though
>>> Eric's Sandeen once pointed out that ENOSPC should be used in this case in
>>> terms of the setxattr(2) man page.
>>> <quote>
>>>        If there is insufficient space remaining to store the extended attribute,
>>>        errno is set to either ENOSPC, or EDQUOT if quota enforcement was the cause.
>>>
>>> However, the "insufficient space" is obscure, it might be the file system
>>> space or no more space in metadata blocks, etc...
>>>
>>> Hence, should we consolidate this scenario to figure out a more clearer
>>> common error?
>>>
>>>
>> Consolidate the erros here would make things much more easier for
>> users/applications which has a heavy usage of xattrs, for example, glusterfs.
>> But, I still think that -ENOSPC or -EDQUOT are the best and easier approaches
>> here. Whether it's FS space or metadata blocks, both of them are lack of space,
>> which properly describes why a tentative to add a new xattr failed.
>> At least for me, this looks much easier than add a new error flag and make all
>> applications using xattrs to change their code to fit new behavior (or maybe
>> they'll need to change it anyway to fit -ENOSPC :).
Yep, we do need an easier approach to unify this situation. :)
> 
> This isn't an "out-of-space" error where there are no more
> resources available to create the ACL. Therefore, ENOSPC is not an
> appropriate error.
> 
> What we are hitting a maximum size limit defined by XATTR
> implemenation used to store the ACLs. Hence the error is equivalent
> to a user trying to set an xattr of larger than XATTR_SIZE_MAX. In
> that case, we have:
> 
> 	setxattr: E2BIG
> 	getxattr: E2BIG
> 	xfs_attrmulti_attr_get: EINVAL
> 	xfs_attrmulti_attr_set: EINVAL
> 
> So, what we see here is that XFS returning EINVAL for too many ACLs
> is internally consistent with it's ioctl-based xattr interfaces, but
> only JFS and OCFS2 are internally consistent with the VFS xattr
> code.
> 
> Hence, IMO, the correct thing to do here is make everything
> consistent with the VFS [gs]etxattr calls and return E2BIG when the
> xattr that holds the ACLs grows too large to fit within the size
> limits bounded by the xattr implementation....
That's would be fine with E2BIG in terms of the semantics.  I played around
with some other file systems with posix acl support, except 9p as well as
JFFS2 as they are not works as expected in my test environment.

Below is the updated results, well, it looks reiserfs is also consistent
with VFS calls. :)

Btrfs: No space left on device (ENOSPC)
Ext2: No space left on device (ENOSPC)
Ext3: No space left on device (ENOSPC)
Ext4: No space left on device (ENOSPC)
F2fs: Numerical result out of range (ERANGE)
GFS2: Invalid Argument (EINVAL)
HFSPlus: Cannot allocate memory (ENOMEM))
JFS: Argument list too long (E2BIG)
OCFS2: Argument list too long (E2BIG)
UBIFS: No space left on device (ENOSPC)
Reiserfs: Argument list too long (E2BIG)
XFS: Invalid argument (EINVAL)
NFS: Invalid argument (EINVAL)


Thanks,
-Jeff