Re: nfs4_getfacl "Failed getxattr operation" when too many ACLentries exist

[Brian De Wolf <bldewolf@csupomona.edu>]

On 04/11/08 12:33, J. Bruce Fields wrote:
> On Thu, Apr 10, 2008 at 06:41:18PM -0400, david m. richter wrote:
>> On Thu, 10 Apr 2008, david m. richter wrote:
>>
>>> On Thu, 10 Apr 2008, Brian De Wolf wrote:
>>>
>>>> Recently we've been prototyping serving Solaris ZFS exports via NFSv4 to some
>>>> Linux hosts.  These will some day be exposed to general users, so I've been
>>>> testing things to see if I can break them.  Anyway, it seems that nfs4_getfacl
>>>> is only able to read ACLs with up to 208 entries.  nfs4_setfacl is able to
>>>> insert a 209th entry, but any attempts to view or edit the ACLs after that
>>>> fail with:
>>>>
>>>> Failed getxattr operation
>>>> : Input/output error
>>>>
>>>> There are two ways to make the ACLs readable again:
>>>> 1) Have someone log in to the Solaris box and remove some of the entries
>>>> 2) Reset the ACLs using nfs4_setfacl -s `some spec`
>>>>
>>>> Has anyone run into this issue before?  Is it fixable?  I didn't reach the
>>>> same problem locally on the Solaris box, nor on another Solaris box with the
>>>> same NFS mount, so it looks like it's a problem specific to Linux.  Here's the
>>>> versions of relevant packages on the test box running Gentoo (did I miss
>>>> any?):
>>>> Kernel: 2.6.23-gentoo-r8
>>>> nfs-utils-1.1.0-r1
>>>> attr-2.4.39
>>>> nfs4-acl-tools-0.3.2
>>> 	honestly, this probably stems from some naive, unrevisited <ahem> 
>>> assumptions still lingering nfs4-acl-tools code that need fixing.  at the 
>>> -very- least, nfs4_setfacl could save the original ACL and attempt to 
>>> restore it if the setxattr() call fails.
>> 	sorry, misread part of your letter the first time around -- it'd 
>> be very bizarre if nfs4_getfacl influenced the ACL in any way, so i 
>> suspect that something's going awry with nfs4_setfacl.  seeing such an 
>> arbitrary limit of 208 or 209 ACEs looks like the tools being dumb.
> 
> I haven't looked at this code in a while.  From a quick look.... It
> appears the kernel limits ACLs to 64K (xdr-encoded).  One ACE has length
> 
> 	16 + (length of user/group name rounded up to multiple of 4)

More or less, yes.  An strace of the ruleset "A::OWNER@:" yields a getxattr 
buffer size of 28 bytes.

> But to be hitting that limit with 208 entries I think you'd have to have
> user/group names (including domain) of about 300 characters.

Unfortunately not.  With 209 lines of "A::OWNER@:", it breaks.  208 lines of 
this makes a getxattr buffer of size 4996.  If I use "A::EVERYONE@:", it ends up 
breaking at 180 lines.  At 179 lines, this requires a buffer of 4988 bytes.  It 
looks like there might be a ceiling at 5000 bytes?

> Anyway, strace'ing nfs4_getfacl/nfs4_setfacl would verify whether the
> error was coming from the kernel or the tools.

This is when the attributes list is too long:

getxattr("hello", "system.nfs4_acl"..., 0x0, 0) = -1 EIO (Input/output error)

I couldn't find a mention of EIO in the man pages for getxattr(2) or stat(2).

> I have to ask: how many acl entries do you need?

We don't plan on using huge ACLs, but it's nice to know they'll work if someone 
tries to use them.  If I could limit the maximum number of ACL entries to 
something smaller, I would have done that instead, but it's not configurable.