ACL and NFSv4 expectations

ACL and NFSv4 expectations

[Tim Rafert]

Looking for assistance if the following are REAL issues or expected behavior using NFSv4 ACL (on linux ol6 and interoperating with solaris).  Thanks in advance.

1) Is there a max # of entries that can be stored in an ACL?  If so - what is it (or is dependent on the device)?

2) Is it up to my own implementation instead of the "NFS Client" on the OS to try and keep the ACLs clean/organized/etc?  See my next question for further clarification

3) If a user adds the same ACE into an ACL multiple times - then it is actually added multiple times?  For example:
 
bash-4.1$ nfs4_setfacl -aA:fd:OWNER@:rwaDdxtTnNCo,A:fdi:EVERYONE@:rxtncy,D::EVERYONE@:DT,A::user1@xyz.com:rwaxtc exampleDir
bash-4.1$ nfs4_getfacl exampleDir
A:fd:OWNER@:rwaDdxtTnNCo
A:fdi:EVERYONE@:rxtncy
D::EVERYONE@:DT
A::user1@xyz.com:rwaxtc
A:fdig:GROUP@:rxtncy
A:fdi:EVERYONE@:rxtncy
A::OWNER@:rwaDxtTnNcCoy
A:g:GROUP@:rxtncy
A::EVERYONE@:rxtncy
bash-4.1$ nfs4_setfacl -aA:fd:OWNER@:rwaDdxtTnNCo,A:fdi:EVERYONE@:rxtncy,D::EVERYONE@:DT,A::user1@xyz.com:rwaxtc exampleDir
bash-4.1$ nfs4_getfacl exampleDir
A:fd:OWNER@:rwaDdxtTnNCo
A:fdi:EVERYONE@:rxtncy
D::EVERYONE@:DT
A::user1@xyz.com:rwaxtc
A:fd:OWNER@:rwaDdxtTnNCo
A:fdi:EVERYONE@:rxtncy
D::EVERYONE@:DT
A::user1@xyz.com:rwaxtc
A:fdig:GROUP@:rxtncy
A:fdi:EVERYONE@:rxtncy
A::OWNER@:rwaDxtTnNcCoy
A:g:GROUP@:rxtncy
A::EVERYONE@:rxtncy

4) ACL "caching" - it appears that if from one client-host - a user alters the ACL and then from another client-host - a user also alters the ACL - then one of the alterations can be lost (if they are within a minute or some cache timeframe).  For example:
 
>From a Linux host:
bash-4.1$ mkdir nfscache
bash-4.1$
bash-4.1$
bash-4.1$ nfs4_getfacl nfscache
A:fdig:GROUP@:rxtncy
A:fdi:EVERYONE@:rxtncy
A::OWNER@:rwaDxtTnNcCoy
A:g:GROUP@:rxtncy
A::EVERYONE@:rxtncy
bash-4.1$ nfs4_setfacl -a A::user1@xyz.com:rwaxtc nfscache
bash-4.1$ nfs4_getfacl nfscache
A::user1@xyz.com:rwaxtc
A:fdig:GROUP@:rxtncy
A:fdi:EVERYONE@:rxtncy
A::OWNER@:rwaDxtTnNcCoy
A:g:GROUP@:rxtncy
A::EVERYONE@:rxtncy
 
Then from Solaris host (we see "user1" was correctly added - but then we add another user: "user2"):
bash-4.1$ ls -aldV nfscache/
drwxr-xr-x+  2 trafert  pditdba        2 Oct 30 19:26 nfscache/
            user:user1:rwxp--a---c---:-------:allow
                 group@:r-x---a-R-c--s:fdi----:allow
              everyone@:r-x---a-R-c--s:fdi----:allow
                 owner@:rwxp-DaARWcCos:-------:allow
                 group@:r-x---a-R-c--s:-------:allow
              everyone@:r-x---a-R-c--s:-------:allow
bash-4.1$ chmod A+user:user2:rwxp--a---c---::allow nfscache
bash-4.1$ ls -aldV nfscache/
drwxr-xr-x+  2 trafert  pditdba        2 Oct 30 19:26 nfscache/
             user:user2:rwxp--a---c---:-------:allow
             user:user1:rwxp--a---c---:-------:allow
                 group@:r-x---a-R-c--s:fdi----:allow
              everyone@:r-x---a-R-c--s:fdi----:allow
                 owner@:rwxp-DaARWcCos:-------:allow
                 group@:r-x---a-R-c--s:-------:allow
              everyone@:r-x---a-R-c--s:-------:allow
 
 
Then come back to the Linux host (we only see the original setting where "user1" was added - "user2" is missing).  Then we add another user "user3".  At which point the "user2" addition is completely lost on either hosts:
 
bash-4.1$ nfs4_getfacl nfscache
A::user1@xyz.com:rwaxtc
A:fdig:GROUP@:rxtncy
A:fdi:EVERYONE@:rxtncy
A::OWNER@:rwaDxtTnNcCoy
A:g:GROUP@:rxtncy
A::EVERYONE@:rxtncy
bash-4.1$ nfs4_setfacl -a A::user3@xyz.com:rwaxtc nfscache
bash-4.1$ nfs4_getfacl nfscache
A::user3@xyz.com:rwaxtc
A::user1@xyz.com:rwaxtc
A:fdig:GROUP@:rxtncy
A:fdi:EVERYONE@:rxtncy
A::OWNER@:rwaDxtTnNcCoy
A:g:GROUP@:rxtncy
A::EVERYONE@:rxtncy
 
Back on Solaris - "user2" is also lost:
bash-4.1$ ls -aldV nfscache/
drwxr-xr-x+  2 trafert  pditdba        2 Oct 30 19:28 nfscache/
             user:user3:rwxp--a---c---:-------:allow
             user:user1:rwxp--a---c---:-------:allow
                 group@:r-x---a-R-c--s:fdi----:allow
              everyone@:r-x---a-R-c--s:fdi----:allow
                 owner@:rwxp-DaARWcCos:-------:allow
                 group@:r-x---a-R-c--s:-------:allow
              everyone@:r-x---a-R-c--s:-------:allow

Re: ACL and NFSv4 expectations

[mdw]

I can't pretend to be familiar with solaris acls, but I have looked at
the linux client while implementing partial acl support in ganesha.
Following are my answers based on that:

> On Fri, Nov 22, 2013 at 11:34:04AM -0800, Tim Rafert wrote:
> > Looking for assistance if the following are REAL issues or expected behavior using NFSv4 ACL (on linux ol6 and interoperating with solaris).  Thanks in advance.
> > 
> > 1) Is there a max # of entries that can be stored in an ACL?  If so - what is it (or is dependent on the device)?

I don't see any mention of maximum in rfc5661.  I'm pretty sure this should
be "device dependent", with some vague notion of "slightly unreasonably large".

> > 
> > 2) Is it up to my own implementation instead of the "NFS Client" on the OS to try and keep the ACLs clean/organized/etc?  See my next question for further clarification

I belive it's mostly an application/user responsibility to keep the acl clean.
If the NFS acl is set indirectly, then whatever does that could change stuff.
There's some complicated wording about trying to preserve as much of what
the user sets as possible (such as permitting things to "BATCH@") while
not pretending to do anything one can't do.  Most of the excuses to do
something indirectly with the acl have to do with posix acls or unix mode bits.

> > 
> > 3) If a user adds the same ACE into an ACL multiple times - then it is actually added multiple times?  For example:

Should be.  Or at least I wouldn't think the server should be attempting
to do this gratuitously.  Certainly any change that changes the semantics of
the list is bad.  I believe if you have only 'allow' entries, you can in fact
reorder without changing the semantics, but as soon as there's any 'deny'
entries, it gets messy.

> > 
> > 4) ACL "caching" - it appears that if from one client-host - a user alters the ACL and then from another client-host - a user also alters the ACL - then one of the alterations can be lost (if they are within a minute or some cache timeframe).  For example:

I have seen weird stuff with caching and acls.  In my case I traced
most of it down to bad server logic, but I certainly believe the linux
client can report something different than what's actually on the server.
If the server is rewriting acls, that certainly sounds like it could
cause visible caching artifacts.  Another complication is that the linux
userland nfs4 tool sometimes hides bits, so what you set and what's stored and
returned, is not in fact what you see.

...

						-Marcus Watts