[RFC v3 00/45] Richacls

[RFC v3 00/45] Richacls

[Andreas Gruenbacher]

Hello,

here's another update of the richacl patch queue.  The changes since the last
posting (https://lwn.net/Articles/638242/) include:

 * The nfs client now allocates pages for received acls on demand like the
   server does.  It no longer caches the acl size between calls.

 * All possible acls consisting of only owner@, group@, and everyone@ entries
   which are equivalent to the file mode permission bits are now recognized.
   This is needed because by the NFSv4 specification, the nfs server must
   translate the file mode permission bits into an acl if it supports acls at
   all.

 * Support for the dacl attribute over NFSv4.1 for Automatic Inheritance, and
   also for the write_retention and write_retention_hold permissions.

 * The richacl_compute_max_masks() documentation has been improved.

 * Various minor bug fixes.

The git version is available here:

  git://git.kernel.org/pub/scm/linux/kernel/git/agruen/linux-richacl.git \
	richacl-2015-04-24

The richacl command-line has been split into getrichacl and setrichacl, in line
with getfacl and setfacl.  Watch out for that when updating the user-space.

Things still to be done, or which I'm not entirely happy with:

 * The nfs server performs some access checking on its own before calling into
   the vfs which is rersponsible for the actual access checking (see where it
   calls inode_permission()).  With the additional MAY_ flags introduced in
   this patch queue, it gets it wrong in some cases; I have yet to figure out
   how to deal with this.

 * We may still need to add back support for the "system.nfs4_acl" attribute on
   nfs mounts for backwards compatibility; it's not clear to me if anyone is
   actually using that.

 * It would be nice if the MAY_DELETE_SELF flag could override the sticky
   directory check as it did in the previous version of this patch queue.  I
   couldn't come up with a clean way of achieving that, though.

 * The base_acl code is still rather ugly.

Thanks,
Andreas


Andreas Gruenbacher (43):
  vfs: Minor documentation fix
  uapi: Remove kernel internal declaration
  nfsd: Checking for acl support does not require fetching any acls
  vfs: Shrink struct posix_acl
  vfs: Add IS_ACL() and IS_RICHACL() tests
  vfs: Add MAY_CREATE_FILE and MAY_CREATE_DIR permission flags
  vfs: Add MAY_DELETE_SELF and MAY_DELETE_CHILD permission flags
  vfs: Make the inode passed to inode_change_ok non-const
  vfs: Add permission flags for setting file attributes
  richacl: In-memory representation and helper functions
  richacl: Permission mapping functions
  richacl: Compute maximum file masks from an acl
  richacl: Update the file masks in chmod()
  richacl: Permission check algorithm
  vfs: Cache base_acl objects in inodes
  vfs: Cache richacl in struct inode
  richacl: Create-time inheritance
  richacl: Check if an acl is equivalent to a file mode
  richacl: Also recognize nontrivial mode-equivalent acls
  richacl: Automatic Inheritance
  richacl: xattr mapping functions
  vfs: Add richacl permission checking
  richacl: acl editing helper functions
  richacl: Move everyone@ aces down the acl
  richacl: Propagate everyone@ permissions to other aces
  richacl: Isolate the owner and group classes
  richacl: Apply the file masks to a richacl
  richacl: Create richacl from mode values
  richacl: Create acl with masks applied in richacl_from_mode()
  nfsd: Remove dead declarations
  nfsd: Keep list of acls to dispose of in compoundargs
  nfsd: Use richacls as internal acl representation
  nfsd: Add richacl support
  NFSv4: Fix GETATTR bitmap verification
  nfs/sunrpc: No more encode and decode function pointer casting
  nfs/sunrpc: Return status code from encode functions
  nfs3: Return posix acl encode errors
  nfs: Remove unused xdr page offsets in getacl/setacl arguments
  rpc: Allow to demand-allocate pages to encode into
  nfs: Add richacl support
  uapi/nfs: Add NFSv4.1 ACL definitions
  nfsd: Add support for the v4.1 dacl attribute
  nfs: Add support for the v4.1 dacl attribute

Aneesh Kumar K.V (2):
  ext4: Add richacl support
  ext4: Add richacl feature flag

 Documentation/filesystems/porting               |    8 +-
 Documentation/filesystems/vfs.txt               |    3 +
 drivers/staging/lustre/lustre/llite/llite_lib.c |    2 +-
 fs/Kconfig                                      |    9 +
 fs/Makefile                                     |    3 +
 fs/attr.c                                       |   81 +-
 fs/ext4/Kconfig                                 |   15 +
 fs/ext4/Makefile                                |    1 +
 fs/ext4/acl.c                                   |    7 +-
 fs/ext4/acl.h                                   |   12 +-
 fs/ext4/ext4.h                                  |    6 +-
 fs/ext4/file.c                                  |    6 +-
 fs/ext4/ialloc.c                                |    7 +-
 fs/ext4/inode.c                                 |   10 +-
 fs/ext4/namei.c                                 |   11 +-
 fs/ext4/richacl.c                               |  211 +++++
 fs/ext4/richacl.h                               |   47 ++
 fs/ext4/super.c                                 |   41 +-
 fs/ext4/xattr.c                                 |    6 +
 fs/ext4/xattr.h                                 |    1 +
 fs/f2fs/acl.c                                   |    4 +-
 fs/inode.c                                      |   15 +-
 fs/lockd/clnt4xdr.c                             |   58 +-
 fs/lockd/clntxdr.c                              |   58 +-
 fs/lockd/mon.c                                  |   26 +-
 fs/namei.c                                      |  108 ++-
 fs/nfs/inode.c                                  |    3 -
 fs/nfs/mount_clnt.c                             |   24 +-
 fs/nfs/nfs2xdr.c                                |  115 ++-
 fs/nfs/nfs3xdr.c                                |  225 +++--
 fs/nfs/nfs4proc.c                               |  357 ++++----
 fs/nfs/nfs4xdr.c                                | 1001 +++++++++++++++--------
 fs/nfs/super.c                                  |    4 +-
 fs/nfs_common/Makefile                          |    1 +
 fs/nfs_common/nfs4acl.c                         |   41 +
 fs/nfsd/Kconfig                                 |    1 +
 fs/nfsd/acl.h                                   |   23 +-
 fs/nfsd/nfs4acl.c                               |  499 +++++------
 fs/nfsd/nfs4callback.c                          |   29 +-
 fs/nfsd/nfs4proc.c                              |   17 +-
 fs/nfsd/nfs4xdr.c                               |  280 ++++---
 fs/nfsd/nfsd.h                                  |    6 +-
 fs/nfsd/xdr4.h                                  |   12 +-
 fs/posix_acl.c                                  |   31 +-
 fs/richacl_base.c                               |  587 +++++++++++++
 fs/richacl_compat.c                             |  841 +++++++++++++++++++
 fs/richacl_inode.c                              |  268 ++++++
 fs/richacl_xattr.c                              |  210 +++++
 fs/xattr.c                                      |   34 +-
 include/linux/fs.h                              |   47 +-
 include/linux/nfs4.h                            |   17 +-
 include/linux/nfs4acl.h                         |    7 +
 include/linux/nfs_fs.h                          |    1 -
 include/linux/nfs_fs_sb.h                       |    2 +
 include/linux/nfs_xdr.h                         |   12 +-
 include/linux/posix_acl.h                       |   12 +-
 include/linux/richacl.h                         |  332 ++++++++
 include/linux/richacl_compat.h                  |   40 +
 include/linux/richacl_xattr.h                   |   52 ++
 include/linux/sunrpc/xdr.h                      |    5 +-
 include/uapi/linux/fs.h                         |    3 +-
 include/uapi/linux/nfs4.h                       |   17 +-
 include/uapi/linux/xattr.h                      |    2 +
 net/sunrpc/auth.c                               |    7 +-
 net/sunrpc/auth_gss/gss_rpc_upcall.c            |    4 +-
 net/sunrpc/auth_gss/gss_rpc_xdr.c               |   11 +-
 net/sunrpc/auth_gss/gss_rpc_xdr.h               |    8 +-
 net/sunrpc/clnt.c                               |    5 +-
 net/sunrpc/rpcb_clnt.c                          |   57 +-
 net/sunrpc/xdr.c                                |    8 +
 70 files changed, 4720 insertions(+), 1294 deletions(-)
 create mode 100644 fs/ext4/richacl.c
 create mode 100644 fs/ext4/richacl.h
 create mode 100644 fs/nfs_common/nfs4acl.c
 create mode 100644 fs/richacl_base.c
 create mode 100644 fs/richacl_compat.c
 create mode 100644 fs/richacl_inode.c
 create mode 100644 fs/richacl_xattr.c
 create mode 100644 include/linux/nfs4acl.h
 create mode 100644 include/linux/richacl.h
 create mode 100644 include/linux/richacl_compat.h
 create mode 100644 include/linux/richacl_xattr.h

-- 
2.1.0

Re: [RFC v3 00/45] Richacls

[Jeremy Allison]

On Fri, Apr 24, 2015 at 01:03:57PM +0200, Andreas Gruenbacher wrote:
> Hello,
> 
> here's another update of the richacl patch queue.  The changes since the last
> posting (https://lwn.net/Articles/638242/) include:
> 
>  * The nfs client now allocates pages for received acls on demand like the
>    server does.  It no longer caches the acl size between calls.
> 
>  * All possible acls consisting of only owner@, group@, and everyone@ entries
>    which are equivalent to the file mode permission bits are now recognized.
>    This is needed because by the NFSv4 specification, the nfs server must
>    translate the file mode permission bits into an acl if it supports acls at
>    all.
> 
>  * Support for the dacl attribute over NFSv4.1 for Automatic Inheritance, and
>    also for the write_retention and write_retention_hold permissions.
> 
>  * The richacl_compute_max_masks() documentation has been improved.
> 
>  * Various minor bug fixes.
> 
> The git version is available here:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/agruen/linux-richacl.git \
> 	richacl-2015-04-24

FYI. I have a mostly (needs test suite adding) working module
for Samba for Andreas's richacls code.

Using it we map incoming Windows ACLs directly to richacls
using the same mapping as we use for existing ZFS ACLs.

Jeremy.

Re: [RFC v3 00/45] Richacls

[Andreas Grünbacher]

Jeremy,

2015-05-23 17:50 GMT+02:00 Jeremy Allison <jra@samba.org>:
> FYI. I have a mostly (needs test suite adding) working module
> for Samba for Andreas's richacls code.

great, thanks. I'll be doing some testing as soon as I get to it.

Andreas

Re: [RFC v3 42/45] nfs: Add richacl support

[Trond Myklebust]

Adding linux-api...

On Fri, May 29, 2015 at 11:00 AM, Andreas Grünbacher
<andreas.gruenbacher@gmail.com> wrote:
> 2015-05-29 15:15 GMT+02:00 Trond Myklebust <trond.myklebust@primarydata.com>:
>> [reply reordered]
>> So having revisited the reasons why I chose the system.nfs4_acl
>> interface when we did NFSv4 ACLs, I'm not sure we should implement
>> system.richacl for the NFS client at all.
>>
>> Your assertion that "when symbolic user@domain and group@domain names
>> are used in the acl, user-space needs to perform ID mapping in the
>> same way as the kernel"  is WRONG. User space needs do no such thing,
>> and that was the whole point of the interface; to allow the user to
>> specify ACLs in a format that is checked only on the _server_, and not
>> on the client.
>
> That's only half true. Right now, user-space applications trying to copy
> permissions between an nfs mount and another file system will fail unless the
> application has explicitly been made nfs aware and supports the
> "system.nfs4_acl"
> attribute (as well as some other acl mechanism if the permissions go beyond the
> file mode).
>
> The same problem exists when trying to make sense of acls.
>
> It seems unreasonable to me to expect applications other than special file
> system maintenance tools to cater to such file system differences; there are
> just too many file systems out there for that to work. Instead, it
> would be better
> to use an interface that can be generalized across file systems.

My point is that system.richacl is not such an interface. It can only
ever work for local filesystems that understand and store local uids
and gids. It has no support for the remote users/groups that are
stored on your NFS/SMB server unless they happen to have a local
mapping into uids and gids, and so the API is inappropriate to replace
the existing NFSv4 acl API on the client.

>> The problem is that you are 100% reliant on an accurate idmapper in
>> order to convert the name@domain to a _correct_ uid/gid. It isn't
>> sufficient to convert to just any valid uid/gid, because if your ACL
>> tool is trying to edit the ACL, you can end up converting all those
>> DENY modes for user 'Johnny_Rotten@blackhats.are.us' into DENY modes
>> for user 'nobody'.
>> ...and yes, libnfsidmap will happily convert all unknown
>> user/groupnames into whatever uid/gid corresponds to 'nobody' without
>> returning an error.
>
> That's indeed a problem, and I can think of two ways of addressing it:
>
> First, acl editors need to be careful about nobody entries; they need to be
> aware that nobody could actually stand for somebody else. (We could map
> unmappable users and groups to something else than nobody, but that
> might just shift the problem without improve anything.)

What is the editor supposed to do about an entry it cannot even
interpret, let alone store back to the server?

> Second, we could add support for passing through unmappable Who values
> as is. But that raises the problem of how to pass thtough and represent
> different kinds of identifiers: NFSv4 users user@domain and group@domain
> strings; smb users SIDs; maybe more.
>

Now you have a mixture of some stuff that is being translated into
local uid/gid format and therefore needs translating back when you're
going to update the ACL, and some stuff that is not. What is the value
of doing the mapping here?

Trond

Re: [RFC v3 42/45] nfs: Add richacl support

[Andreas Grünbacher]

2015-05-29 17:24 GMT+02:00 Trond Myklebust <trond.myklebust@primarydata.com>:
>> It seems unreasonable to me to expect applications other than special file
>> system maintenance tools to cater to such file system differences; there are
>> just too many file systems out there for that to work. Instead, it
>> would be better
>> to use an interface that can be generalized across file systems.
>
> My point is that system.richacl is not such an interface. It can only
> ever work for local filesystems that understand and store local uids
> and gids. It has no support for the remote users/groups that are
> stored on your NFS/SMB server unless they happen to have a local
> mapping into uids and gids, and so the API is inappropriate to replace
> the existing NFSv4 acl API on the client.

That can be changed if we find a reasonable solution.

>>> The problem is that you are 100% reliant on an accurate idmapper in
>>> order to convert the name@domain to a _correct_ uid/gid. It isn't
>>> sufficient to convert to just any valid uid/gid, because if your ACL
>>> tool is trying to edit the ACL, you can end up converting all those
>>> DENY modes for user 'Johnny_Rotten@blackhats.are.us' into DENY modes
>>> for user 'nobody'.
>>> ...and yes, libnfsidmap will happily convert all unknown
>>> user/groupnames into whatever uid/gid corresponds to 'nobody' without
>>> returning an error.
>>
>> That's indeed a problem, and I can think of two ways of addressing it:
>>
>> First, acl editors need to be careful about nobody entries; they need to be
>> aware that nobody could actually stand for somebody else. (We could map
>> unmappable users and groups to something else than nobody, but that
>> might just shift the problem without improve anything.)
>
> What is the editor supposed to do about an entry it cannot even
> interpret, let alone store back to the server?

In the end, it will have to be a user decision what to do about such entries,
the editor can warn the user and make it harder to make mistakes though.

>> Second, we could add support for passing through unmappable Who values
>> as is. But that raises the problem of how to pass thtough and represent
>> different kinds of identifiers: NFSv4 users user@domain and group@domain
>> strings; smb users SIDs; maybe more.
>
> Now you have a mixture of some stuff that is being translated into
> local uid/gid format and therefore needs translating back when you're
> going to update the ACL, and some stuff that is not. What is the value
> of doing the mapping here?

If you don't translate, you cannot copy the permissions to another file
system. In the ideal case, everything can be translated; where that fails,
the user probably wants to know.

Thanks,
Andreas

Re: [RFC v3 42/45] nfs: Add richacl support

[Trond Myklebust]

On Fri, May 29, 2015 at 11:45 AM, Andreas Grünbacher
<andreas.gruenbacher@gmail.com> wrote:
> 2015-05-29 17:24 GMT+02:00 Trond Myklebust <trond.myklebust@primarydata.com>:
>>> It seems unreasonable to me to expect applications other than special file
>>> system maintenance tools to cater to such file system differences; there are
>>> just too many file systems out there for that to work. Instead, it
>>> would be better
>>> to use an interface that can be generalized across file systems.
>>
>> My point is that system.richacl is not such an interface. It can only
>> ever work for local filesystems that understand and store local uids
>> and gids. It has no support for the remote users/groups that are
>> stored on your NFS/SMB server unless they happen to have a local
>> mapping into uids and gids, and so the API is inappropriate to replace
>> the existing NFSv4 acl API on the client.
>
> That can be changed if we find a reasonable solution.
>
>>>> The problem is that you are 100% reliant on an accurate idmapper in
>>>> order to convert the name@domain to a _correct_ uid/gid. It isn't
>>>> sufficient to convert to just any valid uid/gid, because if your ACL
>>>> tool is trying to edit the ACL, you can end up converting all those
>>>> DENY modes for user 'Johnny_Rotten@blackhats.are.us' into DENY modes
>>>> for user 'nobody'.
>>>> ...and yes, libnfsidmap will happily convert all unknown
>>>> user/groupnames into whatever uid/gid corresponds to 'nobody' without
>>>> returning an error.
>>>
>>> That's indeed a problem, and I can think of two ways of addressing it:
>>>
>>> First, acl editors need to be careful about nobody entries; they need to be
>>> aware that nobody could actually stand for somebody else. (We could map
>>> unmappable users and groups to something else than nobody, but that
>>> might just shift the problem without improve anything.)
>>
>> What is the editor supposed to do about an entry it cannot even
>> interpret, let alone store back to the server?
>
> In the end, it will have to be a user decision what to do about such entries,
> the editor can warn the user and make it harder to make mistakes though.
>
>>> Second, we could add support for passing through unmappable Who values
>>> as is. But that raises the problem of how to pass thtough and represent
>>> different kinds of identifiers: NFSv4 users user@domain and group@domain
>>> strings; smb users SIDs; maybe more.
>>
>> Now you have a mixture of some stuff that is being translated into
>> local uid/gid format and therefore needs translating back when you're
>> going to update the ACL, and some stuff that is not. What is the value
>> of doing the mapping here?
>
> If you don't translate, you cannot copy the permissions to another file
> system. In the ideal case, everything can be translated; where that fails,
> the user probably wants to know.

Why are we more worried about a hypothetical copy of the file to
another filesystem than we are about representing the ACL correctly
for the filesystem that the file is actually on? Your ACL on the
remote SMB server windows.publicserver.com may not be fully
represented when you copy the file to your local filesystem; get over
it... However if you are authorised to edit the ACL on
windows.publicserver.com, and you can't set it to something that can
be enforced correctly by windows.publicserver.com, then you have a
real problem.

Trond
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [RFC v3 42/45] nfs: Add richacl support

[Frank Filz]

> On Fri, May 29, 2015 at 11:00 AM, Andreas Grünbacher
> <andreas.gruenbacher-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > 2015-05-29 15:15 GMT+02:00 Trond Myklebust
> <trond.myklebust-7I+n7zu2hftEKMMhf/gKZA@public.gmane.org>:
> >> [reply reordered]
> >> So having revisited the reasons why I chose the system.nfs4_acl
> >> interface when we did NFSv4 ACLs, I'm not sure we should implement
> >> system.richacl for the NFS client at all.
> >>
> >> Your assertion that "when symbolic user@domain and group@domain
> names
> >> are used in the acl, user-space needs to perform ID mapping in the
> >> same way as the kernel"  is WRONG. User space needs do no such thing,
> >> and that was the whole point of the interface; to allow the user to
> >> specify ACLs in a format that is checked only on the _server_, and
> >> not on the client.
> >
> > That's only half true. Right now, user-space applications trying to
> > copy permissions between an nfs mount and another file system will
> > fail unless the application has explicitly been made nfs aware and
> > supports the "system.nfs4_acl"
> > attribute (as well as some other acl mechanism if the permissions go
> > beyond the file mode).
> >
> > The same problem exists when trying to make sense of acls.
> >
> > It seems unreasonable to me to expect applications other than special
> > file system maintenance tools to cater to such file system
> > differences; there are just too many file systems out there for that
> > to work. Instead, it would be better to use an interface that can be
> > generalized across file systems.
> 
> My point is that system.richacl is not such an interface. It can only ever work
> for local filesystems that understand and store local uids and gids. It has no
> support for the remote users/groups that are stored on your NFS/SMB
> server unless they happen to have a local mapping into uids and gids, and so
> the API is inappropriate to replace the existing NFSv4 acl API on the client.

Could we have both xattrs? Or a mount option that specifies which xattr to have?

That way folks who don't have local idmapping for every remote identity can use system.nfs4_acl while those who have local mapping for all remote identies and need to use a wide variety of tools can use system.richacl? system.richacl would obviously need to be documented that this issue can arise. But that will forever be an issue, unless we store the ACL with symbolic names, copying from a remote server to a local filesystem will always be lossy if the idmapping is incomplete.

Maybe that's too messy...

Frank


--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC v3 00/45] Richacls

[Stefan (metze) Metzmacher]

[-- Attachment #1: Type: text/plain, Size: 1512 bytes --]

Hi Andreas,

>> here's another update of the richacl patch queue.  The changes since the last
>> posting (https://lwn.net/Articles/638242/) include:
>>
>>  * The nfs client now allocates pages for received acls on demand like the
>>    server does.  It no longer caches the acl size between calls.
>>
>>  * All possible acls consisting of only owner@, group@, and everyone@ entries
>>    which are equivalent to the file mode permission bits are now recognized.
>>    This is needed because by the NFSv4 specification, the nfs server must
>>    translate the file mode permission bits into an acl if it supports acls at
>>    all.
>>
>>  * Support for the dacl attribute over NFSv4.1 for Automatic Inheritance, and
>>    also for the write_retention and write_retention_hold permissions.
>>
>>  * The richacl_compute_max_masks() documentation has been improved.
>>
>>  * Various minor bug fixes.
>>
>> The git version is available here:
>>
>>   git://git.kernel.org/pub/scm/linux/kernel/git/agruen/linux-richacl.git \
>> 	richacl-2015-04-24
> 
> FYI. I have a mostly (needs test suite adding) working module
> for Samba for Andreas's richacls code.
> 
> Using it we map incoming Windows ACLs directly to richacls
> using the same mapping as we use for existing ZFS ACLs.

Yes, we need to make sure the code supports everything we require,
proved by working code.

We should avoid the disaster with the mostly unuseable F_SETLEASE
code for kernel oplocks.

Thanks!
metze


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]