Problems with POSIX ACL <=> NFSv4 ACL mapping in mainline

Problems with POSIX ACL <=> NFSv4 ACL mapping in mainline

[Andreas Gruenbacher]

Hello,

we have a problem with the POSIX ACL <=> NFSv4 ACL mapping as currently 
implemented in NFSv4 in mainline: it is implemented on the server side only, 
which leads to interesting problems. I'll try to explain.

POSIX ACLs [1] and NFSv4 ACLs [2] are based on very different concepts, and so 
they don't map onto each other very well. Marius and others are proposing a 
mapping between the two formats [3], and an implementation of that mapping 
seems to have found its way into the mainline nfsv4 implementation. (Note 
that I didn't actually verify the implemented mapping actually reflects draft 
[3].)

The mapping from POSIX ACLs to NFSv4 ACLs and back is implemented only on the 
server side only right now: POSIX ACLs on the server are presented as NFSv4 
ACLs over the network, and as "system.nfs4_acl" extended attributes to 
user-space on the client. This is no problem when moving or copying files 
between NFSv4 mounts, but other filesystems on the client don't know how to 
deal with "system.nfs4_acl" attributes, and so preserving those attributes 
will inevitably fail. What's really bad is that this will even fail for 
filesystems that support POSIX ACLs. 

The CITI has patches that try to work around this in libacl by trying both 
POSIX ACLs and "system.nfs4_acl" attributes, and converting between the two 
if necessary. This approach has a number of disadvantages: (1) applications 
that don't use libacl will still fail, (2) always trying both formats without 
knowing the properties of the underlying filesystem will always be slow, (3) 
it exposes a deficiency of NFSv4 instead of trying to hide it.

In addition, I could imagine that we'll optionally support local NFSv4-style 
ACLs on specific file systems somewhen in the future. (Samba would benefit 
greatly from that for example; Solaris already can do that.) Not being able 
to tell the one kind of ACLs from the other in a world where we may have both 
sounds like a recipe for disaster to me.

I could imagine the following solutions to this problem:

* Don't map between POSIX ACLs and NFSv4 ACLs at all, and use a different
  protocol for POSIX ACLs and for NFSv4 ACLs instead (e.g., version 4 of the
  NFSACL protocol extension). The protocol extension is not currently
  implemented in Solaris because current Solaris supports NFSv4 ACLs natively
  instead of POSIX ACLs, and so they don't seem to have as strong a motivation
  to keep POSIX ACLs working well as we currently do. The NFSACL protocol has
  the huge advantage of supporting POSIX ACLs pretty well, and so a lot of the
  overhead and complexity of doing the NFSv4 mapping would go away though.

* Map back from NFSv4 ACLs to POSIX ACLs on the client side, so that
  POSIX ACLs on the server will appear as POSIX ACLs on the client. This
  would take away all the mess we currently have in user-space and limit the
  mapping problems to the NFSv4 protocol, where it really occurs. We could
  define an additional NFsv4 attribute that the server could use to declare
  if it supports POSIX ACLs only, so that the client won't have to try to
  convert from NFSv4 to POSIX ACLs for servers that have native NFSv4 ACLs.

* Go with the user-space solution, and implement the mapping in libacl. This
  solution is ugly, slow, and adds complexity to an unrelated piece of
  software. Some applications will break, and there is no hope that we'll ever
  get rid of this complex hack in the future, for as long as libacl will still
  exist. Yuck!

* Don't support POSIX ACLs over NFSv4 at all.

What do you think?

Thanks,
Andreas


[1] POSIX 1003.1e/2c draft 17, withdrawn, 
http://wt.xpilot.org/publications/posix.1e/download.html

[2] Spencer Shepler et al.: Network File System (NFS) version 4 Protocol, 
section 5.11. Access Control Lists, http://www.ietf.org/rfc/rfc3530.txt

[3] Marius Aamodt Eriksen et al.: Mapping Between NFSv4 and Posix Draft ACLs,
http://www.ietf.org/internet-drafts/draft-ietf-nfsv4-acl-mapping-04.txt

-- 
Andreas Gruenbacher <agruen@suse.de>
Novell / SUSE Labs

Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: Problems with POSIX ACL <=> NFSv4 ACL mapping in mainline

[J. Bruce Fields]

On Fri, Jun 23, 2006 at 05:45:22PM +0200, Andreas Gruenbacher wrote:
> they don't map onto each other very well. Marius and others are proposing a 
> mapping between the two formats [3], and an implementation of that mapping 
> seems to have found its way into the mainline nfsv4 implementation. (Note 
> that I didn't actually verify the implemented mapping actually reflects draft 
> [3].)

It doesn't; the implementation is based on the previous version.  The newer
version requires the server to perform a (possibly lossy) mapping of virtually
any NFSv4 ACL, whereas the previous (currently implemented) version only
accepted a very small subset of possible NFSv4 ACLs.

> The CITI has patches that try to work around this in libacl by trying both 
> POSIX ACLs and "system.nfs4_acl" attributes, and converting between the two 
> if necessary. This approach has a number of disadvantages: (1) applications 
> that don't use libacl will still fail

Are there particular examples of such applications that you're thinking of?

> (2) always trying both formats without knowing the properties of the
> underlying filesystem will always be slow,

How much slower?  I'd expect the cost of one system call (which just
immediately returns -ENOTSUPP) to be insignificant.

> In addition, I could imagine that we'll optionally support local NFSv4-style
> ACLs on specific file systems somewhen in the future. (Samba would benefit
> greatly from that for example; Solaris already can do that.) Not being able
> to tell the one kind of ACLs from the other in a world where we may have both
> sounds like a recipe for disaster to me.

I'm not following you.  Currently nfsv4 and posix acls are being presented to
userspace via different extended attributes, so you can tell one type from the
other.

> I could imagine the following solutions to this problem:
> 
> * Don't map between POSIX ACLs and NFSv4 ACLs at all, and use a different
> protocol for POSIX ACLs and for NFSv4 ACLs instead (e.g., version 4 of the
> NFSACL protocol extension). The protocol extension is not currently
> implemented in Solaris because current Solaris supports NFSv4 ACLs natively
> instead of POSIX ACLs, and so they don't seem to have as strong a motivation
> to keep POSIX ACLs working well as we currently do. The NFSACL protocol has
> the huge advantage of supporting POSIX ACLs pretty well, and so a lot of the
> overhead and complexity of doing the NFSv4 mapping would go away though.

If you wanted to support *both* POSIX and NFSv4 ACLs, then you'd still have the
same problem.  Unless you expect it to be possible to set both types of ACLs on
the same file independently, and make the kernel enforce the AND of the two, or
something like that.  I don't think that would be helpful.

> * Map back from NFSv4 ACLs to POSIX ACLs on the client side, so that POSIX
> ACLs on the server will appear as POSIX ACLs on the client. This would take
> away all the mess we currently have in user-space

And put it in the kernel instead?  We originally did exactly that, and Trond
argued that we should do this in userspace instead.  I still think that was the
right decision.

I'm not satisfied with the current situation either.  But I don't think Samba
or NFSv4 will ever be completely happy until we provide good support for
NFSv4/Windows-style ACLs.  We can't rip out POSIX ACLs in the process, so we're
going to have to support both.  I don't see how to support both well without
mapping between them.  And if we've got to do this complicated mapping, let's
do it in userspace whenever possible.

--b.

Re: [NFS] Problems with POSIX ACL <=> NFSv4 ACL mapping in mainline

[Christoph Hellwig]

On Fri, Jun 23, 2006 at 05:45:22PM +0200, Andreas Gruenbacher wrote:
> In addition, I could imagine that we'll optionally support local NFSv4-style 
> ACLs on specific file systems somewhen in the future.

Most likely not.  Let's not spread the NFSv4 ACL cancer more than absolutely
nessecary.

> * Don't map between POSIX ACLs and NFSv4 ACLs at all, and use a different
>   protocol for POSIX ACLs and for NFSv4 ACLs instead (e.g., version 4 of the
>   NFSACL protocol extension). The protocol extension is not currently
>   implemented in Solaris because current Solaris supports NFSv4 ACLs natively
>   instead of POSIX ACLs, and so they don't seem to have as strong a motivation
>   to keep POSIX ACLs working well as we currently do. The NFSACL protocol has
>   the huge advantage of supporting POSIX ACLs pretty well, and so a lot of the
>   overhead and complexity of doing the NFSv4 mapping would go away though.

We definitly should support NFSACL for Linux to Linux deployments to avoid
all that mess, yes.

> * Map back from NFSv4 ACLs to POSIX ACLs on the client side, so that
>   POSIX ACLs on the server will appear as POSIX ACLs on the client. This
>   would take away all the mess we currently have in user-space and limit the
>   mapping problems to the NFSv4 protocol, where it really occurs. We could
>   define an additional NFsv4 attribute that the server could use to declare
>   if it supports POSIX ACLs only, so that the client won't have to try to
>   convert from NFSv4 to POSIX ACLs for servers that have native NFSv4 ACLs.

That's probably useful if we want to support non-Linux NFSv4 servers.

Re: Problems with POSIX ACL <=> NFSv4 ACL mapping in mainline

[Andreas Gruenbacher]

On Friday, 23 June 2006 18:06, J. Bruce Fields wrote:

> > The CITI has patches that try to work around this in libacl by trying
> > both POSIX ACLs and "system.nfs4_acl" attributes, and converting between
> > the two if necessary. This approach has a number of disadvantages: (1)
> > applications that don't use libacl will still fail
>
> Are there particular examples of such applications that you're thinking of?

Anything that simply copies xattrs from a file on NFSv4 to a file on another 
filesystem fails. The specific case I ran into was that I didn't apply the 
CITI libacl patches because I am unconvinced about the general approach, and 
I didn't feel well with adding the whole mess to libacl late in the SLES10 
game, let alone testing the code. The result were errors when copying files 
with cp -p (which preserves xattrs).

I much dislike adding a whole lot of complexity to libacl for a specific 
corner case only because of the NFSv4 to POSIX ACL mapping. This mapping 
should really be hidden from applications wherever possible (and only 
applications that specifically deal with implementing the mapping should have 
to deal with it).

> > (2) always trying both formats without knowing the properties of the
> > underlying filesystem will always be slow,
>
> How much slower?  I'd expect the cost of one system call (which just
> immediately returns -ENOTSUPP) to be insignificant.

The time overhead probably is low, yes. It's still a mess, and it will fail if 
ever we'll have a file system that understands both forms of ACLs. (How to 
resolve such conflicts is another question of course.)

> > In addition, I could imagine that we'll optionally support local
> > NFSv4-style ACLs on specific file systems somewhen in the future. (Samba
> > would benefit greatly from that for example; Solaris already can do
> > that.) Not being able to tell the one kind of ACLs from the other in a
> > world where we may have both sounds like a recipe for disaster to me.
>
> I'm not following you.  Currently nfsv4 and posix acls are being presented
> to userspace via different extended attributes, so you can tell one type
> from the other.

Except on NFSv4, we see POSIX ACLs represented as "system.nfs4_acl". I'd be 
much happier to see POSIX ACLs represented as POSIX ACLs, and NFSv4 ACLs and 
NFSv4 ACLs.

> > I could imagine the following solutions to this problem:
> >
> > * Don't map between POSIX ACLs and NFSv4 ACLs at all, and use a different
> > protocol for POSIX ACLs and for NFSv4 ACLs instead (e.g., version 4 of
> > the NFSACL protocol extension). The protocol extension is not currently
> > implemented in Solaris because current Solaris supports NFSv4 ACLs
> > natively instead of POSIX ACLs, and so they don't seem to have as strong
> > a motivation to keep POSIX ACLs working well as we currently do. The
> > NFSACL protocol has the huge advantage of supporting POSIX ACLs pretty
> > well, and so a lot of the overhead and complexity of doing the NFSv4
> > mapping would go away though.
>
> If you wanted to support *both* POSIX and NFSv4 ACLs, then you'd still have
> the same problem.  Unless you expect it to be possible to set both types of
> ACLs on the same file independently, and make the kernel enforce the AND of
> the two, or something like that.  I don't think that would be helpful.

Both at the same time probably is not a good idea, agreed. Perhaps storing an 
NFSv4 ACL when NFSv4 ACL tools are used and POSIX ACLs when POSIX ACL tools 
are used might benting version 4 of the NFSACL side-band protocol and dumping 
the NFSv4 ACL <=> POSIX ACL mapping in nfsd would completely get rid of no-op 
(hopefully) rounds of mappings. That would mean less complexity in the kernel 
(we already have version 3 NFSACL), and complexity in user spaceecome an 
option, or some such.

> > * Map back from NFSv4 ACLs to POSIX ACLs on the client side, so that
> > POSIX ACLs on the server will appear as POSIX ACLs on the client. This
> > would take away all the mess we currently have in user-space
>
> And put it in the kernel instead?

Yes.

> We originally did exactly that, and Trond argued that we should do this in
> userspace instead.  I still think that was the right decision.

Things would look different with native NFSv4 ACLs as on Solaris, but we are 
not in this situation.

> I'm not satisfied with the current situation either.  But I don't think
> Samba or NFSv4 will ever be completely happy until we provide good support
> for NFSv4/Windows-style ACLs.  We can't rip out POSIX ACLs in the process,
> so we're going to have to support both.  I don't see how to support both
> well without mapping between them.  And if we've got to do this complicated
> mapping, let's do it in userspace whenever possible.

Implementing version 4 of the NFSACL side-band protocol and dumping the NFSv4 
ACL <=> POSIX ACL mapping in nfsd would completely get rid of no-op 
(hopefully) rounds of mappings. That would mean less complexity in the kernel 
(we already have version 3 NFSACL, and version 4 is essentially the same).

Andreas

Re: Problems with POSIX ACL <=> NFSv4 ACL mapping in mainline

[J. Bruce Fields]

On Mon, Jun 26, 2006 at 05:30:09PM +0200, Andreas Gruenbacher wrote:
> I much dislike adding a whole lot of complexity to libacl for a specific 
> corner case only because of the NFSv4 to POSIX ACL mapping.

That's certainly understandable, but...

> This mapping should really be hidden from applications wherever possible (and
> only applications that specifically deal with implementing the mapping should
> have to deal with it).

... short of putting the mapping into the kernel, putting it in libacl seems
the closest we can come to hiding it from applications.  Maybe there are other
ways we could connect libacl with the NFSv4 mapping code that would minimize
the impact on libacl?

The current libacl patches need some work in any case.

> > > (2) always trying both formats without knowing the properties of the
> > > underlying filesystem will always be slow,
> >
> > How much slower?  I'd expect the cost of one system call (which just
> > immediately returns -ENOTSUPP) to be insignificant.
> 
> The time overhead probably is low, yes. It's still a mess, and it will fail
> if ever we'll have a file system that understands both forms of ACLs. (How to
> resolve such conflicts is another question of course.)

You've referred several times to the case of a file system that understand both
forms of ACLs.  How are you imagining that would work?  That's a problem that
forces some hard choices, and without knowing how you're thinking of making
those choices, I find it hard to judge whether the current system would help or
hurt that case.

> > > In addition, I could imagine that we'll optionally support local
> > > NFSv4-style ACLs on specific file systems somewhen in the future. (Samba
> > > would benefit greatly from that for example; Solaris already can do
> > > that.) Not being able to tell the one kind of ACLs from the other in a
> > > world where we may have both sounds like a recipe for disaster to me.
> >
> > I'm not following you.  Currently nfsv4 and posix acls are being presented
> > to userspace via different extended attributes, so you can tell one type
> > from the other.
> 
> Except on NFSv4, we see POSIX ACLs represented as "system.nfs4_acl". I'd be 
> much happier to see POSIX ACLs represented as POSIX ACLs, and NFSv4 ACLs and 
> NFSv4 ACLs.

The stuff stored in "system.nfs4_acl" clearly represents an NFSv4 ACL, not a
POSIX ACL, so I'm not sure what you mean here.

> Both at the same time probably is not a good idea, agreed. Perhaps storing an 
> NFSv4 ACL when NFSv4 ACL tools are used and POSIX ACLs when POSIX ACL tools 
> are used might benting version 4 of the NFSACL side-band protocol

(I think there was some email-editing mistake here?)

If what you're proposing is storing *either* NFSv4 ACLs, *or* POSIX ACLs, but
never both, that could work.  It has the problem that if two users using the
two different types of ACLs ever operate on the same file, then they won't be
able to see each others ACLs, and setting one type will completely lose the
other.

If we'd been willing to go for that kind of solution from the start, we could
have tossed mode bits and chosen ACLs that were simpler than both NFSv4 and
POSIX ACLs.  But backwards compatibility seems too strong a requirement....

> Implementing version 4 of the NFSACL side-band protocol and dumping the NFSv4 
> ACL <=> POSIX ACL mapping in nfsd would completely get rid of no-op 
> (hopefully) rounds of mappings. That would mean less complexity in the kernel 
> (we already have version 3 NFSACL, and version 4 is essentially the same).

I'm not as familiar with the NFSACL protocol as I should be.  It can't accept
NFSv4 filehandles, though, can it?

--b.

Re: Problems with POSIX ACL <=> NFSv4 ACL mapping in mainline

[Andreas Dilger]

On Jun 26, 2006  17:30 +0200, Andreas Gruenbacher wrote:
> The result were errors when copying files with cp -p (which preserves xattrs).

Is that true of "cp" in general, or only SuSE "cp"?  We get problems with
Lustre when root is copying files, because SuSE cp is incorrectly trying to
copy all of the EAs by default.  If I have an EA that is specific to a
file I don't necessarily want it to be copied.

It would probably be preferrable that cp only copy EAs that it understands.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.


Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: Problems with POSIX ACL <=> NFSv4 ACL mapping in mainline

[Andreas Gruenbacher]

On Monday, 26 June 2006 19:52, Andreas Dilger wrote:
> On Jun 26, 2006  17:30 +0200, Andreas Gruenbacher wrote:
> > The result were errors when copying files with cp -p (which preserves
> > xattrs).
>
> Is that true of "cp" in general, or only SuSE "cp"?

It's our cp only right now.

> We get problems with 
> Lustre when root is copying files, because SuSE cp is incorrectly trying to
> copy all of the EAs by default.  If I have an EA that is specific to a
> file I don't necessarily want it to be copied.

There's a blacklist in libattr which we could add Lustre attributes to. A 
blacklist somewhere in /etc would be even better though I guess.

> It would probably be preferrable that cp only copy EAs that it understands.

What to do with unknown attributes? Warn? Err out?

Andreas

Re: Problems with POSIX ACL <=> NFSv4 ACL mapping in mainline

[Andreas Dilger]

On Jun 26, 2006  20:07 +0200, Andreas Gruenbacher wrote:
> On Monday, 26 June 2006 19:52, Andreas Dilger wrote:
> > On Jun 26, 2006  17:30 +0200, Andreas Gruenbacher wrote:
> > > The result were errors when copying files with cp -p (which preserves
> > > xattrs).
> >
> > Is that true of "cp" in general, or only SuSE "cp"?
> 
> It's our cp only right now.
> 
> > We get problems with 
> > Lustre when root is copying files, because SuSE cp is incorrectly trying to
> > copy all of the EAs by default.  If I have an EA that is specific to a
> > file I don't necessarily want it to be copied.
> 
> There's a blacklist in libattr which we could add Lustre attributes to.

I figured as much after digging through the SLES-patched cp, I just hadn't
gotten as far as grabbing the libattr source and seeing what that was.  Is
this blacklist specific to the SLES libattr, or is this in the released
version?

> A blacklist somewhere in /etc would be even better though I guess.

Definitely, yes.  Given that libselinx gropes around in a several files
already, adding one more won't hurt.  Something like /etc/libattr.conf?

> > It would probably be preferrable that cp only copy EAs that it understands.
> 
> What to do with unknown attributes? Warn? Err out?

Warn once, maybe with pointer to /etc/libattr.conf for adding to include or
exclude list.  Given that copying file attributes isn't guaranteed in any
case, we can probably safely live without copying everything.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.