Re: NFSv3 and xprtsec policies

[Scott Mayhew <smayhew@redhat.com>]

On Fri, 03 May 2024, Chuck Lever wrote:

> On Thu, May 02, 2024 at 02:51:23PM -0400, Scott Mayhew wrote:
> > On Thu, 02 May 2024, Chuck Lever III wrote:
> > 
> > > 
> > > 
> > > > On May 2, 2024, at 1:37 PM, Scott Mayhew <smayhew@redhat.com> wrote:
> > > > 
> > > > On Thu, 02 May 2024, Chuck Lever III wrote:
> > > > 
> > > >>> On May 2, 2024, at 11:54 AM, Scott Mayhew <smayhew@redhat.com> wrote:
> > > >>> 
> > > >>> Red Hat QE identified an "interesting" issue with NFSv3 and TLS, in that an
> > > >>> NFSv3 client can mount with "xprtsec=none" a filesystem exported with
> > > >>> "xprtsec=tls:mtls" (in the sense that the client gets the filehandle and adds a
> > > >>> mount to its mount table - it can't actually access the mount).
> > > >>> 
> > > >>> Here's an example using machines from the recent Bakeathon.
> > > >>> 
> > > >>> Mounting a server with TLS enabled:
> > > >>> 
> > > >>> # mount -o v4.2,sec=sys,xprtsec=tls oracle-102.chuck.lever.oracle.com.nfsv4.dev:/export/tls /mnt
> > > >>> # umount /mnt
> > > >>> 
> > > >>> Trying to mount without "xprtsec=tls" shows that the filesystem is not exported with "xprtsec=none":
> > > >>> 
> > > >>> # mount -o v4.2,sec=sys oracle-102.chuck.lever.oracle.com.nfsv4.dev:/export/tls /mnt
> > > >>> mount.nfs: Operation not permitted for oracle-102.chuck.lever.oracle.com.nfsv4.dev:/export/tls on /mnt
> > > >>> 
> > > >>> Yet a v3 mount without "xprtsec=tls" works:
> > > >>> 
> > > >>> # mount -o v3,sec=sys oracle-102.chuck.lever.oracle.com.nfsv4.dev:/export/tls /mnt
> > > >>> # umount /mnt
> > > >>> 
> > > >>> and a mount with no explicit version and without "xprtsec=tls" falls back to
> > > >>> v3 and also "works":
> > > >>> 
> > > >>> # mount -o sec=sys oracle-102.chuck.lever.oracle.com.nfsv4.dev:/export/tls /mnt
> > > >>> # grep ora /proc/mounts
> > > >>> oracle-102.chuck.lever.oracle.com.nfsv4.dev:/export/tls /mnt nfs
> > > >>> +rw,relatime,vers=3,rsize=524288,wsize=524288,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=100.64.0.49,mountvers=3,mountport=20048,mountproto=udp,local_lock=none,addr=100.64.0.49 0 0
> > > >>> 
> > > >>> Even though the filesystem is mounted, the client can't do anything with it:
> > > >>> 
> > > >>> # ls /mnt
> > > >>> ls: cannot open directory '/mnt': Permission denied
> > > >>> 
> > > >>> When krb5 is used with NFSv3, the server returns a list of pseudoflavors in
> > > >>> mountres3_ok (https://datatracker.ietf.org/doc/html/rfc1813#section-5.2.1).
> > > >>> The client compares that list with its own list of auth flavors parsed from the
> > > >>> mount request and returns -EACCES if no match is found (see
> > > >>> nfs_verify_authflavors()).
> > > >>> 
> > > >>> Perhaps we should be doing something similar with xprtsec policies?
> > > >> 
> > > >> The problem might be in how you've set up the exports. With NFSv3,
> > > >> the parent export needs the "crossmnt" export option in order for
> > > >> NFSv3 to behave like NFSv4 in this regard, although I could have
> > > >> missed something.
> > > > 
> > > > I was mounting your server though :)
> > > 
> > > OK, then not the same bug that Olga found last year.
> > > 
> > > We should find out what FreeBSD does in this case.
> > 
> > I thought about that.  Rick's servers from the BAT are offline, and I
> > don't think he was exporting v3 anyway.
> > 
> > > 
> > > 
> > > >>> Should
> > > >>> there be an errata to RFC 9289 and a request from IANA for assigned numbers for
> > > >>> pseudo-flavors corresponding to xprtsec policies?
> > > >> 
> > > >> No. Transport-layer security is not an RPC security flavor or
> > > >> pseudo-flavor. These two things are not related.
> > > >> 
> > > >> (And in fact, I proposed something like this for NFSv4 SECINFO,
> > > >> but it was rejected).
> > > > 
> > > > I thought it might be a stretch to try to use mountres3.auth_flavors for
> > > > this, but since RFC 9289 does refer to AUTH_TLS as an authentication
> > > > flavor and https://www.iana.org/assignments/rpc-authentication-numbers/rpc-authentication-numbers.xhtml
> > > > also lists TLS under the Flavor Name column I thought it might make
> > > > sense to treat xprtsec policies as if they were pseudo-flavors even
> > > > though they're not, if only to give the client a way to determine that
> > > > the mount should fail.
> > > 
> > > RPC_AUTH_TLS is used only when a client probes a server to see if
> > > it supports RPC-with-TLS. At all other times, the client uses one
> > > of the normal, legitimate flavors. It does not represent a security
> > > flavor that can be used during regular operation.
> > > 
> > > NFSv3 mount failover logic is still open for discussion (ie, incomplete).
> > > 
> > > Would it help if rpc.mountd stuck RPC_AUTH_TLS in the auth_flavors
> > > list? I think clients that don't recognize it should ignore it,
> > > but I'm not sure. What should a client do if it sees that flavor in
> > > the list? It's not one that can be used for any other procedure than
> > > a NULL RPC.
> > 
> > Maybe?  After the client gets the filehandle it's calling FSINFO and
> > PATHCONF.  The latter get NFS3ERR_ACCES, but nfs_probe_fsinfo() isn't
> > checking for a negative return code from the PATHCONF operation.  If it
> > did, it could maybe use the -EACCES coupled with the knowledge that the
> > server had RPC_AUTH_TLS enabled to emit an error message saying to check
> > the xprtsec policies (but I don't think that would be as definitive as
> > what I had in mind) and to fail the mount.
> 
> If Linux is the only implementation of NFSv3 with TLS so far, then
> we have some latitude for innovation.
> 
> I would like to hear from the client maintainers about what they
> would prefer the client user experience to look like. Then NFSD's
> behavior can be adjusted to accommodate.
> 
> In this case, Steve would have to sign off on an rpc.mountd change
> to return AUTH_TLS in the auth_flavors list.

Maybe instead of messing with the auth_flavors list, we just have the
client check the status from the PATHCONF operation and leave it at
that.

---8<---
diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index 44eca51b2808..09d28dae0f06 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -867,8 +867,10 @@ static int nfs_probe_fsinfo(struct nfs_server *server, struct nfs_fh *mntfh, str
 		pathinfo.fattr = fattr;
 		nfs_fattr_init(fattr);
 
-		if (clp->rpc_ops->pathconf(server, mntfh, &pathinfo) >= 0)
-			server->namelen = pathinfo.max_namelen;
+		error = clp->rpc_ops->pathconf(server, mntfh, &pathinfo);
+		if (error < 0)
+			return error;
+		server->namelen = pathinfo.max_namelen;
 	}
 
 	if (clp->rpc_ops->discover_trunking != NULL &&
---8<---

That's sufficient to make the mount fail with EACCES, which should be
enough of a clue for someone that they should compare the client's mount
options with the server's export options.

[root@rawhide-client ~]# mount -o v3,sec=sys oracle-102.chuck.lever.oracle.com.nfsv4.dev:/export/tls /mnt
mount.nfs: access denied by server while mounting oracle-102.chuck.lever.oracle.com.nfsv4.dev:/export/tls

-Scott

> 
> 
> -- 
> Chuck Lever
>