[PATCH v2] nfsd: enforce export security for NFSv4 filehandles and stateids

[VeroFess <coderzeng@gmail.com>]

An AUTH_SYS NFSv4.0 client can read or write a file on a sec=krb5p
export using another client's valid OPEN stateid by inserting
SAVEFH between PUTFH and READ or WRITE.

PUTFH resolves the handle with NFSD_MAY_BYPASS_GSS, expecting
a later security check. SAVEFH carries OP_HANDLES_WRONGSEC even though
it only saves the handle, so PUTFH skips NFS4ERR_WRONGSEC. Stateid I/O
can then reuse a cached nfsd_file.

The original report exported /srv/nfs-sec to * with these options:

  rw,sync,no_subtree_check,fsid=0,sec=krb5p,secure,root_squash

The 0600, nobody:nogroup file /srv/nfs-sec/secret.txt held 32 random
bytes. A Kerberos privacy NFSv4.0 client ran SETCLIENTID, OPEN and
OPEN_CONFIRM, then kept its read/write stateid live. A separate
AUTH_SYS client used UID/GID 65534 and a reserved source port.

The AUTH_SYS client enumerated a bounded range of nearby clientid counters,
stateid numbers and seqids to find the live OPEN stateid. With it,
PUTFH, READ returned NFS4ERR_WRONGSEC, while PUTFH, SAVEFH, READ
returned the file data and PUTFH, SAVEFH, WRITE changed it. The
Kerberos client subsequently read the changed contents.

SAVEFH leaves CURRENT_FH in place. When deciding whether PUTFH must
report NFS4ERR_WRONGSEC, look past SAVEFH to the first operation that
uses or replaces CURRENT_FH. If another PUTFH replaces it, leave the
earlier PUTFH unchecked, as required by RFC 8881 Section 2.6.3.1.1.

Local COPY and CLONE consume SAVED_FH as the source. Check the
saved export with check_nfsd_access() before using its stateid:
a cached open stateid can otherwise bypass the sec= policy. Map
NFS4ERR_WRONGSEC to NFS4ERR_ACCESS at COPY or CLONE, since neither
operation permits NFS4ERR_WRONGSEC.

An inter-server COPY carries a foreign SAVED_FH. The source server
validates it; the destination must not apply its own export policy
to that handle. The destination PUTFH still checks its local export.

A bind-mounted regular file can have a stricter export than its parent.
LOOKUP crosses into that export, but nfsd_cross_mnt() checks the
parent export. Check the selected export in nfsd_lookup() before
composing a filehandle, and in READDIR before encoding child attributes.
When rdattr_error is not requested, return NFS4ERR_ACCESS for a flavor
mismatch; NFS4ERR_WRONGSEC is not a READDIR status. Keep the child-export
check out of nfsd_cross_mnt(): SECINFO uses it to discover the child's
supported security flavors.

The LOOKUP check controls whether a client obtains a new child handle.
It does not cover a handle and OPEN stateid retained while the export's
sec= policy changes. The cached nfsd_file branch of nfs4_check_file()
calls nfsd_permission() without checking the current export's flavor.
Check that export before reusing the file for stateid I/O.

COPY_NOTIFY validates a stateid without acquiring an nfsd_file, so it
misses that check. After an export was changed to sec=krb5p, an
AUTH_SYS COPY_NOTIFY using an existing stateid still issued a copy
stateid although AUTH_SYS READ was denied. In the test, a separate
user-space GSS client used that stateid to read the file. Check the
current export before issuing it and return NFS4ERR_ACCESS on a
flavor mismatch.

Fixes: 68d93184352f ("nfsd4: fix wrongsec handling for PUTFH + op cases")
Fixes: 624322f1adc5 ("NFSD add COPY_NOTIFY operation")
Fixes: f0d6ad5a2f67 ("nfsd: move check_nfsd_access() call into nfsd_cross_mnt()")
Assisted-by: LLM
Signed-off-by: VeroFess <coderzeng@gmail.com>
---
Changes since v1:
  Check the selected export after LOOKUP and READDIR mount crossing.
  Map READDIR's operation-level WRONGSEC to ACCESS when rdattr_error
  was not requested. Add guards for cached stateid I/O and COPY_NOTIFY.
  Restore the RFC comments, drop a redundant assignment, and add the
  missing Fixes tags. Regenerate the diff after v1's whitespace damage.

Tests and review notes:
  W=1 object builds and an x86_64 KASAN bzImage completed. Runtime
  tests used bcd97658ca9b; the diff also applied without fuzz to
  cel/nfsd-testing at 944812cf2b77.

  PUTFH, SAVEFH, READ/WRITE returned NFS4ERR_WRONGSEC at PUTFH.
  SAVEFH alone, a replacement PUTFH, and SECINFO_NO_NAME remained
  allowed; RESTOREFH, READ checked at RESTOREFH.

  An ordinary file export kept the parent handle on LOOKUP; a bind
  mount selected the stricter child export. With parent sec=sys:krb5p
  and child sec=krb5p, AUTH_SYS LOOKUP, READ/WRITE succeeded before
  this fix using a GSS-opened stateid. The v4.0 test used another
  client; v4.1 and v4.2 changed credentials within one session. The
  final kernel returned NFS4ERR_WRONGSEC at LOOKUP on all three
  versions. Direct PUTFH already returned NFS4ERR_WRONGSEC. GSS
  LOOKUP, AUTH_SYS SECINFO, and access to a public sibling succeeded.
  SECINFO returned krb5p for the strict child. Source review found
  that the Linux client calls SECINFO(parent, name) after LOOKUP
  WRONGSEC and retries LOOKUP with the selected flavor.
  Cross-client stateids were not tested on v4.1 or v4.2.

  On v4.0, v4.1, and v4.2, READDIR reported the strict child's
  NFS4ERR_WRONGSEC in rdattr_error and kept public entries intact.
  With size but no rdattr_error, READDIR returned NFS4ERR_ACCESS;
  name-only READDIR succeeded. COPY into the strict child copied
  4096 bytes before this fix; the final LOOKUP check now prevents
  that destination from being selected under AUTH_SYS. CLONE reached
  ext4 but returned NFS4ERR_NOTSUPP before the fix, so no clone
  completed. Protected saved-source COPY/CLONE still returned
  NFS4ERR_ACCESS.

  Before restoring the LOOKUP check, the cached-file guard returned
  NFS4ERR_ACCESS for READ/WRITE, READ_PLUS, SEEK, ALLOCATE, DEALLOCATE,
  and COPY with the strict child handle. GETATTR, ACCESS, and OPEN
  already returned NFS4ERR_WRONGSEC. Source review found fh_verify()
  in SETATTR, READLINK, VERIFY/NVERIFY, and xattrs; COMMIT acquires
  an nfsd_file. LINK/RENAME saved-FH probes returned NFS4ERR_WRONGSEC.

  After a live AUTH_SYS OPEN, changing the export to sec=krb5p and
  mode 000 denied AUTH_SYS READ but still allowed COPY_NOTIFY to
  issue a copy stateid before this fix. A separate user-space GSS
  client could not OPEN the file, yet read 4096 bytes with that
  stateid and wrote them to another export. An intermediate build
  rejected COPY_NOTIFY with NFS4ERR_ACCESS before issuing a stateid.
  With the final kernel, both READ and COPY_NOTIFY compounds stop at
  LOOKUP with NFS4ERR_WRONGSEC. Source review and repeated denial
  found no notification allocation or retained stateid reference
  on COPY_NOTIFY's own error path.

  Stock NFSD-to-NFSD COPY from the strict export ended with
  NFS4ERR_OFFLOAD_DENIED and zero bytes copied; the permissive-source
  control copied the file. With the final kernel, an authorized foreign-FH
  inter-server COPY completed and the destination matched the source.
  A local-only COPY with a foreign FH returned NFS4ERR_STALE at PUTFH.

  The targeted KASAN guests logged no KASAN fault or NFSD warning.
  pNFS, other architectures, and the full pynfs suite were not tested.

base-commit: bcd97658ca9be5a691eff14065a8f2dcfef6bbaf

 fs/nfsd/nfs4proc.c  | 43 ++++++++++++++++++++++++++++++-------------
 fs/nfsd/nfs4state.c |  6 ++++++
 fs/nfsd/nfs4xdr.c   |  8 +++++++-
 fs/nfsd/vfs.c       |  4 ++++
 4 files changed, 47 insertions(+), 14 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index a50f9dcb3..3a7f5e96a 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -1491,6 +1491,11 @@ nfsd4_verify_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	if (!cstate->save_fh.fh_dentry)
 		return nfserr_nofilehandle;
 
+	/* A cached open stateid does not authorize the saved source export. */
+	status = check_nfsd_access(cstate->save_fh.fh_export, rqstp);
+	if (status)
+		return nfserr_acces;
+
 	status = nfs4_preprocess_stateid_op(rqstp, cstate, &cstate->save_fh,
 					    src_stateid, RD_STATE, src, NULL);
 	if (status)
@@ -2532,6 +2537,12 @@ nfsd4_copy_notify(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	if (!stid)
 		return nfserr_bad_stateid;
 
+	status = check_nfsd_access(cstate->current_fh.fh_export, rqstp);
+	if (status) {
+		status = nfserr_acces;
+		goto out;
+	}
+
 	cn->cpn_lease_time.tv_sec = nn->nfsd4_lease;
 	cn->cpn_lease_time.tv_nsec = 0;
 
@@ -3237,32 +3248,38 @@ static bool need_wrongsec_check(struct svc_rqst *rqstp)
 	struct nfsd4_compoundres *resp = rqstp->rq_resp;
 	struct nfsd4_compoundargs *argp = rqstp->rq_argp;
 	struct nfsd4_op *this = &argp->ops[resp->opcnt - 1];
-	struct nfsd4_op *next = &argp->ops[resp->opcnt];
+	struct nfsd4_op *next;
 	const struct nfsd4_operation *thisd = OPDESC(this);
 	const struct nfsd4_operation *nextd;
+	unsigned int i;
 
-	/*
-	 * Most ops check wronsec on our own; only the putfh-like ops
-	 * have special rules.
-	 */
+	/* Only put-filehandle operations need a deferred WRONGSEC check. */
 	if (!(thisd->op_flags & OP_IS_PUTFH_LIKE))
 		return false;
+
 	/*
-	 * rfc 5661 2.6.3.1.1.6: don't bother erroring out a
-	 * put-filehandle operation if we're not going to use the
-	 * result:
+	 * RFC 8881 2.6.3.1.1.1: ignore SAVEFH when choosing the operation
+	 * that determines whether the put-filehandle operation checks WRONGSEC.
 	 */
-	if (argp->opcnt == resp->opcnt)
+	for (i = resp->opcnt; i < argp->opcnt; i++) {
+		next = &argp->ops[i];
+		if (next->opnum != OP_SAVEFH)
+			break;
+	}
+
+	/* RFC 8881 2.6.3.1.1.6: the filehandle is not used. */
+	if (i == argp->opcnt)
 		return false;
+
 	if (next->opnum == OP_ILLEGAL)
 		return false;
 	nextd = OPDESC(next);
 	/*
-	 * Rest of 2.6.3.1.1: certain operations will return WRONGSEC
-	 * errors themselves as necessary; others should check for them
-	 * now:
+	 * RFC 8881 2.6.3.1.1.2: a later put-filehandle operation replaces
+	 * this one. Other marked operations check WRONGSEC themselves.
 	 */
-	return !(nextd->op_flags & OP_HANDLES_WRONGSEC);
+	return !(nextd->op_flags &
+		  (OP_HANDLES_WRONGSEC | OP_IS_PUTFH_LIKE));
 }
 
 #ifdef CONFIG_NFSD_V4_2_INTER_SSC
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index f2b52ba90..27f5b1a58 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -8486,6 +8486,12 @@ nfs4_check_file(struct svc_rqst *rqstp, struct svc_fh *fhp, struct nfs4_stid *s,
 
 	nf = nfs4_find_file(s, flags);
 	if (nf) {
+		/* A cached file still needs the current export security check. */
+		status = check_nfsd_access(fhp->fh_export, rqstp);
+		if (status) {
+			nfsd_file_put(nf);
+			return nfserr_acces;
+		}
 		status = nfsd_permission(&rqstp->rq_cred,
 					 fhp->fh_export, fhp->fh_dentry,
 				acc | NFSD_MAY_OWNER_OVERRIDE);
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 4d9834446..daf2ad77d 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -4677,6 +4677,9 @@ nfsd4_encode_entry4_fattr(struct nfsd4_readdir *cd, const char *name,
 		 * At least, add a comment here to explain....
 		 */
 		nfserr = nfsd_cross_mnt(cd->rd_rqstp, &dentry, &exp);
+		if (nfserr)
+			goto out_put;
+		nfserr = check_nfsd_access(exp, cd->rd_rqstp);
 		if (nfserr)
 			goto out_put;
 		crossed = true;
@@ -4775,8 +4778,11 @@ nfsd4_encode_entry4(void *ccdv, const char *name, int namlen,
 		 * then in accordance with the spec, we fail the
 		 * entire READDIR operation(!)
 		 */
-		if (!(cd->rd_bmval[0] & FATTR4_WORD0_RDATTR_ERROR))
+		if (!(cd->rd_bmval[0] & FATTR4_WORD0_RDATTR_ERROR)) {
+			if (nfserr == nfserr_wrongsec)
+				nfserr = nfserr_acces;
 			goto fail;
+		}
 		if (nfsd4_encode_entry4_rdattr_error(xdr, nfserr)) {
 			nfserr = nfserr_toosmall;
 			goto fail;
diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c
index f59a55f89..8935a6fcc 100644
--- a/fs/nfsd/vfs.c
+++ b/fs/nfsd/vfs.c
@@ -340,6 +340,9 @@ nfsd_lookup(struct svc_rqst *rqstp, struct svc_fh *fhp, const char *name,
 	err = nfsd_lookup_dentry(rqstp, fhp, name, len, &exp, &dentry);
 	if (err)
 		return err;
+	err = check_nfsd_access(exp, rqstp);
+	if (err)
+		goto out;
 	/*
 	 * Note: we compose the file handle now, but as the
 	 * dentry may be negative, it may need to be updated.
@@ -348,6 +351,7 @@ nfsd_lookup(struct svc_rqst *rqstp, struct svc_fh *fhp, const char *name,
 	if (!err && d_really_is_negative(dentry))
 		err = nfserr_noent;
 
+out:
 	dput(dentry);
 	exp_put(exp);
 	return err;
-- 
2.43.0