[PATCH v2 0/2] nfsd: close potential race between open and delegation

[PATCH v2 0/2] nfsd: close potential race between open and delegation

[Jeff Layton]

v2:
- use nfsd_lookup_dentry instead of lookup_one_len

Here's a respin of the patches to fix up the potential race between an
open and delegation. I took Neil's advice an changed over to use
nfsd_lookup_dentry.

This patchset is based on top of Neil's recent patchset entitled:

    [PATCH 0/8] NFSD: clean up locking.

Tested with xfstests and it seemed to behave. I haven't done any testing
to ensure that the race is actually fixed, mainly because I don't have a
way to reliably reproduce it.

Jeff Layton (2):
  nfsd: drop fh argument from alloc_init_deleg
  nfsd: vet the opened dentry after setting a delegation

 fs/nfsd/nfs4state.c | 58 ++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 49 insertions(+), 9 deletions(-)

-- 
2.36.1

[PATCH v2 1/2] nfsd: drop fh argument from alloc_init_deleg

[Jeff Layton]

Currently, we pass the fh of the opened file down through several
functions so that alloc_init_deleg can pass it to delegation_blocked.
The filehandle of the open file is available in the nfs4_file however,
so there's no need to pass it in a separate argument.

Drop the argument from alloc_init_deleg, nfs4_open_delegation and
nfs4_set_delegation.

Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
 fs/nfsd/nfs4state.c | 14 ++++++--------
 1 file changed, 6 insertions(+), 8 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 8201d716a557..d2d21fdf5c41 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1131,7 +1131,6 @@ static void block_delegations(struct knfsd_fh *fh)
 
 static struct nfs4_delegation *
 alloc_init_deleg(struct nfs4_client *clp, struct nfs4_file *fp,
-		 struct svc_fh *current_fh,
 		 struct nfs4_clnt_odstate *odstate)
 {
 	struct nfs4_delegation *dp;
@@ -1141,7 +1140,7 @@ alloc_init_deleg(struct nfs4_client *clp, struct nfs4_file *fp,
 	n = atomic_long_inc_return(&num_delegations);
 	if (n < 0 || n > max_delegations)
 		goto out_dec;
-	if (delegation_blocked(&current_fh->fh_handle))
+	if (delegation_blocked(&fp->fi_fhandle))
 		goto out_dec;
 	dp = delegstateid(nfs4_alloc_stid(clp, deleg_slab, nfs4_free_deleg));
 	if (dp == NULL)
@@ -5269,7 +5268,7 @@ static int nfsd4_check_conflicting_opens(struct nfs4_client *clp,
 }
 
 static struct nfs4_delegation *
-nfs4_set_delegation(struct nfs4_client *clp, struct svc_fh *fh,
+nfs4_set_delegation(struct nfs4_client *clp,
 		    struct nfs4_file *fp, struct nfs4_clnt_odstate *odstate)
 {
 	int status = 0;
@@ -5314,7 +5313,7 @@ nfs4_set_delegation(struct nfs4_client *clp, struct svc_fh *fh,
 		return ERR_PTR(status);
 
 	status = -ENOMEM;
-	dp = alloc_init_deleg(clp, fp, fh, odstate);
+	dp = alloc_init_deleg(clp, fp, odstate);
 	if (!dp)
 		goto out_delegees;
 
@@ -5382,8 +5381,7 @@ static void nfsd4_open_deleg_none_ext(struct nfsd4_open *open, int status)
  * proper support for them.
  */
 static void
-nfs4_open_delegation(struct svc_fh *fh, struct nfsd4_open *open,
-			struct nfs4_ol_stateid *stp)
+nfs4_open_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
 {
 	struct nfs4_delegation *dp;
 	struct nfs4_openowner *oo = openowner(stp->st_stateowner);
@@ -5415,7 +5413,7 @@ nfs4_open_delegation(struct svc_fh *fh, struct nfsd4_open *open,
 		default:
 			goto out_no_deleg;
 	}
-	dp = nfs4_set_delegation(clp, fh, stp->st_stid.sc_file, stp->st_clnt_odstate);
+	dp = nfs4_set_delegation(clp, stp->st_stid.sc_file, stp->st_clnt_odstate);
 	if (IS_ERR(dp))
 		goto out_no_deleg;
 
@@ -5547,7 +5545,7 @@ nfsd4_process_open2(struct svc_rqst *rqstp, struct svc_fh *current_fh, struct nf
 	* Attempt to hand out a delegation. No error return, because the
 	* OPEN succeeds even if we fail.
 	*/
-	nfs4_open_delegation(current_fh, open, stp);
+	nfs4_open_delegation(open, stp);
 nodeleg:
 	status = nfs_ok;
 	trace_nfsd_open(&stp->st_stid.sc_stateid);
-- 
2.36.1

[PATCH v2 2/2] nfsd: vet the opened dentry after setting a delegation

[Jeff Layton]

Between opening a file and setting a delegation on it, someone could
rename or unlink the dentry. If this happens, we do not want to grant a
delegation on the open.

On a CLAIM_NULL open, we're opening by filename, and we'll hold the
i_rwsem while when attempting to set a delegation. After getting a
lease, redo the lookup of the file being opened and validate that the
resulting dentry matches the one in the open file description.

Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
 fs/nfsd/nfs4state.c | 52 ++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 47 insertions(+), 5 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index d2d21fdf5c41..a0258214c279 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -5267,11 +5267,42 @@ static int nfsd4_check_conflicting_opens(struct nfs4_client *clp,
 	return 0;
 }
 
+/*
+ * It's possible that between opening the dentry and setting the delegation,
+ * that it has been renamed or unlinked. Redo the lookup to validate that this
+ * hasn't happened.
+ */
+static int
+nfsd4_vet_deleg_dentry(struct svc_rqst *rqstp, struct nfsd4_open *open,
+		       struct nfs4_file *fp, struct svc_fh *parent)
+{
+	__be32 err;
+	struct dentry *child;
+	struct svc_export *exp;
+
+	lockdep_assert_held(&d_inode(parent->fh_dentry)->i_rwsem);
+
+	err = nfsd_lookup_dentry(rqstp, parent, open->op_fname, open->op_fnamelen,
+				 &exp, &child, true);
+	if (err)
+		return -EAGAIN;
+
+	dput(child);
+	exp_put(exp);
+
+	if (child != file_dentry(fp->fi_deleg_file->nf_file))
+		return -EAGAIN;
+	return 0;
+}
+
 static struct nfs4_delegation *
-nfs4_set_delegation(struct nfs4_client *clp,
-		    struct nfs4_file *fp, struct nfs4_clnt_odstate *odstate)
+nfs4_set_delegation(struct svc_rqst *rqstp, struct nfsd4_open *open, struct nfs4_ol_stateid *stp,
+		    struct svc_fh *parent)
 {
 	int status = 0;
+	struct nfs4_client *clp = stp->st_stid.sc_client;
+	struct nfs4_file *fp = stp->st_stid.sc_file;
+	struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate;
 	struct nfs4_delegation *dp;
 	struct nfsd_file *nf;
 	struct file_lock *fl;
@@ -5326,6 +5357,13 @@ nfs4_set_delegation(struct nfs4_client *clp,
 		locks_free_lock(fl);
 	if (status)
 		goto out_clnt_odstate;
+
+	if (parent) {
+		status = nfsd4_vet_deleg_dentry(rqstp, open, fp, parent);
+		if (status)
+			goto out_unlock;
+	}
+
 	status = nfsd4_check_conflicting_opens(clp, fp);
 	if (status)
 		goto out_unlock;
@@ -5381,11 +5419,13 @@ static void nfsd4_open_deleg_none_ext(struct nfsd4_open *open, int status)
  * proper support for them.
  */
 static void
-nfs4_open_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
+nfs4_open_delegation(struct svc_rqst *rqstp, struct nfsd4_open *open, struct nfs4_ol_stateid *stp,
+		     struct svc_fh *parent)
 {
 	struct nfs4_delegation *dp;
 	struct nfs4_openowner *oo = openowner(stp->st_stateowner);
 	struct nfs4_client *clp = stp->st_stid.sc_client;
+	struct svc_fh *deleg_parent = NULL;
 	int cb_up;
 	int status = 0;
 
@@ -5399,6 +5439,8 @@ nfs4_open_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
 				goto out_no_deleg;
 			break;
 		case NFS4_OPEN_CLAIM_NULL:
+			deleg_parent = parent;
+			fallthrough;
 		case NFS4_OPEN_CLAIM_FH:
 			/*
 			 * Let's not give out any delegations till everyone's
@@ -5413,7 +5455,7 @@ nfs4_open_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
 		default:
 			goto out_no_deleg;
 	}
-	dp = nfs4_set_delegation(clp, stp->st_stid.sc_file, stp->st_clnt_odstate);
+	dp = nfs4_set_delegation(rqstp, open, stp, deleg_parent);
 	if (IS_ERR(dp))
 		goto out_no_deleg;
 
@@ -5545,7 +5587,7 @@ nfsd4_process_open2(struct svc_rqst *rqstp, struct svc_fh *current_fh, struct nf
 	* Attempt to hand out a delegation. No error return, because the
 	* OPEN succeeds even if we fail.
 	*/
-	nfs4_open_delegation(open, stp);
+	nfs4_open_delegation(rqstp, open, stp, &resp->cstate.current_fh);
 nodeleg:
 	status = nfs_ok;
 	trace_nfsd_open(&stp->st_stid.sc_stateid);
-- 
2.36.1

Re: [PATCH v2 0/2] nfsd: close potential race between open and delegation

[Chuck Lever III]

> On Jul 15, 2022, at 2:32 PM, Jeff Layton <jlayton@kernel.org> wrote:
> 
> v2:
> - use nfsd_lookup_dentry instead of lookup_one_len
> 
> Here's a respin of the patches to fix up the potential race between an
> open and delegation. I took Neil's advice an changed over to use
> nfsd_lookup_dentry.
> 
> This patchset is based on top of Neil's recent patchset entitled:
> 
>    [PATCH 0/8] NFSD: clean up locking.

Thanks to both of you for pursuing this work! I think there are
some good improvements here.

Note that there are some outstanding review comments (aside from
the disagreement about how to refactor nfsd_create) so I expect
Neil will be reposting his series. This is just to note that, as
long as your series is based on his, I will consider your series
as RFC until his base series is stable and pulled.

I'll review again today or over the weekend.


> Tested with xfstests and it seemed to behave. I haven't done any testing
> to ensure that the race is actually fixed, mainly because I don't have a
> way to reliably reproduce it.

That's the thing: we don't have many tests that use multiple clients
targeting the same set of files.


> Jeff Layton (2):
>  nfsd: drop fh argument from alloc_init_deleg
>  nfsd: vet the opened dentry after setting a delegation
> 
> fs/nfsd/nfs4state.c | 58 ++++++++++++++++++++++++++++++++++++++-------
> 1 file changed, 49 insertions(+), 9 deletions(-)
> 
> -- 
> 2.36.1
> 

--
Chuck Lever

Re: [PATCH v2 0/2] nfsd: close potential race between open and delegation

[Jeff Layton]

On Fri, 2022-07-15 at 18:42 +0000, Chuck Lever III wrote:
> 
> > On Jul 15, 2022, at 2:32 PM, Jeff Layton <jlayton@kernel.org> wrote:
> > 
> > v2:
> > - use nfsd_lookup_dentry instead of lookup_one_len
> > 
> > Here's a respin of the patches to fix up the potential race between an
> > open and delegation. I took Neil's advice an changed over to use
> > nfsd_lookup_dentry.
> > 
> > This patchset is based on top of Neil's recent patchset entitled:
> > 
> >    [PATCH 0/8] NFSD: clean up locking.
> 
> Thanks to both of you for pursuing this work! I think there are
> some good improvements here.
> 
> Note that there are some outstanding review comments (aside from
> the disagreement about how to refactor nfsd_create) so I expect
> Neil will be reposting his series. This is just to note that, as
> long as your series is based on his, I will consider your series
> as RFC until his base series is stable and pulled.
> 
> I'll review again today or over the weekend.
> 

Thanks. I think I'll be able to adapt this approach on top of whatever
Neil comes up with.

> 
> > Tested with xfstests and it seemed to behave. I haven't done any testing
> > to ensure that the race is actually fixed, mainly because I don't have a
> > way to reliably reproduce it.
> 
> That's the thing: we don't have many tests that use multiple clients
> targeting the same set of files.
> 
> 

Yeah, it's a difficult problem. Testing delegation behavior is
particularly difficult since the client doesn't have a lot of control
over them being granted in the first place.

> > Jeff Layton (2):
> >  nfsd: drop fh argument from alloc_init_deleg
> >  nfsd: vet the opened dentry after setting a delegation
> > 
> > fs/nfsd/nfs4state.c | 58 ++++++++++++++++++++++++++++++++++++++-------
> > 1 file changed, 49 insertions(+), 9 deletions(-)
> > 
> > -- 
> > 2.36.1
> > 
> 
> --
> Chuck Lever
> 
> 
> 

-- 
Jeff Layton <jlayton@kernel.org>