[PATCH 00/19] OPEN optimisations and Attribute delegations

[PATCH 00/19] OPEN optimisations and Attribute delegations

[trondmy]

From: Trond Myklebust <trond.myklebust@hammerspace.com>

Now that https://datatracker.ietf.org/doc/draft-ietf-nfsv4-delstid/ is
mostly done with the review process, it is time to look at pushing the
client implementation that we've been working on upstream.

The following patch series therefore adds support for the NFSv4.2
extension to OP_OPEN to allow the client to request that the server
return either an open stateid or a delegation instead of always sending
the open stateid whether or not a delegation is returned.
This allows us to optimise away CLOSE, and hence makes small or cached
file access significantly more efficient.

It also adds support for attribute delegations, which allow the client
to manage the atime and mtime, and simply inform the server at file
close time what the values should be. This means that most GETATTR
operations to retrieve the atime/mtime values while the file is under
I/O can be optimised away.

Finally, we also add support for the detection mechanism that allows the
client to determine whether or not the server supports the above
functionality.

Lance Shelton (1):
  NFS: Add a generic callback to return the delegation

Trond Myklebust (18):
  NFSv4: Clean up open delegation return structure
  NFSv4: Refactor nfs4_opendata_check_deleg()
  NFSv4: Add new attribute delegation definitions
  NFSv4: Plumb in XDR support for the new delegation-only setattr op
  NFSv4: Add CB_GETATTR support for delegated attributes
  NFSv4: Add a flags argument to the 'have_delegation' callback
  NFSv4: Add support for delegated atime and mtime attributes
  NFSv4: Add recovery of attribute delegations
  NFSv4: Add a capability for delegated attributes
  NFSv4: Enable attribute delegations
  NFSv4: Delegreturn must set m/atime when they are delegated
  NFSv4: Fix up delegated attributes in nfs_setattr
  NFSv4: Don't request atime/mtime/size if they are delegated to us
  NFSv4: Add support for the FATTR4_OPEN_ARGUMENTS attribute
  NFSv4: Detect support for OPEN4_SHARE_ACCESS_WANT_OPEN_XOR_DELEGATION
  NFSv4: Add support for OPEN4_RESULT_NO_OPEN_STATEID
  NFSv4: Ask for a delegation or an open stateid in OPEN
  Return the delegation when deleting the sillyrenamed file

 fs/nfs/callback.h         |   5 +-
 fs/nfs/callback_proc.c    |  14 ++-
 fs/nfs/callback_xdr.c     |  39 ++++++-
 fs/nfs/delegation.c       |  59 ++++++----
 fs/nfs/delegation.h       |  45 +++++++-
 fs/nfs/dir.c              |   2 +-
 fs/nfs/file.c             |   4 +-
 fs/nfs/inode.c            | 104 +++++++++++++++--
 fs/nfs/nfs3proc.c         |  10 +-
 fs/nfs/nfs4proc.c         | 230 ++++++++++++++++++++++++++++----------
 fs/nfs/nfs4xdr.c          | 131 +++++++++++++++++-----
 fs/nfs/proc.c             |  10 +-
 fs/nfs/read.c             |   3 +
 fs/nfs/unlink.c           |   2 +
 fs/nfs/write.c            |  11 +-
 include/linux/nfs4.h      |  11 ++
 include/linux/nfs_fs_sb.h |   2 +
 include/linux/nfs_xdr.h   |  45 +++++++-
 include/uapi/linux/nfs4.h |   4 +
 19 files changed, 586 insertions(+), 145 deletions(-)

-- 
2.45.2

[PATCH 01/19] NFSv4: Clean up open delegation return structure

[trondmy]

From: Trond Myklebust <trond.myklebust@primarydata.com>

Instead of having the fields open coded in the struct nfs_openres,
add a separate structure for them so that we can reuse that code
for the WANT_DELEGATION case.

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: Lance Shelton <lance.shelton@hammerspace.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/nfs4proc.c       | 30 ++++++++++++++++++------------
 fs/nfs/nfs4xdr.c        | 38 +++++++++++++++++++-------------------
 include/linux/nfs_xdr.h | 21 +++++++++++++++++----
 3 files changed, 54 insertions(+), 35 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 94c07875aa3f..d3781ce7e0a5 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -1960,6 +1960,13 @@ nfs4_opendata_check_deleg(struct nfs4_opendata *data, struct nfs4_state *state)
 	struct nfs_delegation *delegation;
 	int delegation_flags = 0;
 
+	switch (data->o_res.delegation.open_delegation_type) {
+	case NFS4_OPEN_DELEGATE_READ:
+	case NFS4_OPEN_DELEGATE_WRITE:
+		break;
+	default:
+		return;
+	};
 	rcu_read_lock();
 	delegation = rcu_dereference(NFS_I(state->inode)->delegation);
 	if (delegation)
@@ -1979,19 +1986,19 @@ nfs4_opendata_check_deleg(struct nfs4_opendata *data, struct nfs4_state *state)
 	if ((delegation_flags & 1UL<<NFS_DELEGATION_NEED_RECLAIM) == 0)
 		nfs_inode_set_delegation(state->inode,
 				data->owner->so_cred,
-				data->o_res.delegation_type,
-				&data->o_res.delegation,
-				data->o_res.pagemod_limit);
+				data->o_res.delegation.type,
+				&data->o_res.delegation.stateid,
+				data->o_res.delegation.pagemod_limit);
 	else
 		nfs_inode_reclaim_delegation(state->inode,
 				data->owner->so_cred,
-				data->o_res.delegation_type,
-				&data->o_res.delegation,
-				data->o_res.pagemod_limit);
+				data->o_res.delegation.type,
+				&data->o_res.delegation.stateid,
+				data->o_res.delegation.pagemod_limit);
 
-	if (data->o_res.do_recall)
+	if (data->o_res.delegation.do_recall)
 		nfs_async_inode_return_delegation(state->inode,
-						  &data->o_res.delegation);
+						  &data->o_res.delegation.stateid);
 }
 
 /*
@@ -2015,8 +2022,7 @@ _nfs4_opendata_reclaim_to_nfs4_state(struct nfs4_opendata *data)
 	if (ret)
 		return ERR_PTR(ret);
 
-	if (data->o_res.delegation_type != 0)
-		nfs4_opendata_check_deleg(data, state);
+	nfs4_opendata_check_deleg(data, state);
 
 	if (!update_open_stateid(state, &data->o_res.stateid,
 				NULL, data->o_arg.fmode))
@@ -2083,7 +2089,7 @@ _nfs4_opendata_to_nfs4_state(struct nfs4_opendata *data)
 	if (IS_ERR(state))
 		goto out;
 
-	if (data->o_res.delegation_type != 0)
+	if (data->o_res.delegation.type != 0)
 		nfs4_opendata_check_deleg(data, state);
 	if (!update_open_stateid(state, &data->o_res.stateid,
 				NULL, data->o_arg.fmode)) {
@@ -3111,7 +3117,7 @@ static int _nfs4_open_and_get_state(struct nfs4_opendata *opendata,
 	case NFS4_OPEN_CLAIM_DELEGATE_PREV:
 		if (!opendata->rpc_done)
 			break;
-		if (opendata->o_res.delegation_type != 0)
+		if (opendata->o_res.delegation.type != 0)
 			dir_verifier = nfs_save_change_attribute(dir);
 		nfs_set_verifier(dentry, dir_verifier);
 	}
diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
index 1416099dfcd1..119061da5298 100644
--- a/fs/nfs/nfs4xdr.c
+++ b/fs/nfs/nfs4xdr.c
@@ -5148,13 +5148,12 @@ static int decode_space_limit(struct xdr_stream *xdr,
 }
 
 static int decode_rw_delegation(struct xdr_stream *xdr,
-		uint32_t delegation_type,
-		struct nfs_openres *res)
+		struct nfs4_open_delegation *res)
 {
 	__be32 *p;
 	int status;
 
-	status = decode_delegation_stateid(xdr, &res->delegation);
+	status = decode_delegation_stateid(xdr, &res->stateid);
 	if (unlikely(status))
 		return status;
 	p = xdr_inline_decode(xdr, 4);
@@ -5162,52 +5161,53 @@ static int decode_rw_delegation(struct xdr_stream *xdr,
 		return -EIO;
 	res->do_recall = be32_to_cpup(p);
 
-	switch (delegation_type) {
+	switch (res->open_delegation_type) {
 	case NFS4_OPEN_DELEGATE_READ:
-		res->delegation_type = FMODE_READ;
+		res->type = FMODE_READ;
 		break;
 	case NFS4_OPEN_DELEGATE_WRITE:
-		res->delegation_type = FMODE_WRITE|FMODE_READ;
+		res->type = FMODE_WRITE|FMODE_READ;
 		if (decode_space_limit(xdr, &res->pagemod_limit) < 0)
 				return -EIO;
 	}
 	return decode_ace(xdr, NULL);
 }
 
-static int decode_no_delegation(struct xdr_stream *xdr, struct nfs_openres *res)
+static int decode_no_delegation(struct xdr_stream *xdr,
+		struct nfs4_open_delegation *res)
 {
 	__be32 *p;
-	uint32_t why_no_delegation;
 
 	p = xdr_inline_decode(xdr, 4);
 	if (unlikely(!p))
 		return -EIO;
-	why_no_delegation = be32_to_cpup(p);
-	switch (why_no_delegation) {
+	res->why_no_delegation = be32_to_cpup(p);
+	switch (res->why_no_delegation) {
 		case WND4_CONTENTION:
 		case WND4_RESOURCE:
-			xdr_inline_decode(xdr, 4);
-			/* Ignore for now */
+			p = xdr_inline_decode(xdr, 4);
+			if (unlikely(!p))
+				return -EIO;
+			res->will_notify = be32_to_cpup(p);
 	}
 	return 0;
 }
 
-static int decode_delegation(struct xdr_stream *xdr, struct nfs_openres *res)
+static int decode_delegation(struct xdr_stream *xdr,
+		struct nfs4_open_delegation *res)
 {
 	__be32 *p;
-	uint32_t delegation_type;
 
 	p = xdr_inline_decode(xdr, 4);
 	if (unlikely(!p))
 		return -EIO;
-	delegation_type = be32_to_cpup(p);
-	res->delegation_type = 0;
-	switch (delegation_type) {
+	res->open_delegation_type = be32_to_cpup(p);
+	switch (res->open_delegation_type) {
 	case NFS4_OPEN_DELEGATE_NONE:
 		return 0;
 	case NFS4_OPEN_DELEGATE_READ:
 	case NFS4_OPEN_DELEGATE_WRITE:
-		return decode_rw_delegation(xdr, delegation_type, res);
+		return decode_rw_delegation(xdr, res);
 	case NFS4_OPEN_DELEGATE_NONE_EXT:
 		return decode_no_delegation(xdr, res);
 	}
@@ -5248,7 +5248,7 @@ static int decode_open(struct xdr_stream *xdr, struct nfs_openres *res)
 	for (; i < NFS4_BITMAP_SIZE; i++)
 		res->attrset[i] = 0;
 
-	return decode_delegation(xdr, res);
+	return decode_delegation(xdr, &res->delegation);
 xdr_error:
 	dprintk("%s: Bitmap too large! Length = %u\n", __func__, bmlen);
 	return -EIO;
diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
index d09b9773b20c..682559e19d9d 100644
--- a/include/linux/nfs_xdr.h
+++ b/include/linux/nfs_xdr.h
@@ -449,6 +449,22 @@ struct stateowner_id {
 	__u32	uniquifier;
 };
 
+struct nfs4_open_delegation {
+	__u32 open_delegation_type;
+	union {
+		struct {
+			fmode_t			type;
+			__u32			do_recall;
+			nfs4_stateid		stateid;
+			unsigned long		pagemod_limit;
+		};
+		struct {
+			__u32			why_no_delegation;
+			__u32			will_notify;
+		};
+	};
+};
+
 /*
  * Arguments to the open call.
  */
@@ -490,13 +506,10 @@ struct nfs_openres {
 	struct nfs_fattr *      f_attr;
 	struct nfs_seqid *	seqid;
 	const struct nfs_server *server;
-	fmode_t			delegation_type;
-	nfs4_stateid		delegation;
-	unsigned long		pagemod_limit;
-	__u32			do_recall;
 	__u32			attrset[NFS4_BITMAP_SIZE];
 	struct nfs4_string	*owner;
 	struct nfs4_string	*group_owner;
+	struct nfs4_open_delegation	delegation;
 	__u32			access_request;
 	__u32			access_supported;
 	__u32			access_result;
-- 
2.45.2

[PATCH 02/19] NFSv4: Refactor nfs4_opendata_check_deleg()

[trondmy]

From: Trond Myklebust <trond.myklebust@primarydata.com>

Modify it to no longer depend directly on the struct opendata.
This will enable sharing with WANT_DELEGATION.

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: Lance Shelton <lance.shelton@hammerspace.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/nfs4proc.c | 66 +++++++++++++++++++++--------------------------
 1 file changed, 30 insertions(+), 36 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index d3781ce7e0a5..639f075e01e9 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -1954,51 +1954,39 @@ static struct nfs4_state *nfs4_try_open_cached(struct nfs4_opendata *opendata)
 }
 
 static void
-nfs4_opendata_check_deleg(struct nfs4_opendata *data, struct nfs4_state *state)
+nfs4_process_delegation(struct inode *inode, const struct cred *cred,
+			enum open_claim_type4 claim,
+			const struct nfs4_open_delegation *delegation)
 {
-	struct nfs_client *clp = NFS_SERVER(state->inode)->nfs_client;
-	struct nfs_delegation *delegation;
-	int delegation_flags = 0;
-
-	switch (data->o_res.delegation.open_delegation_type) {
+	switch (delegation->open_delegation_type) {
 	case NFS4_OPEN_DELEGATE_READ:
 	case NFS4_OPEN_DELEGATE_WRITE:
 		break;
 	default:
 		return;
-	};
-	rcu_read_lock();
-	delegation = rcu_dereference(NFS_I(state->inode)->delegation);
-	if (delegation)
-		delegation_flags = delegation->flags;
-	rcu_read_unlock();
-	switch (data->o_arg.claim) {
-	default:
-		break;
+	}
+	switch (claim) {
 	case NFS4_OPEN_CLAIM_DELEGATE_CUR:
 	case NFS4_OPEN_CLAIM_DELEG_CUR_FH:
 		pr_err_ratelimited("NFS: Broken NFSv4 server %s is "
 				   "returning a delegation for "
 				   "OPEN(CLAIM_DELEGATE_CUR)\n",
-				   clp->cl_hostname);
-		return;
+				   NFS_SERVER(inode)->nfs_client->cl_hostname);
+		break;
+	case NFS4_OPEN_CLAIM_PREVIOUS:
+		nfs_inode_reclaim_delegation(inode, cred,
+				delegation->type,
+				&delegation->stateid,
+				delegation->pagemod_limit);
+		break;
+	default:
+		nfs_inode_set_delegation(inode, cred,
+				delegation->type,
+				&delegation->stateid,
+				delegation->pagemod_limit);
 	}
-	if ((delegation_flags & 1UL<<NFS_DELEGATION_NEED_RECLAIM) == 0)
-		nfs_inode_set_delegation(state->inode,
-				data->owner->so_cred,
-				data->o_res.delegation.type,
-				&data->o_res.delegation.stateid,
-				data->o_res.delegation.pagemod_limit);
-	else
-		nfs_inode_reclaim_delegation(state->inode,
-				data->owner->so_cred,
-				data->o_res.delegation.type,
-				&data->o_res.delegation.stateid,
-				data->o_res.delegation.pagemod_limit);
-
-	if (data->o_res.delegation.do_recall)
-		nfs_async_inode_return_delegation(state->inode,
-						  &data->o_res.delegation.stateid);
+	if (delegation->do_recall)
+		nfs_async_inode_return_delegation(inode, &delegation->stateid);
 }
 
 /*
@@ -2022,7 +2010,10 @@ _nfs4_opendata_reclaim_to_nfs4_state(struct nfs4_opendata *data)
 	if (ret)
 		return ERR_PTR(ret);
 
-	nfs4_opendata_check_deleg(data, state);
+	nfs4_process_delegation(state->inode,
+				data->owner->so_cred,
+				data->o_arg.claim,
+				&data->o_res.delegation);
 
 	if (!update_open_stateid(state, &data->o_res.stateid,
 				NULL, data->o_arg.fmode))
@@ -2089,8 +2080,11 @@ _nfs4_opendata_to_nfs4_state(struct nfs4_opendata *data)
 	if (IS_ERR(state))
 		goto out;
 
-	if (data->o_res.delegation.type != 0)
-		nfs4_opendata_check_deleg(data, state);
+	nfs4_process_delegation(state->inode,
+				data->owner->so_cred,
+				data->o_arg.claim,
+				&data->o_res.delegation);
+
 	if (!update_open_stateid(state, &data->o_res.stateid,
 				NULL, data->o_arg.fmode)) {
 		nfs4_put_open_state(state);
-- 
2.45.2

[PATCH 03/19] NFSv4: Add new attribute delegation definitions

[trondmy]

From: Trond Myklebust <trond.myklebust@primarydata.com>

Signed-off-by: Tom Haynes <loghyr@primarydata.com>
Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: Lance Shelton <lance.shelton@hammerspace.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/nfs4proc.c         | 2 +-
 include/linux/nfs4.h      | 9 +++++++++
 include/uapi/linux/nfs4.h | 2 ++
 3 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 639f075e01e9..ce47cf2f9301 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -3852,7 +3852,7 @@ static void nfs4_close_context(struct nfs_open_context *ctx, int is_sync)
 
 #define FATTR4_WORD1_NFS40_MASK (2*FATTR4_WORD1_MOUNTED_ON_FILEID - 1UL)
 #define FATTR4_WORD2_NFS41_MASK (2*FATTR4_WORD2_SUPPATTR_EXCLCREAT - 1UL)
-#define FATTR4_WORD2_NFS42_MASK (2*FATTR4_WORD2_XATTR_SUPPORT - 1UL)
+#define FATTR4_WORD2_NFS42_MASK (2*FATTR4_WORD2_TIME_DELEG_MODIFY - 1UL)
 
 static int _nfs4_server_capabilities(struct nfs_server *server, struct nfs_fh *fhandle)
 {
diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
index ef8d2d618d5b..bc13d7f04e8d 100644
--- a/include/linux/nfs4.h
+++ b/include/linux/nfs4.h
@@ -367,6 +367,8 @@ enum open_delegation_type4 {
 	NFS4_OPEN_DELEGATE_READ = 1,
 	NFS4_OPEN_DELEGATE_WRITE = 2,
 	NFS4_OPEN_DELEGATE_NONE_EXT = 3, /* 4.1 */
+	NFS4_OPEN_DELEGATE_READ_ATTRS_DELEG = 4,
+	NFS4_OPEN_DELEGATE_WRITE_ATTRS_DELEG = 5,
 };
 
 enum why_no_delegation4 { /* new to v4.1 */
@@ -507,6 +509,11 @@ enum {
 	FATTR4_XATTR_SUPPORT		= 82,
 };
 
+enum {
+	FATTR4_TIME_DELEG_ACCESS	= 84,
+	FATTR4_TIME_DELEG_MODIFY	= 85,
+};
+
 /*
  * The following internal definitions enable processing the above
  * attribute bits within 32-bit word boundaries.
@@ -586,6 +593,8 @@ enum {
 #define FATTR4_WORD2_SECURITY_LABEL     BIT(FATTR4_SEC_LABEL - 64)
 #define FATTR4_WORD2_MODE_UMASK		BIT(FATTR4_MODE_UMASK - 64)
 #define FATTR4_WORD2_XATTR_SUPPORT	BIT(FATTR4_XATTR_SUPPORT - 64)
+#define FATTR4_WORD2_TIME_DELEG_ACCESS	BIT(FATTR4_TIME_DELEG_ACCESS - 64)
+#define FATTR4_WORD2_TIME_DELEG_MODIFY	BIT(FATTR4_TIME_DELEG_MODIFY - 64)
 
 /* MDS threshold bitmap bits */
 #define THRESHOLD_RD                    (1UL << 0)
diff --git a/include/uapi/linux/nfs4.h b/include/uapi/linux/nfs4.h
index 1d2043708bf1..afd7e32906c3 100644
--- a/include/uapi/linux/nfs4.h
+++ b/include/uapi/linux/nfs4.h
@@ -69,6 +69,8 @@
 #define NFS4_SHARE_SIGNAL_DELEG_WHEN_RESRC_AVAIL	0x10000
 #define NFS4_SHARE_PUSH_DELEG_WHEN_UNCONTENDED		0x20000
 
+#define NFS4_SHARE_WANT_DELEG_TIMESTAMPS		0x100000
+
 #define NFS4_CDFC4_FORE	0x1
 #define NFS4_CDFC4_BACK 0x2
 #define NFS4_CDFC4_BOTH 0x3
-- 
2.45.2

[PATCH 04/19] NFSv4: Plumb in XDR support for the new delegation-only setattr op

[trondmy]

From: Trond Myklebust <trond.myklebust@primarydata.com>

We want to send the updated atime and mtime as part of the delegreturn
compound. Add a special structure to hold those variables.

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: Lance Shelton <lance.shelton@hammerspace.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/nfs4proc.c       | 25 ++++++++++++++++++++
 fs/nfs/nfs4xdr.c        | 51 +++++++++++++++++++++++++++++++++++++++++
 include/linux/nfs_xdr.h | 10 ++++++++
 3 files changed, 86 insertions(+)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index ce47cf2f9301..0ed734ab448e 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -6554,6 +6554,7 @@ struct nfs4_delegreturndata {
 		u32 roc_barrier;
 		bool roc;
 	} lr;
+	struct nfs4_delegattr sattr;
 	struct nfs_fattr fattr;
 	int rpc_status;
 	struct inode *inode;
@@ -6578,6 +6579,30 @@ static void nfs4_delegreturn_done(struct rpc_task *task, void *calldata)
 			  &data->res.lr_ret) == -EAGAIN)
 		goto out_restart;
 
+	if (data->args.sattr_args && task->tk_status != 0) {
+		switch(data->res.sattr_ret) {
+		case 0:
+			data->args.sattr_args = NULL;
+			data->res.sattr_res = false;
+			break;
+		case -NFS4ERR_ADMIN_REVOKED:
+		case -NFS4ERR_DELEG_REVOKED:
+		case -NFS4ERR_EXPIRED:
+		case -NFS4ERR_BAD_STATEID:
+			/* Let the main handler below do stateid recovery */
+			break;
+		case -NFS4ERR_OLD_STATEID:
+			if (nfs4_refresh_delegation_stateid(&data->stateid,
+						data->inode))
+				goto out_restart;
+			fallthrough;
+		default:
+			data->args.sattr_args = NULL;
+			data->res.sattr_res = false;
+			goto out_restart;
+		}
+	}
+
 	switch (task->tk_status) {
 	case 0:
 		renew_lease(data->res.server, data->timestamp);
diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
index 119061da5298..4c22b865b9c9 100644
--- a/fs/nfs/nfs4xdr.c
+++ b/fs/nfs/nfs4xdr.c
@@ -224,6 +224,11 @@ static int decode_layoutget(struct xdr_stream *xdr, struct rpc_rqst *req,
 				 encode_attrs_maxsz)
 #define decode_setattr_maxsz	(op_decode_hdr_maxsz + \
 				 nfs4_fattr_bitmap_maxsz)
+#define encode_delegattr_maxsz	(op_encode_hdr_maxsz + \
+				 encode_stateid_maxsz + \
+				nfs4_fattr_bitmap_maxsz + \
+				2*nfstime4_maxsz)
+#define decode_delegattr_maxsz	(decode_setattr_maxsz)
 #define encode_read_maxsz	(op_encode_hdr_maxsz + \
 				 encode_stateid_maxsz + 3)
 #define decode_read_maxsz	(op_decode_hdr_maxsz + 2 + pagepad_maxsz)
@@ -758,12 +763,14 @@ static int decode_layoutget(struct xdr_stream *xdr, struct rpc_rqst *req,
 				encode_sequence_maxsz + \
 				encode_putfh_maxsz + \
 				encode_layoutreturn_maxsz + \
+				encode_delegattr_maxsz + \
 				encode_delegreturn_maxsz + \
 				encode_getattr_maxsz)
 #define NFS4_dec_delegreturn_sz (compound_decode_hdr_maxsz + \
 				decode_sequence_maxsz + \
 				decode_putfh_maxsz + \
 				decode_layoutreturn_maxsz + \
+				decode_delegattr_maxsz + \
 				decode_delegreturn_maxsz + \
 				decode_getattr_maxsz)
 #define NFS4_enc_getacl_sz	(compound_encode_hdr_maxsz + \
@@ -1735,6 +1742,33 @@ static void encode_setattr(struct xdr_stream *xdr, const struct nfs_setattrargs
 			server->attr_bitmask);
 }
 
+static void encode_delegattr(struct xdr_stream *xdr,
+		const nfs4_stateid *stateid,
+		const struct nfs4_delegattr *attr,
+		struct compound_hdr *hdr)
+{
+	uint32_t bitmap[3] = { 0 };
+	uint32_t len = 0;
+	__be32 *p;
+
+	encode_op_hdr(xdr, OP_SETATTR, encode_delegattr_maxsz, hdr);
+	encode_nfs4_stateid(xdr, stateid);
+	if (attr->atime_set) {
+		bitmap[2] |= FATTR4_WORD2_TIME_DELEG_ACCESS;
+		len += (nfstime4_maxsz << 2);
+	}
+	if (attr->mtime_set) {
+		bitmap[2] |= FATTR4_WORD2_TIME_DELEG_MODIFY;
+		len += (nfstime4_maxsz << 2);
+	}
+	xdr_encode_bitmap4(xdr, bitmap, ARRAY_SIZE(bitmap));
+	xdr_stream_encode_opaque_inline(xdr, (void **)&p, len);
+	if (bitmap[2] & FATTR4_WORD2_TIME_DELEG_ACCESS)
+		p = xdr_encode_nfstime4(p, &attr->atime);
+	if (bitmap[2] & FATTR4_WORD2_TIME_DELEG_MODIFY)
+		p = xdr_encode_nfstime4(p, &attr->mtime);
+}
+
 static void encode_setclientid(struct xdr_stream *xdr, const struct nfs4_setclientid *setclientid, struct compound_hdr *hdr)
 {
 	__be32 *p;
@@ -2812,6 +2846,8 @@ static void nfs4_xdr_enc_delegreturn(struct rpc_rqst *req,
 	encode_putfh(xdr, args->fhandle, &hdr);
 	if (args->lr_args)
 		encode_layoutreturn(xdr, args->lr_args, &hdr);
+	if (args->sattr_args)
+		encode_delegattr(xdr, args->stateid, args->sattr_args, &hdr);
 	if (args->bitmask)
 		encode_getfattr(xdr, args->bitmask, &hdr);
 	encode_delegreturn(xdr, args->stateid, &hdr);
@@ -5163,9 +5199,11 @@ static int decode_rw_delegation(struct xdr_stream *xdr,
 
 	switch (res->open_delegation_type) {
 	case NFS4_OPEN_DELEGATE_READ:
+	case NFS4_OPEN_DELEGATE_READ_ATTRS_DELEG:
 		res->type = FMODE_READ;
 		break;
 	case NFS4_OPEN_DELEGATE_WRITE:
+	case NFS4_OPEN_DELEGATE_WRITE_ATTRS_DELEG:
 		res->type = FMODE_WRITE|FMODE_READ;
 		if (decode_space_limit(xdr, &res->pagemod_limit) < 0)
 				return -EIO;
@@ -5207,6 +5245,8 @@ static int decode_delegation(struct xdr_stream *xdr,
 		return 0;
 	case NFS4_OPEN_DELEGATE_READ:
 	case NFS4_OPEN_DELEGATE_WRITE:
+	case NFS4_OPEN_DELEGATE_READ_ATTRS_DELEG:
+	case NFS4_OPEN_DELEGATE_WRITE_ATTRS_DELEG:
 		return decode_rw_delegation(xdr, res);
 	case NFS4_OPEN_DELEGATE_NONE_EXT:
 		return decode_no_delegation(xdr, res);
@@ -5480,6 +5520,11 @@ static int decode_setattr(struct xdr_stream *xdr)
 	return -EIO;
 }
 
+static int decode_delegattr(struct xdr_stream *xdr)
+{
+	return decode_setattr(xdr);
+}
+
 static int decode_setclientid(struct xdr_stream *xdr, struct nfs4_setclientid_res *res)
 {
 	__be32 *p;
@@ -7052,6 +7097,12 @@ static int nfs4_xdr_dec_delegreturn(struct rpc_rqst *rqstp,
 		if (status)
 			goto out;
 	}
+	if (res->sattr_res) {
+		status = decode_delegattr(xdr);
+		res->sattr_ret = status;
+		if (status)
+			goto out;
+	}
 	if (res->fattr) {
 		status = decode_getfattr(xdr, res->fattr, res->server);
 		if (status != 0)
diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
index 682559e19d9d..f40be64ce942 100644
--- a/include/linux/nfs_xdr.h
+++ b/include/linux/nfs_xdr.h
@@ -622,6 +622,13 @@ struct nfs_release_lockowner_res {
 	struct nfs4_sequence_res	seq_res;
 };
 
+struct nfs4_delegattr {
+	struct timespec64	atime;
+	struct timespec64	mtime;
+	bool			atime_set;
+	bool			mtime_set;
+};
+
 struct nfs4_delegreturnargs {
 	struct nfs4_sequence_args	seq_args;
 	const struct nfs_fh *fhandle;
@@ -629,6 +636,7 @@ struct nfs4_delegreturnargs {
 	const u32 *bitmask;
 	u32 bitmask_store[NFS_BITMASK_SZ];
 	struct nfs4_layoutreturn_args *lr_args;
+	struct nfs4_delegattr *sattr_args;
 };
 
 struct nfs4_delegreturnres {
@@ -637,6 +645,8 @@ struct nfs4_delegreturnres {
 	struct nfs_server *server;
 	struct nfs4_layoutreturn_res *lr_res;
 	int lr_ret;
+	bool sattr_res;
+	int sattr_ret;
 };
 
 /*
-- 
2.45.2

[PATCH 05/19] NFSv4: Add CB_GETATTR support for delegated attributes

[trondmy]

From: Trond Myklebust <trond.myklebust@primarydata.com>

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: Lance Shelton <lance.shelton@hammerspace.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/callback.h      |  5 +++--
 fs/nfs/callback_proc.c | 14 +++++++++-----
 fs/nfs/callback_xdr.c  | 39 +++++++++++++++++++++++++++++++++++++--
 3 files changed, 49 insertions(+), 9 deletions(-)

diff --git a/fs/nfs/callback.h b/fs/nfs/callback.h
index 650758ee0d5f..154a6ed1299f 100644
--- a/fs/nfs/callback.h
+++ b/fs/nfs/callback.h
@@ -46,14 +46,15 @@ struct cb_compound_hdr_res {
 
 struct cb_getattrargs {
 	struct nfs_fh fh;
-	uint32_t bitmap[2];
+	uint32_t bitmap[3];
 };
 
 struct cb_getattrres {
 	__be32 status;
-	uint32_t bitmap[2];
+	uint32_t bitmap[3];
 	uint64_t size;
 	uint64_t change_attr;
+	struct timespec64 atime;
 	struct timespec64 ctime;
 	struct timespec64 mtime;
 };
diff --git a/fs/nfs/callback_proc.c b/fs/nfs/callback_proc.c
index 76cea34477ae..199c52788640 100644
--- a/fs/nfs/callback_proc.c
+++ b/fs/nfs/callback_proc.c
@@ -37,7 +37,7 @@ __be32 nfs4_callback_getattr(void *argp, void *resp,
 	if (!cps->clp) /* Always set for v4.0. Set in cb_sequence for v4.1 */
 		goto out;
 
-	res->bitmap[0] = res->bitmap[1] = 0;
+	memset(res->bitmap, 0, sizeof(res->bitmap));
 	res->status = htonl(NFS4ERR_BADHANDLE);
 
 	dprintk_rcu("NFS: GETATTR callback request from %s\n",
@@ -59,12 +59,16 @@ __be32 nfs4_callback_getattr(void *argp, void *resp,
 	res->change_attr = delegation->change_attr;
 	if (nfs_have_writebacks(inode))
 		res->change_attr++;
+	res->atime = inode_get_atime(inode);
 	res->ctime = inode_get_ctime(inode);
 	res->mtime = inode_get_mtime(inode);
-	res->bitmap[0] = (FATTR4_WORD0_CHANGE|FATTR4_WORD0_SIZE) &
-		args->bitmap[0];
-	res->bitmap[1] = (FATTR4_WORD1_TIME_METADATA|FATTR4_WORD1_TIME_MODIFY) &
-		args->bitmap[1];
+	res->bitmap[0] = (FATTR4_WORD0_CHANGE | FATTR4_WORD0_SIZE) &
+			 args->bitmap[0];
+	res->bitmap[1] = (FATTR4_WORD1_TIME_ACCESS |
+			  FATTR4_WORD1_TIME_METADATA |
+			  FATTR4_WORD1_TIME_MODIFY) & args->bitmap[1];
+	res->bitmap[2] = (FATTR4_WORD2_TIME_DELEG_ACCESS |
+			  FATTR4_WORD2_TIME_DELEG_MODIFY) & args->bitmap[2];
 	res->status = 0;
 out_iput:
 	rcu_read_unlock();
diff --git a/fs/nfs/callback_xdr.c b/fs/nfs/callback_xdr.c
index 9369488f2ed4..29c49a7e5fe1 100644
--- a/fs/nfs/callback_xdr.c
+++ b/fs/nfs/callback_xdr.c
@@ -25,8 +25,9 @@
 #define CB_OP_GETATTR_BITMAP_MAXSZ	(4 * 4) // bitmap length, 3 bitmaps
 #define CB_OP_GETATTR_RES_MAXSZ		(CB_OP_HDR_RES_MAXSZ + \
 					 CB_OP_GETATTR_BITMAP_MAXSZ + \
-					 /* change, size, ctime, mtime */\
-					 (2 + 2 + 3 + 3) * 4)
+					 /* change, size, atime, ctime,
+					  * mtime, deleg_atime, deleg_mtime */\
+					 (2 + 2 + 3 + 3 + 3 + 3 + 3) * 4)
 #define CB_OP_RECALL_RES_MAXSZ		(CB_OP_HDR_RES_MAXSZ)
 
 #if defined(CONFIG_NFS_V4_1)
@@ -635,6 +636,13 @@ static __be32 encode_attr_time(struct xdr_stream *xdr, const struct timespec64 *
 	return 0;
 }
 
+static __be32 encode_attr_atime(struct xdr_stream *xdr, const uint32_t *bitmap, const struct timespec64 *time)
+{
+	if (!(bitmap[1] & FATTR4_WORD1_TIME_ACCESS))
+		return 0;
+	return encode_attr_time(xdr,time);
+}
+
 static __be32 encode_attr_ctime(struct xdr_stream *xdr, const uint32_t *bitmap, const struct timespec64 *time)
 {
 	if (!(bitmap[1] & FATTR4_WORD1_TIME_METADATA))
@@ -649,6 +657,24 @@ static __be32 encode_attr_mtime(struct xdr_stream *xdr, const uint32_t *bitmap,
 	return encode_attr_time(xdr,time);
 }
 
+static __be32 encode_attr_delegatime(struct xdr_stream *xdr,
+				     const uint32_t *bitmap,
+				     const struct timespec64 *time)
+{
+	if (!(bitmap[2] & FATTR4_WORD2_TIME_DELEG_ACCESS))
+		return 0;
+	return encode_attr_time(xdr,time);
+}
+
+static __be32 encode_attr_delegmtime(struct xdr_stream *xdr,
+				     const uint32_t *bitmap,
+				     const struct timespec64 *time)
+{
+	if (!(bitmap[2] & FATTR4_WORD2_TIME_DELEG_MODIFY))
+		return 0;
+	return encode_attr_time(xdr,time);
+}
+
 static __be32 encode_compound_hdr_res(struct xdr_stream *xdr, struct cb_compound_hdr_res *hdr)
 {
 	__be32 status;
@@ -697,12 +723,21 @@ static __be32 encode_getattr_res(struct svc_rqst *rqstp, struct xdr_stream *xdr,
 	if (unlikely(status != 0))
 		goto out;
 	status = encode_attr_size(xdr, res->bitmap, res->size);
+	if (unlikely(status != 0))
+		goto out;
+	status = encode_attr_atime(xdr, res->bitmap, &res->atime);
 	if (unlikely(status != 0))
 		goto out;
 	status = encode_attr_ctime(xdr, res->bitmap, &res->ctime);
 	if (unlikely(status != 0))
 		goto out;
 	status = encode_attr_mtime(xdr, res->bitmap, &res->mtime);
+	if (unlikely(status != 0))
+		goto out;
+	status = encode_attr_delegatime(xdr, res->bitmap, &res->atime);
+	if (unlikely(status != 0))
+		goto out;
+	status = encode_attr_delegmtime(xdr, res->bitmap, &res->mtime);
 	*savep = htonl((unsigned int)((char *)xdr->p - (char *)(savep+1)));
 out:
 	return status;
-- 
2.45.2

[PATCH 06/19] NFSv4: Add a flags argument to the 'have_delegation' callback

[trondmy]

From: Trond Myklebust <trond.myklebust@primarydata.com>

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: Lance Shelton <lance.shelton@hammerspace.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/delegation.c     | 26 +++++++++++++-------------
 fs/nfs/delegation.h     | 16 +++++++++++++---
 fs/nfs/dir.c            |  2 +-
 fs/nfs/file.c           |  4 ++--
 fs/nfs/inode.c          |  7 +++----
 fs/nfs/nfs3proc.c       |  2 +-
 fs/nfs/nfs4proc.c       | 14 +++++++-------
 fs/nfs/proc.c           |  2 +-
 fs/nfs/write.c          |  2 +-
 include/linux/nfs_xdr.h |  2 +-
 10 files changed, 43 insertions(+), 34 deletions(-)

diff --git a/fs/nfs/delegation.c b/fs/nfs/delegation.c
index 6bace5fece04..6fdffd25cb2b 100644
--- a/fs/nfs/delegation.c
+++ b/fs/nfs/delegation.c
@@ -82,11 +82,10 @@ static void nfs_mark_return_delegation(struct nfs_server *server,
 	set_bit(NFS4CLNT_DELEGRETURN, &server->nfs_client->cl_state);
 }
 
-static bool
-nfs4_is_valid_delegation(const struct nfs_delegation *delegation,
-		fmode_t flags)
+static bool nfs4_is_valid_delegation(const struct nfs_delegation *delegation,
+				     fmode_t type)
 {
-	if (delegation != NULL && (delegation->type & flags) == flags &&
+	if (delegation != NULL && (delegation->type & type) == type &&
 	    !test_bit(NFS_DELEGATION_REVOKED, &delegation->flags) &&
 	    !test_bit(NFS_DELEGATION_RETURNING, &delegation->flags))
 		return true;
@@ -103,16 +102,16 @@ struct nfs_delegation *nfs4_get_valid_delegation(const struct inode *inode)
 	return NULL;
 }
 
-static int
-nfs4_do_check_delegation(struct inode *inode, fmode_t flags, bool mark)
+static int nfs4_do_check_delegation(struct inode *inode, fmode_t type,
+				    int flags, bool mark)
 {
 	struct nfs_delegation *delegation;
 	int ret = 0;
 
-	flags &= FMODE_READ|FMODE_WRITE;
+	type &= FMODE_READ|FMODE_WRITE;
 	rcu_read_lock();
 	delegation = rcu_dereference(NFS_I(inode)->delegation);
-	if (nfs4_is_valid_delegation(delegation, flags)) {
+	if (nfs4_is_valid_delegation(delegation, type)) {
 		if (mark)
 			nfs_mark_delegation_referenced(delegation);
 		ret = 1;
@@ -124,22 +123,23 @@ nfs4_do_check_delegation(struct inode *inode, fmode_t flags, bool mark)
  * nfs4_have_delegation - check if inode has a delegation, mark it
  * NFS_DELEGATION_REFERENCED if there is one.
  * @inode: inode to check
- * @flags: delegation types to check for
+ * @type: delegation types to check for
+ * @flags: various modifiers
  *
  * Returns one if inode has the indicated delegation, otherwise zero.
  */
-int nfs4_have_delegation(struct inode *inode, fmode_t flags)
+int nfs4_have_delegation(struct inode *inode, fmode_t type, int flags)
 {
-	return nfs4_do_check_delegation(inode, flags, true);
+	return nfs4_do_check_delegation(inode, type, flags, true);
 }
 
 /*
  * nfs4_check_delegation - check if inode has a delegation, do not mark
  * NFS_DELEGATION_REFERENCED if it has one.
  */
-int nfs4_check_delegation(struct inode *inode, fmode_t flags)
+int nfs4_check_delegation(struct inode *inode, fmode_t type)
 {
-	return nfs4_do_check_delegation(inode, flags, false);
+	return nfs4_do_check_delegation(inode, type, 0, false);
 }
 
 static int nfs_delegation_claim_locks(struct nfs4_state *state, const nfs4_stateid *stateid)
diff --git a/fs/nfs/delegation.h b/fs/nfs/delegation.h
index a6f495d012cf..257b3d726043 100644
--- a/fs/nfs/delegation.h
+++ b/fs/nfs/delegation.h
@@ -75,8 +75,8 @@ bool nfs4_refresh_delegation_stateid(nfs4_stateid *dst, struct inode *inode);
 
 struct nfs_delegation *nfs4_get_valid_delegation(const struct inode *inode);
 void nfs_mark_delegation_referenced(struct nfs_delegation *delegation);
-int nfs4_have_delegation(struct inode *inode, fmode_t flags);
-int nfs4_check_delegation(struct inode *inode, fmode_t flags);
+int nfs4_have_delegation(struct inode *inode, fmode_t type, int flags);
+int nfs4_check_delegation(struct inode *inode, fmode_t type);
 bool nfs4_delegation_flush_on_close(const struct inode *inode);
 void nfs_inode_find_delegation_state_and_recover(struct inode *inode,
 		const nfs4_stateid *stateid);
@@ -84,9 +84,19 @@ int nfs4_inode_make_writeable(struct inode *inode);
 
 #endif
 
+static inline int nfs_have_read_or_write_delegation(struct inode *inode)
+{
+	return NFS_PROTO(inode)->have_delegation(inode, FMODE_READ, 0);
+}
+
+static inline int nfs_have_write_delegation(struct inode *inode)
+{
+	return NFS_PROTO(inode)->have_delegation(inode, FMODE_WRITE, 0);
+}
+
 static inline int nfs_have_delegated_attributes(struct inode *inode)
 {
-	return NFS_PROTO(inode)->have_delegation(inode, FMODE_READ);
+	return NFS_PROTO(inode)->have_delegation(inode, FMODE_READ, 0);
 }
 
 #endif
diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c
index 788077a4feb9..6b63d1ecd2c2 100644
--- a/fs/nfs/dir.c
+++ b/fs/nfs/dir.c
@@ -1437,7 +1437,7 @@ static void nfs_set_verifier_locked(struct dentry *dentry, unsigned long verf)
 
 	if (!dir || !nfs_verify_change_attribute(dir, verf))
 		return;
-	if (inode && NFS_PROTO(inode)->have_delegation(inode, FMODE_READ))
+	if (inode && NFS_PROTO(inode)->have_delegation(inode, FMODE_READ, 0))
 		nfs_set_verifier_delegated(&verf);
 	dentry->d_time = verf;
 }
diff --git a/fs/nfs/file.c b/fs/nfs/file.c
index 407c6e15afe2..8c13b9a41aaa 100644
--- a/fs/nfs/file.c
+++ b/fs/nfs/file.c
@@ -730,7 +730,7 @@ do_getlk(struct file *filp, int cmd, struct file_lock *fl, int is_local)
 	}
 	fl->c.flc_type = saved_type;
 
-	if (NFS_PROTO(inode)->have_delegation(inode, FMODE_READ))
+	if (nfs_have_read_or_write_delegation(inode))
 		goto out_noconflict;
 
 	if (is_local)
@@ -813,7 +813,7 @@ do_setlk(struct file *filp, int cmd, struct file_lock *fl, int is_local)
 	 * This makes locking act as a cache coherency point.
 	 */
 	nfs_sync_mapping(filp->f_mapping);
-	if (!NFS_PROTO(inode)->have_delegation(inode, FMODE_READ)) {
+	if (!nfs_have_read_or_write_delegation(inode)) {
 		nfs_zap_caches(inode);
 		if (mapping_mapped(filp->f_mapping))
 			nfs_revalidate_mapping(inode, filp->f_mapping);
diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
index acef52ecb1bb..89722919b463 100644
--- a/fs/nfs/inode.c
+++ b/fs/nfs/inode.c
@@ -190,9 +190,8 @@ static bool nfs_has_xattr_cache(const struct nfs_inode *nfsi)
 void nfs_set_cache_invalid(struct inode *inode, unsigned long flags)
 {
 	struct nfs_inode *nfsi = NFS_I(inode);
-	bool have_delegation = NFS_PROTO(inode)->have_delegation(inode, FMODE_READ);
 
-	if (have_delegation) {
+	if (nfs_have_delegated_attributes(inode)) {
 		if (!(flags & NFS_INO_REVAL_FORCED))
 			flags &= ~(NFS_INO_INVALID_MODE |
 				   NFS_INO_INVALID_OTHER |
@@ -1012,7 +1011,7 @@ void nfs_close_context(struct nfs_open_context *ctx, int is_sync)
 	if (!is_sync)
 		return;
 	inode = d_inode(ctx->dentry);
-	if (NFS_PROTO(inode)->have_delegation(inode, FMODE_READ))
+	if (nfs_have_read_or_write_delegation(inode))
 		return;
 	nfsi = NFS_I(inode);
 	if (inode->i_mapping->nrpages == 0)
@@ -1482,7 +1481,7 @@ static int nfs_check_inode_attributes(struct inode *inode, struct nfs_fattr *fat
 	unsigned long invalid = 0;
 	struct timespec64 ts;
 
-	if (NFS_PROTO(inode)->have_delegation(inode, FMODE_READ))
+	if (nfs_have_delegated_attributes(inode))
 		return 0;
 
 	if (!(fattr->valid & NFS_ATTR_FATTR_FILEID)) {
diff --git a/fs/nfs/nfs3proc.c b/fs/nfs/nfs3proc.c
index 74bda639a7cf..cab6c73d25d6 100644
--- a/fs/nfs/nfs3proc.c
+++ b/fs/nfs/nfs3proc.c
@@ -979,7 +979,7 @@ nfs3_proc_lock(struct file *filp, int cmd, struct file_lock *fl)
 	return status;
 }
 
-static int nfs3_have_delegation(struct inode *inode, fmode_t flags)
+static int nfs3_have_delegation(struct inode *inode, fmode_t type, int flags)
 {
 	return 0;
 }
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 0ed734ab448e..27fb40653f1d 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -293,7 +293,7 @@ static void nfs4_bitmap_copy_adjust(__u32 *dst, const __u32 *src,
 	unsigned long cache_validity;
 
 	memcpy(dst, src, NFS4_BITMASK_SZ*sizeof(*dst));
-	if (!inode || !nfs4_have_delegation(inode, FMODE_READ))
+	if (!inode || !nfs_have_read_or_write_delegation(inode))
 		return;
 
 	cache_validity = READ_ONCE(NFS_I(inode)->cache_validity) | flags;
@@ -1264,7 +1264,7 @@ nfs4_update_changeattr_locked(struct inode *inode,
 		if (S_ISDIR(inode->i_mode))
 			nfs_force_lookup_revalidate(inode);
 
-		if (!NFS_PROTO(inode)->have_delegation(inode, FMODE_READ))
+		if (!nfs_have_delegated_attributes(inode))
 			cache_validity |=
 				NFS_INO_INVALID_ACCESS | NFS_INO_INVALID_ACL |
 				NFS_INO_INVALID_SIZE | NFS_INO_INVALID_OTHER |
@@ -3700,7 +3700,7 @@ static void nfs4_close_prepare(struct rpc_task *task, void *data)
 
 	if (calldata->arg.fmode == 0 || calldata->arg.fmode == FMODE_READ) {
 		/* Close-to-open cache consistency revalidation */
-		if (!nfs4_have_delegation(inode, FMODE_READ)) {
+		if (!nfs4_have_delegation(inode, FMODE_READ, 0)) {
 			nfs4_bitmask_set(calldata->arg.bitmask_store,
 					 server->cache_consistency_bitmask,
 					 inode, 0);
@@ -4617,7 +4617,7 @@ static int _nfs4_proc_access(struct inode *inode, struct nfs_access_entry *entry
 	};
 	int status = 0;
 
-	if (!nfs4_have_delegation(inode, FMODE_READ)) {
+	if (!nfs4_have_delegation(inode, FMODE_READ, 0)) {
 		res.fattr = nfs_alloc_fattr();
 		if (res.fattr == NULL)
 			return -ENOMEM;
@@ -5586,7 +5586,7 @@ bool nfs4_write_need_cache_consistency_data(struct nfs_pgio_header *hdr)
 	/* Otherwise, request attributes if and only if we don't hold
 	 * a delegation
 	 */
-	return nfs4_have_delegation(hdr->inode, FMODE_READ) == 0;
+	return nfs4_have_delegation(hdr->inode, FMODE_READ, 0) == 0;
 }
 
 void nfs4_bitmask_set(__u32 bitmask[], const __u32 src[],
@@ -7633,10 +7633,10 @@ static int nfs4_add_lease(struct file *file, int arg, struct file_lease **lease,
 	int ret;
 
 	/* No delegation, no lease */
-	if (!nfs4_have_delegation(inode, type))
+	if (!nfs4_have_delegation(inode, type, 0))
 		return -EAGAIN;
 	ret = generic_setlease(file, arg, lease, priv);
-	if (ret || nfs4_have_delegation(inode, type))
+	if (ret || nfs4_have_delegation(inode, type, 0))
 		return ret;
 	/* We raced with a delegation return */
 	nfs4_delete_lease(file, priv);
diff --git a/fs/nfs/proc.c b/fs/nfs/proc.c
index d105e5b2659d..995cc42b0fa0 100644
--- a/fs/nfs/proc.c
+++ b/fs/nfs/proc.c
@@ -687,7 +687,7 @@ static int nfs_lock_check_bounds(const struct file_lock *fl)
 	return -EINVAL;
 }
 
-static int nfs_have_delegation(struct inode *inode, fmode_t flags)
+static int nfs_have_delegation(struct inode *inode, fmode_t type, int flags)
 {
 	return 0;
 }
diff --git a/fs/nfs/write.c b/fs/nfs/write.c
index 5de85d725fb9..a9186b113fe7 100644
--- a/fs/nfs/write.c
+++ b/fs/nfs/write.c
@@ -1320,7 +1320,7 @@ static int nfs_can_extend_write(struct file *file, struct folio *folio,
 		return 0;
 	if (!nfs_folio_write_uptodate(folio, pagelen))
 		return 0;
-	if (NFS_PROTO(inode)->have_delegation(inode, FMODE_WRITE))
+	if (nfs_have_write_delegation(inode))
 		return 1;
 	if (!flctx || (list_empty_careful(&flctx->flc_flock) &&
 		       list_empty_careful(&flctx->flc_posix)))
diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
index f40be64ce942..51611583af51 100644
--- a/include/linux/nfs_xdr.h
+++ b/include/linux/nfs_xdr.h
@@ -1830,7 +1830,7 @@ struct nfs_rpc_ops {
 				int open_flags,
 				struct iattr *iattr,
 				int *);
-	int (*have_delegation)(struct inode *, fmode_t);
+	int (*have_delegation)(struct inode *, fmode_t, int);
 	struct nfs_client *(*alloc_client) (const struct nfs_client_initdata *);
 	struct nfs_client *(*init_client) (struct nfs_client *,
 				const struct nfs_client_initdata *);
-- 
2.45.2

[PATCH 07/19] NFSv4: Add support for delegated atime and mtime attributes

[trondmy]

From: Trond Myklebust <trond.myklebust@primarydata.com>

Ensure that we update the mtime and atime correctly when we read
or write data to the file and when we truncate. Let the server manage
ctime on other attribute updates.

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: Lance Shelton <lance.shelton@hammerspace.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/delegation.c | 24 +++++++++++++++-----
 fs/nfs/delegation.h | 25 +++++++++++++++++++--
 fs/nfs/inode.c      | 54 +++++++++++++++++++++++++++++++++++++++++----
 fs/nfs/nfs4proc.c   | 21 ++++++++++--------
 fs/nfs/read.c       |  3 +++
 fs/nfs/write.c      |  9 ++++++++
 6 files changed, 116 insertions(+), 20 deletions(-)

diff --git a/fs/nfs/delegation.c b/fs/nfs/delegation.c
index 6fdffd25cb2b..e72eead06c08 100644
--- a/fs/nfs/delegation.c
+++ b/fs/nfs/delegation.c
@@ -115,6 +115,9 @@ static int nfs4_do_check_delegation(struct inode *inode, fmode_t type,
 		if (mark)
 			nfs_mark_delegation_referenced(delegation);
 		ret = 1;
+		if ((flags & NFS_DELEGATION_FLAG_TIME) &&
+		    !test_bit(NFS_DELEGATION_DELEGTIME, &delegation->flags))
+			ret = 0;
 	}
 	rcu_read_unlock();
 	return ret;
@@ -221,11 +224,12 @@ static int nfs_delegation_claim_opens(struct inode *inode,
  * @type: delegation type
  * @stateid: delegation stateid
  * @pagemod_limit: write delegation "space_limit"
+ * @deleg_type: raw delegation type
  *
  */
 void nfs_inode_reclaim_delegation(struct inode *inode, const struct cred *cred,
 				  fmode_t type, const nfs4_stateid *stateid,
-				  unsigned long pagemod_limit)
+				  unsigned long pagemod_limit, u32 deleg_type)
 {
 	struct nfs_delegation *delegation;
 	const struct cred *oldcred = NULL;
@@ -250,7 +254,7 @@ void nfs_inode_reclaim_delegation(struct inode *inode, const struct cred *cred,
 	} else {
 		rcu_read_unlock();
 		nfs_inode_set_delegation(inode, cred, type, stateid,
-					 pagemod_limit);
+					 pagemod_limit, deleg_type);
 	}
 }
 
@@ -418,13 +422,13 @@ nfs_update_inplace_delegation(struct nfs_delegation *delegation,
  * @type: delegation type
  * @stateid: delegation stateid
  * @pagemod_limit: write delegation "space_limit"
+ * @deleg_type: raw delegation type
  *
  * Returns zero on success, or a negative errno value.
  */
 int nfs_inode_set_delegation(struct inode *inode, const struct cred *cred,
-				  fmode_t type,
-				  const nfs4_stateid *stateid,
-				  unsigned long pagemod_limit)
+			     fmode_t type, const nfs4_stateid *stateid,
+			     unsigned long pagemod_limit, u32 deleg_type)
 {
 	struct nfs_server *server = NFS_SERVER(inode);
 	struct nfs_client *clp = server->nfs_client;
@@ -444,6 +448,11 @@ int nfs_inode_set_delegation(struct inode *inode, const struct cred *cred,
 	delegation->cred = get_cred(cred);
 	delegation->inode = inode;
 	delegation->flags = 1<<NFS_DELEGATION_REFERENCED;
+	switch (deleg_type) {
+	case NFS4_OPEN_DELEGATE_READ_ATTRS_DELEG:
+	case NFS4_OPEN_DELEGATE_WRITE_ATTRS_DELEG:
+		delegation->flags |= BIT(NFS_DELEGATION_DELEGTIME);
+	}
 	delegation->test_gen = 0;
 	spin_lock_init(&delegation->lock);
 
@@ -508,6 +517,11 @@ int nfs_inode_set_delegation(struct inode *inode, const struct cred *cred,
 	atomic_long_inc(&nfs_active_delegations);
 
 	trace_nfs4_set_delegation(inode, type);
+
+	/* If we hold writebacks and have delegated mtime then update */
+	if (deleg_type == NFS4_OPEN_DELEGATE_WRITE_ATTRS_DELEG &&
+	    nfs_have_writebacks(inode))
+		nfs_update_delegated_mtime(inode);
 out:
 	spin_unlock(&clp->cl_lock);
 	if (delegation != NULL)
diff --git a/fs/nfs/delegation.h b/fs/nfs/delegation.h
index 257b3d726043..2e9ad83acf2c 100644
--- a/fs/nfs/delegation.h
+++ b/fs/nfs/delegation.h
@@ -38,12 +38,17 @@ enum {
 	NFS_DELEGATION_TEST_EXPIRED,
 	NFS_DELEGATION_INODE_FREEING,
 	NFS_DELEGATION_RETURN_DELAYED,
+	NFS_DELEGATION_DELEGTIME,
 };
 
+#define NFS_DELEGATION_FLAG_TIME	BIT(1)
+
 int nfs_inode_set_delegation(struct inode *inode, const struct cred *cred,
-		fmode_t type, const nfs4_stateid *stateid, unsigned long pagemod_limit);
+			     fmode_t type, const nfs4_stateid *stateid,
+			     unsigned long pagemod_limit, u32 deleg_type);
 void nfs_inode_reclaim_delegation(struct inode *inode, const struct cred *cred,
-		fmode_t type, const nfs4_stateid *stateid, unsigned long pagemod_limit);
+				  fmode_t type, const nfs4_stateid *stateid,
+				  unsigned long pagemod_limit, u32 deleg_type);
 int nfs4_inode_return_delegation(struct inode *inode);
 void nfs4_inode_return_delegation_on_close(struct inode *inode);
 int nfs_async_inode_return_delegation(struct inode *inode, const nfs4_stateid *stateid);
@@ -84,6 +89,10 @@ int nfs4_inode_make_writeable(struct inode *inode);
 
 #endif
 
+void nfs_update_delegated_atime(struct inode *inode);
+void nfs_update_delegated_mtime(struct inode *inode);
+void nfs_update_delegated_mtime_locked(struct inode *inode);
+
 static inline int nfs_have_read_or_write_delegation(struct inode *inode)
 {
 	return NFS_PROTO(inode)->have_delegation(inode, FMODE_READ, 0);
@@ -99,4 +108,16 @@ static inline int nfs_have_delegated_attributes(struct inode *inode)
 	return NFS_PROTO(inode)->have_delegation(inode, FMODE_READ, 0);
 }
 
+static inline int nfs_have_delegated_atime(struct inode *inode)
+{
+	return NFS_PROTO(inode)->have_delegation(inode, FMODE_READ,
+						 NFS_DELEGATION_FLAG_TIME);
+}
+
+static inline int nfs_have_delegated_mtime(struct inode *inode)
+{
+	return NFS_PROTO(inode)->have_delegation(inode, FMODE_WRITE,
+						 NFS_DELEGATION_FLAG_TIME);
+}
+
 #endif
diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
index 89722919b463..91c0aeaf6c1e 100644
--- a/fs/nfs/inode.c
+++ b/fs/nfs/inode.c
@@ -275,6 +275,8 @@ EXPORT_SYMBOL_GPL(nfs_zap_acl_cache);
 
 void nfs_invalidate_atime(struct inode *inode)
 {
+	if (nfs_have_delegated_atime(inode))
+		return;
 	spin_lock(&inode->i_lock);
 	nfs_set_cache_invalid(inode, NFS_INO_INVALID_ATIME);
 	spin_unlock(&inode->i_lock);
@@ -603,6 +605,33 @@ nfs_fhget(struct super_block *sb, struct nfs_fh *fh, struct nfs_fattr *fattr)
 }
 EXPORT_SYMBOL_GPL(nfs_fhget);
 
+void nfs_update_delegated_atime(struct inode *inode)
+{
+	spin_lock(&inode->i_lock);
+	if (nfs_have_delegated_atime(inode)) {
+		inode_update_timestamps(inode, S_ATIME);
+		NFS_I(inode)->cache_validity &= ~NFS_INO_INVALID_ATIME;
+	}
+	spin_unlock(&inode->i_lock);
+}
+
+void nfs_update_delegated_mtime_locked(struct inode *inode)
+{
+	if (nfs_have_delegated_mtime(inode)) {
+		inode_update_timestamps(inode, S_CTIME | S_MTIME);
+		NFS_I(inode)->cache_validity &= ~(NFS_INO_INVALID_CTIME |
+						  NFS_INO_INVALID_MTIME);
+	}
+}
+
+void nfs_update_delegated_mtime(struct inode *inode)
+{
+	spin_lock(&inode->i_lock);
+	nfs_update_delegated_mtime_locked(inode);
+	spin_unlock(&inode->i_lock);
+}
+EXPORT_SYMBOL_GPL(nfs_update_delegated_mtime);
+
 #define NFS_VALID_ATTRS (ATTR_MODE|ATTR_UID|ATTR_GID|ATTR_SIZE|ATTR_ATIME|ATTR_ATIME_SET|ATTR_MTIME|ATTR_MTIME_SET|ATTR_FILE|ATTR_OPEN)
 
 int
@@ -630,6 +659,17 @@ nfs_setattr(struct mnt_idmap *idmap, struct dentry *dentry,
 			attr->ia_valid &= ~ATTR_SIZE;
 	}
 
+	if (nfs_have_delegated_mtime(inode)) {
+		if (attr->ia_valid & ATTR_MTIME) {
+			nfs_update_delegated_mtime(inode);
+			attr->ia_valid &= ~ATTR_MTIME;
+		}
+		if (attr->ia_valid & ATTR_ATIME) {
+			nfs_update_delegated_atime(inode);
+			attr->ia_valid &= ~ATTR_ATIME;
+		}
+	}
+
 	/* Optimization: if the end result is no change, don't RPC */
 	if (((attr->ia_valid & NFS_VALID_ATTRS) & ~(ATTR_FILE|ATTR_OPEN)) == 0)
 		return 0;
@@ -685,6 +725,7 @@ static int nfs_vmtruncate(struct inode * inode, loff_t offset)
 
 	spin_unlock(&inode->i_lock);
 	truncate_pagecache(inode, offset);
+	nfs_update_delegated_mtime_locked(inode);
 	spin_lock(&inode->i_lock);
 out:
 	return err;
@@ -708,8 +749,9 @@ void nfs_setattr_update_inode(struct inode *inode, struct iattr *attr,
 	spin_lock(&inode->i_lock);
 	NFS_I(inode)->attr_gencount = fattr->gencount;
 	if ((attr->ia_valid & ATTR_SIZE) != 0) {
-		nfs_set_cache_invalid(inode, NFS_INO_INVALID_MTIME |
-						     NFS_INO_INVALID_BLOCKS);
+		if (!nfs_have_delegated_mtime(inode))
+			nfs_set_cache_invalid(inode, NFS_INO_INVALID_MTIME);
+		nfs_set_cache_invalid(inode, NFS_INO_INVALID_BLOCKS);
 		nfs_inc_stats(inode, NFSIOS_SETATTRTRUNC);
 		nfs_vmtruncate(inode, attr->ia_size);
 	}
@@ -855,8 +897,12 @@ int nfs_getattr(struct mnt_idmap *idmap, const struct path *path,
 
 	/* Flush out writes to the server in order to update c/mtime/version.  */
 	if ((request_mask & (STATX_CTIME | STATX_MTIME | STATX_CHANGE_COOKIE)) &&
-	    S_ISREG(inode->i_mode))
-		filemap_write_and_wait(inode->i_mapping);
+	    S_ISREG(inode->i_mode)) {
+		if (nfs_have_delegated_mtime(inode))
+			filemap_fdatawrite(inode->i_mapping);
+		else
+			filemap_write_and_wait(inode->i_mapping);
+	}
 
 	/*
 	 * We may force a getattr if the user cares about atime.
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 27fb40653f1d..83edbc7a3bcc 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -1245,7 +1245,8 @@ nfs4_update_changeattr_locked(struct inode *inode,
 	struct nfs_inode *nfsi = NFS_I(inode);
 	u64 change_attr = inode_peek_iversion_raw(inode);
 
-	cache_validity |= NFS_INO_INVALID_CTIME | NFS_INO_INVALID_MTIME;
+	if (!nfs_have_delegated_mtime(inode))
+		cache_validity |= NFS_INO_INVALID_CTIME | NFS_INO_INVALID_MTIME;
 	if (S_ISDIR(inode->i_mode))
 		cache_validity |= NFS_INO_INVALID_DATA;
 
@@ -1961,6 +1962,8 @@ nfs4_process_delegation(struct inode *inode, const struct cred *cred,
 	switch (delegation->open_delegation_type) {
 	case NFS4_OPEN_DELEGATE_READ:
 	case NFS4_OPEN_DELEGATE_WRITE:
+	case NFS4_OPEN_DELEGATE_READ_ATTRS_DELEG:
+	case NFS4_OPEN_DELEGATE_WRITE_ATTRS_DELEG:
 		break;
 	default:
 		return;
@@ -1974,16 +1977,16 @@ nfs4_process_delegation(struct inode *inode, const struct cred *cred,
 				   NFS_SERVER(inode)->nfs_client->cl_hostname);
 		break;
 	case NFS4_OPEN_CLAIM_PREVIOUS:
-		nfs_inode_reclaim_delegation(inode, cred,
-				delegation->type,
-				&delegation->stateid,
-				delegation->pagemod_limit);
+		nfs_inode_reclaim_delegation(inode, cred, delegation->type,
+					     &delegation->stateid,
+					     delegation->pagemod_limit,
+					     delegation->open_delegation_type);
 		break;
 	default:
-		nfs_inode_set_delegation(inode, cred,
-				delegation->type,
-				&delegation->stateid,
-				delegation->pagemod_limit);
+		nfs_inode_set_delegation(inode, cred, delegation->type,
+					 &delegation->stateid,
+					 delegation->pagemod_limit,
+					 delegation->open_delegation_type);
 	}
 	if (delegation->do_recall)
 		nfs_async_inode_return_delegation(inode, &delegation->stateid);
diff --git a/fs/nfs/read.c b/fs/nfs/read.c
index a142287d86f6..1b0e06c11983 100644
--- a/fs/nfs/read.c
+++ b/fs/nfs/read.c
@@ -28,6 +28,7 @@
 #include "fscache.h"
 #include "pnfs.h"
 #include "nfstrace.h"
+#include "delegation.h"
 
 #define NFSDBG_FACILITY		NFSDBG_PAGECACHE
 
@@ -372,6 +373,7 @@ int nfs_read_folio(struct file *file, struct folio *folio)
 		goto out_put;
 
 	nfs_pageio_complete_read(&pgio);
+	nfs_update_delegated_atime(inode);
 	ret = pgio.pg_error < 0 ? pgio.pg_error : 0;
 	if (!ret) {
 		ret = folio_wait_locked_killable(folio);
@@ -428,6 +430,7 @@ void nfs_readahead(struct readahead_control *ractl)
 	}
 
 	nfs_pageio_complete_read(&pgio);
+	nfs_update_delegated_atime(inode);
 
 	put_nfs_open_context(ctx);
 out:
diff --git a/fs/nfs/write.c b/fs/nfs/write.c
index a9186b113fe7..c0cd644b97ff 100644
--- a/fs/nfs/write.c
+++ b/fs/nfs/write.c
@@ -289,6 +289,8 @@ static void nfs_grow_file(struct folio *folio, unsigned int offset,
 	NFS_I(inode)->cache_validity &= ~NFS_INO_INVALID_SIZE;
 	nfs_inc_stats(inode, NFSIOS_EXTENDWRITE);
 out:
+	/* Atomically update timestamps if they are delegated to us. */
+	nfs_update_delegated_mtime_locked(inode);
 	spin_unlock(&inode->i_lock);
 	nfs_fscache_invalidate(inode, 0);
 }
@@ -1514,6 +1516,13 @@ void nfs_writeback_update_inode(struct nfs_pgio_header *hdr)
 	struct nfs_fattr *fattr = &hdr->fattr;
 	struct inode *inode = hdr->inode;
 
+	if (nfs_have_delegated_mtime(inode)) {
+		spin_lock(&inode->i_lock);
+		nfs_set_cache_invalid(inode, NFS_INO_INVALID_BLOCKS);
+		spin_unlock(&inode->i_lock);
+		return;
+	}
+
 	spin_lock(&inode->i_lock);
 	nfs_writeback_check_extend(hdr, fattr);
 	nfs_post_op_update_inode_force_wcc_locked(inode, fattr);
-- 
2.45.2

[PATCH 08/19] NFSv4: Add recovery of attribute delegations

[trondmy]

From: Trond Myklebust <trond.myklebust@primarydata.com>

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: Lance Shelton <lance.shelton@hammerspace.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/nfs4proc.c       | 18 +++++++++++++++---
 fs/nfs/nfs4xdr.c        | 18 ++++++++----------
 include/linux/nfs_xdr.h |  2 +-
 3 files changed, 24 insertions(+), 14 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 83edbc7a3bcc..b613c11fac09 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -2225,7 +2225,7 @@ static int _nfs4_do_open_reclaim(struct nfs_open_context *ctx, struct nfs4_state
 {
 	struct nfs_delegation *delegation;
 	struct nfs4_opendata *opendata;
-	fmode_t delegation_type = 0;
+	u32 delegation_type = NFS4_OPEN_DELEGATE_NONE;
 	int status;
 
 	opendata = nfs4_open_recoverdata_alloc(ctx, state,
@@ -2234,8 +2234,20 @@ static int _nfs4_do_open_reclaim(struct nfs_open_context *ctx, struct nfs4_state
 		return PTR_ERR(opendata);
 	rcu_read_lock();
 	delegation = rcu_dereference(NFS_I(state->inode)->delegation);
-	if (delegation != NULL && test_bit(NFS_DELEGATION_NEED_RECLAIM, &delegation->flags) != 0)
-		delegation_type = delegation->type;
+	if (delegation != NULL && test_bit(NFS_DELEGATION_NEED_RECLAIM, &delegation->flags) != 0) {
+		switch(delegation->type) {
+		case FMODE_READ:
+			delegation_type = NFS4_OPEN_DELEGATE_READ;
+			if (test_bit(NFS_DELEGATION_DELEGTIME, &delegation->flags))
+				delegation_type = NFS4_OPEN_DELEGATE_READ_ATTRS_DELEG;
+			break;
+		case FMODE_WRITE:
+		case FMODE_READ|FMODE_WRITE:
+			delegation_type = NFS4_OPEN_DELEGATE_WRITE;
+			if (test_bit(NFS_DELEGATION_DELEGTIME, &delegation->flags))
+				delegation_type = NFS4_OPEN_DELEGATE_WRITE_ATTRS_DELEG;
+		}
+	}
 	rcu_read_unlock();
 	opendata->o_arg.u.delegation_type = delegation_type;
 	status = nfs4_open_recover(opendata, state);
diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
index 4c22b865b9c9..e160a275ad4a 100644
--- a/fs/nfs/nfs4xdr.c
+++ b/fs/nfs/nfs4xdr.c
@@ -1475,20 +1475,18 @@ static void encode_opentype(struct xdr_stream *xdr, const struct nfs_openargs *a
 	}
 }
 
-static inline void encode_delegation_type(struct xdr_stream *xdr, fmode_t delegation_type)
+static inline void encode_delegation_type(struct xdr_stream *xdr, u32 delegation_type)
 {
 	__be32 *p;
 
 	p = reserve_space(xdr, 4);
 	switch (delegation_type) {
-	case 0:
-		*p = cpu_to_be32(NFS4_OPEN_DELEGATE_NONE);
-		break;
-	case FMODE_READ:
-		*p = cpu_to_be32(NFS4_OPEN_DELEGATE_READ);
-		break;
-	case FMODE_WRITE|FMODE_READ:
-		*p = cpu_to_be32(NFS4_OPEN_DELEGATE_WRITE);
+	case NFS4_OPEN_DELEGATE_NONE:
+	case NFS4_OPEN_DELEGATE_READ:
+	case NFS4_OPEN_DELEGATE_WRITE:
+	case NFS4_OPEN_DELEGATE_READ_ATTRS_DELEG:
+	case NFS4_OPEN_DELEGATE_WRITE_ATTRS_DELEG:
+		*p = cpu_to_be32(delegation_type);
 		break;
 	default:
 		BUG();
@@ -1504,7 +1502,7 @@ static inline void encode_claim_null(struct xdr_stream *xdr, const struct qstr *
 	encode_string(xdr, name->len, name->name);
 }
 
-static inline void encode_claim_previous(struct xdr_stream *xdr, fmode_t type)
+static inline void encode_claim_previous(struct xdr_stream *xdr, u32 type)
 {
 	__be32 *p;
 
diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
index 51611583af51..d8cfa956d24c 100644
--- a/include/linux/nfs_xdr.h
+++ b/include/linux/nfs_xdr.h
@@ -484,7 +484,7 @@ struct nfs_openargs {
 			nfs4_verifier   verifier; /* EXCLUSIVE */
 		};
 		nfs4_stateid	delegation;		/* CLAIM_DELEGATE_CUR */
-		fmode_t		delegation_type;	/* CLAIM_PREVIOUS */
+		__u32		delegation_type;	/* CLAIM_PREVIOUS */
 	} u;
 	const struct qstr *	name;
 	const struct nfs_server *server;	 /* Needed for ID mapping */
-- 
2.45.2

[PATCH 09/19] NFSv4: Add a capability for delegated attributes

[trondmy]

From: Trond Myklebust <trond.myklebust@primarydata.com>

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: Lance Shelton <lance.shelton@hammerspace.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/nfs4proc.c         | 2 ++
 include/linux/nfs_fs_sb.h | 1 +
 2 files changed, 3 insertions(+)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index b613c11fac09..efa07c275338 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -3930,6 +3930,8 @@ static int _nfs4_server_capabilities(struct nfs_server *server, struct nfs_fh *f
 #endif
 		if (res.attr_bitmask[0] & FATTR4_WORD0_FS_LOCATIONS)
 			server->caps |= NFS_CAP_FS_LOCATIONS;
+		if (res.attr_bitmask[2] & FATTR4_WORD2_TIME_DELEG_MODIFY)
+			server->caps |= NFS_CAP_DELEGTIME;
 		if (!(res.attr_bitmask[0] & FATTR4_WORD0_FILEID))
 			server->fattr_valid &= ~NFS_ATTR_FATTR_FILEID;
 		if (!(res.attr_bitmask[1] & FATTR4_WORD1_MODE))
diff --git a/include/linux/nfs_fs_sb.h b/include/linux/nfs_fs_sb.h
index 92de074e63b9..5a76a87cd924 100644
--- a/include/linux/nfs_fs_sb.h
+++ b/include/linux/nfs_fs_sb.h
@@ -278,6 +278,7 @@ struct nfs_server {
 #define NFS_CAP_LGOPEN		(1U << 5)
 #define NFS_CAP_CASE_INSENSITIVE	(1U << 6)
 #define NFS_CAP_CASE_PRESERVING	(1U << 7)
+#define NFS_CAP_DELEGTIME	(1U << 13)
 #define NFS_CAP_POSIX_LOCK	(1U << 14)
 #define NFS_CAP_UIDGID_NOMAP	(1U << 15)
 #define NFS_CAP_STATEID_NFSV41	(1U << 16)
-- 
2.45.2

[PATCH 10/19] NFSv4: Enable attribute delegations

[trondmy]

From: Trond Myklebust <trond.myklebust@primarydata.com>

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: Lance Shelton <lance.shelton@hammerspace.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/nfs4proc.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index efa07c275338..140ff1d75320 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -1339,8 +1339,13 @@ nfs4_map_atomic_open_share(struct nfs_server *server,
 	if (!(server->caps & NFS_CAP_ATOMIC_OPEN_V1))
 		goto out;
 	/* Want no delegation if we're using O_DIRECT */
-	if (openflags & O_DIRECT)
+	if (openflags & O_DIRECT) {
 		res |= NFS4_SHARE_WANT_NO_DELEG;
+		goto out;
+	}
+	/* res |= NFS4_SHARE_WANT_NO_PREFERENCE; */
+	if (server->caps & NFS_CAP_DELEGTIME)
+		res |= NFS4_SHARE_WANT_DELEG_TIMESTAMPS;
 out:
 	return res;
 }
-- 
2.45.2

[PATCH 11/19] NFSv4: Delegreturn must set m/atime when they are delegated

[trondmy]

From: Trond Myklebust <trond.myklebust@primarydata.com>

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: Lance Shelton <lance.shelton@hammerspace.com>
---
 fs/nfs/delegation.c |  9 +++++----
 fs/nfs/delegation.h |  4 +++-
 fs/nfs/nfs4proc.c   | 27 ++++++++++++++++++++++++---
 3 files changed, 32 insertions(+), 8 deletions(-)

diff --git a/fs/nfs/delegation.c b/fs/nfs/delegation.c
index e72eead06c08..d8f4a1cdbc8e 100644
--- a/fs/nfs/delegation.c
+++ b/fs/nfs/delegation.c
@@ -258,7 +258,9 @@ void nfs_inode_reclaim_delegation(struct inode *inode, const struct cred *cred,
 	}
 }
 
-static int nfs_do_return_delegation(struct inode *inode, struct nfs_delegation *delegation, int issync)
+static int nfs_do_return_delegation(struct inode *inode,
+				    struct nfs_delegation *delegation,
+				    int issync)
 {
 	const struct cred *cred;
 	int res = 0;
@@ -267,9 +269,8 @@ static int nfs_do_return_delegation(struct inode *inode, struct nfs_delegation *
 		spin_lock(&delegation->lock);
 		cred = get_cred(delegation->cred);
 		spin_unlock(&delegation->lock);
-		res = nfs4_proc_delegreturn(inode, cred,
-				&delegation->stateid,
-				issync);
+		res = nfs4_proc_delegreturn(inode, cred, &delegation->stateid,
+					    delegation, issync);
 		put_cred(cred);
 	}
 	return res;
diff --git a/fs/nfs/delegation.h b/fs/nfs/delegation.h
index 2e9ad83acf2c..da910e2e98a4 100644
--- a/fs/nfs/delegation.h
+++ b/fs/nfs/delegation.h
@@ -72,7 +72,9 @@ void nfs_test_expired_all_delegations(struct nfs_client *clp);
 void nfs_reap_expired_delegations(struct nfs_client *clp);
 
 /* NFSv4 delegation-related procedures */
-int nfs4_proc_delegreturn(struct inode *inode, const struct cred *cred, const nfs4_stateid *stateid, int issync);
+int nfs4_proc_delegreturn(struct inode *inode, const struct cred *cred,
+			  const nfs4_stateid *stateid,
+			  struct nfs_delegation *delegation, int issync);
 int nfs4_open_delegation_recall(struct nfs_open_context *ctx, struct nfs4_state *state, const nfs4_stateid *stateid);
 int nfs4_lock_delegation_recall(struct file_lock *fl, struct nfs4_state *state, const nfs4_stateid *stateid);
 bool nfs4_copy_delegation_stateid(struct inode *inode, fmode_t flags, nfs4_stateid *dst, const struct cred **cred);
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 140ff1d75320..b0c1564a7bc7 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -6718,7 +6718,10 @@ static const struct rpc_call_ops nfs4_delegreturn_ops = {
 	.rpc_release = nfs4_delegreturn_release,
 };
 
-static int _nfs4_proc_delegreturn(struct inode *inode, const struct cred *cred, const nfs4_stateid *stateid, int issync)
+static int _nfs4_proc_delegreturn(struct inode *inode, const struct cred *cred,
+				  const nfs4_stateid *stateid,
+				  struct nfs_delegation *delegation,
+				  int issync)
 {
 	struct nfs4_delegreturndata *data;
 	struct nfs_server *server = NFS_SERVER(inode);
@@ -6770,12 +6773,27 @@ static int _nfs4_proc_delegreturn(struct inode *inode, const struct cred *cred,
 		}
 	}
 
+	if (delegation &&
+	    test_bit(NFS_DELEGATION_DELEGTIME, &delegation->flags)) {
+		if (delegation->type & FMODE_READ) {
+			data->sattr.atime = inode_get_atime(inode);
+			data->sattr.atime_set = true;
+		}
+		if (delegation->type & FMODE_WRITE) {
+			data->sattr.mtime = inode_get_mtime(inode);
+			data->sattr.mtime_set = true;
+		}
+		data->args.sattr_args = &data->sattr;
+		data->res.sattr_res = true;
+	}
+
 	if (!data->inode)
 		nfs4_init_sequence(&data->args.seq_args, &data->res.seq_res, 1,
 				   1);
 	else
 		nfs4_init_sequence(&data->args.seq_args, &data->res.seq_res, 1,
 				   0);
+
 	task_setup_data.callback_data = data;
 	msg.rpc_argp = &data->args;
 	msg.rpc_resp = &data->res;
@@ -6793,13 +6811,16 @@ static int _nfs4_proc_delegreturn(struct inode *inode, const struct cred *cred,
 	return status;
 }
 
-int nfs4_proc_delegreturn(struct inode *inode, const struct cred *cred, const nfs4_stateid *stateid, int issync)
+int nfs4_proc_delegreturn(struct inode *inode, const struct cred *cred,
+			  const nfs4_stateid *stateid,
+			  struct nfs_delegation *delegation, int issync)
 {
 	struct nfs_server *server = NFS_SERVER(inode);
 	struct nfs4_exception exception = { };
 	int err;
 	do {
-		err = _nfs4_proc_delegreturn(inode, cred, stateid, issync);
+		err = _nfs4_proc_delegreturn(inode, cred, stateid,
+					     delegation, issync);
 		trace_nfs4_delegreturn(inode, stateid, err);
 		switch (err) {
 			case -NFS4ERR_STALE_STATEID:
-- 
2.45.2

[PATCH 12/19] NFSv4: Fix up delegated attributes in nfs_setattr

[trondmy]

From: Trond Myklebust <trond.myklebust@primarydata.com>

nfs_setattr calls nfs_update_inode() directly, so we have to reset the
m/ctime there.

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: Lance Shelton <lance.shelton@hammerspace.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/inode.c | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
index 91c0aeaf6c1e..e03c512c8535 100644
--- a/fs/nfs/inode.c
+++ b/fs/nfs/inode.c
@@ -605,6 +605,46 @@ nfs_fhget(struct super_block *sb, struct nfs_fh *fh, struct nfs_fattr *fattr)
 }
 EXPORT_SYMBOL_GPL(nfs_fhget);
 
+static void
+nfs_fattr_fixup_delegated(struct inode *inode, struct nfs_fattr *fattr)
+{
+	unsigned long cache_validity = NFS_I(inode)->cache_validity;
+
+	if (!nfs_have_read_or_write_delegation(inode))
+		return;
+
+	if (!(cache_validity & NFS_INO_REVAL_FORCED))
+		cache_validity &= ~(NFS_INO_INVALID_ATIME
+				| NFS_INO_INVALID_CHANGE
+				| NFS_INO_INVALID_CTIME
+				| NFS_INO_INVALID_MTIME
+				| NFS_INO_INVALID_SIZE);
+
+	if (!(cache_validity & NFS_INO_INVALID_SIZE))
+		fattr->valid &= ~(NFS_ATTR_FATTR_PRESIZE
+				| NFS_ATTR_FATTR_SIZE);
+
+	if (!(cache_validity & NFS_INO_INVALID_CHANGE))
+		fattr->valid &= ~(NFS_ATTR_FATTR_PRECHANGE
+				| NFS_ATTR_FATTR_CHANGE);
+
+	if (nfs_have_delegated_mtime(inode)) {
+		if (!(cache_validity & NFS_INO_INVALID_CTIME))
+			fattr->valid &= ~(NFS_ATTR_FATTR_PRECTIME
+					| NFS_ATTR_FATTR_CTIME);
+
+		if (!(cache_validity & NFS_INO_INVALID_MTIME))
+			fattr->valid &= ~(NFS_ATTR_FATTR_PREMTIME
+					| NFS_ATTR_FATTR_MTIME);
+
+		if (!(cache_validity & NFS_INO_INVALID_ATIME))
+			fattr->valid &= ~NFS_ATTR_FATTR_ATIME;
+	} else if (nfs_have_delegated_atime(inode)) {
+		if (!(cache_validity & NFS_INO_INVALID_ATIME))
+			fattr->valid &= ~NFS_ATTR_FATTR_ATIME;
+	}
+}
+
 void nfs_update_delegated_atime(struct inode *inode)
 {
 	spin_lock(&inode->i_lock);
@@ -2163,6 +2203,9 @@ static int nfs_update_inode(struct inode *inode, struct nfs_fattr *fattr)
 	 */
 	nfsi->read_cache_jiffies = fattr->time_start;
 
+	/* Fix up any delegated attributes in the struct nfs_fattr */
+	nfs_fattr_fixup_delegated(inode, fattr);
+
 	save_cache_validity = nfsi->cache_validity;
 	nfsi->cache_validity &= ~(NFS_INO_INVALID_ATTR
 			| NFS_INO_INVALID_ATIME
-- 
2.45.2

[PATCH 13/19] NFSv4: Don't request atime/mtime/size if they are delegated to us

[trondmy]

From: Trond Myklebust <trond.myklebust@primarydata.com>

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: Lance Shelton <lance.shelton@hammerspace.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/nfs4proc.c | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index b0c1564a7bc7..512268c732a1 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -310,6 +310,18 @@ static void nfs4_bitmap_copy_adjust(__u32 *dst, const __u32 *src,
 		dst[1] &= ~FATTR4_WORD1_MODE;
 	if (!(cache_validity & NFS_INO_INVALID_OTHER))
 		dst[1] &= ~(FATTR4_WORD1_OWNER | FATTR4_WORD1_OWNER_GROUP);
+
+	if (nfs_have_delegated_mtime(inode)) {
+		if (!(cache_validity & NFS_INO_INVALID_ATIME))
+			dst[1] &= ~FATTR4_WORD1_TIME_ACCESS;
+		if (!(cache_validity & NFS_INO_INVALID_MTIME))
+			dst[1] &= ~FATTR4_WORD1_TIME_MODIFY;
+		if (!(cache_validity & NFS_INO_INVALID_CTIME))
+			dst[1] &= ~FATTR4_WORD1_TIME_METADATA;
+	} else if (nfs_have_delegated_atime(inode)) {
+		if (!(cache_validity & NFS_INO_INVALID_ATIME))
+			dst[1] &= ~FATTR4_WORD1_TIME_ACCESS;
+	}
 }
 
 static void nfs4_setup_readdir(u64 cookie, __be32 *verifier, struct dentry *dentry,
@@ -3414,7 +3426,8 @@ static int nfs4_do_setattr(struct inode *inode, const struct cred *cred,
 		.inode = inode,
 		.stateid = &arg.stateid,
 	};
-	unsigned long adjust_flags = NFS_INO_INVALID_CHANGE;
+	unsigned long adjust_flags = NFS_INO_INVALID_CHANGE |
+				     NFS_INO_INVALID_CTIME;
 	int err;
 
 	if (sattr->ia_valid & (ATTR_MODE | ATTR_KILL_SUID | ATTR_KILL_SGID))
@@ -4958,7 +4971,7 @@ static int _nfs4_proc_link(struct inode *inode, struct inode *dir, const struct
 
 	nfs4_inode_make_writeable(inode);
 	nfs4_bitmap_copy_adjust(bitmask, nfs4_bitmask(server, res.fattr->label), inode,
-				NFS_INO_INVALID_CHANGE);
+				NFS_INO_INVALID_CHANGE | NFS_INO_INVALID_CTIME);
 	status = nfs4_call_sync(server->client, server, &msg, &arg.seq_args, &res.seq_res, 1);
 	if (!status) {
 		nfs4_update_changeattr(dir, &res.cinfo, res.fattr->time_start,
-- 
2.45.2

[PATCH 14/19] NFSv4: Add support for the FATTR4_OPEN_ARGUMENTS attribute

[trondmy]

From: Trond Myklebust <trond.myklebust@primarydata.com>

Query the server for the OPEN arguments that it supports so that
we can figure out which extensions we can use.

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: Lance Shelton <lance.shelton@hammerspace.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/nfs4proc.c       | 20 ++++++++++++++++++--
 fs/nfs/nfs4xdr.c        | 24 ++++++++++++++++++++++++
 include/linux/nfs4.h    |  2 ++
 include/linux/nfs_xdr.h |  9 +++++++++
 4 files changed, 53 insertions(+), 2 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 512268c732a1..ca2c115b6545 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -3885,11 +3885,14 @@ static void nfs4_close_context(struct nfs_open_context *ctx, int is_sync)
 
 #define FATTR4_WORD1_NFS40_MASK (2*FATTR4_WORD1_MOUNTED_ON_FILEID - 1UL)
 #define FATTR4_WORD2_NFS41_MASK (2*FATTR4_WORD2_SUPPATTR_EXCLCREAT - 1UL)
-#define FATTR4_WORD2_NFS42_MASK (2*FATTR4_WORD2_TIME_DELEG_MODIFY - 1UL)
+#define FATTR4_WORD2_NFS42_MASK (2*FATTR4_WORD2_OPEN_ARGUMENTS - 1UL)
 
 static int _nfs4_server_capabilities(struct nfs_server *server, struct nfs_fh *fhandle)
 {
-	u32 bitmask[3] = {}, minorversion = server->nfs_client->cl_minorversion;
+	u32 minorversion = server->nfs_client->cl_minorversion;
+	u32 bitmask[3] = {
+		[0] = FATTR4_WORD0_SUPPORTED_ATTRS,
+	};
 	struct nfs4_server_caps_arg args = {
 		.fhandle = fhandle,
 		.bitmask = bitmask,
@@ -3915,6 +3918,14 @@ static int _nfs4_server_capabilities(struct nfs_server *server, struct nfs_fh *f
 
 	status = nfs4_call_sync(server->client, server, &msg, &args.seq_args, &res.seq_res, 0);
 	if (status == 0) {
+		bitmask[0] = (FATTR4_WORD0_SUPPORTED_ATTRS |
+			      FATTR4_WORD0_FH_EXPIRE_TYPE |
+			      FATTR4_WORD0_LINK_SUPPORT |
+			      FATTR4_WORD0_SYMLINK_SUPPORT |
+			      FATTR4_WORD0_ACLSUPPORT |
+			      FATTR4_WORD0_CASE_INSENSITIVE |
+			      FATTR4_WORD0_CASE_PRESERVING) &
+			     res.attr_bitmask[0];
 		/* Sanity check the server answers */
 		switch (minorversion) {
 		case 0:
@@ -3923,9 +3934,14 @@ static int _nfs4_server_capabilities(struct nfs_server *server, struct nfs_fh *f
 			break;
 		case 1:
 			res.attr_bitmask[2] &= FATTR4_WORD2_NFS41_MASK;
+			bitmask[2] = FATTR4_WORD2_SUPPATTR_EXCLCREAT &
+				     res.attr_bitmask[2];
 			break;
 		case 2:
 			res.attr_bitmask[2] &= FATTR4_WORD2_NFS42_MASK;
+			bitmask[2] = (FATTR4_WORD2_SUPPATTR_EXCLCREAT |
+				      FATTR4_WORD2_OPEN_ARGUMENTS) &
+				     res.attr_bitmask[2];
 		}
 		memcpy(server->attr_bitmask, res.attr_bitmask, sizeof(server->attr_bitmask));
 		server->caps &= ~(NFS_CAP_ACLS | NFS_CAP_HARDLINKS |
diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
index e160a275ad4a..98aab2c324c9 100644
--- a/fs/nfs/nfs4xdr.c
+++ b/fs/nfs/nfs4xdr.c
@@ -4337,6 +4337,28 @@ static int decode_attr_xattrsupport(struct xdr_stream *xdr, uint32_t *bitmap,
 	return 0;
 }
 
+static int decode_attr_open_arguments(struct xdr_stream *xdr, uint32_t *bitmap,
+		struct nfs4_open_caps *res)
+{
+	memset(res, 0, sizeof(*res));
+	if (unlikely(bitmap[2] & (FATTR4_WORD2_OPEN_ARGUMENTS - 1U)))
+		return -EIO;
+	if (likely(bitmap[2] & FATTR4_WORD2_OPEN_ARGUMENTS)) {
+		if (decode_bitmap4(xdr, res->oa_share_access, ARRAY_SIZE(res->oa_share_access)) < 0)
+			return -EIO;
+		if (decode_bitmap4(xdr, res->oa_share_deny, ARRAY_SIZE(res->oa_share_deny)) < 0)
+			return -EIO;
+		if (decode_bitmap4(xdr, res->oa_share_access_want, ARRAY_SIZE(res->oa_share_access_want)) < 0)
+			return -EIO;
+		if (decode_bitmap4(xdr, res->oa_open_claim, ARRAY_SIZE(res->oa_open_claim)) < 0)
+			return -EIO;
+		if (decode_bitmap4(xdr, res->oa_createmode, ARRAY_SIZE(res->oa_createmode)) < 0)
+			return -EIO;
+		bitmap[2] &= ~FATTR4_WORD2_OPEN_ARGUMENTS;
+	}
+	return 0;
+}
+
 static int verify_attr_len(struct xdr_stream *xdr, unsigned int savep, uint32_t attrlen)
 {
 	unsigned int attrwords = XDR_QUADLEN(attrlen);
@@ -4511,6 +4533,8 @@ static int decode_server_caps(struct xdr_stream *xdr, struct nfs4_server_caps_re
 	if ((status = decode_attr_exclcreat_supported(xdr, bitmap,
 				res->exclcreat_bitmask)) != 0)
 		goto xdr_error;
+	if ((status = decode_attr_open_arguments(xdr, bitmap, &res->open_caps)) != 0)
+		goto xdr_error;
 	status = verify_attr_len(xdr, savep, attrlen);
 xdr_error:
 	dprintk("%s: xdr returned %d!\n", __func__, -status);
diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
index bc13d7f04e8d..79b23ad674c8 100644
--- a/include/linux/nfs4.h
+++ b/include/linux/nfs4.h
@@ -512,6 +512,7 @@ enum {
 enum {
 	FATTR4_TIME_DELEG_ACCESS	= 84,
 	FATTR4_TIME_DELEG_MODIFY	= 85,
+	FATTR4_OPEN_ARGUMENTS		= 86,
 };
 
 /*
@@ -595,6 +596,7 @@ enum {
 #define FATTR4_WORD2_XATTR_SUPPORT	BIT(FATTR4_XATTR_SUPPORT - 64)
 #define FATTR4_WORD2_TIME_DELEG_ACCESS	BIT(FATTR4_TIME_DELEG_ACCESS - 64)
 #define FATTR4_WORD2_TIME_DELEG_MODIFY	BIT(FATTR4_TIME_DELEG_MODIFY - 64)
+#define FATTR4_WORD2_OPEN_ARGUMENTS	BIT(FATTR4_OPEN_ARGUMENTS - 64)
 
 /* MDS threshold bitmap bits */
 #define THRESHOLD_RD                    (1UL << 0)
diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
index d8cfa956d24c..af510a7ec46a 100644
--- a/include/linux/nfs_xdr.h
+++ b/include/linux/nfs_xdr.h
@@ -1213,6 +1213,14 @@ struct nfs4_statfs_res {
 	struct nfs_fsstat	       *fsstat;
 };
 
+struct nfs4_open_caps {
+	u32				oa_share_access[1];
+	u32				oa_share_deny[1];
+	u32				oa_share_access_want[1];
+	u32				oa_open_claim[1];
+	u32				oa_createmode[1];
+};
+
 struct nfs4_server_caps_arg {
 	struct nfs4_sequence_args	seq_args;
 	struct nfs_fh		       *fhandle;
@@ -1229,6 +1237,7 @@ struct nfs4_server_caps_res {
 	u32				fh_expire_type;
 	u32				case_insensitive;
 	u32				case_preserving;
+	struct nfs4_open_caps		open_caps;
 };
 
 #define NFS4_PATHNAME_MAXCOMPONENTS 512
-- 
2.45.2

[PATCH 15/19] NFSv4: Detect support for OPEN4_SHARE_ACCESS_WANT_OPEN_XOR_DELEGATION

[trondmy]

From: Trond Myklebust <trond.myklebust@primarydata.com>

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: Lance Shelton <lance.shelton@hammerspace.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/nfs4proc.c         | 4 ++++
 include/linux/nfs_fs_sb.h | 1 +
 include/uapi/linux/nfs4.h | 2 ++
 3 files changed, 7 insertions(+)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index ca2c115b6545..87a197864277 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -3990,6 +3990,10 @@ static int _nfs4_server_capabilities(struct nfs_server *server, struct nfs_fh *f
 				sizeof(server->attr_bitmask));
 		server->attr_bitmask_nl[2] &= ~FATTR4_WORD2_SECURITY_LABEL;
 
+		if (res.open_caps.oa_share_access_want[0] &
+		    NFS4_SHARE_WANT_OPEN_XOR_DELEGATION)
+			server->caps |= NFS_CAP_OPEN_XOR;
+
 		memcpy(server->cache_consistency_bitmask, res.attr_bitmask, sizeof(server->cache_consistency_bitmask));
 		server->cache_consistency_bitmask[0] &= FATTR4_WORD0_CHANGE|FATTR4_WORD0_SIZE;
 		server->cache_consistency_bitmask[1] &= FATTR4_WORD1_TIME_METADATA|FATTR4_WORD1_TIME_MODIFY;
diff --git a/include/linux/nfs_fs_sb.h b/include/linux/nfs_fs_sb.h
index 5a76a87cd924..fe5b1a8bd723 100644
--- a/include/linux/nfs_fs_sb.h
+++ b/include/linux/nfs_fs_sb.h
@@ -278,6 +278,7 @@ struct nfs_server {
 #define NFS_CAP_LGOPEN		(1U << 5)
 #define NFS_CAP_CASE_INSENSITIVE	(1U << 6)
 #define NFS_CAP_CASE_PRESERVING	(1U << 7)
+#define NFS_CAP_OPEN_XOR	(1U << 12)
 #define NFS_CAP_DELEGTIME	(1U << 13)
 #define NFS_CAP_POSIX_LOCK	(1U << 14)
 #define NFS_CAP_UIDGID_NOMAP	(1U << 15)
diff --git a/include/uapi/linux/nfs4.h b/include/uapi/linux/nfs4.h
index afd7e32906c3..caf4db2fcbb9 100644
--- a/include/uapi/linux/nfs4.h
+++ b/include/uapi/linux/nfs4.h
@@ -46,6 +46,7 @@
 #define NFS4_OPEN_RESULT_CONFIRM		0x0002
 #define NFS4_OPEN_RESULT_LOCKTYPE_POSIX		0x0004
 #define NFS4_OPEN_RESULT_PRESERVE_UNLINKED	0x0008
+#define NFS4_OPEN_RESULT_NO_OPEN_STATEID	0x0010
 #define NFS4_OPEN_RESULT_MAY_NOTIFY_LOCK	0x0020
 
 #define NFS4_SHARE_ACCESS_MASK	0x000F
@@ -70,6 +71,7 @@
 #define NFS4_SHARE_PUSH_DELEG_WHEN_UNCONTENDED		0x20000
 
 #define NFS4_SHARE_WANT_DELEG_TIMESTAMPS		0x100000
+#define NFS4_SHARE_WANT_OPEN_XOR_DELEGATION		0x200000
 
 #define NFS4_CDFC4_FORE	0x1
 #define NFS4_CDFC4_BACK 0x2
-- 
2.45.2

[PATCH 16/19] NFSv4: Add support for OPEN4_RESULT_NO_OPEN_STATEID

[trondmy]

From: Trond Myklebust <trond.myklebust@primarydata.com>

If the server returns a delegation stateid only, then don't try to set
an open stateid.

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: Lance Shelton <lance.shelton@hammerspace.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/nfs4proc.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 87a197864277..23947fca78fe 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -2035,8 +2035,11 @@ _nfs4_opendata_reclaim_to_nfs4_state(struct nfs4_opendata *data)
 				data->o_arg.claim,
 				&data->o_res.delegation);
 
-	if (!update_open_stateid(state, &data->o_res.stateid,
-				NULL, data->o_arg.fmode))
+	if (!(data->o_res.rflags & NFS4_OPEN_RESULT_NO_OPEN_STATEID)) {
+		if (!update_open_stateid(state, &data->o_res.stateid,
+					 NULL, data->o_arg.fmode))
+			return ERR_PTR(-EAGAIN);
+	} else if (!update_open_stateid(state, NULL, NULL, data->o_arg.fmode))
 		return ERR_PTR(-EAGAIN);
 	refcount_inc(&state->count);
 
@@ -2105,8 +2108,13 @@ _nfs4_opendata_to_nfs4_state(struct nfs4_opendata *data)
 				data->o_arg.claim,
 				&data->o_res.delegation);
 
-	if (!update_open_stateid(state, &data->o_res.stateid,
-				NULL, data->o_arg.fmode)) {
+	if (!(data->o_res.rflags & NFS4_OPEN_RESULT_NO_OPEN_STATEID)) {
+		if (!update_open_stateid(state, &data->o_res.stateid,
+					 NULL, data->o_arg.fmode)) {
+			nfs4_put_open_state(state);
+			state = ERR_PTR(-EAGAIN);
+		}
+	} else if (!update_open_stateid(state, NULL, NULL, data->o_arg.fmode)) {
 		nfs4_put_open_state(state);
 		state = ERR_PTR(-EAGAIN);
 	}
-- 
2.45.2

[PATCH 17/19] NFSv4: Ask for a delegation or an open stateid in OPEN

[trondmy]

From: Trond Myklebust <trond.myklebust@primarydata.com>

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: Lance Shelton <lance.shelton@hammerspace.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/nfs4proc.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 23947fca78fe..d41d86c713ea 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -1358,6 +1358,8 @@ nfs4_map_atomic_open_share(struct nfs_server *server,
 	/* res |= NFS4_SHARE_WANT_NO_PREFERENCE; */
 	if (server->caps & NFS_CAP_DELEGTIME)
 		res |= NFS4_SHARE_WANT_DELEG_TIMESTAMPS;
+	if (server->caps & NFS_CAP_OPEN_XOR)
+		res |= NFS4_SHARE_WANT_OPEN_XOR_DELEGATION;
 out:
 	return res;
 }
-- 
2.45.2

[PATCH 18/19] NFS: Add a generic callback to return the delegation

[trondmy]

From: Lance Shelton <lance.shelton@primarydata.com>

Allow generic NFS code to return the delegation when appropriate.

Signed-off-by: Lance Shelton <lance.shelton@hammerspace.com>
Signed-off-by: Trond Myklebust <Trond.Myklebust@hammerspace.com>
---
 fs/nfs/nfs3proc.c       | 8 ++++++++
 fs/nfs/nfs4proc.c       | 1 +
 fs/nfs/proc.c           | 8 ++++++++
 include/linux/nfs_xdr.h | 1 +
 4 files changed, 18 insertions(+)

diff --git a/fs/nfs/nfs3proc.c b/fs/nfs/nfs3proc.c
index cab6c73d25d6..1566163c6d85 100644
--- a/fs/nfs/nfs3proc.c
+++ b/fs/nfs/nfs3proc.c
@@ -984,6 +984,13 @@ static int nfs3_have_delegation(struct inode *inode, fmode_t type, int flags)
 	return 0;
 }
 
+static int nfs3_return_delegation(struct inode *inode)
+{
+	if (S_ISREG(inode->i_mode))
+		nfs_wb_all(inode);
+	return 0;
+}
+
 static const struct inode_operations nfs3_dir_inode_operations = {
 	.create		= nfs_create,
 	.atomic_open	= nfs_atomic_open_v23,
@@ -1062,6 +1069,7 @@ const struct nfs_rpc_ops nfs_v3_clientops = {
 	.clear_acl_cache = forget_all_cached_acls,
 	.close_context	= nfs_close_context,
 	.have_delegation = nfs3_have_delegation,
+	.return_delegation = nfs3_return_delegation,
 	.alloc_client	= nfs_alloc_client,
 	.init_client	= nfs_init_client,
 	.free_client	= nfs_free_client,
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index d41d86c713ea..a4f85af880c2 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -10849,6 +10849,7 @@ const struct nfs_rpc_ops nfs_v4_clientops = {
 	.close_context  = nfs4_close_context,
 	.open_context	= nfs4_atomic_open,
 	.have_delegation = nfs4_have_delegation,
+	.return_delegation = nfs4_inode_return_delegation,
 	.alloc_client	= nfs4_alloc_client,
 	.init_client	= nfs4_init_client,
 	.free_client	= nfs4_free_client,
diff --git a/fs/nfs/proc.c b/fs/nfs/proc.c
index 995cc42b0fa0..6c09cd090c34 100644
--- a/fs/nfs/proc.c
+++ b/fs/nfs/proc.c
@@ -692,6 +692,13 @@ static int nfs_have_delegation(struct inode *inode, fmode_t type, int flags)
 	return 0;
 }
 
+static int nfs_return_delegation(struct inode *inode)
+{
+	if (S_ISREG(inode->i_mode))
+		nfs_wb_all(inode);
+	return 0;
+}
+
 static const struct inode_operations nfs_dir_inode_operations = {
 	.create		= nfs_create,
 	.lookup		= nfs_lookup,
@@ -757,6 +764,7 @@ const struct nfs_rpc_ops nfs_v2_clientops = {
 	.lock_check_bounds = nfs_lock_check_bounds,
 	.close_context	= nfs_close_context,
 	.have_delegation = nfs_have_delegation,
+	.return_delegation = nfs_return_delegation,
 	.alloc_client	= nfs_alloc_client,
 	.init_client	= nfs_init_client,
 	.free_client	= nfs_free_client,
diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
index af510a7ec46a..01efacae4634 100644
--- a/include/linux/nfs_xdr.h
+++ b/include/linux/nfs_xdr.h
@@ -1840,6 +1840,7 @@ struct nfs_rpc_ops {
 				struct iattr *iattr,
 				int *);
 	int (*have_delegation)(struct inode *, fmode_t, int);
+	int (*return_delegation)(struct inode *);
 	struct nfs_client *(*alloc_client) (const struct nfs_client_initdata *);
 	struct nfs_client *(*init_client) (struct nfs_client *,
 				const struct nfs_client_initdata *);
-- 
2.45.2

[PATCH 19/19] Return the delegation when deleting the sillyrenamed file

[trondmy]

From: Trond Myklebust <trond.myklebust@primarydata.com>

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: Lance Shelton <lance.shelton@hammerspace.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/unlink.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/fs/nfs/unlink.c b/fs/nfs/unlink.c
index 0110299643a2..bf77399696a7 100644
--- a/fs/nfs/unlink.c
+++ b/fs/nfs/unlink.c
@@ -232,6 +232,8 @@ nfs_complete_unlink(struct dentry *dentry, struct inode *inode)
 	dentry->d_fsdata = NULL;
 	spin_unlock(&dentry->d_lock);
 
+	NFS_PROTO(inode)->return_delegation(inode);
+
 	if (NFS_STALE(inode) || !nfs_call_unlink(dentry, inode, data))
 		nfs_free_unlinkdata(data);
 }
-- 
2.45.2

Re: [PATCH 12/19] NFSv4: Fix up delegated attributes in nfs_setattr

[Anna Schumaker]

Hi Trond,

On Thu, Jun 13, 2024 at 12:18 AM <trondmy@gmail.com> wrote:
>
> From: Trond Myklebust <trond.myklebust@primarydata.com>
>
> nfs_setattr calls nfs_update_inode() directly, so we have to reset the
> m/ctime there.
>
> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
> Signed-off-by: Lance Shelton <lance.shelton@hammerspace.com>
> Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
> ---
>  fs/nfs/inode.c | 43 +++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 43 insertions(+)

After applying this patch I see a handful of new failures on xfstests:
generic/075, generic/086, generic/112, generic/332, generic/346,
generic/647, and generic/729. I see the first five on NFS v4.2, but
647 and 729 both fail on v4.1 in addition to v4.2.

I hope this helps!
Anna

>
> diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
> index 91c0aeaf6c1e..e03c512c8535 100644
> --- a/fs/nfs/inode.c
> +++ b/fs/nfs/inode.c
> @@ -605,6 +605,46 @@ nfs_fhget(struct super_block *sb, struct nfs_fh *fh, struct nfs_fattr *fattr)
>  }
>  EXPORT_SYMBOL_GPL(nfs_fhget);
>
> +static void
> +nfs_fattr_fixup_delegated(struct inode *inode, struct nfs_fattr *fattr)
> +{
> +       unsigned long cache_validity = NFS_I(inode)->cache_validity;
> +
> +       if (!nfs_have_read_or_write_delegation(inode))
> +               return;
> +
> +       if (!(cache_validity & NFS_INO_REVAL_FORCED))
> +               cache_validity &= ~(NFS_INO_INVALID_ATIME
> +                               | NFS_INO_INVALID_CHANGE
> +                               | NFS_INO_INVALID_CTIME
> +                               | NFS_INO_INVALID_MTIME
> +                               | NFS_INO_INVALID_SIZE);
> +
> +       if (!(cache_validity & NFS_INO_INVALID_SIZE))
> +               fattr->valid &= ~(NFS_ATTR_FATTR_PRESIZE
> +                               | NFS_ATTR_FATTR_SIZE);
> +
> +       if (!(cache_validity & NFS_INO_INVALID_CHANGE))
> +               fattr->valid &= ~(NFS_ATTR_FATTR_PRECHANGE
> +                               | NFS_ATTR_FATTR_CHANGE);
> +
> +       if (nfs_have_delegated_mtime(inode)) {
> +               if (!(cache_validity & NFS_INO_INVALID_CTIME))
> +                       fattr->valid &= ~(NFS_ATTR_FATTR_PRECTIME
> +                                       | NFS_ATTR_FATTR_CTIME);
> +
> +               if (!(cache_validity & NFS_INO_INVALID_MTIME))
> +                       fattr->valid &= ~(NFS_ATTR_FATTR_PREMTIME
> +                                       | NFS_ATTR_FATTR_MTIME);
> +
> +               if (!(cache_validity & NFS_INO_INVALID_ATIME))
> +                       fattr->valid &= ~NFS_ATTR_FATTR_ATIME;
> +       } else if (nfs_have_delegated_atime(inode)) {
> +               if (!(cache_validity & NFS_INO_INVALID_ATIME))
> +                       fattr->valid &= ~NFS_ATTR_FATTR_ATIME;
> +       }
> +}
> +
>  void nfs_update_delegated_atime(struct inode *inode)
>  {
>         spin_lock(&inode->i_lock);
> @@ -2163,6 +2203,9 @@ static int nfs_update_inode(struct inode *inode, struct nfs_fattr *fattr)
>          */
>         nfsi->read_cache_jiffies = fattr->time_start;
>
> +       /* Fix up any delegated attributes in the struct nfs_fattr */
> +       nfs_fattr_fixup_delegated(inode, fattr);
> +
>         save_cache_validity = nfsi->cache_validity;
>         nfsi->cache_validity &= ~(NFS_INO_INVALID_ATTR
>                         | NFS_INO_INVALID_ATIME
> --
> 2.45.2
>
>

Re: [PATCH 12/19] NFSv4: Fix up delegated attributes in nfs_setattr

[Trond Myklebust]

On Fri, 2024-06-14 at 12:32 -0400, Anna Schumaker wrote:
> Hi Trond,
> 
> On Thu, Jun 13, 2024 at 12:18 AM <trondmy@gmail.com> wrote:
> > 
> > From: Trond Myklebust <trond.myklebust@primarydata.com>
> > 
> > nfs_setattr calls nfs_update_inode() directly, so we have to reset
> > the
> > m/ctime there.
> > 
> > Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
> > Signed-off-by: Lance Shelton <lance.shelton@hammerspace.com>
> > Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
> > ---
> >  fs/nfs/inode.c | 43 +++++++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 43 insertions(+)
> 
> After applying this patch I see a handful of new failures on
> xfstests:
> generic/075, generic/086, generic/112, generic/332, generic/346,
> generic/647, and generic/729. I see the first five on NFS v4.2, but
> 647 and 729 both fail on v4.1 in addition to v4.2.

Thanks Anna!

Yes, I think that is a consequence of the changes that were made in
commit cc7f2dae63bc ("NFS: Don't store NFS_INO_REVAL_FORCED"). I can
just remove that "if (!(cache_validity & NFS_INO_REVAL_FORCED))"
altogether with that change applied.

Version 2 will be forthcoming.

> 
> I hope this helps!
> Anna
> 
> > 
> > diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
> > index 91c0aeaf6c1e..e03c512c8535 100644
> > --- a/fs/nfs/inode.c
> > +++ b/fs/nfs/inode.c
> > @@ -605,6 +605,46 @@ nfs_fhget(struct super_block *sb, struct
> > nfs_fh *fh, struct nfs_fattr *fattr)
> >  }
> >  EXPORT_SYMBOL_GPL(nfs_fhget);
> > 
> > +static void
> > +nfs_fattr_fixup_delegated(struct inode *inode, struct nfs_fattr
> > *fattr)
> > +{
> > +       unsigned long cache_validity = NFS_I(inode)-
> > >cache_validity;
> > +
> > +       if (!nfs_have_read_or_write_delegation(inode))
> > +               return;
> > +
> > +       if (!(cache_validity & NFS_INO_REVAL_FORCED))
> > +               cache_validity &= ~(NFS_INO_INVALID_ATIME
> > +                               | NFS_INO_INVALID_CHANGE
> > +                               | NFS_INO_INVALID_CTIME
> > +                               | NFS_INO_INVALID_MTIME
> > +                               | NFS_INO_INVALID_SIZE);
> > +
> > +       if (!(cache_validity & NFS_INO_INVALID_SIZE))
> > +               fattr->valid &= ~(NFS_ATTR_FATTR_PRESIZE
> > +                               | NFS_ATTR_FATTR_SIZE);
> > +
> > +       if (!(cache_validity & NFS_INO_INVALID_CHANGE))
> > +               fattr->valid &= ~(NFS_ATTR_FATTR_PRECHANGE
> > +                               | NFS_ATTR_FATTR_CHANGE);
> > +
> > +       if (nfs_have_delegated_mtime(inode)) {
> > +               if (!(cache_validity & NFS_INO_INVALID_CTIME))
> > +                       fattr->valid &= ~(NFS_ATTR_FATTR_PRECTIME
> > +                                       | NFS_ATTR_FATTR_CTIME);
> > +
> > +               if (!(cache_validity & NFS_INO_INVALID_MTIME))
> > +                       fattr->valid &= ~(NFS_ATTR_FATTR_PREMTIME
> > +                                       | NFS_ATTR_FATTR_MTIME);
> > +
> > +               if (!(cache_validity & NFS_INO_INVALID_ATIME))
> > +                       fattr->valid &= ~NFS_ATTR_FATTR_ATIME;
> > +       } else if (nfs_have_delegated_atime(inode)) {
> > +               if (!(cache_validity & NFS_INO_INVALID_ATIME))
> > +                       fattr->valid &= ~NFS_ATTR_FATTR_ATIME;
> > +       }
> > +}
> > +
> >  void nfs_update_delegated_atime(struct inode *inode)
> >  {
> >         spin_lock(&inode->i_lock);
> > @@ -2163,6 +2203,9 @@ static int nfs_update_inode(struct inode
> > *inode, struct nfs_fattr *fattr)
> >          */
> >         nfsi->read_cache_jiffies = fattr->time_start;
> > 
> > +       /* Fix up any delegated attributes in the struct nfs_fattr
> > */
> > +       nfs_fattr_fixup_delegated(inode, fattr);
> > +
> >         save_cache_validity = nfsi->cache_validity;
> >         nfsi->cache_validity &= ~(NFS_INO_INVALID_ATTR
> >                         | NFS_INO_INVALID_ATIME
> > --
> > 2.45.2
> > 
> > 

-- 
Trond Myklebust
Linux NFS client maintainer, Hammerspace
trond.myklebust@hammerspace.com

Re: [PATCH 12/19] NFSv4: Fix up delegated attributes in nfs_setattr

[Trond Myklebust]

On Fri, 2024-06-14 at 15:59 -0400, Trond Myklebust wrote:
> On Fri, 2024-06-14 at 12:32 -0400, Anna Schumaker wrote:
> > Hi Trond,
> > 
> > On Thu, Jun 13, 2024 at 12:18 AM <trondmy@gmail.com> wrote:
> > > 
> > > From: Trond Myklebust <trond.myklebust@primarydata.com>
> > > 
> > > nfs_setattr calls nfs_update_inode() directly, so we have to
> > > reset
> > > the
> > > m/ctime there.
> > > 
> > > Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
> > > Signed-off-by: Lance Shelton <lance.shelton@hammerspace.com>
> > > Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
> > > ---
> > >  fs/nfs/inode.c | 43 +++++++++++++++++++++++++++++++++++++++++++
> > >  1 file changed, 43 insertions(+)
> > 
> > After applying this patch I see a handful of new failures on
> > xfstests:
> > generic/075, generic/086, generic/112, generic/332, generic/346,
> > generic/647, and generic/729. I see the first five on NFS v4.2, but
> > 647 and 729 both fail on v4.1 in addition to v4.2.
> 
> Thanks Anna!
> 
> Yes, I think that is a consequence of the changes that were made in
> commit cc7f2dae63bc ("NFS: Don't store NFS_INO_REVAL_FORCED"). I can
> just remove that "if (!(cache_validity & NFS_INO_REVAL_FORCED))"
> altogether with that change applied.

Sigh... Never mind. I discovered a discrepancy between what I had in
the topic branch for these features vs. what I had in our master
branch. At some point in the last few years, I must have updated the
latter without remembering to update the former.

> 
> Version 2 will be forthcoming.
> 

...and it will not affect the behaviour of the regular delegations.


> > 
> > I hope this helps!
> > Anna

Yes. I've checked both the patchsets that I sent out the other day, and
I believe this is the only patch which deviates from our master branch.

Again thanks!

> > 
> > > 
> > > diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
> > > index 91c0aeaf6c1e..e03c512c8535 100644
> > > --- a/fs/nfs/inode.c
> > > +++ b/fs/nfs/inode.c
> > > @@ -605,6 +605,46 @@ nfs_fhget(struct super_block *sb, struct
> > > nfs_fh *fh, struct nfs_fattr *fattr)
> > >  }
> > >  EXPORT_SYMBOL_GPL(nfs_fhget);
> > > 
> > > +static void
> > > +nfs_fattr_fixup_delegated(struct inode *inode, struct nfs_fattr
> > > *fattr)
> > > +{
> > > +       unsigned long cache_validity = NFS_I(inode)-
> > > > cache_validity;
> > > +
> > > +       if (!nfs_have_read_or_write_delegation(inode))
> > > +               return;
> > > +
> > > +       if (!(cache_validity & NFS_INO_REVAL_FORCED))
> > > +               cache_validity &= ~(NFS_INO_INVALID_ATIME
> > > +                               | NFS_INO_INVALID_CHANGE
> > > +                               | NFS_INO_INVALID_CTIME
> > > +                               | NFS_INO_INVALID_MTIME
> > > +                               | NFS_INO_INVALID_SIZE);
> > > +
> > > +       if (!(cache_validity & NFS_INO_INVALID_SIZE))
> > > +               fattr->valid &= ~(NFS_ATTR_FATTR_PRESIZE
> > > +                               | NFS_ATTR_FATTR_SIZE);
> > > +
> > > +       if (!(cache_validity & NFS_INO_INVALID_CHANGE))
> > > +               fattr->valid &= ~(NFS_ATTR_FATTR_PRECHANGE
> > > +                               | NFS_ATTR_FATTR_CHANGE);
> > > +
> > > +       if (nfs_have_delegated_mtime(inode)) {
> > > +               if (!(cache_validity & NFS_INO_INVALID_CTIME))
> > > +                       fattr->valid &= ~(NFS_ATTR_FATTR_PRECTIME
> > > +                                       | NFS_ATTR_FATTR_CTIME);
> > > +
> > > +               if (!(cache_validity & NFS_INO_INVALID_MTIME))
> > > +                       fattr->valid &= ~(NFS_ATTR_FATTR_PREMTIME
> > > +                                       | NFS_ATTR_FATTR_MTIME);
> > > +
> > > +               if (!(cache_validity & NFS_INO_INVALID_ATIME))
> > > +                       fattr->valid &= ~NFS_ATTR_FATTR_ATIME;
> > > +       } else if (nfs_have_delegated_atime(inode)) {
> > > +               if (!(cache_validity & NFS_INO_INVALID_ATIME))
> > > +                       fattr->valid &= ~NFS_ATTR_FATTR_ATIME;
> > > +       }
> > > +}
> > > +
> > >  void nfs_update_delegated_atime(struct inode *inode)
> > >  {
> > >         spin_lock(&inode->i_lock);
> > > @@ -2163,6 +2203,9 @@ static int nfs_update_inode(struct inode
> > > *inode, struct nfs_fattr *fattr)
> > >          */
> > >         nfsi->read_cache_jiffies = fattr->time_start;
> > > 
> > > +       /* Fix up any delegated attributes in the struct
> > > nfs_fattr
> > > */
> > > +       nfs_fattr_fixup_delegated(inode, fattr);
> > > +
> > >         save_cache_validity = nfsi->cache_validity;
> > >         nfsi->cache_validity &= ~(NFS_INO_INVALID_ATTR
> > >                         | NFS_INO_INVALID_ATIME
> > > --
> > > 2.45.2
> > > 
> > > 
> 

-- 
Trond Myklebust
Linux NFS client maintainer, Hammerspace
trond.myklebust@hammerspace.com

Re: [PATCH 07/19] NFSv4: Add support for delegated atime and mtime attributes

[Anna Schumaker]

Hi Trond,

On Thu, Jun 13, 2024 at 12:17 AM <trondmy@gmail.com> wrote:
>
> From: Trond Myklebust <trond.myklebust@primarydata.com>
>
> Ensure that we update the mtime and atime correctly when we read
> or write data to the file and when we truncate. Let the server manage
> ctime on other attribute updates.
>
> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
> Signed-off-by: Lance Shelton <lance.shelton@hammerspace.com>
> Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
> ---
>  fs/nfs/delegation.c | 24 +++++++++++++++-----
>  fs/nfs/delegation.h | 25 +++++++++++++++++++--
>  fs/nfs/inode.c      | 54 +++++++++++++++++++++++++++++++++++++++++----
>  fs/nfs/nfs4proc.c   | 21 ++++++++++--------
>  fs/nfs/read.c       |  3 +++
>  fs/nfs/write.c      |  9 ++++++++
>  6 files changed, 116 insertions(+), 20 deletions(-)
>
> diff --git a/fs/nfs/delegation.c b/fs/nfs/delegation.c
> index 6fdffd25cb2b..e72eead06c08 100644
> --- a/fs/nfs/delegation.c
> +++ b/fs/nfs/delegation.c
> @@ -115,6 +115,9 @@ static int nfs4_do_check_delegation(struct inode *inode, fmode_t type,
>                 if (mark)
>                         nfs_mark_delegation_referenced(delegation);
>                 ret = 1;
> +               if ((flags & NFS_DELEGATION_FLAG_TIME) &&
> +                   !test_bit(NFS_DELEGATION_DELEGTIME, &delegation->flags))
> +                       ret = 0;
>         }
>         rcu_read_unlock();
>         return ret;
> @@ -221,11 +224,12 @@ static int nfs_delegation_claim_opens(struct inode *inode,
>   * @type: delegation type
>   * @stateid: delegation stateid
>   * @pagemod_limit: write delegation "space_limit"
> + * @deleg_type: raw delegation type
>   *
>   */
>  void nfs_inode_reclaim_delegation(struct inode *inode, const struct cred *cred,
>                                   fmode_t type, const nfs4_stateid *stateid,
> -                                 unsigned long pagemod_limit)
> +                                 unsigned long pagemod_limit, u32 deleg_type)
>  {
>         struct nfs_delegation *delegation;
>         const struct cred *oldcred = NULL;
> @@ -250,7 +254,7 @@ void nfs_inode_reclaim_delegation(struct inode *inode, const struct cred *cred,
>         } else {
>                 rcu_read_unlock();
>                 nfs_inode_set_delegation(inode, cred, type, stateid,
> -                                        pagemod_limit);
> +                                        pagemod_limit, deleg_type);
>         }
>  }
>
> @@ -418,13 +422,13 @@ nfs_update_inplace_delegation(struct nfs_delegation *delegation,
>   * @type: delegation type
>   * @stateid: delegation stateid
>   * @pagemod_limit: write delegation "space_limit"
> + * @deleg_type: raw delegation type
>   *
>   * Returns zero on success, or a negative errno value.
>   */
>  int nfs_inode_set_delegation(struct inode *inode, const struct cred *cred,
> -                                 fmode_t type,
> -                                 const nfs4_stateid *stateid,
> -                                 unsigned long pagemod_limit)
> +                            fmode_t type, const nfs4_stateid *stateid,
> +                            unsigned long pagemod_limit, u32 deleg_type)
>  {
>         struct nfs_server *server = NFS_SERVER(inode);
>         struct nfs_client *clp = server->nfs_client;
> @@ -444,6 +448,11 @@ int nfs_inode_set_delegation(struct inode *inode, const struct cred *cred,
>         delegation->cred = get_cred(cred);
>         delegation->inode = inode;
>         delegation->flags = 1<<NFS_DELEGATION_REFERENCED;
> +       switch (deleg_type) {
> +       case NFS4_OPEN_DELEGATE_READ_ATTRS_DELEG:
> +       case NFS4_OPEN_DELEGATE_WRITE_ATTRS_DELEG:
> +               delegation->flags |= BIT(NFS_DELEGATION_DELEGTIME);
> +       }
>         delegation->test_gen = 0;
>         spin_lock_init(&delegation->lock);
>
> @@ -508,6 +517,11 @@ int nfs_inode_set_delegation(struct inode *inode, const struct cred *cred,
>         atomic_long_inc(&nfs_active_delegations);
>
>         trace_nfs4_set_delegation(inode, type);
> +
> +       /* If we hold writebacks and have delegated mtime then update */
> +       if (deleg_type == NFS4_OPEN_DELEGATE_WRITE_ATTRS_DELEG &&
> +           nfs_have_writebacks(inode))
> +               nfs_update_delegated_mtime(inode);
>  out:
>         spin_unlock(&clp->cl_lock);
>         if (delegation != NULL)
> diff --git a/fs/nfs/delegation.h b/fs/nfs/delegation.h
> index 257b3d726043..2e9ad83acf2c 100644
> --- a/fs/nfs/delegation.h
> +++ b/fs/nfs/delegation.h
> @@ -38,12 +38,17 @@ enum {
>         NFS_DELEGATION_TEST_EXPIRED,
>         NFS_DELEGATION_INODE_FREEING,
>         NFS_DELEGATION_RETURN_DELAYED,
> +       NFS_DELEGATION_DELEGTIME,
>  };
>
> +#define NFS_DELEGATION_FLAG_TIME       BIT(1)

This flag is defined under an "#if IS_ENABLED(CONFIG_NFS_V4)" guard,
so it doesn't exist for the functions farther down in this file when
v4 is disabled.

Anna

> +
>  int nfs_inode_set_delegation(struct inode *inode, const struct cred *cred,
> -               fmode_t type, const nfs4_stateid *stateid, unsigned long pagemod_limit);
> +                            fmode_t type, const nfs4_stateid *stateid,
> +                            unsigned long pagemod_limit, u32 deleg_type);
>  void nfs_inode_reclaim_delegation(struct inode *inode, const struct cred *cred,
> -               fmode_t type, const nfs4_stateid *stateid, unsigned long pagemod_limit);
> +                                 fmode_t type, const nfs4_stateid *stateid,
> +                                 unsigned long pagemod_limit, u32 deleg_type);
>  int nfs4_inode_return_delegation(struct inode *inode);
>  void nfs4_inode_return_delegation_on_close(struct inode *inode);
>  int nfs_async_inode_return_delegation(struct inode *inode, const nfs4_stateid *stateid);
> @@ -84,6 +89,10 @@ int nfs4_inode_make_writeable(struct inode *inode);
>
>  #endif
>
> +void nfs_update_delegated_atime(struct inode *inode);
> +void nfs_update_delegated_mtime(struct inode *inode);
> +void nfs_update_delegated_mtime_locked(struct inode *inode);
> +
>  static inline int nfs_have_read_or_write_delegation(struct inode *inode)
>  {
>         return NFS_PROTO(inode)->have_delegation(inode, FMODE_READ, 0);
> @@ -99,4 +108,16 @@ static inline int nfs_have_delegated_attributes(struct inode *inode)
>         return NFS_PROTO(inode)->have_delegation(inode, FMODE_READ, 0);
>  }
>
> +static inline int nfs_have_delegated_atime(struct inode *inode)
> +{
> +       return NFS_PROTO(inode)->have_delegation(inode, FMODE_READ,
> +                                                NFS_DELEGATION_FLAG_TIME);
> +}
> +
> +static inline int nfs_have_delegated_mtime(struct inode *inode)
> +{
> +       return NFS_PROTO(inode)->have_delegation(inode, FMODE_WRITE,
> +                                                NFS_DELEGATION_FLAG_TIME);
> +}
> +
>  #endif
> diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
> index 89722919b463..91c0aeaf6c1e 100644
> --- a/fs/nfs/inode.c
> +++ b/fs/nfs/inode.c
> @@ -275,6 +275,8 @@ EXPORT_SYMBOL_GPL(nfs_zap_acl_cache);
>
>  void nfs_invalidate_atime(struct inode *inode)
>  {
> +       if (nfs_have_delegated_atime(inode))
> +               return;
>         spin_lock(&inode->i_lock);
>         nfs_set_cache_invalid(inode, NFS_INO_INVALID_ATIME);
>         spin_unlock(&inode->i_lock);
> @@ -603,6 +605,33 @@ nfs_fhget(struct super_block *sb, struct nfs_fh *fh, struct nfs_fattr *fattr)
>  }
>  EXPORT_SYMBOL_GPL(nfs_fhget);
>
> +void nfs_update_delegated_atime(struct inode *inode)
> +{
> +       spin_lock(&inode->i_lock);
> +       if (nfs_have_delegated_atime(inode)) {
> +               inode_update_timestamps(inode, S_ATIME);
> +               NFS_I(inode)->cache_validity &= ~NFS_INO_INVALID_ATIME;
> +       }
> +       spin_unlock(&inode->i_lock);
> +}
> +
> +void nfs_update_delegated_mtime_locked(struct inode *inode)
> +{
> +       if (nfs_have_delegated_mtime(inode)) {
> +               inode_update_timestamps(inode, S_CTIME | S_MTIME);
> +               NFS_I(inode)->cache_validity &= ~(NFS_INO_INVALID_CTIME |
> +                                                 NFS_INO_INVALID_MTIME);
> +       }
> +}
> +
> +void nfs_update_delegated_mtime(struct inode *inode)
> +{
> +       spin_lock(&inode->i_lock);
> +       nfs_update_delegated_mtime_locked(inode);
> +       spin_unlock(&inode->i_lock);
> +}
> +EXPORT_SYMBOL_GPL(nfs_update_delegated_mtime);
> +
>  #define NFS_VALID_ATTRS (ATTR_MODE|ATTR_UID|ATTR_GID|ATTR_SIZE|ATTR_ATIME|ATTR_ATIME_SET|ATTR_MTIME|ATTR_MTIME_SET|ATTR_FILE|ATTR_OPEN)
>
>  int
> @@ -630,6 +659,17 @@ nfs_setattr(struct mnt_idmap *idmap, struct dentry *dentry,
>                         attr->ia_valid &= ~ATTR_SIZE;
>         }
>
> +       if (nfs_have_delegated_mtime(inode)) {
> +               if (attr->ia_valid & ATTR_MTIME) {
> +                       nfs_update_delegated_mtime(inode);
> +                       attr->ia_valid &= ~ATTR_MTIME;
> +               }
> +               if (attr->ia_valid & ATTR_ATIME) {
> +                       nfs_update_delegated_atime(inode);
> +                       attr->ia_valid &= ~ATTR_ATIME;
> +               }
> +       }
> +
>         /* Optimization: if the end result is no change, don't RPC */
>         if (((attr->ia_valid & NFS_VALID_ATTRS) & ~(ATTR_FILE|ATTR_OPEN)) == 0)
>                 return 0;
> @@ -685,6 +725,7 @@ static int nfs_vmtruncate(struct inode * inode, loff_t offset)
>
>         spin_unlock(&inode->i_lock);
>         truncate_pagecache(inode, offset);
> +       nfs_update_delegated_mtime_locked(inode);
>         spin_lock(&inode->i_lock);
>  out:
>         return err;
> @@ -708,8 +749,9 @@ void nfs_setattr_update_inode(struct inode *inode, struct iattr *attr,
>         spin_lock(&inode->i_lock);
>         NFS_I(inode)->attr_gencount = fattr->gencount;
>         if ((attr->ia_valid & ATTR_SIZE) != 0) {
> -               nfs_set_cache_invalid(inode, NFS_INO_INVALID_MTIME |
> -                                                    NFS_INO_INVALID_BLOCKS);
> +               if (!nfs_have_delegated_mtime(inode))
> +                       nfs_set_cache_invalid(inode, NFS_INO_INVALID_MTIME);
> +               nfs_set_cache_invalid(inode, NFS_INO_INVALID_BLOCKS);
>                 nfs_inc_stats(inode, NFSIOS_SETATTRTRUNC);
>                 nfs_vmtruncate(inode, attr->ia_size);
>         }
> @@ -855,8 +897,12 @@ int nfs_getattr(struct mnt_idmap *idmap, const struct path *path,
>
>         /* Flush out writes to the server in order to update c/mtime/version.  */
>         if ((request_mask & (STATX_CTIME | STATX_MTIME | STATX_CHANGE_COOKIE)) &&
> -           S_ISREG(inode->i_mode))
> -               filemap_write_and_wait(inode->i_mapping);
> +           S_ISREG(inode->i_mode)) {
> +               if (nfs_have_delegated_mtime(inode))
> +                       filemap_fdatawrite(inode->i_mapping);
> +               else
> +                       filemap_write_and_wait(inode->i_mapping);
> +       }
>
>         /*
>          * We may force a getattr if the user cares about atime.
> diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
> index 27fb40653f1d..83edbc7a3bcc 100644
> --- a/fs/nfs/nfs4proc.c
> +++ b/fs/nfs/nfs4proc.c
> @@ -1245,7 +1245,8 @@ nfs4_update_changeattr_locked(struct inode *inode,
>         struct nfs_inode *nfsi = NFS_I(inode);
>         u64 change_attr = inode_peek_iversion_raw(inode);
>
> -       cache_validity |= NFS_INO_INVALID_CTIME | NFS_INO_INVALID_MTIME;
> +       if (!nfs_have_delegated_mtime(inode))
> +               cache_validity |= NFS_INO_INVALID_CTIME | NFS_INO_INVALID_MTIME;
>         if (S_ISDIR(inode->i_mode))
>                 cache_validity |= NFS_INO_INVALID_DATA;
>
> @@ -1961,6 +1962,8 @@ nfs4_process_delegation(struct inode *inode, const struct cred *cred,
>         switch (delegation->open_delegation_type) {
>         case NFS4_OPEN_DELEGATE_READ:
>         case NFS4_OPEN_DELEGATE_WRITE:
> +       case NFS4_OPEN_DELEGATE_READ_ATTRS_DELEG:
> +       case NFS4_OPEN_DELEGATE_WRITE_ATTRS_DELEG:
>                 break;
>         default:
>                 return;
> @@ -1974,16 +1977,16 @@ nfs4_process_delegation(struct inode *inode, const struct cred *cred,
>                                    NFS_SERVER(inode)->nfs_client->cl_hostname);
>                 break;
>         case NFS4_OPEN_CLAIM_PREVIOUS:
> -               nfs_inode_reclaim_delegation(inode, cred,
> -                               delegation->type,
> -                               &delegation->stateid,
> -                               delegation->pagemod_limit);
> +               nfs_inode_reclaim_delegation(inode, cred, delegation->type,
> +                                            &delegation->stateid,
> +                                            delegation->pagemod_limit,
> +                                            delegation->open_delegation_type);
>                 break;
>         default:
> -               nfs_inode_set_delegation(inode, cred,
> -                               delegation->type,
> -                               &delegation->stateid,
> -                               delegation->pagemod_limit);
> +               nfs_inode_set_delegation(inode, cred, delegation->type,
> +                                        &delegation->stateid,
> +                                        delegation->pagemod_limit,
> +                                        delegation->open_delegation_type);
>         }
>         if (delegation->do_recall)
>                 nfs_async_inode_return_delegation(inode, &delegation->stateid);
> diff --git a/fs/nfs/read.c b/fs/nfs/read.c
> index a142287d86f6..1b0e06c11983 100644
> --- a/fs/nfs/read.c
> +++ b/fs/nfs/read.c
> @@ -28,6 +28,7 @@
>  #include "fscache.h"
>  #include "pnfs.h"
>  #include "nfstrace.h"
> +#include "delegation.h"
>
>  #define NFSDBG_FACILITY                NFSDBG_PAGECACHE
>
> @@ -372,6 +373,7 @@ int nfs_read_folio(struct file *file, struct folio *folio)
>                 goto out_put;
>
>         nfs_pageio_complete_read(&pgio);
> +       nfs_update_delegated_atime(inode);
>         ret = pgio.pg_error < 0 ? pgio.pg_error : 0;
>         if (!ret) {
>                 ret = folio_wait_locked_killable(folio);
> @@ -428,6 +430,7 @@ void nfs_readahead(struct readahead_control *ractl)
>         }
>
>         nfs_pageio_complete_read(&pgio);
> +       nfs_update_delegated_atime(inode);
>
>         put_nfs_open_context(ctx);
>  out:
> diff --git a/fs/nfs/write.c b/fs/nfs/write.c
> index a9186b113fe7..c0cd644b97ff 100644
> --- a/fs/nfs/write.c
> +++ b/fs/nfs/write.c
> @@ -289,6 +289,8 @@ static void nfs_grow_file(struct folio *folio, unsigned int offset,
>         NFS_I(inode)->cache_validity &= ~NFS_INO_INVALID_SIZE;
>         nfs_inc_stats(inode, NFSIOS_EXTENDWRITE);
>  out:
> +       /* Atomically update timestamps if they are delegated to us. */
> +       nfs_update_delegated_mtime_locked(inode);
>         spin_unlock(&inode->i_lock);
>         nfs_fscache_invalidate(inode, 0);
>  }
> @@ -1514,6 +1516,13 @@ void nfs_writeback_update_inode(struct nfs_pgio_header *hdr)
>         struct nfs_fattr *fattr = &hdr->fattr;
>         struct inode *inode = hdr->inode;
>
> +       if (nfs_have_delegated_mtime(inode)) {
> +               spin_lock(&inode->i_lock);
> +               nfs_set_cache_invalid(inode, NFS_INO_INVALID_BLOCKS);
> +               spin_unlock(&inode->i_lock);
> +               return;
> +       }
> +
>         spin_lock(&inode->i_lock);
>         nfs_writeback_check_extend(hdr, fattr);
>         nfs_post_op_update_inode_force_wcc_locked(inode, fattr);
> --
> 2.45.2
>
>

Re: [PATCH 00/19] OPEN optimisations and Attribute delegations

[Jeff Layton]

On Thu, 2024-06-13 at 00:11 -0400, trondmy@gmail.com wrote:
> From: Trond Myklebust <trond.myklebust@hammerspace.com>
> 
> Now that https://datatracker.ietf.org/doc/draft-ietf-nfsv4-delstid/ is
> mostly done with the review process, it is time to look at pushing the
> client implementation that we've been working on upstream.
> 
> The following patch series therefore adds support for the NFSv4.2
> extension to OP_OPEN to allow the client to request that the server
> return either an open stateid or a delegation instead of always sending
> the open stateid whether or not a delegation is returned.
> This allows us to optimise away CLOSE, and hence makes small or cached
> file access significantly more efficient.
> 
> It also adds support for attribute delegations, which allow the client
> to manage the atime and mtime, and simply inform the server at file
> close time what the values should be. This means that most GETATTR
> operations to retrieve the atime/mtime values while the file is under
> I/O can be optimised away.
> 
> Finally, we also add support for the detection mechanism that allows the
> client to determine whether or not the server supports the above
> functionality.
> 
> Lance Shelton (1):
>   NFS: Add a generic callback to return the delegation
> 
> Trond Myklebust (18):
>   NFSv4: Clean up open delegation return structure
>   NFSv4: Refactor nfs4_opendata_check_deleg()
>   NFSv4: Add new attribute delegation definitions
>   NFSv4: Plumb in XDR support for the new delegation-only setattr op
>   NFSv4: Add CB_GETATTR support for delegated attributes
>   NFSv4: Add a flags argument to the 'have_delegation' callback
>   NFSv4: Add support for delegated atime and mtime attributes
>   NFSv4: Add recovery of attribute delegations
>   NFSv4: Add a capability for delegated attributes
>   NFSv4: Enable attribute delegations
>   NFSv4: Delegreturn must set m/atime when they are delegated
>   NFSv4: Fix up delegated attributes in nfs_setattr
>   NFSv4: Don't request atime/mtime/size if they are delegated to us
>   NFSv4: Add support for the FATTR4_OPEN_ARGUMENTS attribute
>   NFSv4: Detect support for OPEN4_SHARE_ACCESS_WANT_OPEN_XOR_DELEGATION
>   NFSv4: Add support for OPEN4_RESULT_NO_OPEN_STATEID
>   NFSv4: Ask for a delegation or an open stateid in OPEN
>   Return the delegation when deleting the sillyrenamed file
> 
>  fs/nfs/callback.h         |   5 +-
>  fs/nfs/callback_proc.c    |  14 ++-
>  fs/nfs/callback_xdr.c     |  39 ++++++-
>  fs/nfs/delegation.c       |  59 ++++++----
>  fs/nfs/delegation.h       |  45 +++++++-
>  fs/nfs/dir.c              |   2 +-
>  fs/nfs/file.c             |   4 +-
>  fs/nfs/inode.c            | 104 +++++++++++++++--
>  fs/nfs/nfs3proc.c         |  10 +-
>  fs/nfs/nfs4proc.c         | 230 ++++++++++++++++++++++++++++----------
>  fs/nfs/nfs4xdr.c          | 131 +++++++++++++++++-----
>  fs/nfs/proc.c             |  10 +-
>  fs/nfs/read.c             |   3 +
>  fs/nfs/unlink.c           |   2 +
>  fs/nfs/write.c            |  11 +-
>  include/linux/nfs4.h      |  11 ++
>  include/linux/nfs_fs_sb.h |   2 +
>  include/linux/nfs_xdr.h   |  45 +++++++-
>  include/uapi/linux/nfs4.h |   4 +
>  19 files changed, 586 insertions(+), 145 deletions(-)
> 

This all looks pretty reasonable except for the last two patches.
Probably, they should be squashed together since there is no caller of
->return_delegation until the last one. There is also nothing
describing the changes there, and I think it could use some explanation
(though I think I get what you're doing).

Finally, I suppose we need to look at implementing support delstid in
knfsd as well. I'll open a new feature request for that the linux-nfs
project on github.

In any case, you can add:

Reviewed-by: Jeff Layton <jlayton@kernel.org>

Re: [PATCH 00/19] OPEN optimisations and Attribute delegations

[Christoph Hellwig]

On Fri, Jun 14, 2024 at 08:34:32AM -0400, Jeff Layton wrote:
> This all looks pretty reasonable except for the last two patches.
> Probably, they should be squashed together since there is no caller of
> ->return_delegation until the last one. There is also nothing
> describing the changes there, and I think it could use some explanation
> (though I think I get what you're doing).
> 
> Finally, I suppose we need to look at implementing support delstid in
> knfsd as well. I'll open a new feature request for that the linux-nfs
> project on github.

I don't think there ever was a formal rule, but having a feature like
this that affects all of the core nfs code without beeing able to test
it against knfsd seems a bit dangerous indeed.

Re: [PATCH 00/19] OPEN optimisations and Attribute delegations

[Trond Myklebust]

On Fri, 2024-06-14 at 23:27 -0700, Christoph Hellwig wrote:
> On Fri, Jun 14, 2024 at 08:34:32AM -0400, Jeff Layton wrote:
> > This all looks pretty reasonable except for the last two patches.
> > Probably, they should be squashed together since there is no caller
> > of
> > ->return_delegation until the last one. There is also nothing
> > describing the changes there, and I think it could use some
> > explanation
> > (though I think I get what you're doing).
> > 
> > Finally, I suppose we need to look at implementing support delstid
> > in
> > knfsd as well. I'll open a new feature request for that the linux-
> > nfs
> > project on github.
> 
> I don't think there ever was a formal rule, but having a feature like
> this that affects all of the core nfs code without beeing able to
> test
> it against knfsd seems a bit dangerous indeed.
> 

We have been going through the IETF process to ensure there is an
agreed upon protocol for this that can be used to test the client code
and validate its correctness. That spec can also be used by others to
contribute the missing knfsd code.
At some point soon, I do hope that Hammerspace will be in the position
where we're able to contribute both client and server code for these
features; for now, we are unfortunately still resource constrained.

-- 
Trond Myklebust
Linux NFS client maintainer, Hammerspace
trond.myklebust@hammerspace.com

Re: [PATCH 00/19] OPEN optimisations and Attribute delegations

[hch]

On Mon, Jun 17, 2024 at 01:39:06AM +0000, Trond Myklebust wrote:
> We have been going through the IETF process to ensure there is an
> agreed upon protocol for this that can be used to test the client code
> and validate its correctness. That spec can also be used by others to
> contribute the missing knfsd code.

I still think it is a bad idea to merge Linux code we can't easily
test on pure Linux setups.

(saying that as someone touching code only exercised by the flexfiles
layout right now, which is a rather annoying)

Re: [PATCH 00/19] OPEN optimisations and Attribute delegations

[Christoph Hellwig]

Btw, your patch mail threading is quite broken, it replies to the
previous patch instead of the cover letter, leading to really long
fake reply chains..

Re: [PATCH 00/19] OPEN optimisations and Attribute delegations

[Trond Myklebust]

On Fri, 2024-06-14 at 23:25 -0700, Christoph Hellwig wrote:
> Btw, your patch mail threading is quite broken, it replies to the
> previous patch instead of the cover letter, leading to really long
> fake reply chains..
> 

I'm just using the 

[sendemail]
	        thread = true

git config option. Is that no longer recommended?

-- 
Trond Myklebust
Linux NFS client maintainer, Hammerspace
trond.myklebust@hammerspace.com

Re: [PATCH 00/19] OPEN optimisations and Attribute delegations

[hch]

On Mon, Jun 17, 2024 at 01:28:26AM +0000, Trond Myklebust wrote:
> On Fri, 2024-06-14 at 23:25 -0700, Christoph Hellwig wrote:
> > Btw, your patch mail threading is quite broken, it replies to the
> > previous patch instead of the cover letter, leading to really long
> > fake reply chains..
> > 
> 
> I'm just using the 
> 
> [sendemail]
> 	        thread = true
> 
> git config option. Is that no longer recommended?

According to the man page thread = true is the same as thread = shallow,
which make git always reply to the cover letter, which is the
recommended form.  Try removing it or replace it with 'thread =
shallow'