[PATCH Version 5 00/10] NFSV4.1,2 session trunking

[PATCH Version 5 00/10] NFSV4.1,2 session trunking

[andros]

From: Andy Adamson <andros@netapp.com>

Version-5
---------

Responded to Anna's code cleanup comments.

--> Andy

Version-4
---------

This patch set enables session trunking configured two ways.

The NFSv4 minorversion > 0 pseudo-fs is probed
with a GETATTR(fs_locations) to retrieve a replicas list.
Each replica address is tested for session trunking. Those
that pass are added as aliases to the mount rpc client.

Try all multipath addresses for a data server. The first address that
successfully connects and creates a session is the mount address.
All subsequent addresses are tested for session trunking and
added as aliases.

TODO:

- Periodically probe the pseudo-fs with the GETATTR(fs_locations)
to pick up server multipath changes. This is why the
GETATTR(fs_locations) is a stand alone call.

- Call BIND_CONN_TO_SESSION for session trunking addresses
established under SP4_MACH_CRED or SP4_SSV. SP4_NONE established
addresses do not require BIND_CONN_TO_SESSION.

TESTING:

The pseudo-fs GETATTR(fs_locations) probe session trunking
was tested against a Linux server with a pseudo-fs
export stanza (e.g. a stanza with the fsid=0 or fsid=root
export option) and a replicas= export option
(replicas=<path1>@<server1>:<path2>@<server2>..)
Note that this configuration is for testing only. A future
patchset will add the replicas= configuration to the
NFSEXP_V4ROOT nfsd and mountd processing.

The pNFS DS multipath session trunking was tested against a
pynfs server which provides DS multipath configuration.


Andy Adamson (10):
  NFS rename default callback ops
  NFS refactor nfs4_match_clientids
  NFS refactor nfs4_check_serverowner_major_id
  NFS detect session trunking
  NFS refactor _nfs4_proc_exchange_id for session trunking
  SUNRPC add flag to rpc_task_release_client
  NFS probe pseudo-fs for replicas
  NFS test and add multipaths for session trunking
  NFS test pnfs data server multipath session trunking aliases
  NFS add multiaddr to nfs_show_nfsv4_options

 fs/nfs/internal.h           |   2 +
 fs/nfs/nfs4_fs.h            |   9 ++
 fs/nfs/nfs4client.c         | 102 +++++++++++++++++---
 fs/nfs/nfs4getroot.c        |   3 +
 fs/nfs/nfs4proc.c           | 222 ++++++++++++++++++++++++++++++++++++++++++--
 fs/nfs/nfs4xdr.c            |  38 +++-----
 fs/nfs/pnfs_nfs.c           |  51 +++++++---
 fs/nfs/super.c              |  26 ++++++
 include/linux/nfs_xdr.h     |   4 +-
 include/linux/sunrpc/clnt.h |   2 +-
 net/sunrpc/clnt.c           |   6 +-
 net/sunrpc/sched.c          |   2 +-
 net/sunrpc/xprtmultipath.c  |   4 +
 13 files changed, 406 insertions(+), 65 deletions(-)

-- 
1.8.3.1

[PATCH Version 5 01/10] NFS rename default callback ops

[andros]

From: Andy Adamson <andros@netapp.com>

Also to be used with refactored exchange_id for session trunking

Signed-off-by: Andy Adamson <andros@netapp.com>
---
 fs/nfs/nfs4proc.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 327b8c3..da2aa2e 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -6784,12 +6784,12 @@ nfs41_same_server_scope(struct nfs41_server_scope *a,
 }
 
 static void
-nfs4_bind_one_conn_to_session_done(struct rpc_task *task, void *calldata)
+nfs4_proc_default_done(struct rpc_task *task, void *calldata)
 {
 }
 
-static const struct rpc_call_ops nfs4_bind_one_conn_to_session_ops = {
-	.rpc_call_done =  &nfs4_bind_one_conn_to_session_done,
+static const struct rpc_call_ops nfs4_proc_default_ops = {
+	.rpc_call_done =  &nfs4_proc_default_done,
 };
 
 /*
@@ -6820,7 +6820,7 @@ int nfs4_proc_bind_one_conn_to_session(struct rpc_clnt *clnt,
 	struct rpc_task_setup task_setup_data = {
 		.rpc_client = clnt,
 		.rpc_xprt = xprt,
-		.callback_ops = &nfs4_bind_one_conn_to_session_ops,
+		.callback_ops = &nfs4_proc_default_ops,
 		.rpc_message = &msg,
 		.flags = RPC_TASK_TIMEOUT,
 	};
-- 
1.8.3.1

[PATCH Version 5 02/10] NFS refactor nfs4_match_clientids

[andros]

From: Andy Adamson <andros@netapp.com>

For session trunking, to compare nfs41_exchange_id_res with
exiting nfs_client.

Signed-off-by: Andy Adamson <andros@netapp.com>
---
 fs/nfs/nfs4client.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/fs/nfs/nfs4client.c b/fs/nfs/nfs4client.c
index 10410e8..c9177a4 100644
--- a/fs/nfs/nfs4client.c
+++ b/fs/nfs/nfs4client.c
@@ -562,15 +562,15 @@ out:
 /*
  * Returns true if the client IDs match
  */
-static bool nfs4_match_clientids(struct nfs_client *a, struct nfs_client *b)
+static bool nfs4_match_clientids(u64 a, u64 b)
 {
-	if (a->cl_clientid != b->cl_clientid) {
+	if (a != b) {
 		dprintk("NFS: --> %s client ID %llx does not match %llx\n",
-			__func__, a->cl_clientid, b->cl_clientid);
+			__func__, a, b);
 		return false;
 	}
 	dprintk("NFS: --> %s client ID %llx matches %llx\n",
-		__func__, a->cl_clientid, b->cl_clientid);
+		__func__, a, b);
 	return true;
 }
 
@@ -650,7 +650,7 @@ int nfs41_walk_client_list(struct nfs_client *new,
 		if (pos->cl_cons_state != NFS_CS_READY)
 			continue;
 
-		if (!nfs4_match_clientids(pos, new))
+		if (!nfs4_match_clientids(pos->cl_clientid, new->cl_clientid))
 			continue;
 
 		/*
-- 
1.8.3.1

[PATCH Version 5 03/10] NFS refactor nfs4_check_serverowner_major_id

[andros]

From: Andy Adamson <andros@netapp.com>

For session trunking, to compare nfs41_exchange_id_res with
existing nfs_client

Signed-off-by: Andy Adamson <andros@netapp.com>
---
 fs/nfs/nfs4client.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/fs/nfs/nfs4client.c b/fs/nfs/nfs4client.c
index c9177a4..98342fc 100644
--- a/fs/nfs/nfs4client.c
+++ b/fs/nfs/nfs4client.c
@@ -578,17 +578,15 @@ static bool nfs4_match_clientids(u64 a, u64 b)
  * Returns true if the server major ids match
  */
 static bool
-nfs4_check_clientid_trunking(struct nfs_client *a, struct nfs_client *b)
+nfs4_check_serverowner_major_id(struct nfs41_server_owner *o1,
+				struct nfs41_server_owner *o2)
 {
-	struct nfs41_server_owner *o1 = a->cl_serverowner;
-	struct nfs41_server_owner *o2 = b->cl_serverowner;
-
 	if (o1->major_id_sz != o2->major_id_sz)
 		goto out_major_mismatch;
 	if (memcmp(o1->major_id, o2->major_id, o1->major_id_sz) != 0)
 		goto out_major_mismatch;
 
-	dprintk("NFS: --> %s server owners match\n", __func__);
+	dprintk("NFS: --> %s server owner major IDs match\n", __func__);
 	return true;
 
 out_major_mismatch:
@@ -658,7 +656,8 @@ int nfs41_walk_client_list(struct nfs_client *new,
 		 * client id trunking. In either case, we want to fall back
 		 * to using the existing nfs_client.
 		 */
-		if (!nfs4_check_clientid_trunking(pos, new))
+		if (!nfs4_check_serverowner_major_id(pos->cl_serverowner,
+						     new->cl_serverowner))
 			continue;
 
 		/* Unlike NFSv4.0, we know that NFSv4.1 always uses the
-- 
1.8.3.1

[PATCH Version 5 04/10] NFS detect session trunking

[andros]

From: Andy Adamson <andros@netapp.com>

Signed-off-by: Andy Adamson <andros@netapp.com>
---
 fs/nfs/nfs4client.c | 81 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 81 insertions(+)

diff --git a/fs/nfs/nfs4client.c b/fs/nfs/nfs4client.c
index 98342fc..2121c1f 100644
--- a/fs/nfs/nfs4client.c
+++ b/fs/nfs/nfs4client.c
@@ -595,6 +595,87 @@ out_major_mismatch:
 	return false;
 }
 
+/*
+ * Returns true if server minor ids match
+ */
+static bool
+nfs4_check_serverowner_minor_id(struct nfs41_server_owner *o1,
+				struct nfs41_server_owner *o2)
+{
+	/* Check eir_server_owner so_minor_id */
+	if (o1->minor_id != o2->minor_id)
+		goto out_minor_mismatch;
+
+	dprintk("NFS: --> %s server owner minor IDs s match\n", __func__);
+	return true;
+
+out_minor_mismatch:
+	dprintk("NFS: --> %s server owner minor IDs do not match\n", __func__);
+	return false;
+}
+
+/*
+ * Returns true if the server scopes match
+ */
+static bool
+nfs4_check_server_scope(struct nfs41_server_scope *s1,
+			struct nfs41_server_scope *s2)
+{
+	if (s1->server_scope_sz != s2->server_scope_sz)
+		goto out_scope_mismatch;
+	if (memcmp(s1->server_scope, s2->server_scope,
+		   s1->server_scope_sz) != 0)
+		goto out_scope_mismatch;
+
+	dprintk("NFS: --> %s server scopes match\n", __func__);
+	return true;
+
+out_scope_mismatch:
+	dprintk("NFS: --> %s server scopes do not match\n",
+		__func__);
+	return false;
+}
+
+/**
+ * nfs4_detect_session_trunking - Checks for ssession trunking called
+ * after a successuful EXCHANGE_ID testing a multi-addr connection to be
+ * potentially added as a session trunk
+ *
+ * @clp:    original mount nfs_client
+ * @res:    result structure from an exchange_id using the original mount
+ *          nfs_client with a new multi_addr transport
+ *
+ * Returns zero on success, otherwise -EINVAL
+ *
+ * Note: since the exchange_id for the new multi_addr transport uses the
+ * same nfs_client from the original mount, the cl_owner_id is reused,
+ * so eir_clientowner is the same.
+ */
+int nfs4_detect_session_trunking(struct nfs_client *clp,
+				 struct nfs41_exchange_id_res *res)
+{
+
+	/* Check eir_clientid */
+	if (!nfs4_match_clientids(clp->cl_clientid, res->clientid))
+		return -EINVAL;
+
+	/* Check eir_server_owner so_major_id */
+	if (!nfs4_check_serverowner_major_id(clp->cl_serverowner,
+					     res->server_owner))
+		return -EINVAL;
+
+	/* Check eir_server_owner so_minor_id */
+	if (!nfs4_check_serverowner_minor_id(clp->cl_serverowner,
+					     res->server_owner))
+		return -EINVAL;
+
+	/* Check eir_server_scope */
+	if (!nfs4_check_server_scope(clp->cl_serverscope, res->server_scope))
+		return -EINVAL;
+
+	return 0;
+}
+
 /**
  * nfs41_walk_client_list - Find nfs_client that matches a client/server owner
  *
-- 
1.8.3.1

[PATCH Version 5 05/10] NFS refactor _nfs4_proc_exchange_id for session trunking

[andros]

From: Andy Adamson <andros@netapp.com>

Add an rpc_xprt parameter to test it's connection for session trunking

Signed-off-by: Andy Adamson <andros@netapp.com>
---
 fs/nfs/nfs4_fs.h  |  2 ++
 fs/nfs/nfs4proc.c | 34 ++++++++++++++++++++++++++++------
 2 files changed, 30 insertions(+), 6 deletions(-)

diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index 4afdee4..ff8cfcd 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
@@ -276,6 +276,8 @@ extern int nfs4_proc_get_lease_time(struct nfs_client *clp,
 		struct nfs_fsinfo *fsinfo);
 extern int nfs4_proc_layoutcommit(struct nfs4_layoutcommit_data *data,
 				  bool sync);
+extern int nfs4_detect_session_trunking(struct nfs_client *clp,
+		struct nfs41_exchange_id_res *res);
 
 static inline bool
 is_ds_only_client(struct nfs_client *clp)
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index da2aa2e..70537f0 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -7024,7 +7024,7 @@ static int nfs4_sp4_select_mode(struct nfs_client *clp,
  * Wrapper for EXCHANGE_ID operation.
  */
 static int _nfs4_proc_exchange_id(struct nfs_client *clp, struct rpc_cred *cred,
-	u32 sp4_how)
+	u32 sp4_how, struct rpc_xprt *xprt)
 {
 	nfs4_verifier verifier;
 	struct nfs41_exchange_id_args args = {
@@ -7049,6 +7049,18 @@ static int _nfs4_proc_exchange_id(struct nfs_client *clp, struct rpc_cred *cred,
 		.rpc_resp = &res,
 		.rpc_cred = cred,
 	};
+	struct rpc_task_setup task_setup_data = {
+		.rpc_client = clp->cl_rpcclient,
+		.rpc_xprt = xprt,
+		.callback_ops = &nfs4_proc_default_ops,
+		.rpc_message = &msg,
+		.flags = RPC_TASK_TIMEOUT,
+	};
+	struct rpc_task *task;
+
+	/* Do not run exchange_id against the established mount connection */
+	if (xprt && xprt == rcu_access_pointer(clp->cl_rpcclient->cl_xprt))
+		return 1;
 
 	nfs4_init_boot_verifier(clp, &verifier);
 
@@ -7096,11 +7108,17 @@ static int _nfs4_proc_exchange_id(struct nfs_client *clp, struct rpc_cred *cred,
 		goto out_impl_id;
 	}
 
-	status = rpc_call_sync(clp->cl_rpcclient, &msg, RPC_TASK_TIMEOUT);
+	task = rpc_run_task(&task_setup_data);
+	if (!IS_ERR(task)) {
+		status = task->tk_status;
+		rpc_put_task(task);
+	} else
+		status = PTR_ERR(task);
 	trace_nfs4_exchange_id(clp, status);
 	if (status == 0)
 		status = nfs4_check_cl_exchange_flags(res.flags);
-
+	if (xprt)
+		goto session_trunk;
 	if (status == 0)
 		status = nfs4_sp4_select_mode(clp, &res.state_protect);
 
@@ -7138,7 +7156,6 @@ static int _nfs4_proc_exchange_id(struct nfs_client *clp, struct rpc_cred *cred,
 			res.server_scope = NULL;
 		}
 	}
-
 out_impl_id:
 	kfree(res.impl_id);
 out_server_scope:
@@ -7154,6 +7171,11 @@ out:
 			clp->cl_implid->date.nseconds);
 	dprintk("NFS reply exchange_id: %d\n", status);
 	return status;
+
+session_trunk:
+	if (status == 0)
+		status = nfs4_detect_session_trunking(clp, &res);
+	goto out_impl_id;
 }
 
 /*
@@ -7176,13 +7198,13 @@ int nfs4_proc_exchange_id(struct nfs_client *clp, struct rpc_cred *cred)
 	/* try SP4_MACH_CRED if krb5i/p	*/
 	if (authflavor == RPC_AUTH_GSS_KRB5I ||
 	    authflavor == RPC_AUTH_GSS_KRB5P) {
-		status = _nfs4_proc_exchange_id(clp, cred, SP4_MACH_CRED);
+		status = _nfs4_proc_exchange_id(clp, cred, SP4_MACH_CRED, NULL);
 		if (!status)
 			return 0;
 	}
 
 	/* try SP4_NONE */
-	return _nfs4_proc_exchange_id(clp, cred, SP4_NONE);
+	return _nfs4_proc_exchange_id(clp, cred, SP4_NONE, NULL);
 }
 
 static int _nfs4_proc_destroy_clientid(struct nfs_client *clp,
-- 
1.8.3.1

[PATCH Version 5 06/10] SUNRPC add flag to rpc_task_release_client

[andros]

From: Andy Adamson <andros@netapp.com>

Want to specify which rpc_xprt to use in rpc_run_task.

Don't pass in an rpc_xprt in rpc_init_task just to have it not used as it
is removed in rpc_task_release_client.

Signed-off-by: Andy Adamson <andros@netapp.com>
---
 include/linux/sunrpc/clnt.h | 2 +-
 net/sunrpc/clnt.c           | 6 +++---
 net/sunrpc/sched.c          | 2 +-
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/include/linux/sunrpc/clnt.h b/include/linux/sunrpc/clnt.h
index 9a7ddba..b2c5b75 100644
--- a/include/linux/sunrpc/clnt.h
+++ b/include/linux/sunrpc/clnt.h
@@ -150,7 +150,7 @@ int		rpc_switch_client_transport(struct rpc_clnt *,
 
 void		rpc_shutdown_client(struct rpc_clnt *);
 void		rpc_release_client(struct rpc_clnt *);
-void		rpc_task_release_client(struct rpc_task *);
+void		rpc_task_release_client(struct rpc_task *, int);
 
 int		rpcb_create_local(struct net *);
 void		rpcb_put_local(struct net *);
diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c
index 7e0c9bf..e157d09 100644
--- a/net/sunrpc/clnt.c
+++ b/net/sunrpc/clnt.c
@@ -942,7 +942,7 @@ out:
 }
 EXPORT_SYMBOL_GPL(rpc_bind_new_program);
 
-void rpc_task_release_client(struct rpc_task *task)
+void rpc_task_release_client(struct rpc_task *task, int rm_xprt)
 {
 	struct rpc_clnt *clnt = task->tk_client;
 	struct rpc_xprt *xprt = task->tk_xprt;
@@ -957,7 +957,7 @@ void rpc_task_release_client(struct rpc_task *task)
 		rpc_release_client(clnt);
 	}
 
-	if (xprt != NULL) {
+	if (rm_xprt && xprt) {
 		task->tk_xprt = NULL;
 
 		xprt_put(xprt);
@@ -969,7 +969,7 @@ void rpc_task_set_client(struct rpc_task *task, struct rpc_clnt *clnt)
 {
 
 	if (clnt != NULL) {
-		rpc_task_release_client(task);
+		rpc_task_release_client(task, 0);
 		if (task->tk_xprt == NULL)
 			task->tk_xprt = xprt_iter_get_next(&clnt->cl_xpi);
 		task->tk_client = clnt;
diff --git a/net/sunrpc/sched.c b/net/sunrpc/sched.c
index fcfd48d..0dacd9c 100644
--- a/net/sunrpc/sched.c
+++ b/net/sunrpc/sched.c
@@ -993,7 +993,7 @@ static void rpc_release_resources_task(struct rpc_task *task)
 		put_rpccred(task->tk_msg.rpc_cred);
 		task->tk_msg.rpc_cred = NULL;
 	}
-	rpc_task_release_client(task);
+	rpc_task_release_client(task, 1);
 }
 
 static void rpc_final_put_task(struct rpc_task *task,
-- 
1.8.3.1

[PATCH Version 5 07/10] NFS probe pseudo-fs for replicas

[andros]

From: Andy Adamson <andros@netapp.com>

Probe an NFSv4 Pseudo-fs for fs_locations replicas. Replicas on a Pseudo-fs
returns multipath addresses for the server which will be tested
for session trunking.

Signed-off-by: Andy Adamson <andros@netapp.com>
---
 fs/nfs/internal.h       |  2 ++
 fs/nfs/nfs4getroot.c    |  3 ++
 fs/nfs/nfs4proc.c       | 82 +++++++++++++++++++++++++++++++++++++++++++++++++
 fs/nfs/nfs4xdr.c        | 38 +++++++++--------------
 include/linux/nfs_xdr.h |  4 +--
 5 files changed, 104 insertions(+), 25 deletions(-)

diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
index f1d1d2c..4a65288 100644
--- a/fs/nfs/internal.h
+++ b/fs/nfs/internal.h
@@ -528,6 +528,8 @@ extern int nfs40_walk_client_list(struct nfs_client *clp,
 extern int nfs41_walk_client_list(struct nfs_client *clp,
 				struct nfs_client **result,
 				struct rpc_cred *cred);
+extern void nfs4_get_pseudofs_replicas(struct nfs_server *server,
+				struct nfs_fh *mntfh);
 
 static inline struct inode *nfs_igrab_and_active(struct inode *inode)
 {
diff --git a/fs/nfs/nfs4getroot.c b/fs/nfs/nfs4getroot.c
index 039b3eb..4c1f5e4 100644
--- a/fs/nfs/nfs4getroot.c
+++ b/fs/nfs/nfs4getroot.c
@@ -36,6 +36,9 @@ int nfs4_get_rootfh(struct nfs_server *server, struct nfs_fh *mntfh, bool auth_p
 	}
 
 	memcpy(&server->fsid, &fsinfo.fattr->fsid, sizeof(server->fsid));
+
+	nfs4_get_pseudofs_replicas(server, mntfh);
+
 out:
 	nfs_free_fattr(fsinfo.fattr);
 	dprintk("<-- nfs4_get_rootfh() = %d\n", ret);
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 70537f0..7b7944a 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -3253,6 +3253,88 @@ static int nfs4_do_find_root_sec(struct nfs_server *server,
 	return nfs_v4_minor_ops[mv]->find_root_sec(server, fhandle, info);
 }
 
+static int _nfs4_proc_fs_locations_probe(struct nfs_server *server,
+					 struct nfs_fh *fhandle,
+					 struct nfs4_fs_locations *fs_locations,
+					 struct page *page)
+{
+	u32 bitmask[3] = {
+		[0] = FATTR4_WORD0_FS_LOCATIONS,
+	};
+	struct nfs4_fs_locations_arg args = {
+		.fh = fhandle,
+		.page = page,
+		.bitmask = bitmask,
+		.replicas = 1,
+	};
+	struct nfs4_fs_locations_res res = {
+		.fs_locations = fs_locations,
+		.replicas = 1,
+	};
+	struct rpc_message msg = {
+		.rpc_proc = &nfs4_procedures[NFSPROC4_CLNT_FS_LOCATIONS],
+		.rpc_argp = &args,
+		.rpc_resp = &res,
+	};
+	struct rpc_clnt *clnt = server->nfs_client->cl_rpcclient;
+
+	nfs_fattr_init(&fs_locations->fattr);
+	fs_locations->nlocations = 0;
+	fs_locations->server = server;
+	return nfs4_call_sync(clnt, server, &msg, &args.seq_args,
+				&res.seq_res, 0);
+}
+
+static int nfs4_proc_fs_locations_probe(struct nfs_server *server,
+					struct nfs_fh *fhandle,
+					struct nfs4_fs_locations *fs_locations,
+					struct page *page)
+{
+	struct nfs4_exception exception = { };
+	int err;
+
+	do {
+		err = _nfs4_proc_fs_locations_probe(server, fhandle,
+						fs_locations, page);
+		err = nfs4_handle_exception(server, err, &exception);
+	} while (exception.retry);
+	return err;
+}
+
+
+/**
+ * Probe the pseudo filesystem for an fs_locations replicas list.
+ * Note: The replicas list on a pseudofs is a list of multipath
+ * addresses
+ */
+void nfs4_get_pseudofs_replicas(struct nfs_server *server,
+				       struct nfs_fh *mntfh)
+{
+	struct page *page = NULL;
+	struct nfs4_fs_locations *locations = NULL;
+	int status;
+
+	if (server->nfs_client->cl_minorversion == 0)
+		return;
+
+	page = alloc_page(GFP_KERNEL);
+	if (page == NULL)
+		goto out;
+	locations = kmalloc(sizeof(struct nfs4_fs_locations), GFP_KERNEL);
+	if (locations == NULL)
+		goto out;
+
+	status = nfs4_proc_fs_locations_probe(server, mntfh, locations, page);
+	if (status != 0)
+		goto out;
+
+	/* test replicas for session trunking here */
+out:
+	if (page)
+		__free_page(page);
+	kfree(locations);
+}
+
 /**
  * nfs4_proc_get_rootfh - get file handle for server's pseudoroot
  * @server: initialized nfs_server handle
diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
index 88474a4..3d06c32 100644
--- a/fs/nfs/nfs4xdr.c
+++ b/fs/nfs/nfs4xdr.c
@@ -2685,18 +2685,17 @@ static void nfs4_xdr_enc_fs_locations(struct rpc_rqst *req,
 
 	encode_compound_hdr(xdr, req, &hdr);
 	encode_sequence(xdr, &args->seq_args, &hdr);
-	if (args->migration) {
+	if (args->replicas || args->migration) {
 		encode_putfh(xdr, args->fh, &hdr);
-		replen = hdr.replen;
-		encode_fs_locations(xdr, args->bitmask, &hdr);
-		if (args->renew)
-			encode_renew(xdr, args->clientid, &hdr);
 	} else {
 		encode_putfh(xdr, args->dir_fh, &hdr);
 		encode_lookup(xdr, args->name, &hdr);
-		replen = hdr.replen;
-		encode_fs_locations(xdr, args->bitmask, &hdr);
 	}
+	replen = hdr.replen;
+	encode_fs_locations(xdr, args->bitmask, &hdr);
+
+	if(args->migration && args->renew)
+		encode_renew(xdr, args->clientid, &hdr);
 
 	/* Set up reply kvec to capture returned fs_locations array. */
 	xdr_inline_pages(&req->rq_rcv_buf, replen << 2, &args->page,
@@ -6916,26 +6915,19 @@ static int nfs4_xdr_dec_fs_locations(struct rpc_rqst *req,
 	status = decode_putfh(xdr);
 	if (status)
 		goto out;
-	if (res->migration) {
-		xdr_enter_page(xdr, PAGE_SIZE);
-		status = decode_getfattr_generic(xdr,
-					&res->fs_locations->fattr,
-					 NULL, res->fs_locations,
-					 NULL, res->fs_locations->server);
-		if (status)
-			goto out;
-		if (res->renew)
-			status = decode_renew(xdr);
-	} else {
+	if (!res->replicas && !res->migration) {
 		status = decode_lookup(xdr);
 		if (status)
 			goto out;
-		xdr_enter_page(xdr, PAGE_SIZE);
-		status = decode_getfattr_generic(xdr,
-					&res->fs_locations->fattr,
-					 NULL, res->fs_locations,
-					 NULL, res->fs_locations->server);
 	}
+	xdr_enter_page(xdr, PAGE_SIZE);
+	status = decode_getfattr_generic(xdr,
+				&res->fs_locations->fattr,
+				NULL, res->fs_locations,
+				NULL, res->fs_locations->server);
+	if (res->migration && res->renew && !status)
+		status = decode_renew(xdr);
+
 out:
 	return status;
 }
diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
index d320906..ab9a08a2 100644
--- a/include/linux/nfs_xdr.h
+++ b/include/linux/nfs_xdr.h
@@ -1120,13 +1120,13 @@ struct nfs4_fs_locations_arg {
 	struct page *page;
 	const u32 *bitmask;
 	clientid4 clientid;
-	unsigned char migration:1, renew:1;
+	unsigned char migration:1, renew:1, replicas:1;
 };
 
 struct nfs4_fs_locations_res {
 	struct nfs4_sequence_res	seq_res;
 	struct nfs4_fs_locations       *fs_locations;
-	unsigned char			migration:1, renew:1;
+	unsigned char			migration:1, renew:1, replicas:1;
 };
 
 struct nfs4_secinfo4 {
-- 
1.8.3.1

[PATCH Version 5 08/10] NFS test and add multipaths for session trunking

[andros]

From: Andy Adamson <andros@netapp.com>

Test the multipath addresses returned by nfs4_get_pseudofs_replicas
for session trunking.

Signed-off-by: Andy Adamson <andros@netapp.com>
---
 fs/nfs/nfs4_fs.h           |   7 ++++
 fs/nfs/nfs4proc.c          | 100 ++++++++++++++++++++++++++++++++++++++++++++-
 net/sunrpc/xprtmultipath.c |   3 ++
 3 files changed, 109 insertions(+), 1 deletion(-)

diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index ff8cfcd..e886aa7 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
@@ -59,6 +59,8 @@ struct nfs4_minor_version_ops {
 			struct nfs4_lock_state *);
 	struct nfs_seqid *
 		(*alloc_seqid)(struct nfs_seqid_counter *, gfp_t);
+	int	(*session_trunk)(struct rpc_clnt *, struct rpc_xprt_switch *,
+			struct rpc_xprt *, void *);
 	const struct rpc_call_ops *call_sync_ops;
 	const struct nfs4_state_recovery_ops *reboot_recovery_ops;
 	const struct nfs4_state_recovery_ops *nograce_recovery_ops;
@@ -214,6 +216,11 @@ struct nfs4_mig_recovery_ops {
 	int (*fsid_present)(struct inode *, struct rpc_cred *);
 };
 
+struct nfs4_add_xprt_data {
+	struct nfs_client	*clp;
+	struct rpc_cred		*cred;
+};
+
 extern const struct dentry_operations nfs4_dentry_operations;
 
 /* dir.c */
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 7b7944a..66e0543 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -44,6 +44,7 @@
 #include <linux/printk.h>
 #include <linux/slab.h>
 #include <linux/sunrpc/clnt.h>
+#include <linux/sunrpc/addr.h>
 #include <linux/nfs.h>
 #include <linux/nfs4.h>
 #include <linux/nfs_fs.h>
@@ -3301,6 +3302,59 @@ static int nfs4_proc_fs_locations_probe(struct nfs_server *server,
 	return err;
 }
 
+/**
+ * Test the multipath addresses returned by nfs4_get_pseudofs_replicas
+ * for session trunking.
+ *
+ * Add session trunking aliases to the cl_rpcclient
+ */
+void nfs4_test_multipath(struct nfs4_fs_locations *locations,
+			struct nfs_client *clp)
+{
+	struct nfs4_add_xprt_data xprtdata = {
+		.clp = clp,
+	};
+	int i;
+
+	if (!clp->cl_mvops->session_trunk || locations == NULL ||
+	    locations->nlocations <= 0)
+		return;
+
+	xprtdata.cred = nfs4_get_clid_cred(clp);
+
+	for (i = 0; i < locations->nlocations; i++) {
+		struct nfs4_fs_location *loc = &locations->locations[i];
+		int i;
+
+		for (i = 0; i < loc->nservers; i++) {
+			struct nfs4_string *server = &loc->servers[i];
+			struct sockaddr_storage addr;
+			size_t addrlen;
+			struct xprt_create xprt_args = {
+				.ident = XPRT_TRANSPORT_TCP,
+				.net = clp->cl_net,
+			};
+
+			addrlen = rpc_pton(clp->cl_net, server->data,
+					server->len, (struct sockaddr *)&addr,
+					sizeof(addr));
+
+			xprt_args.dstaddr = (struct sockaddr *)&addr;
+			xprt_args.addrlen = addrlen;
+			xprt_args.servername = server->data;
+
+			/**
+			 * Check for session trunking. Add this address as
+			 * an alias if session trunking is permitted.
+			 */
+			rpc_clnt_add_xprt(clp->cl_rpcclient, &xprt_args,
+					clp->cl_mvops->session_trunk,
+					&xprtdata);
+		}
+	}
+	if (xprtdata.cred)
+		put_rpccred(xprtdata.cred);
+}
 
 /**
  * Probe the pseudo filesystem for an fs_locations replicas list.
@@ -3328,7 +3382,8 @@ void nfs4_get_pseudofs_replicas(struct nfs_server *server,
 	if (status != 0)
 		goto out;
 
-	/* test replicas for session trunking here */
+	/* test replicas for session trunking */
+	nfs4_test_multipath(locations, server->nfs_client);
 out:
 	if (page)
 		__free_page(page);
@@ -7289,6 +7344,47 @@ int nfs4_proc_exchange_id(struct nfs_client *clp, struct rpc_cred *cred)
 	return _nfs4_proc_exchange_id(clp, cred, SP4_NONE, NULL);
 }
 
+/**
+ * nfs4_test_session_trunk - Test for session trunking with a
+ * synchronous exchange_id call. Upon success, add a new transport
+ * to the rpc_clnt
+ *
+ * @clnt: struct rpc_clnt to get new transport
+ * @xps:  the rpc_xprt_switch to hold the new transport
+ * @xprt: the rpc_xprt to test
+ * @data: call data for _nfs4_proc_exchange_id.
+ *
+ */
+int nfs4_test_session_trunk(struct rpc_clnt *clnt, struct rpc_xprt_switch *xps,
+			    struct rpc_xprt *xprt, void *data)
+{
+	struct nfs4_add_xprt_data *xdata = (struct nfs4_add_xprt_data *)data;
+	u32 sp4_how;
+	int status;
+
+	sp4_how = (xdata->clp->cl_sp4_flags == 0 ? SP4_NONE : SP4_MACH_CRED);
+
+	/* Ensure these stick around for the rpc call */
+	xps = xprt_switch_get(xps);
+	xprt = xprt_get(xprt);
+
+	/* Sync call */
+	status = _nfs4_proc_exchange_id(xdata->clp, xdata->cred, sp4_how, xprt);
+
+	xprt_put(xprt);
+	xprt_switch_put(xps);
+
+	if (status)
+		pr_info("NFS:   %s: Session trunking failed for %s status %d\n",
+			xdata->clp->cl_hostname,
+			xprt->address_strings[RPC_DISPLAY_ADDR], status);
+	else
+		pr_info("NFS:   %s: Session trunking succeeded for %s\n",
+			xdata->clp->cl_hostname,
+			xprt->address_strings[RPC_DISPLAY_ADDR]);
+	return status;
+}
+
 static int _nfs4_proc_destroy_clientid(struct nfs_client *clp,
 		struct rpc_cred *cred)
 {
@@ -8880,6 +8976,7 @@ static const struct nfs4_minor_version_ops nfs_v4_1_minor_ops = {
 	.find_root_sec = nfs41_find_root_sec,
 	.free_lock_state = nfs41_free_lock_state,
 	.alloc_seqid = nfs_alloc_no_seqid,
+	.session_trunk = nfs4_test_session_trunk,
 	.call_sync_ops = &nfs41_call_sync_ops,
 	.reboot_recovery_ops = &nfs41_reboot_recovery_ops,
 	.nograce_recovery_ops = &nfs41_nograce_recovery_ops,
@@ -8908,6 +9005,7 @@ static const struct nfs4_minor_version_ops nfs_v4_2_minor_ops = {
 	.free_lock_state = nfs41_free_lock_state,
 	.call_sync_ops = &nfs41_call_sync_ops,
 	.alloc_seqid = nfs_alloc_no_seqid,
+	.session_trunk = nfs4_test_session_trunk,
 	.reboot_recovery_ops = &nfs41_reboot_recovery_ops,
 	.nograce_recovery_ops = &nfs41_nograce_recovery_ops,
 	.state_renewal_ops = &nfs41_state_renewal_ops,
diff --git a/net/sunrpc/xprtmultipath.c b/net/sunrpc/xprtmultipath.c
index e7fd769..360f64c 100644
--- a/net/sunrpc/xprtmultipath.c
+++ b/net/sunrpc/xprtmultipath.c
@@ -53,6 +53,7 @@ void rpc_xprt_switch_add_xprt(struct rpc_xprt_switch *xps,
 		xprt_switch_add_xprt_locked(xps, xprt);
 	spin_unlock(&xps->xps_lock);
 }
+EXPORT_SYMBOL_GPL(rpc_xprt_switch_add_xprt);
 
 static void xprt_switch_remove_xprt_locked(struct rpc_xprt_switch *xps,
 		struct rpc_xprt *xprt)
@@ -145,6 +146,7 @@ struct rpc_xprt_switch *xprt_switch_get(struct rpc_xprt_switch *xps)
 		return xps;
 	return NULL;
 }
+EXPORT_SYMBOL_GPL(xprt_switch_get);
 
 /**
  * xprt_switch_put - Release a reference to a rpc_xprt_switch
@@ -157,6 +159,7 @@ void xprt_switch_put(struct rpc_xprt_switch *xps)
 	if (xps != NULL)
 		kref_put(&xps->xps_kref, xprt_switch_free);
 }
+EXPORT_SYMBOL_GPL(xprt_switch_put);
 
 /**
  * rpc_xprt_switch_set_roundrobin - Set a round-robin policy on rpc_xprt_switch
-- 
1.8.3.1

Re: [PATCH Version 5 08/10] NFS test and add multipaths for session trunking

[Adamson, Andy]

DQo+IE9uIE1heSAyMCwgMjAxNiwgYXQgMTA6NDMgQU0sIEFkYW1zb24sIEFuZHkgPFdpbGxpYW0u
QWRhbXNvbkBuZXRhcHAuY29tPiB3cm90ZToNCj4gDQo+IEZyb206IEFuZHkgQWRhbXNvbiA8YW5k
cm9zQG5ldGFwcC5jb20+DQo+IA0KPiBUZXN0IHRoZSBtdWx0aXBhdGggYWRkcmVzc2VzIHJldHVy
bmVkIGJ5IG5mczRfZ2V0X3BzZXVkb2ZzX3JlcGxpY2FzDQo+IGZvciBzZXNzaW9uIHRydW5raW5n
Lg0KPiANCj4gU2lnbmVkLW9mZi1ieTogQW5keSBBZGFtc29uIDxhbmRyb3NAbmV0YXBwLmNvbT4N
Cj4gLS0tDQo+IGZzL25mcy9uZnM0X2ZzLmggICAgICAgICAgIHwgICA3ICsrKysNCj4gZnMvbmZz
L25mczRwcm9jLmMgICAgICAgICAgfCAxMDAgKysrKysrKysrKysrKysrKysrKysrKysrKysrKysr
KysrKysrKysrKysrKystDQo+IG5ldC9zdW5ycGMveHBydG11bHRpcGF0aC5jIHwgICAzICsrDQo+
IDMgZmlsZXMgY2hhbmdlZCwgMTA5IGluc2VydGlvbnMoKyksIDEgZGVsZXRpb24oLSkNCj4gDQo+
IGRpZmYgLS1naXQgYS9mcy9uZnMvbmZzNF9mcy5oIGIvZnMvbmZzL25mczRfZnMuaA0KPiBpbmRl
eCBmZjhjZmNkLi5lODg2YWE3IDEwMDY0NA0KPiAtLS0gYS9mcy9uZnMvbmZzNF9mcy5oDQo+ICsr
KyBiL2ZzL25mcy9uZnM0X2ZzLmgNCj4gQEAgLTU5LDYgKzU5LDggQEAgc3RydWN0IG5mczRfbWlu
b3JfdmVyc2lvbl9vcHMgew0KPiAJCQlzdHJ1Y3QgbmZzNF9sb2NrX3N0YXRlICopOw0KPiAJc3Ry
dWN0IG5mc19zZXFpZCAqDQo+IAkJKCphbGxvY19zZXFpZCkoc3RydWN0IG5mc19zZXFpZF9jb3Vu
dGVyICosIGdmcF90KTsNCj4gKwlpbnQJKCpzZXNzaW9uX3RydW5rKShzdHJ1Y3QgcnBjX2NsbnQg
Kiwgc3RydWN0IHJwY194cHJ0X3N3aXRjaCAqLA0KPiArCQkJc3RydWN0IHJwY194cHJ0ICosIHZv
aWQgKik7DQo+IAljb25zdCBzdHJ1Y3QgcnBjX2NhbGxfb3BzICpjYWxsX3N5bmNfb3BzOw0KPiAJ
Y29uc3Qgc3RydWN0IG5mczRfc3RhdGVfcmVjb3Zlcnlfb3BzICpyZWJvb3RfcmVjb3Zlcnlfb3Bz
Ow0KPiAJY29uc3Qgc3RydWN0IG5mczRfc3RhdGVfcmVjb3Zlcnlfb3BzICpub2dyYWNlX3JlY292
ZXJ5X29wczsNCj4gQEAgLTIxNCw2ICsyMTYsMTEgQEAgc3RydWN0IG5mczRfbWlnX3JlY292ZXJ5
X29wcyB7DQo+IAlpbnQgKCpmc2lkX3ByZXNlbnQpKHN0cnVjdCBpbm9kZSAqLCBzdHJ1Y3QgcnBj
X2NyZWQgKik7DQo+IH07DQo+IA0KPiArc3RydWN0IG5mczRfYWRkX3hwcnRfZGF0YSB7DQo+ICsJ
c3RydWN0IG5mc19jbGllbnQJKmNscDsNCj4gKwlzdHJ1Y3QgcnBjX2NyZWQJCSpjcmVkOw0KPiAr
fTsNCj4gKw0KPiBleHRlcm4gY29uc3Qgc3RydWN0IGRlbnRyeV9vcGVyYXRpb25zIG5mczRfZGVu
dHJ5X29wZXJhdGlvbnM7DQo+IA0KPiAvKiBkaXIuYyAqLw0KPiBkaWZmIC0tZ2l0IGEvZnMvbmZz
L25mczRwcm9jLmMgYi9mcy9uZnMvbmZzNHByb2MuYw0KPiBpbmRleCA3Yjc5NDRhLi42NmUwNTQz
IDEwMDY0NA0KPiAtLS0gYS9mcy9uZnMvbmZzNHByb2MuYw0KPiArKysgYi9mcy9uZnMvbmZzNHBy
b2MuYw0KPiBAQCAtNDQsNiArNDQsNyBAQA0KPiAjaW5jbHVkZSA8bGludXgvcHJpbnRrLmg+DQo+
ICNpbmNsdWRlIDxsaW51eC9zbGFiLmg+DQo+ICNpbmNsdWRlIDxsaW51eC9zdW5ycGMvY2xudC5o
Pg0KPiArI2luY2x1ZGUgPGxpbnV4L3N1bnJwYy9hZGRyLmg+DQo+ICNpbmNsdWRlIDxsaW51eC9u
ZnMuaD4NCj4gI2luY2x1ZGUgPGxpbnV4L25mczQuaD4NCj4gI2luY2x1ZGUgPGxpbnV4L25mc19m
cy5oPg0KPiBAQCAtMzMwMSw2ICszMzAyLDU5IEBAIHN0YXRpYyBpbnQgbmZzNF9wcm9jX2ZzX2xv
Y2F0aW9uc19wcm9iZShzdHJ1Y3QgbmZzX3NlcnZlciAqc2VydmVyLA0KPiAJcmV0dXJuIGVycjsN
Cj4gfQ0KPiANCj4gKy8qKg0KPiArICogVGVzdCB0aGUgbXVsdGlwYXRoIGFkZHJlc3NlcyByZXR1
cm5lZCBieSBuZnM0X2dldF9wc2V1ZG9mc19yZXBsaWNhcw0KPiArICogZm9yIHNlc3Npb24gdHJ1
bmtpbmcuDQo+ICsgKg0KPiArICogQWRkIHNlc3Npb24gdHJ1bmtpbmcgYWxpYXNlcyB0byB0aGUg
Y2xfcnBjY2xpZW50DQo+ICsgKi8NCj4gK3ZvaWQgbmZzNF90ZXN0X211bHRpcGF0aChzdHJ1Y3Qg
bmZzNF9mc19sb2NhdGlvbnMgKmxvY2F0aW9ucywNCj4gKwkJCXN0cnVjdCBuZnNfY2xpZW50ICpj
bHApDQo+ICt7DQo+ICsJc3RydWN0IG5mczRfYWRkX3hwcnRfZGF0YSB4cHJ0ZGF0YSA9IHsNCj4g
KwkJLmNscCA9IGNscCwNCj4gKwl9Ow0KPiArCWludCBpOw0KPiArDQo+ICsJaWYgKCFjbHAtPmNs
X212b3BzLT5zZXNzaW9uX3RydW5rIHx8IGxvY2F0aW9ucyA9PSBOVUxMIHx8DQo+ICsJICAgIGxv
Y2F0aW9ucy0+bmxvY2F0aW9ucyA8PSAwKQ0KPiArCQlyZXR1cm47DQo+ICsNCj4gKwl4cHJ0ZGF0
YS5jcmVkID0gbmZzNF9nZXRfY2xpZF9jcmVkKGNscCk7DQo+ICsNCj4gKwlmb3IgKGkgPSAwOyBp
IDwgbG9jYXRpb25zLT5ubG9jYXRpb25zOyBpKyspIHsNCj4gKwkJc3RydWN0IG5mczRfZnNfbG9j
YXRpb24gKmxvYyA9ICZsb2NhdGlvbnMtPmxvY2F0aW9uc1tpXTsNCj4gKwkJaW50IGk7DQo+ICsN
Cj4gKwkJZm9yIChpID0gMDsgaSA8IGxvYy0+bnNlcnZlcnM7IGkrKykgew0KPiArCQkJc3RydWN0
IG5mczRfc3RyaW5nICpzZXJ2ZXIgPSAmbG9jLT5zZXJ2ZXJzW2ldOw0KPiArCQkJc3RydWN0IHNv
Y2thZGRyX3N0b3JhZ2UgYWRkcjsNCj4gKwkJCXNpemVfdCBhZGRybGVuOw0KPiArCQkJc3RydWN0
IHhwcnRfY3JlYXRlIHhwcnRfYXJncyA9IHsNCj4gKwkJCQkuaWRlbnQgPSBYUFJUX1RSQU5TUE9S
VF9UQ1AsDQo+ICsJCQkJLm5ldCA9IGNscC0+Y2xfbmV0LA0KPiArCQkJfTsNCj4gKw0KPiArCQkJ
YWRkcmxlbiA9IHJwY19wdG9uKGNscC0+Y2xfbmV0LCBzZXJ2ZXItPmRhdGEsDQo+ICsJCQkJCXNl
cnZlci0+bGVuLCAoc3RydWN0IHNvY2thZGRyICopJmFkZHIsDQo+ICsJCQkJCXNpemVvZihhZGRy
KSk7DQoNCkkgbmVlZCB0byBjaGVjayB0aGUgcmV0dXJuIG9mIHJwY19wdG9uIGhlcmUuDQoNCuKA
lD5BbmR5DQoNCg0KPiArDQo+ICsJCQl4cHJ0X2FyZ3MuZHN0YWRkciA9IChzdHJ1Y3Qgc29ja2Fk
ZHIgKikmYWRkcjsNCj4gKwkJCXhwcnRfYXJncy5hZGRybGVuID0gYWRkcmxlbjsNCj4gKwkJCXhw
cnRfYXJncy5zZXJ2ZXJuYW1lID0gc2VydmVyLT5kYXRhOw0KPiArDQo+ICsJCQkvKioNCj4gKwkJ
CSAqIENoZWNrIGZvciBzZXNzaW9uIHRydW5raW5nLiBBZGQgdGhpcyBhZGRyZXNzIGFzDQo+ICsJ
CQkgKiBhbiBhbGlhcyBpZiBzZXNzaW9uIHRydW5raW5nIGlzIHBlcm1pdHRlZC4NCj4gKwkJCSAq
Lw0KPiArCQkJcnBjX2NsbnRfYWRkX3hwcnQoY2xwLT5jbF9ycGNjbGllbnQsICZ4cHJ0X2FyZ3Ms
DQo+ICsJCQkJCWNscC0+Y2xfbXZvcHMtPnNlc3Npb25fdHJ1bmssDQo+ICsJCQkJCSZ4cHJ0ZGF0
YSk7DQo+ICsJCX0NCj4gKwl9DQo+ICsJaWYgKHhwcnRkYXRhLmNyZWQpDQo+ICsJCXB1dF9ycGNj
cmVkKHhwcnRkYXRhLmNyZWQpOw0KPiArfQ0KPiANCj4gLyoqDQo+ICAqIFByb2JlIHRoZSBwc2V1
ZG8gZmlsZXN5c3RlbSBmb3IgYW4gZnNfbG9jYXRpb25zIHJlcGxpY2FzIGxpc3QuDQo+IEBAIC0z
MzI4LDcgKzMzODIsOCBAQCB2b2lkIG5mczRfZ2V0X3BzZXVkb2ZzX3JlcGxpY2FzKHN0cnVjdCBu
ZnNfc2VydmVyICpzZXJ2ZXIsDQo+IAlpZiAoc3RhdHVzICE9IDApDQo+IAkJZ290byBvdXQ7DQo+
IA0KPiAtCS8qIHRlc3QgcmVwbGljYXMgZm9yIHNlc3Npb24gdHJ1bmtpbmcgaGVyZSAqLw0KPiAr
CS8qIHRlc3QgcmVwbGljYXMgZm9yIHNlc3Npb24gdHJ1bmtpbmcgKi8NCj4gKwluZnM0X3Rlc3Rf
bXVsdGlwYXRoKGxvY2F0aW9ucywgc2VydmVyLT5uZnNfY2xpZW50KTsNCj4gb3V0Og0KPiAJaWYg
KHBhZ2UpDQo+IAkJX19mcmVlX3BhZ2UocGFnZSk7DQo+IEBAIC03Mjg5LDYgKzczNDQsNDcgQEAg
aW50IG5mczRfcHJvY19leGNoYW5nZV9pZChzdHJ1Y3QgbmZzX2NsaWVudCAqY2xwLCBzdHJ1Y3Qg
cnBjX2NyZWQgKmNyZWQpDQo+IAlyZXR1cm4gX25mczRfcHJvY19leGNoYW5nZV9pZChjbHAsIGNy
ZWQsIFNQNF9OT05FLCBOVUxMKTsNCj4gfQ0KPiANCj4gKy8qKg0KPiArICogbmZzNF90ZXN0X3Nl
c3Npb25fdHJ1bmsgLSBUZXN0IGZvciBzZXNzaW9uIHRydW5raW5nIHdpdGggYQ0KPiArICogc3lu
Y2hyb25vdXMgZXhjaGFuZ2VfaWQgY2FsbC4gVXBvbiBzdWNjZXNzLCBhZGQgYSBuZXcgdHJhbnNw
b3J0DQo+ICsgKiB0byB0aGUgcnBjX2NsbnQNCj4gKyAqDQo+ICsgKiBAY2xudDogc3RydWN0IHJw
Y19jbG50IHRvIGdldCBuZXcgdHJhbnNwb3J0DQo+ICsgKiBAeHBzOiAgdGhlIHJwY194cHJ0X3N3
aXRjaCB0byBob2xkIHRoZSBuZXcgdHJhbnNwb3J0DQo+ICsgKiBAeHBydDogdGhlIHJwY194cHJ0
IHRvIHRlc3QNCj4gKyAqIEBkYXRhOiBjYWxsIGRhdGEgZm9yIF9uZnM0X3Byb2NfZXhjaGFuZ2Vf
aWQuDQo+ICsgKg0KPiArICovDQo+ICtpbnQgbmZzNF90ZXN0X3Nlc3Npb25fdHJ1bmsoc3RydWN0
IHJwY19jbG50ICpjbG50LCBzdHJ1Y3QgcnBjX3hwcnRfc3dpdGNoICp4cHMsDQo+ICsJCQkgICAg
c3RydWN0IHJwY194cHJ0ICp4cHJ0LCB2b2lkICpkYXRhKQ0KPiArew0KPiArCXN0cnVjdCBuZnM0
X2FkZF94cHJ0X2RhdGEgKnhkYXRhID0gKHN0cnVjdCBuZnM0X2FkZF94cHJ0X2RhdGEgKilkYXRh
Ow0KPiArCXUzMiBzcDRfaG93Ow0KPiArCWludCBzdGF0dXM7DQo+ICsNCj4gKwlzcDRfaG93ID0g
KHhkYXRhLT5jbHAtPmNsX3NwNF9mbGFncyA9PSAwID8gU1A0X05PTkUgOiBTUDRfTUFDSF9DUkVE
KTsNCj4gKw0KPiArCS8qIEVuc3VyZSB0aGVzZSBzdGljayBhcm91bmQgZm9yIHRoZSBycGMgY2Fs
bCAqLw0KPiArCXhwcyA9IHhwcnRfc3dpdGNoX2dldCh4cHMpOw0KPiArCXhwcnQgPSB4cHJ0X2dl
dCh4cHJ0KTsNCj4gKw0KPiArCS8qIFN5bmMgY2FsbCAqLw0KPiArCXN0YXR1cyA9IF9uZnM0X3By
b2NfZXhjaGFuZ2VfaWQoeGRhdGEtPmNscCwgeGRhdGEtPmNyZWQsIHNwNF9ob3csIHhwcnQpOw0K
PiArDQo+ICsJeHBydF9wdXQoeHBydCk7DQo+ICsJeHBydF9zd2l0Y2hfcHV0KHhwcyk7DQo+ICsN
Cj4gKwlpZiAoc3RhdHVzKQ0KPiArCQlwcl9pbmZvKCJORlM6ICAgJXM6IFNlc3Npb24gdHJ1bmtp
bmcgZmFpbGVkIGZvciAlcyBzdGF0dXMgJWRcbiIsDQo+ICsJCQl4ZGF0YS0+Y2xwLT5jbF9ob3N0
bmFtZSwNCj4gKwkJCXhwcnQtPmFkZHJlc3Nfc3RyaW5nc1tSUENfRElTUExBWV9BRERSXSwgc3Rh
dHVzKTsNCj4gKwllbHNlDQo+ICsJCXByX2luZm8oIk5GUzogICAlczogU2Vzc2lvbiB0cnVua2lu
ZyBzdWNjZWVkZWQgZm9yICVzXG4iLA0KPiArCQkJeGRhdGEtPmNscC0+Y2xfaG9zdG5hbWUsDQo+
ICsJCQl4cHJ0LT5hZGRyZXNzX3N0cmluZ3NbUlBDX0RJU1BMQVlfQUREUl0pOw0KPiArCXJldHVy
biBzdGF0dXM7DQo+ICt9DQo+ICsNCj4gc3RhdGljIGludCBfbmZzNF9wcm9jX2Rlc3Ryb3lfY2xp
ZW50aWQoc3RydWN0IG5mc19jbGllbnQgKmNscCwNCj4gCQlzdHJ1Y3QgcnBjX2NyZWQgKmNyZWQp
DQo+IHsNCj4gQEAgLTg4ODAsNiArODk3Niw3IEBAIHN0YXRpYyBjb25zdCBzdHJ1Y3QgbmZzNF9t
aW5vcl92ZXJzaW9uX29wcyBuZnNfdjRfMV9taW5vcl9vcHMgPSB7DQo+IAkuZmluZF9yb290X3Nl
YyA9IG5mczQxX2ZpbmRfcm9vdF9zZWMsDQo+IAkuZnJlZV9sb2NrX3N0YXRlID0gbmZzNDFfZnJl
ZV9sb2NrX3N0YXRlLA0KPiAJLmFsbG9jX3NlcWlkID0gbmZzX2FsbG9jX25vX3NlcWlkLA0KPiAr
CS5zZXNzaW9uX3RydW5rID0gbmZzNF90ZXN0X3Nlc3Npb25fdHJ1bmssDQo+IAkuY2FsbF9zeW5j
X29wcyA9ICZuZnM0MV9jYWxsX3N5bmNfb3BzLA0KPiAJLnJlYm9vdF9yZWNvdmVyeV9vcHMgPSAm
bmZzNDFfcmVib290X3JlY292ZXJ5X29wcywNCj4gCS5ub2dyYWNlX3JlY292ZXJ5X29wcyA9ICZu
ZnM0MV9ub2dyYWNlX3JlY292ZXJ5X29wcywNCj4gQEAgLTg5MDgsNiArOTAwNSw3IEBAIHN0YXRp
YyBjb25zdCBzdHJ1Y3QgbmZzNF9taW5vcl92ZXJzaW9uX29wcyBuZnNfdjRfMl9taW5vcl9vcHMg
PSB7DQo+IAkuZnJlZV9sb2NrX3N0YXRlID0gbmZzNDFfZnJlZV9sb2NrX3N0YXRlLA0KPiAJLmNh
bGxfc3luY19vcHMgPSAmbmZzNDFfY2FsbF9zeW5jX29wcywNCj4gCS5hbGxvY19zZXFpZCA9IG5m
c19hbGxvY19ub19zZXFpZCwNCj4gKwkuc2Vzc2lvbl90cnVuayA9IG5mczRfdGVzdF9zZXNzaW9u
X3RydW5rLA0KPiAJLnJlYm9vdF9yZWNvdmVyeV9vcHMgPSAmbmZzNDFfcmVib290X3JlY292ZXJ5
X29wcywNCj4gCS5ub2dyYWNlX3JlY292ZXJ5X29wcyA9ICZuZnM0MV9ub2dyYWNlX3JlY292ZXJ5
X29wcywNCj4gCS5zdGF0ZV9yZW5ld2FsX29wcyA9ICZuZnM0MV9zdGF0ZV9yZW5ld2FsX29wcywN
Cj4gZGlmZiAtLWdpdCBhL25ldC9zdW5ycGMveHBydG11bHRpcGF0aC5jIGIvbmV0L3N1bnJwYy94
cHJ0bXVsdGlwYXRoLmMNCj4gaW5kZXggZTdmZDc2OS4uMzYwZjY0YyAxMDA2NDQNCj4gLS0tIGEv
bmV0L3N1bnJwYy94cHJ0bXVsdGlwYXRoLmMNCj4gKysrIGIvbmV0L3N1bnJwYy94cHJ0bXVsdGlw
YXRoLmMNCj4gQEAgLTUzLDYgKzUzLDcgQEAgdm9pZCBycGNfeHBydF9zd2l0Y2hfYWRkX3hwcnQo
c3RydWN0IHJwY194cHJ0X3N3aXRjaCAqeHBzLA0KPiAJCXhwcnRfc3dpdGNoX2FkZF94cHJ0X2xv
Y2tlZCh4cHMsIHhwcnQpOw0KPiAJc3Bpbl91bmxvY2soJnhwcy0+eHBzX2xvY2spOw0KPiB9DQo+
ICtFWFBPUlRfU1lNQk9MX0dQTChycGNfeHBydF9zd2l0Y2hfYWRkX3hwcnQpOw0KPiANCj4gc3Rh
dGljIHZvaWQgeHBydF9zd2l0Y2hfcmVtb3ZlX3hwcnRfbG9ja2VkKHN0cnVjdCBycGNfeHBydF9z
d2l0Y2ggKnhwcywNCj4gCQlzdHJ1Y3QgcnBjX3hwcnQgKnhwcnQpDQo+IEBAIC0xNDUsNiArMTQ2
LDcgQEAgc3RydWN0IHJwY194cHJ0X3N3aXRjaCAqeHBydF9zd2l0Y2hfZ2V0KHN0cnVjdCBycGNf
eHBydF9zd2l0Y2ggKnhwcykNCj4gCQlyZXR1cm4geHBzOw0KPiAJcmV0dXJuIE5VTEw7DQo+IH0N
Cj4gK0VYUE9SVF9TWU1CT0xfR1BMKHhwcnRfc3dpdGNoX2dldCk7DQo+IA0KPiAvKioNCj4gICog
eHBydF9zd2l0Y2hfcHV0IC0gUmVsZWFzZSBhIHJlZmVyZW5jZSB0byBhIHJwY194cHJ0X3N3aXRj
aA0KPiBAQCAtMTU3LDYgKzE1OSw3IEBAIHZvaWQgeHBydF9zd2l0Y2hfcHV0KHN0cnVjdCBycGNf
eHBydF9zd2l0Y2ggKnhwcykNCj4gCWlmICh4cHMgIT0gTlVMTCkNCj4gCQlrcmVmX3B1dCgmeHBz
LT54cHNfa3JlZiwgeHBydF9zd2l0Y2hfZnJlZSk7DQo+IH0NCj4gK0VYUE9SVF9TWU1CT0xfR1BM
KHhwcnRfc3dpdGNoX3B1dCk7DQo+IA0KPiAvKioNCj4gICogcnBjX3hwcnRfc3dpdGNoX3NldF9y
b3VuZHJvYmluIC0gU2V0IGEgcm91bmQtcm9iaW4gcG9saWN5IG9uIHJwY194cHJ0X3N3aXRjaA0K
PiAtLSANCj4gMS44LjMuMQ0KPiANCg0K

[PATCH Version 5 09/10] NFS test pnfs data server multipath session trunking aliases

[andros]

From: Andy Adamson <andros@netapp.com>

Try all multipath addresses for a data server. The first address that
successfully connects and creates a session is the mount address.
All subsequent addresses are tested for session trunking and
added as aliases.

Signed-off-by: Andy Adamson <andros@netapp.com>
---
 fs/nfs/pnfs_nfs.c | 51 +++++++++++++++++++++++++++++++++++++--------------
 1 file changed, 37 insertions(+), 14 deletions(-)

diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c
index 4aaed89..b1e2b94 100644
--- a/fs/nfs/pnfs_nfs.c
+++ b/fs/nfs/pnfs_nfs.c
@@ -654,13 +654,43 @@ static int _nfs4_pnfs_v4_ds_connect(struct nfs_server *mds_srv,
 		dprintk("%s: DS %s: trying address %s\n",
 			__func__, ds->ds_remotestr, da->da_remotestr);
 
-		clp = nfs4_set_ds_client(mds_srv->nfs_client,
-					(struct sockaddr *)&da->da_addr,
-					da->da_addrlen, IPPROTO_TCP,
-					timeo, retrans, minor_version,
-					au_flavor);
-		if (!IS_ERR(clp))
-			break;
+		if (!IS_ERR(clp) && clp->cl_mvops->session_trunk) {
+			struct xprt_create xprt_args = {
+				.ident = XPRT_TRANSPORT_TCP,
+				.net = clp->cl_net,
+				.dstaddr = (struct sockaddr *)&da->da_addr,
+				.addrlen = da->da_addrlen,
+				.servername = clp->cl_hostname,
+			};
+			struct nfs4_add_xprt_data xprtdata = {
+				.clp = clp,
+			};
+			xprtdata.cred = nfs4_get_clid_cred(clp);
+
+			/**
+			 * Test this address for session trunking and
+			 * add as an alias
+			 */
+			rpc_clnt_add_xprt(clp->cl_rpcclient, &xprt_args,
+					  clp->cl_mvops->session_trunk,
+					  &xprtdata);
+		} else {
+			clp = nfs4_set_ds_client(mds_srv->nfs_client,
+						(struct sockaddr *)&da->da_addr,
+						da->da_addrlen, IPPROTO_TCP,
+						timeo, retrans, minor_version,
+						au_flavor);
+			if (IS_ERR(clp))
+				continue;
+
+			status = nfs4_init_ds_session(clp,
+					mds_srv->nfs_client->cl_lease_time);
+			if (status) {
+				nfs_put_client(clp);
+				clp = ERR_PTR(-EIO);
+				continue;
+			}
+		}
 	}
 
 	if (IS_ERR(clp)) {
@@ -668,18 +698,11 @@ static int _nfs4_pnfs_v4_ds_connect(struct nfs_server *mds_srv,
 		goto out;
 	}
 
-	status = nfs4_init_ds_session(clp, mds_srv->nfs_client->cl_lease_time);
-	if (status)
-		goto out_put;
-
 	smp_wmb();
 	ds->ds_clp = clp;
 	dprintk("%s [new] addr: %s\n", __func__, ds->ds_remotestr);
 out:
 	return status;
-out_put:
-	nfs_put_client(clp);
-	goto out;
 }
 
 /*
-- 
1.8.3.1

[PATCH Version 5 10/10] NFS add multiaddr to nfs_show_nfsv4_options

[andros]

From: Andy Adamson <andros@netapp.com>

Signed-off-by: Andy Adamson <andros@netapp.com>
---
 fs/nfs/super.c             | 26 ++++++++++++++++++++++++++
 net/sunrpc/xprtmultipath.c |  1 +
 2 files changed, 27 insertions(+)

diff --git a/fs/nfs/super.c b/fs/nfs/super.c
index f126828..8d96025 100644
--- a/fs/nfs/super.c
+++ b/fs/nfs/super.c
@@ -603,8 +603,34 @@ static void nfs_show_nfsv4_options(struct seq_file *m, struct nfs_server *nfss,
 				    int showdefaults)
 {
 	struct nfs_client *clp = nfss->nfs_client;
+	struct rpc_clnt *clnt = clp->cl_rpcclient;
+	struct rpc_xprt_switch *xps;
+	struct rpc_xprt *pos;
+	unsigned int nxprts;
 
 	seq_printf(m, ",clientaddr=%s", clp->cl_ipaddr);
+	rcu_read_lock();
+	xps = xprt_switch_get(rcu_dereference(clnt->cl_xpi.xpi_xpswitch));
+	pos = xprt_iter_xprt(&clnt->cl_xpi);
+	if (xps == NULL || pos == NULL) {
+		rcu_read_unlock();
+		return;
+	}
+	nxprts = xps->xps_nxprts;
+	list_for_each_entry_rcu(pos, &xps->xps_xprt_list, xprt_switch) {
+		if (nxprts == 0)
+			break;
+		if (pos->address_strings[RPC_DISPLAY_ADDR] == NULL)
+			break;
+		/* Do not display mount xprt as a multiaddr */
+		if (pos == rcu_access_pointer(clp->cl_rpcclient->cl_xprt))
+			continue;
+		seq_printf(m, ",multiaddr=%s",
+			pos->address_strings[RPC_DISPLAY_ADDR]);
+		nxprts--;
+	}
+	rcu_read_unlock();
+	xprt_switch_put(xps);
 }
 #else
 static void nfs_show_nfsv4_options(struct seq_file *m, struct nfs_server *nfss,
diff --git a/net/sunrpc/xprtmultipath.c b/net/sunrpc/xprtmultipath.c
index 360f64c..708c22a 100644
--- a/net/sunrpc/xprtmultipath.c
+++ b/net/sunrpc/xprtmultipath.c
@@ -403,6 +403,7 @@ struct rpc_xprt *xprt_iter_xprt(struct rpc_xprt_iter *xpi)
 	WARN_ON_ONCE(!rcu_read_lock_held());
 	return xprt_iter_ops(xpi)->xpi_xprt(xpi);
 }
+EXPORT_SYMBOL_GPL(xprt_iter_xprt);
 
 static
 struct rpc_xprt *xprt_iter_get_helper(struct rpc_xprt_iter *xpi,
-- 
1.8.3.1

Re: [PATCH Version 5 00/10] NFSV4.1,2 session trunking

[Martin Houry]

Greeting,

I have just tested the new version of the patch. I tried to shutdown
an interface to see how it react. I have seen that the transfert is
suspended until I up the interface back. This is a threat and I would
like to share with you some clue to resolve it.
1 - Could we use the depth of the requests queue to analyse if it stay
at the same value during a determined amout of time. If this time is
too long, we switch all the requests of this queue to another.
2 - Stay aware of the state of each TCP connection and switch all the
request of this queue to another if the TCP connection goes down.
3 - Check the RTT of the request and consider the connection down if
the RTT is too long. If it's too long  switch all the request of this
queue to another.

What do you think of these propositions?


Martin

2016-05-20 16:43 GMT+02:00  <andros@netapp.com>:
> From: Andy Adamson <andros@netapp.com>
>
> Version-5
> ---------
>
> Responded to Anna's code cleanup comments.
>
> --> Andy
>
> Version-4
> ---------
>
> This patch set enables session trunking configured two ways.
>
> The NFSv4 minorversion > 0 pseudo-fs is probed
> with a GETATTR(fs_locations) to retrieve a replicas list.
> Each replica address is tested for session trunking. Those
> that pass are added as aliases to the mount rpc client.
>
> Try all multipath addresses for a data server. The first address that
> successfully connects and creates a session is the mount address.
> All subsequent addresses are tested for session trunking and
> added as aliases.
>
> TODO:
>
> - Periodically probe the pseudo-fs with the GETATTR(fs_locations)
> to pick up server multipath changes. This is why the
> GETATTR(fs_locations) is a stand alone call.
>
> - Call BIND_CONN_TO_SESSION for session trunking addresses
> established under SP4_MACH_CRED or SP4_SSV. SP4_NONE established
> addresses do not require BIND_CONN_TO_SESSION.
>
> TESTING:
>
> The pseudo-fs GETATTR(fs_locations) probe session trunking
> was tested against a Linux server with a pseudo-fs
> export stanza (e.g. a stanza with the fsid=0 or fsid=root
> export option) and a replicas= export option
> (replicas=<path1>@<server1>:<path2>@<server2>..)
> Note that this configuration is for testing only. A future
> patchset will add the replicas= configuration to the
> NFSEXP_V4ROOT nfsd and mountd processing.
>
> The pNFS DS multipath session trunking was tested against a
> pynfs server which provides DS multipath configuration.
>
>
> Andy Adamson (10):
>   NFS rename default callback ops
>   NFS refactor nfs4_match_clientids
>   NFS refactor nfs4_check_serverowner_major_id
>   NFS detect session trunking
>   NFS refactor _nfs4_proc_exchange_id for session trunking
>   SUNRPC add flag to rpc_task_release_client
>   NFS probe pseudo-fs for replicas
>   NFS test and add multipaths for session trunking
>   NFS test pnfs data server multipath session trunking aliases
>   NFS add multiaddr to nfs_show_nfsv4_options
>
>  fs/nfs/internal.h           |   2 +
>  fs/nfs/nfs4_fs.h            |   9 ++
>  fs/nfs/nfs4client.c         | 102 +++++++++++++++++---
>  fs/nfs/nfs4getroot.c        |   3 +
>  fs/nfs/nfs4proc.c           | 222 ++++++++++++++++++++++++++++++++++++++++++--
>  fs/nfs/nfs4xdr.c            |  38 +++-----
>  fs/nfs/pnfs_nfs.c           |  51 +++++++---
>  fs/nfs/super.c              |  26 ++++++
>  include/linux/nfs_xdr.h     |   4 +-
>  include/linux/sunrpc/clnt.h |   2 +-
>  net/sunrpc/clnt.c           |   6 +-
>  net/sunrpc/sched.c          |   2 +-
>  net/sunrpc/xprtmultipath.c  |   4 +
>  13 files changed, 406 insertions(+), 65 deletions(-)
>
> --
> 1.8.3.1
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html