[PATCH 00/11] NFS: Support binding to source address

[PATCH 00/11] NFS: Support binding to source address

[greearb]

From: Ben Greear <greearb@candelatech.com>

This lets one specify the source IP address for
sockets, allowing users to leverage routing rules
on multi-homed systems.

Patches are also needed to nfs-utils.  These have been
posted and will be accepted if the kernel support is
accepted.

Ben Greear (11):
  sunrpc:  Don't attempt to bind to AF_UNSPEC address.
  nfs:  Two AF_UNSPEC addresses should always match each other.
  nfs:  Add srcaddr member to nfs_client.
  nfs:  Use request destination addr as callback source addr.
  nfs:  Pay attention to srcaddr in v4.1 callback logic.
  nfs:  Use srcaddr in nfs_match_client.
  nfs:  Add srcaddr to /proc/fs/nfsfs/servers
  nfs:  Pass srcaddr into mount request.
  nfs:  Propagate src-addr in client code.
  nfs: Bind to srcaddr in rpcb_create.
  nfs: Support srcaddr= to bind to specific IP address.

 fs/nfs/callback.h          |    3 ++
 fs/nfs/callback_proc.c     |    3 +-
 fs/nfs/callback_xdr.c      |    3 ++
 fs/nfs/client.c            |   72 +++++++++++++++++++++++++++++++++++++++++--
 fs/nfs/internal.h          |   10 +++++-
 fs/nfs/mount_clnt.c        |    1 +
 fs/nfs/super.c             |   33 +++++++++++++++++++-
 include/linux/nfs_fs_sb.h  |    4 ++
 include/linux/sunrpc/svc.h |    5 +++
 net/sunrpc/rpcb_clnt.c     |   28 ++++++++++++++++-
 net/sunrpc/sunrpc.h        |   45 +++++++++++++++++++++++++++
 net/sunrpc/xprtsock.c      |   47 +---------------------------
 12 files changed, 200 insertions(+), 54 deletions(-)

-- 
1.7.3.4

[PATCH 01/11] sunrpc: Don't attempt to bind to AF_UNSPEC address.

[greearb]

From: Ben Greear <greearb@candelatech.com>

This lets calling code simply pass AF_UNSPEC addresses
instead of adding lots of checks to conditionally pass
NULL addresses.

Signed-off-by: Ben Greear <greearb@candelatech.com>
---
:100644 100644 72abb73... 37299c0... M	net/sunrpc/xprtsock.c
 net/sunrpc/xprtsock.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c
index 72abb73..37299c0 100644
--- a/net/sunrpc/xprtsock.c
+++ b/net/sunrpc/xprtsock.c
@@ -2509,9 +2509,9 @@ static struct rpc_xprt *xs_setup_xprt(struct xprt_create *args,
 	new = container_of(xprt, struct sock_xprt, xprt);
 	memcpy(&xprt->addr, args->dstaddr, args->addrlen);
 	xprt->addrlen = args->addrlen;
-	if (args->srcaddr)
+	if (args->srcaddr && (args->srcaddr->sa_family != AF_UNSPEC)) {
 		memcpy(&new->srcaddr, args->srcaddr, args->addrlen);
-	else {
+	} else {
 		int err;
 		err = xs_init_anyaddr(args->dstaddr->sa_family,
 					(struct sockaddr *)&new->srcaddr);
-- 
1.7.3.4

[PATCH 02/11] nfs: Two AF_UNSPEC addresses should always match each other.

[greearb]

From: Ben Greear <greearb@candelatech.com>

Signed-off-by: Ben Greear <greearb@candelatech.com>
---
:100644 100644 b3dc2b8... 8ba7e1a... M	fs/nfs/client.c
 fs/nfs/client.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index b3dc2b8..8ba7e1a 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -403,6 +403,8 @@ static int nfs_sockaddr_match_ipaddr(const struct sockaddr *sa1,
 		return nfs_sockaddr_match_ipaddr4(sa1, sa2);
 	case AF_INET6:
 		return nfs_sockaddr_match_ipaddr6(sa1, sa2);
+	case AF_UNSPEC:
+		return 1; /* two UNSPEC addresses match */
 	}
 	return 0;
 }
-- 
1.7.3.4

[PATCH 03/11] nfs: Add srcaddr member to nfs_client.

[greearb]

From: Ben Greear <greearb@candelatech.com>


Signed-off-by: Ben Greear <greearb@candelatech.com>
---
:100644 100644 2a55347... 93979a9... M	fs/nfs/internal.h
:100644 100644 ce40e5c... cb41131... M	fs/nfs/super.c
:100644 100644 87694ca... 9db4473... M	include/linux/nfs_fs_sb.h
 fs/nfs/internal.h         |    5 +++++
 fs/nfs/super.c            |    2 ++
 include/linux/nfs_fs_sb.h |    4 ++++
 3 files changed, 11 insertions(+), 0 deletions(-)

diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
index 2a55347..93979a9 100644
--- a/fs/nfs/internal.h
+++ b/fs/nfs/internal.h
@@ -107,6 +107,11 @@ struct nfs_parsed_mount_data {
 	struct {
 		struct sockaddr_storage	address;
 		size_t			addrlen;
+	} srcaddr;
+
+	struct {
+		struct sockaddr_storage address;
+		size_t                  addrlen;
 		char			*hostname;
 		u32			version;
 		int			port;
diff --git a/fs/nfs/super.c b/fs/nfs/super.c
index ce40e5c..cb41131 100644
--- a/fs/nfs/super.c
+++ b/fs/nfs/super.c
@@ -904,6 +904,8 @@ static struct nfs_parsed_mount_data *nfs_alloc_parsed_mount_data(unsigned int ve
 		data->auth_flavor_len	= 1;
 		data->version		= version;
 		data->minorversion	= 0;
+		data->srcaddr.address.ss_family = AF_UNSPEC;
+		data->srcaddr.addrlen = sizeof(data->srcaddr.address);
 	}
 	return data;
 }
diff --git a/include/linux/nfs_fs_sb.h b/include/linux/nfs_fs_sb.h
index 87694ca..9db4473 100644
--- a/include/linux/nfs_fs_sb.h
+++ b/include/linux/nfs_fs_sb.h
@@ -80,6 +80,10 @@ struct nfs_client {
 	struct list_head	cl_layouts;
 #endif /* CONFIG_NFS_V4 */
 
+	/* If we should bind to a local IP, it should be specified below. */
+	struct sockaddr_storage	srcaddr;
+	size_t			srcaddrlen;
+
 #ifdef CONFIG_NFS_FSCACHE
 	struct fscache_cookie	*fscache;	/* client index cache cookie */
 #endif
-- 
1.7.3.4

[PATCH 04/11] nfs: Use request destination addr as callback source addr.

[greearb]

From: Ben Greear <greearb@candelatech.com>

This helps make sure that responses use the correct
source address on multi-homed machines.

Signed-off-by: Ben Greear <greearb@candelatech.com>
---
:100644 100644 b257383... 00902ec... M	fs/nfs/callback.h
:100644 100644 c6c86a7... ed5be89... M	fs/nfs/callback_xdr.c
:100644 100644 ea29330... 45d5d7c... M	include/linux/sunrpc/svc.h
 fs/nfs/callback.h          |    3 +++
 fs/nfs/callback_xdr.c      |    3 +++
 include/linux/sunrpc/svc.h |    5 +++++
 3 files changed, 11 insertions(+), 0 deletions(-)

diff --git a/fs/nfs/callback.h b/fs/nfs/callback.h
index b257383..00902ec 100644
--- a/fs/nfs/callback.h
+++ b/fs/nfs/callback.h
@@ -57,6 +57,7 @@ struct cb_compound_hdr_res {
 
 struct cb_getattrargs {
 	struct sockaddr *addr;
+	struct sockaddr *srcaddr;
 	struct nfs_fh fh;
 	uint32_t bitmap[2];
 };
@@ -72,6 +73,7 @@ struct cb_getattrres {
 
 struct cb_recallargs {
 	struct sockaddr *addr;
+	struct sockaddr *srcaddr;
 	struct nfs_fh fh;
 	nfs4_stateid stateid;
 	uint32_t truncate;
@@ -92,6 +94,7 @@ struct referring_call_list {
 
 struct cb_sequenceargs {
 	struct sockaddr			*csa_addr;
+	struct sockaddr                 *csa_daddr;
 	struct nfs4_sessionid		csa_sessionid;
 	uint32_t			csa_sequenceid;
 	uint32_t			csa_slotid;
diff --git a/fs/nfs/callback_xdr.c b/fs/nfs/callback_xdr.c
index c6c86a7..ed5be89 100644
--- a/fs/nfs/callback_xdr.c
+++ b/fs/nfs/callback_xdr.c
@@ -196,6 +196,7 @@ static __be32 decode_getattr_args(struct svc_rqst *rqstp, struct xdr_stream *xdr
 	if (unlikely(status != 0))
 		goto out;
 	args->addr = svc_addr(rqstp);
+	args->srcaddr = svc_daddr(rqstp);
 	status = decode_bitmap(xdr, args->bitmap);
 out:
 	dprintk("%s: exit with status = %d\n", __func__, ntohl(status));
@@ -208,6 +209,7 @@ static __be32 decode_recall_args(struct svc_rqst *rqstp, struct xdr_stream *xdr,
 	__be32 status;
 
 	args->addr = svc_addr(rqstp);
+	args->srcaddr = svc_daddr(rqstp);
 	status = decode_stateid(xdr, &args->stateid);
 	if (unlikely(status != 0))
 		goto out;
@@ -442,6 +444,7 @@ static __be32 decode_cb_sequence_args(struct svc_rqst *rqstp,
 		goto out;
 
 	args->csa_addr = svc_addr(rqstp);
+	args->csa_daddr = svc_daddr(rqstp);
 	args->csa_sequenceid = ntohl(*p++);
 	args->csa_slotid = ntohl(*p++);
 	args->csa_highestslotid = ntohl(*p++);
diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h
index ea29330..45d5d7c 100644
--- a/include/linux/sunrpc/svc.h
+++ b/include/linux/sunrpc/svc.h
@@ -299,6 +299,11 @@ static inline struct sockaddr *svc_addr(const struct svc_rqst *rqst)
 	return (struct sockaddr *) &rqst->rq_addr;
 }
 
+static inline struct sockaddr *svc_daddr(struct svc_rqst *rqst)
+{
+	return (struct sockaddr *) &rqst->rq_daddr;
+}
+
 /*
  * Check buffer bounds after decoding arguments
  */
-- 
1.7.3.4

[PATCH 05/11] nfs: Pay attention to srcaddr in v4.1 callback logic.

[greearb]

From: Ben Greear <greearb@candelatech.com>

Signed-off-by: Ben Greear <greearb@candelatech.com>
---
:100644 100644 d4d1954... 5b20987... M	fs/nfs/callback_proc.c
:100644 100644 8ba7e1a... c571a97... M	fs/nfs/client.c
:100644 100644 93979a9... 60a36a2... M	fs/nfs/internal.h
 fs/nfs/callback_proc.c |    3 ++-
 fs/nfs/client.c        |   27 +++++++++++++++++++++++++--
 fs/nfs/internal.h      |    4 +++-
 3 files changed, 30 insertions(+), 4 deletions(-)

diff --git a/fs/nfs/callback_proc.c b/fs/nfs/callback_proc.c
index d4d1954..5b20987 100644
--- a/fs/nfs/callback_proc.c
+++ b/fs/nfs/callback_proc.c
@@ -424,7 +424,8 @@ __be32 nfs4_callback_sequence(struct cb_sequenceargs *args,
 
 	cps->clp = NULL;
 
-	clp = nfs4_find_client_sessionid(args->csa_addr, &args->csa_sessionid);
+	clp = nfs4_find_client_sessionid(args->csa_daddr, args->csa_addr,
+					 &args->csa_sessionid);
 	if (clp == NULL)
 		goto out;
 
diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index 8ba7e1a..c571a97 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -1233,10 +1233,12 @@ nfs4_find_client_ident(int cb_ident)
  * Returns NULL if no such client
  */
 struct nfs_client *
-nfs4_find_client_sessionid(const struct sockaddr *addr,
+nfs4_find_client_sessionid(const struct sockaddr *srcaddr,
+			   const struct sockaddr *addr,
 			   struct nfs4_sessionid *sid)
 {
 	struct nfs_client *clp;
+	struct nfs_client *ok_fit = NULL;
 
 	spin_lock(&nfs_client_lock);
 	list_for_each_entry(clp, &nfs_client_list, cl_share_link) {
@@ -1251,10 +1253,30 @@ nfs4_find_client_sessionid(const struct sockaddr *addr,
 		    sid->data, NFS4_MAX_SESSIONID_LEN) != 0)
 			continue;
 
+		if (srcaddr) {
+			const struct sockaddr *sa;
+			sa = (const struct sockaddr *)&clp->cl_addr;
+			if (!nfs_sockaddr_match_ipaddr(srcaddr, sa)) {
+				/* If clp doesn't bind to srcaddr, then
+				 * it is a potential match if we don't find
+				 * a better one.
+				 */
+				if (sa->sa_family == AF_UNSPEC && !ok_fit)
+					ok_fit = clp;
+				continue;
+			}
+		}
+found_one:
 		atomic_inc(&clp->cl_count);
 		spin_unlock(&nfs_client_lock);
 		return clp;
 	}
+
+	if (ok_fit) {
+		clp = ok_fit;
+		goto found_one;
+	}
+
 	spin_unlock(&nfs_client_lock);
 	return NULL;
 }
@@ -1262,7 +1284,8 @@ nfs4_find_client_sessionid(const struct sockaddr *addr,
 #else /* CONFIG_NFS_V4_1 */
 
 struct nfs_client *
-nfs4_find_client_sessionid(const struct sockaddr *addr,
+nfs4_find_client_sessionid(const struct sockaddr *srcaddr,
+			   const struct sockaddr *addr,
 			   struct nfs4_sessionid *sid)
 {
 	return NULL;
diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
index 93979a9..60a36a2 100644
--- a/fs/nfs/internal.h
+++ b/fs/nfs/internal.h
@@ -155,7 +155,9 @@ extern void nfs_put_client(struct nfs_client *);
 extern struct nfs_client *nfs4_find_client_no_ident(const struct sockaddr *);
 extern struct nfs_client *nfs4_find_client_ident(int);
 extern struct nfs_client *
-nfs4_find_client_sessionid(const struct sockaddr *, struct nfs4_sessionid *);
+nfs4_find_client_sessionid(const struct sockaddr *srcaddr,
+			   const struct sockaddr *addr,
+			   struct nfs4_sessionid *);
 extern struct nfs_server *nfs_create_server(
 					const struct nfs_parsed_mount_data *,
 					struct nfs_fh *);
-- 
1.7.3.4

[PATCH 06/11] nfs: Use srcaddr in nfs_match_client.

[greearb]

From: Ben Greear <greearb@candelatech.com>

For bound clients, we have to make sure we do not re-use
a client that is bound differently.

Signed-off-by: Ben Greear <greearb@candelatech.com>
---
:100644 100644 c571a97... 5d8998a... M	fs/nfs/client.c
 fs/nfs/client.c |   12 ++++++++++++
 1 files changed, 12 insertions(+), 0 deletions(-)

diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index c571a97..5d8998a 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -131,7 +131,9 @@ struct rpc_program		nfsacl_program = {
 struct nfs_client_initdata {
 	const char *hostname;
 	const struct sockaddr *addr;
+	const struct sockaddr *srcaddr;
 	size_t addrlen;
+	size_t srcaddrlen;
 	const struct nfs_rpc_ops *rpc_ops;
 	int proto;
 	u32 minorversion;
@@ -463,6 +465,9 @@ static struct nfs_client *nfs_match_client(const struct nfs_client_initdata *dat
 
 	list_for_each_entry(clp, &nfs_client_list, cl_share_link) {
 	        const struct sockaddr *clap = (struct sockaddr *)&clp->cl_addr;
+		const struct sockaddr *sa;
+		sa = (const struct sockaddr *)&clp->srcaddr;
+
 		/* Don't match clients that failed to initialise properly */
 		if (clp->cl_cons_state < 0)
 			continue;
@@ -480,6 +485,13 @@ static struct nfs_client *nfs_match_client(const struct nfs_client_initdata *dat
 		if (!nfs_sockaddr_cmp(sap, clap))
 			continue;
 
+		/* Check to make sure local-IP bindings match,
+		 * but just the IP-addr.
+		 */
+		if (data->srcaddr &&
+		    !nfs_sockaddr_match_ipaddr(data->srcaddr, sa))
+			continue;
+
 		atomic_inc(&clp->cl_count);
 		return clp;
 	}
-- 
1.7.3.4

[PATCH 07/11] nfs: Add srcaddr to /proc/fs/nfsfs/servers

[greearb]

From: Ben Greear <greearb@candelatech.com>

Helps users understand whether a server instance is
bound to a local IP or not.

Signed-off-by: Ben Greear <greearb@candelatech.com>
---
:100644 100644 5d8998a... 223785f... M	fs/nfs/client.c
 fs/nfs/client.c |   18 ++++++++++++++++--
 1 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index 5d8998a..223785f 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -1895,20 +1895,34 @@ static int nfs_server_list_show(struct seq_file *m, void *v)
 
 	/* display header on line 1 */
 	if (v == &nfs_client_list) {
-		seq_puts(m, "NV SERVER   PORT USE HOSTNAME\n");
+		seq_puts(m, "NV SERVER   PORT USE HOSTNAME"
+			 "           SRCADDR\n");
 		return 0;
 	}
 
 	/* display one transport per line on subsequent lines */
 	clp = list_entry(v, struct nfs_client, cl_share_link);
 
-	seq_printf(m, "v%u %s %s %3d %s\n",
+	seq_printf(m, "v%u %s %s %3d %s",
 		   clp->rpc_ops->version,
 		   rpc_peeraddr2str(clp->cl_rpcclient, RPC_DISPLAY_HEX_ADDR),
 		   rpc_peeraddr2str(clp->cl_rpcclient, RPC_DISPLAY_HEX_PORT),
 		   atomic_read(&clp->cl_count),
 		   clp->cl_hostname);
 
+	if (clp->srcaddr.ss_family == AF_INET) {
+		const struct sockaddr_in *sin;
+		sin = (const struct sockaddr_in *)&clp->srcaddr;
+		seq_printf(m, "   %pI4\n", &sin->sin_addr.s_addr);
+	} else if (clp->srcaddr.ss_family == AF_INET6) {
+		const struct sockaddr_in6 *sin6;
+		sin6 = (const struct sockaddr_in6 *)&clp->srcaddr;
+		seq_printf(m, "   %pI6c\n", &sin6->sin6_addr);
+	} else if (clp->srcaddr.ss_family == AF_UNSPEC)
+		seq_printf(m, "   ANY\n");
+	else
+		seq_printf(m, "   UNKNOWN_%i\n", (int)(clp->srcaddr.ss_family));
+
 	return 0;
 }
 
-- 
1.7.3.4

[PATCH 08/11] nfs: Pass srcaddr into mount request.

[greearb]

From: Ben Greear <greearb@candelatech.com>

This starts tying the srcaddr logic together.

Signed-off-by: Ben Greear <greearb@candelatech.com>
---
:100644 100644 60a36a2... eaf9c66... M	fs/nfs/internal.h
:100644 100644 d4c2d6b... 5ca6e12... M	fs/nfs/mount_clnt.c
:100644 100644 cb41131... e45f616... M	fs/nfs/super.c
 fs/nfs/internal.h   |    1 +
 fs/nfs/mount_clnt.c |    1 +
 fs/nfs/super.c      |    2 ++
 3 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
index 60a36a2..eaf9c66 100644
--- a/fs/nfs/internal.h
+++ b/fs/nfs/internal.h
@@ -133,6 +133,7 @@ struct nfs_parsed_mount_data {
 /* mount_clnt.c */
 struct nfs_mount_request {
 	struct sockaddr		*sap;
+	struct sockaddr         *srcaddr;
 	size_t			salen;
 	char			*hostname;
 	char			*dirpath;
diff --git a/fs/nfs/mount_clnt.c b/fs/nfs/mount_clnt.c
index d4c2d6b..5ca6e12 100644
--- a/fs/nfs/mount_clnt.c
+++ b/fs/nfs/mount_clnt.c
@@ -155,6 +155,7 @@ int nfs_mount(struct nfs_mount_request *info)
 	struct rpc_create_args args = {
 		.net		= &init_net,
 		.protocol	= info->protocol,
+		.saddress	= info->srcaddr,
 		.address	= info->sap,
 		.addrsize	= info->salen,
 		.servername	= info->hostname,
diff --git a/fs/nfs/super.c b/fs/nfs/super.c
index cb41131..e45f616 100644
--- a/fs/nfs/super.c
+++ b/fs/nfs/super.c
@@ -1600,6 +1600,8 @@ static int nfs_try_mount(struct nfs_parsed_mount_data *args,
 	struct nfs_mount_request request = {
 		.sap		= (struct sockaddr *)
 						&args->mount_server.address,
+		.salen		= args->mount_server.addrlen,
+		.srcaddr	= (struct sockaddr *)&args->srcaddr.address,
 		.dirpath	= args->nfs_server.export_path,
 		.protocol	= args->mount_server.protocol,
 		.fh		= root_fh,
-- 
1.7.3.4

[PATCH 09/11] nfs: Propagate src-addr in client code.

[greearb]

From: Ben Greear <greearb@candelatech.com>

This ties the srcaddr supplied by mount to the
rpc code.

Signed-off-by: Ben Greear <greearb@candelatech.com>
---
:100644 100644 223785f... 8b8766a... M	fs/nfs/client.c
 fs/nfs/client.c |   13 +++++++++++++
 1 files changed, 13 insertions(+), 0 deletions(-)

diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index 223785f..8b8766a 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -161,6 +161,8 @@ static struct nfs_client *nfs_alloc_client(const struct nfs_client_initdata *cl_
 
 	memcpy(&clp->cl_addr, cl_init->addr, cl_init->addrlen);
 	clp->cl_addrlen = cl_init->addrlen;
+	memcpy(&clp->srcaddr, cl_init->srcaddr, cl_init->srcaddrlen);
+	clp->srcaddrlen = cl_init->srcaddrlen;
 
 	if (cl_init->hostname) {
 		err = -ENOMEM;
@@ -655,6 +657,7 @@ static int nfs_create_rpc_client(struct nfs_client *clp,
 		.net		= &init_net,
 		.protocol	= clp->cl_proto,
 		.address	= (struct sockaddr *)&clp->cl_addr,
+		.saddress	= (struct sockaddr *)&clp->srcaddr,
 		.addrsize	= clp->cl_addrlen,
 		.timeout	= timeparms,
 		.servername	= clp->cl_hostname,
@@ -839,6 +842,8 @@ static int nfs_init_server(struct nfs_server *server,
 		.hostname = data->nfs_server.hostname,
 		.addr = (const struct sockaddr *)&data->nfs_server.address,
 		.addrlen = data->nfs_server.addrlen,
+		.srcaddr = (const struct sockaddr *)&data->srcaddr.address,
+		.srcaddrlen = data->srcaddr.addrlen,
 		.rpc_ops = &nfs_v2_clientops,
 		.proto = data->nfs_server.protocol,
 	};
@@ -1419,6 +1424,8 @@ static int nfs4_set_client(struct nfs_server *server,
 		const struct sockaddr *addr,
 		const size_t addrlen,
 		const char *ip_addr,
+		const struct sockaddr *srcaddr,
+		const size_t srcaddrlen,
 		rpc_authflavor_t authflavour,
 		int proto, const struct rpc_timeout *timeparms,
 		u32 minorversion)
@@ -1430,6 +1437,8 @@ static int nfs4_set_client(struct nfs_server *server,
 		.rpc_ops = &nfs_v4_clientops,
 		.proto = proto,
 		.minorversion = minorversion,
+		.srcaddr = srcaddr,
+		.srcaddrlen = srcaddrlen,
 	};
 	struct nfs_client *clp;
 	int error;
@@ -1604,6 +1613,8 @@ static int nfs4_init_server(struct nfs_server *server,
 			(const struct sockaddr *)&data->nfs_server.address,
 			data->nfs_server.addrlen,
 			data->client_address,
+			(const struct sockaddr *)&data->srcaddr.address,
+			data->srcaddr.addrlen,
 			data->auth_flavors[0],
 			data->nfs_server.protocol,
 			&timeparms,
@@ -1701,6 +1712,8 @@ struct nfs_server *nfs4_create_referral_server(struct nfs_clone_mount *data,
 				data->addr,
 				data->addrlen,
 				parent_client->cl_ipaddr,
+				(const struct sockaddr *)&parent_client->srcaddr,
+				parent_client->srcaddrlen,
 				data->authflavor,
 				parent_server->client->cl_xprt->prot,
 				parent_server->client->cl_timeout,
-- 
1.7.3.4

[PATCH 10/11] nfs: Bind to srcaddr in rpcb_create.

[greearb]

From: Ben Greear <greearb@candelatech.com>

This more properly clones the existing client connection
settings, allowing rpc calls to take advantage of routing
rules based on source IP addresses.

Signed-off-by: Ben Greear <greearb@candelatech.com>
---
:100644 100644 9a80a92... 4985e18... M	net/sunrpc/rpcb_clnt.c
:100644 100644 90c292e... 22841e1... M	net/sunrpc/sunrpc.h
:100644 100644 37299c0... 72408c2... M	net/sunrpc/xprtsock.c
 net/sunrpc/rpcb_clnt.c |   28 ++++++++++++++++++++++++++--
 net/sunrpc/sunrpc.h    |   45 +++++++++++++++++++++++++++++++++++++++++++++
 net/sunrpc/xprtsock.c  |   43 -------------------------------------------
 3 files changed, 71 insertions(+), 45 deletions(-)

diff --git a/net/sunrpc/rpcb_clnt.c b/net/sunrpc/rpcb_clnt.c
index 9a80a92..4985e18 100644
--- a/net/sunrpc/rpcb_clnt.c
+++ b/net/sunrpc/rpcb_clnt.c
@@ -28,6 +28,7 @@
 #include <linux/sunrpc/clnt.h>
 #include <linux/sunrpc/sched.h>
 #include <linux/sunrpc/xprtsock.h>
+#include "sunrpc.h"
 
 #ifdef RPC_DEBUG
 # define RPCDBG_FACILITY	RPCDBG_BIND
@@ -292,11 +293,13 @@ out:
 }
 
 static struct rpc_clnt *rpcb_create(char *hostname, struct sockaddr *srvaddr,
-				    size_t salen, int proto, u32 version)
+				    size_t salen, int proto, u32 version,
+				    struct sockaddr *srcaddr)
 {
 	struct rpc_create_args args = {
 		.net		= &init_net,
 		.protocol	= proto,
+		.saddress	= srcaddr,
 		.address	= srvaddr,
 		.addrsize	= salen,
 		.servername	= hostname,
@@ -601,11 +604,32 @@ void rpcb_getport_async(struct rpc_task *task)
 	struct rpc_task	*child;
 	struct sockaddr_storage addr;
 	struct sockaddr *sap = (struct sockaddr *)&addr;
+	struct sockaddr_storage srcaddr;
+	struct sockaddr *srcaddra = NULL;
 	size_t salen;
 	int status;
+	struct sock_xprt *sxprt;
 
 	clnt = rpcb_find_transport_owner(task->tk_client);
 	xprt = clnt->cl_xprt;
+	sxprt = container_of(xprt, struct sock_xprt, xprt);
+
+	/* We just want to bind to the IP, not the port */
+	memset(&srcaddr, 0, sizeof(srcaddr));
+	srcaddr.ss_family = sxprt->srcaddr.ss_family;
+	if (sxprt->srcaddr.ss_family == AF_INET) {
+		struct sockaddr_in *si = (struct sockaddr_in *)(&srcaddr);
+		struct sockaddr_in *si2;
+		si2 = (struct sockaddr_in *)(&sxprt->srcaddr);
+		si->sin_addr.s_addr = si2->sin_addr.s_addr;
+		srcaddra = (struct sockaddr *)(&srcaddr);
+	} else if (sxprt->srcaddr.ss_family == AF_INET6) {
+		struct sockaddr_in6 *si = (struct sockaddr_in6 *)(&srcaddr);
+		struct sockaddr_in6 *si2;
+		si2 = (struct sockaddr_in6 *)(&sxprt->srcaddr);
+		memcpy(&si->sin6_addr, &si2->sin6_addr, sizeof(si2->sin6_addr));
+		srcaddra = (struct sockaddr *)(&srcaddr);
+	}
 
 	dprintk("RPC: %5u %s(%s, %u, %u, %d)\n",
 		task->tk_pid, __func__,
@@ -660,7 +684,7 @@ void rpcb_getport_async(struct rpc_task *task)
 		task->tk_pid, __func__, bind_version);
 
 	rpcb_clnt = rpcb_create(clnt->cl_server, sap, salen, xprt->prot,
-				bind_version);
+				bind_version, srcaddra);
 	if (IS_ERR(rpcb_clnt)) {
 		status = PTR_ERR(rpcb_clnt);
 		dprintk("RPC: %5u %s: rpcb_create failed, error %ld\n",
diff --git a/net/sunrpc/sunrpc.h b/net/sunrpc/sunrpc.h
index 90c292e..22841e1 100644
--- a/net/sunrpc/sunrpc.h
+++ b/net/sunrpc/sunrpc.h
@@ -37,6 +37,51 @@ struct rpc_buffer {
 	char	data[];
 };
 
+struct sock_xprt {
+	struct rpc_xprt		xprt;
+
+	/*
+	 * Network layer
+	 */
+	struct socket		*sock;
+	struct sock		*inet;
+
+	/*
+	 * State of TCP reply receive
+	 */
+	__be32			tcp_fraghdr,
+				tcp_xid,
+				tcp_calldir;
+
+	u32			tcp_offset,
+				tcp_reclen;
+
+	unsigned long		tcp_copied,
+				tcp_flags;
+
+	/*
+	 * Connection of transports
+	 */
+	struct delayed_work	connect_worker;
+	struct sockaddr_storage	srcaddr;
+	unsigned short		srcport;
+
+	/*
+	 * UDP socket buffer size parameters
+	 */
+	size_t			rcvsize,
+				sndsize;
+
+	/*
+	 * Saved socket callback addresses
+	 */
+	void			(*old_data_ready)(struct sock *, int);
+	void			(*old_state_change)(struct sock *);
+	void			(*old_write_space)(struct sock *);
+	void			(*old_error_report)(struct sock *);
+};
+
+
 static inline int rpc_reply_expected(struct rpc_task *task)
 {
 	return (task->tk_msg.rpc_proc != NULL) &&
diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c
index 37299c0..72408c2 100644
--- a/net/sunrpc/xprtsock.c
+++ b/net/sunrpc/xprtsock.c
@@ -202,49 +202,6 @@ static inline void xs_pktdump(char *msg, u32 *packet, unsigned int count)
 }
 #endif
 
-struct sock_xprt {
-	struct rpc_xprt		xprt;
-
-	/*
-	 * Network layer
-	 */
-	struct socket *		sock;
-	struct sock *		inet;
-
-	/*
-	 * State of TCP reply receive
-	 */
-	__be32			tcp_fraghdr,
-				tcp_xid,
-				tcp_calldir;
-
-	u32			tcp_offset,
-				tcp_reclen;
-
-	unsigned long		tcp_copied,
-				tcp_flags;
-
-	/*
-	 * Connection of transports
-	 */
-	struct delayed_work	connect_worker;
-	struct sockaddr_storage	srcaddr;
-	unsigned short		srcport;
-
-	/*
-	 * UDP socket buffer size parameters
-	 */
-	size_t			rcvsize,
-				sndsize;
-
-	/*
-	 * Saved socket callback addresses
-	 */
-	void			(*old_data_ready)(struct sock *, int);
-	void			(*old_state_change)(struct sock *);
-	void			(*old_write_space)(struct sock *);
-	void			(*old_error_report)(struct sock *);
-};
 
 /*
  * TCP receive state flags
-- 
1.7.3.4

[PATCH 11/11] nfs: Support srcaddr= to bind to specific IP address.

[greearb]

From: Ben Greear <greearb@candelatech.com>

In order to have more control on multi-homed machines, it is
nice to be able to bind to a specific IP address.  This can aid
with interface selection, policy based routing, multiple unique
mounts to the same server, and similar things.

This patch allows srcaddr= option for NFS.  The key 'srcaddr'
was chosen to match the similar patch for cifs.

For NFSv4, if one is specifying clientaddr, it must be the same as
srcaddr or things may not work properly.

NFSv3, NFSv4 over TCP/IPv4 and TCP/IPv6 has been successfully tested.

Usage:
  mount -t nfs4 [2002::1]:/rpool/ben /mnt/foo/ben-1 -o srcaddr=2002::2,clientaddr=2002::2
  mount -t nfs4 192.168.100.3:/foo /mnt/foo/ben-2 -o srcaddr=192.168.100.174

Signed-off-by: Ben Greear <greearb@candelatech.com>
---
:100644 100644 e45f616... 5d88e7b... M	fs/nfs/super.c
 fs/nfs/super.c |   29 ++++++++++++++++++++++++++++-
 1 files changed, 28 insertions(+), 1 deletions(-)

diff --git a/fs/nfs/super.c b/fs/nfs/super.c
index e45f616..5d88e7b 100644
--- a/fs/nfs/super.c
+++ b/fs/nfs/super.c
@@ -103,7 +103,7 @@ enum {
 
 	/* Mount options that take string arguments */
 	Opt_sec, Opt_proto, Opt_mountproto, Opt_mounthost,
-	Opt_addr, Opt_mountaddr, Opt_clientaddr,
+	Opt_addr, Opt_mountaddr, Opt_clientaddr, Opt_srcaddr,
 	Opt_lookupcache,
 	Opt_fscache_uniq,
 	Opt_local_lock,
@@ -173,6 +173,7 @@ static const match_table_t nfs_mount_option_tokens = {
 	{ Opt_mountproto, "mountproto=%s" },
 	{ Opt_addr, "addr=%s" },
 	{ Opt_clientaddr, "clientaddr=%s" },
+	{ Opt_srcaddr, "srcaddr=%s" },
 	{ Opt_mounthost, "mounthost=%s" },
 	{ Opt_mountaddr, "mountaddr=%s" },
 
@@ -695,6 +696,15 @@ static void nfs_show_mount_options(struct seq_file *m, struct nfs_server *nfss,
 	else
 		nfs_show_nfsv4_options(m, nfss, showdefaults);
 
+	if (clp->srcaddr.ss_family == AF_INET6) {
+		struct sockaddr_in6 *sin6;
+		sin6 = (struct sockaddr_in6 *)(&clp->srcaddr);
+		seq_printf(m, ",srcaddr=%pI6c", &sin6->sin6_addr);
+	} else if (clp->srcaddr.ss_family == AF_INET) {
+		struct sockaddr_in *sin = (struct sockaddr_in *)&clp->srcaddr;
+		seq_printf(m, ",srcaddr=%pI4", &sin->sin_addr.s_addr);
+	}
+
 	if (nfss->options & NFS_OPTION_FSCACHE)
 		seq_printf(m, ",fsc");
 
@@ -1448,6 +1458,23 @@ static int nfs_parse_mount_options(char *raw,
 				goto out_nomem;
 			mnt->options |= NFS_OPTION_FSCACHE;
 			break;
+		case Opt_srcaddr:
+			string = match_strdup(args);
+			if (string == NULL)
+				goto out_nomem;
+			mnt->srcaddr.addrlen =
+				rpc_pton(string, strlen(string),
+					 (struct sockaddr *)
+					 &mnt->srcaddr.address,
+					 sizeof(mnt->srcaddr.address));
+			kfree(string);
+			if (mnt->srcaddr.addrlen == 0) {
+				printk(KERN_WARNING
+				       "nfs: Could not parse srcaddr: %s\n",
+				       string);
+				goto out_invalid_address;
+			}
+			break;
 		case Opt_local_lock:
 			string = match_strdup(args);
 			if (string == NULL)
-- 
1.7.3.4

Re: [PATCH 00/11] NFS: Support binding to source address

[Chuck Lever]

On Jun 22, 2011, at 3:36 PM, greearb@candelatech.com wrote:

> From: Ben Greear <greearb@candelatech.com>
> 
> This lets one specify the source IP address for
> sockets, allowing users to leverage routing rules
> on multi-homed systems.

Do you need changes to send NLM requests from the right source address?

> Patches are also needed to nfs-utils.  These have been
> posted and will be accepted if the kernel support is
> accepted.
> 
> Ben Greear (11):
>  sunrpc:  Don't attempt to bind to AF_UNSPEC address.
>  nfs:  Two AF_UNSPEC addresses should always match each other.
>  nfs:  Add srcaddr member to nfs_client.
>  nfs:  Use request destination addr as callback source addr.
>  nfs:  Pay attention to srcaddr in v4.1 callback logic.
>  nfs:  Use srcaddr in nfs_match_client.
>  nfs:  Add srcaddr to /proc/fs/nfsfs/servers
>  nfs:  Pass srcaddr into mount request.
>  nfs:  Propagate src-addr in client code.
>  nfs: Bind to srcaddr in rpcb_create.
>  nfs: Support srcaddr= to bind to specific IP address.
> 
> fs/nfs/callback.h          |    3 ++
> fs/nfs/callback_proc.c     |    3 +-
> fs/nfs/callback_xdr.c      |    3 ++
> fs/nfs/client.c            |   72 +++++++++++++++++++++++++++++++++++++++++--
> fs/nfs/internal.h          |   10 +++++-
> fs/nfs/mount_clnt.c        |    1 +
> fs/nfs/super.c             |   33 +++++++++++++++++++-
> include/linux/nfs_fs_sb.h  |    4 ++
> include/linux/sunrpc/svc.h |    5 +++
> net/sunrpc/rpcb_clnt.c     |   28 ++++++++++++++++-
> net/sunrpc/sunrpc.h        |   45 +++++++++++++++++++++++++++
> net/sunrpc/xprtsock.c      |   47 +---------------------------
> 12 files changed, 200 insertions(+), 54 deletions(-)
> 
> -- 
> 1.7.3.4
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Re: [PATCH 00/11] NFS: Support binding to source address

[Ben Greear]

On 06/23/2011 07:59 AM, Chuck Lever wrote:
>
> On Jun 22, 2011, at 3:36 PM, greearb@candelatech.com wrote:
>
>> From: Ben Greear<greearb@candelatech.com>
>>
>> This lets one specify the source IP address for
>> sockets, allowing users to leverage routing rules
>> on multi-homed systems.
>
> Do you need changes to send NLM requests from the right source address?

Not for my testing so far...but maybe I'm not utilizing whatever NLM
does.  What sort of things would require/use it?

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: [PATCH 00/11] NFS: Support binding to source address

[Chuck Lever]

Hi Ben-

On Jun 23, 2011, at 9:29 AM, Ben Greear wrote:

> On 06/23/2011 07:59 AM, Chuck Lever wrote:
>> 
>> On Jun 22, 2011, at 3:36 PM, greearb@candelatech.com wrote:
>> 
>>> From: Ben Greear<greearb@candelatech.com>
>>> 
>>> This lets one specify the source IP address for
>>> sockets, allowing users to leverage routing rules
>>> on multi-homed systems.
>> 
>> Do you need changes to send NLM requests from the right source address?
> 
> Not for my testing so far...but maybe I'm not utilizing whatever NLM
> does.  What sort of things would require/use it?

File locking.  NLM support should be a requirement for a complete implementation of srcaddr=, IMO.

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Re: [PATCH 00/11] NFS: Support binding to source address

[Ben Greear]

On 06/23/2011 08:46 AM, Chuck Lever wrote:
> Hi Ben-
>
> On Jun 23, 2011, at 9:29 AM, Ben Greear wrote:
>
>> On 06/23/2011 07:59 AM, Chuck Lever wrote:
>>>
>>> On Jun 22, 2011, at 3:36 PM, greearb@candelatech.com wrote:
>>>
>>>> From: Ben Greear<greearb@candelatech.com>
>>>>
>>>> This lets one specify the source IP address for
>>>> sockets, allowing users to leverage routing rules
>>>> on multi-homed systems.
>>>
>>> Do you need changes to send NLM requests from the right source address?
>>
>> Not for my testing so far...but maybe I'm not utilizing whatever NLM
>> does.  What sort of things would require/use it?
>
> File locking.  NLM support should be a requirement for a complete implementation of srcaddr=, IMO.

Ok, I think I have some patches for this already, but didn't re-add them when
breaking up the big patch for submission.  I'll set up a test case with
file locking and double-check that I have it working correctly.

Thanks,
Ben

>
> --
> Chuck Lever
> chuck[dot]lever[at]oracle[dot]com
>
>


-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com