[PATCH 00/11] [RFC] possible NFSv2/v3 lock recovery enhancements

[PATCH 00/11] [RFC] possible NFSv2/v3 lock recovery enhancements

[Chuck Lever]

For the purpose of discussion, here are 11 WIP patches.  These have
seen very little testing, and are presented here as a proof of concept
only.

There are three main goals of this work:

1)  Provide an API for an NFSv2/v3 server administrator to release
    stale locks

2)  Provide an alternate API that doesn't depend on the existance of
    a particular network loopback interface to allow statd to perform
    NLM callbacks (ie statd won't have to care whether to use IPv4
    or IPv6 loopback to make NLM downcalls)

3)  Allow clients to send correct caller_name strings in
    NLMPROC_LOCK requests.

1 and 2 are met by implementing a set of directories under /sys that
expose the kernel's nlm_host and nsm_handle cache.  Each entry in the
nsm_handle cache has a reboot attribute that would allow a user space
process to trigger reboot recovery on the remote peer represented by
that entry.

3 is met by adding a new NFSv2/v3 mount option, similar to clientaddr=
for NFSv4 mounts, that allows the mount command to specify the
caller_name string to use for lock requests on this mount point.

There is still a chicken-and-egg type problem for statd.  These new
/sys files are owned and readable by only root.  The reboot attribute
is owned and writable by only root.  statd runs as "rpcuser" usually,
so unless something special is done, it wouldn't be able to use a /sys
file to trigger reboot recovery.

A possible way to address this is to add support for AF_LOCAL to the
kernel.  The kernel can create an RPC socket that statd can use to
perform downcalls.  AF_LOCAL can also be used in other cases, like
when the kernel has to contact rpcbind, so kernel AF_LOCAL support
wouldn't be a single-use type of thing.

---

Chuck Lever (11):
      lockd: Allow mount option to specify caller_name
      lockd: use sm_my_name for nsm_handle lookups
      lockd: Keep my_name in nsm_handle
      lockd: Add "reboot" attribute to nsm_handles
      lockd: Refactor nlm_host_rebooted()
      lockd: Add /sys/fs/lockd/nsm_handle/*
      lockd: Add /sys/fs/lockd/mon
      lockd: Add attributes to /sys/fs/lockd/hosts/*/
      lockd: Add /sys/fs/lockd/hosts/*
      lockd: Add /sys/fs/lockd/hosts/
      lockd: Add /sys/fs/lockd


 fs/lockd/clntlock.c         |    3 
 fs/lockd/clntproc.c         |    6 -
 fs/lockd/host.c             |  319 +++++++++++++++++++++++++++++++++++++++++--
 fs/lockd/mon.c              |  253 ++++++++++++++++++++++++++++++++--
 fs/lockd/svc.c              |   31 ++++
 fs/lockd/svclock.c          |    2 
 fs/nfs/client.c             |   11 +
 fs/nfs/internal.h           |    1 
 fs/nfs/super.c              |    9 +
 include/linux/lockd/bind.h  |    1 
 include/linux/lockd/lockd.h |   19 ++-
 include/linux/nfs_fs_sb.h   |    5 +
 12 files changed, 620 insertions(+), 40 deletions(-)

-- 
Chuck Lever

[PATCH 01/11] lockd: Add /sys/fs/lockd

[Chuck Lever]

Create the parent directory for all lockd-related sysfs entries.  The
directory is created when lockd.ko is loaded, and removed when it is
unloaded.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 fs/lockd/svc.c              |   16 +++++++++++++---
 include/linux/lockd/lockd.h |    1 +
 2 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/fs/lockd/svc.c b/fs/lockd/svc.c
index 7d15051..94e0d4b 100644
--- a/fs/lockd/svc.c
+++ b/fs/lockd/svc.c
@@ -76,6 +76,8 @@ static const int		nlm_port_min = 0, nlm_port_max = 65535;
 static struct ctl_table_header * nlm_sysctl_table;
 #endif
 
+struct kobject			*nlm_kobj;
+
 static unsigned long get_lockd_grace_period(void)
 {
 	/* Note: nlm_timeout should always be nonzero */
@@ -515,12 +517,19 @@ module_param(nlm_max_connections, uint, 0644);
 
 static int __init init_nlm(void)
 {
+	nlm_kobj = kobject_create_and_add("lockd", fs_kobj);
+	if (!nlm_kobj)
+		return -ENOMEM;
+
 #ifdef CONFIG_SYSCTL
 	nlm_sysctl_table = register_sysctl_table(nlm_sysctl_root);
-	return nlm_sysctl_table ? 0 : -ENOMEM;
-#else
-	return 0;
+	if (!nlm_sysctl_table) {
+		kobject_put(nlm_kobj);
+		return -ENOMEM;
+	}
 #endif
+
+	return 0;
 }
 
 static void __exit exit_nlm(void)
@@ -530,6 +539,7 @@ static void __exit exit_nlm(void)
 #ifdef CONFIG_SYSCTL
 	unregister_sysctl_table(nlm_sysctl_table);
 #endif
+	kobject_put(nlm_kobj);
 }
 
 module_init(init_nlm);
diff --git a/include/linux/lockd/lockd.h b/include/linux/lockd/lockd.h
index a34dea4..56c749e 100644
--- a/include/linux/lockd/lockd.h
+++ b/include/linux/lockd/lockd.h
@@ -196,6 +196,7 @@ extern int			nlmsvc_grace_period;
 extern unsigned long		nlmsvc_timeout;
 extern int			nsm_use_hostnames;
 extern u32			nsm_local_state;
+extern struct kobject		*nlm_kobj;
 
 /*
  * Lockd client functions

[PATCH 02/11] lockd: Add /sys/fs/lockd/hosts/

[Chuck Lever]

Add parent directory to contain directories representing entries in
lockd's host cache.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 fs/lockd/host.c             |   46 +++++++++++++++++++++++++++++++++++++++++++
 fs/lockd/svc.c              |    7 +++++++
 include/linux/lockd/lockd.h |    2 ++
 3 files changed, 55 insertions(+), 0 deletions(-)

diff --git a/fs/lockd/host.c b/fs/lockd/host.c
index 4600c20..3fbb0a5 100644
--- a/fs/lockd/host.c
+++ b/fs/lockd/host.c
@@ -45,6 +45,10 @@ struct nlm_lookup_host_info {
 	const int		noresvport;	/* use non-priv port */
 };
 
+static struct kobject		*nlm_hosts_kobj;
+static struct kobject		*nlm_hosts_client_kobj;
+static struct kobject		*nlm_hosts_server_kobj;
+
 /*
  * Hash function must work well on big- and little-endian platforms
  */
@@ -567,3 +571,45 @@ nlm_gc_hosts(void)
 
 	next_gc = jiffies + NLM_HOST_COLLECT;
 }
+
+/**
+ * nlm_host_init - Called when lockd module is loaded
+ *
+ */
+int nlm_host_init(void)
+{
+	nlm_hosts_kobj = kobject_create_and_add("hosts", nlm_kobj);
+	if (nlm_hosts_kobj == NULL)
+		return -ENOMEM;
+
+	nlm_hosts_server_kobj = kobject_create_and_add("server", nlm_hosts_kobj);
+	if (nlm_hosts_server_kobj == NULL) {
+		kobject_put(nlm_hosts_kobj);
+		return -ENOMEM;
+	}
+
+	nlm_hosts_client_kobj = kobject_create_and_add("client", nlm_hosts_kobj);
+	if (nlm_hosts_client_kobj == NULL) {
+		kobject_put(nlm_hosts_server_kobj);
+		kobject_put(nlm_hosts_kobj);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
+
+/**
+ * nlm_host_shutdown - Called when lockd module is about to be unloaded
+ *
+ */
+void nlm_host_shutdown(void)
+{
+	if (nrhosts != 0) {
+		printk(KERN_WARNING "NLM hosts remain at shutdown");
+		return;
+ 	}
+
+	kobject_put(nlm_hosts_client_kobj);
+	kobject_put(nlm_hosts_server_kobj);
+	kobject_put(nlm_hosts_kobj);
+}
diff --git a/fs/lockd/svc.c b/fs/lockd/svc.c
index 94e0d4b..acfb6d7 100644
--- a/fs/lockd/svc.c
+++ b/fs/lockd/svc.c
@@ -521,9 +521,15 @@ static int __init init_nlm(void)
 	if (!nlm_kobj)
 		return -ENOMEM;
 
+	if (nlm_host_init()) {
+		kobject_put(nlm_kobj);
+		return -ENOMEM;
+	}
+
 #ifdef CONFIG_SYSCTL
 	nlm_sysctl_table = register_sysctl_table(nlm_sysctl_root);
 	if (!nlm_sysctl_table) {
+		nlm_host_shutdown();
 		kobject_put(nlm_kobj);
 		return -ENOMEM;
 	}
@@ -539,6 +545,7 @@ static void __exit exit_nlm(void)
 #ifdef CONFIG_SYSCTL
 	unregister_sysctl_table(nlm_sysctl_table);
 #endif
+	nlm_host_shutdown();
 	kobject_put(nlm_kobj);
 }
 
diff --git a/include/linux/lockd/lockd.h b/include/linux/lockd/lockd.h
index 56c749e..2d89cdb 100644
--- a/include/linux/lockd/lockd.h
+++ b/include/linux/lockd/lockd.h
@@ -217,6 +217,8 @@ void		  nlmclnt_next_cookie(struct nlm_cookie *);
 /*
  * Host cache
  */
+int		  nlm_host_init(void);
+void		  nlm_host_shutdown(void);
 struct nlm_host  *nlmclnt_lookup_host(const struct sockaddr *sap,
 					const size_t salen,
 					const unsigned short protocol,

[PATCH 03/11] lockd: Add /sys/fs/lockd/hosts/*

[Chuck Lever]

Populate the new hosts/ directory as nlm_hosts come and go.
The new subdirectories are named after the IP address, transport
protocol, and NLM protocol version of each entry in the nlm_hosts
cache.

Introduce a simple-minded approach to managing the kref in the
nlm_host's kobject.  Unfortunately the existing nlm_host reference
counting is more complex than other typical kref usage in the kernel,
so we retain the h_count field.  The kref is always 1 until the
nlm_host is ready to be destroyed.


Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 fs/lockd/host.c             |   89 ++++++++++++++++++++++++++++++++++++++++++-
 include/linux/lockd/lockd.h |    1 
 2 files changed, 87 insertions(+), 3 deletions(-)

diff --git a/fs/lockd/host.c b/fs/lockd/host.c
index 3fbb0a5..2414ef7 100644
--- a/fs/lockd/host.c
+++ b/fs/lockd/host.c
@@ -30,6 +30,7 @@ static unsigned long		next_gc;
 static int			nrhosts;
 static DEFINE_MUTEX(nlm_host_mutex);
 
+static void			nlm_kobj_release(struct kobject *kobj);
 static void			nlm_gc_hosts(void);
 
 struct nlm_lookup_host_info {
@@ -49,6 +50,48 @@ static struct kobject		*nlm_hosts_kobj;
 static struct kobject		*nlm_hosts_client_kobj;
 static struct kobject		*nlm_hosts_server_kobj;
 
+struct nlm_host_attr {
+	struct attribute	attr;
+	ssize_t (*show)(struct nlm_host *host, char *buf);
+	ssize_t (*store)(struct nlm_host *host, const char *buf, size_t count);
+};
+
+static ssize_t nlm_kobj_show(struct kobject *kobj, struct attribute *attr,
+			     char *buf)
+{
+	struct nlm_host_attr *hattr = container_of(attr,
+						struct nlm_host_attr, attr);
+	struct nlm_host *host = container_of(kobj, struct nlm_host, h_kobj);
+	if (hattr->show)
+		return hattr->show(host, buf);
+	return 0;
+}
+
+static ssize_t nlm_kobj_store(struct kobject *kobj, struct attribute *attr,
+			      const char *buf, size_t count)
+{
+	struct nlm_host_attr *hattr = container_of(attr,
+						struct nlm_host_attr, attr);
+	struct nlm_host *host = container_of(kobj, struct nlm_host, h_kobj);
+	if (hattr->store)
+		return hattr->store(host, buf, count);
+	return 0;
+}
+
+static struct sysfs_ops nlm_kobj_sysfs_ops = {
+	.show		= nlm_kobj_show,
+	.store		= nlm_kobj_store,
+};
+
+static struct attribute *nlm_kobj_attrs[] = {
+};
+
+static struct kobj_type nlm_host_ktype = {
+	.release		= nlm_kobj_release,
+	.sysfs_ops		= &nlm_kobj_sysfs_ops,
+	.default_attrs		= nlm_kobj_attrs,
+};
+
 /*
  * Hash function must work well on big- and little-endian platforms
  */
@@ -92,6 +135,35 @@ static unsigned int nlm_hash_address(const struct sockaddr *sap)
 }
 
 /*
+ * New kobjects are placed in either client/ or server/
+ * subdirectory and are named "ip-proto-vers".
+ */
+int nlm_add_kobject(struct nlm_host *host)
+{
+	struct kobject *parent;
+	char *proto_name;
+
+	parent = nlm_hosts_server_kobj;
+	if (host->h_server)
+		parent = nlm_hosts_client_kobj;
+
+	switch (host->h_proto) {
+	case IPPROTO_UDP:
+		proto_name = "udp";
+		break;
+	case IPPROTO_TCP:
+		proto_name = "tcp";
+		break;
+	default:
+		return -EPROTONOSUPPORT;
+	}
+
+	return kobject_init_and_add(&host->h_kobj, &nlm_host_ktype,
+				parent, "%s-%s-%u", host->h_addrbuf,
+				proto_name, host->h_version);
+}
+
+/*
  * Common host lookup routine for server & client
  */
 static struct nlm_host *nlm_lookup_host(struct nlm_lookup_host_info *ni)
@@ -100,6 +172,7 @@ static struct nlm_host *nlm_lookup_host(struct nlm_lookup_host_info *ni)
 	struct hlist_node *pos;
 	struct nlm_host	*host;
 	struct nsm_handle *nsm = NULL;
+	int ret;
 
 	mutex_lock(&nlm_host_mutex);
 
@@ -191,6 +264,16 @@ static struct nlm_host *nlm_lookup_host(struct nlm_lookup_host_info *ni)
 	INIT_LIST_HEAD(&host->h_granted);
 	INIT_LIST_HEAD(&host->h_reclaim);
 
+	ret = nlm_add_kobject(host);
+	if (unlikely(ret != 0)) {
+		atomic_set(&host->h_count, 0);
+		kobject_put(&host->h_kobj);	/* releases nsm too */
+		dprintk("lockd: nlm_lookup_host "
+				"could not add nlm_host kobject\n");
+		host = NULL;
+		goto out;
+	}
+
 	nrhosts++;
 
 	dprintk("lockd: nlm_lookup_host created host %s\n",
@@ -204,9 +287,9 @@ out:
 /*
  * Destroy a host
  */
-static void
-nlm_destroy_host(struct nlm_host *host)
+static void nlm_kobj_release(struct kobject *kobj)
 {
+	struct nlm_host *host = container_of(kobj, struct nlm_host, h_kobj);
 	struct rpc_clnt	*clnt;
 
 	BUG_ON(!list_empty(&host->h_lockowners));
@@ -564,7 +647,7 @@ nlm_gc_hosts(void)
 			dprintk("lockd: delete host %s\n", host->h_name);
 			hlist_del_init(&host->h_hash);
 
-			nlm_destroy_host(host);
+			kobject_put(&host->h_kobj);
 			nrhosts--;
 		}
 	}
diff --git a/include/linux/lockd/lockd.h b/include/linux/lockd/lockd.h
index 2d89cdb..bdb87a6 100644
--- a/include/linux/lockd/lockd.h
+++ b/include/linux/lockd/lockd.h
@@ -39,6 +39,7 @@
  * Lockd host handle (used both by the client and server personality).
  */
 struct nlm_host {
+	struct kobject		h_kobj;
 	struct hlist_node	h_hash;		/* doubly linked list */
 	struct sockaddr_storage	h_addr;		/* peer address */
 	size_t			h_addrlen;

[PATCH 04/11] lockd: Add attributes to /sys/fs/lockd/hosts/*/

[Chuck Lever]

Expose fields in each nlm_host cache entry via files under
/sys/fs/lockd/hosts/{client,server}/*/.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 fs/lockd/host.c |  137 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 137 insertions(+), 0 deletions(-)

diff --git a/fs/lockd/host.c b/fs/lockd/host.c
index 2414ef7..b15579b 100644
--- a/fs/lockd/host.c
+++ b/fs/lockd/host.c
@@ -83,7 +83,144 @@ static struct sysfs_ops nlm_kobj_sysfs_ops = {
 	.store		= nlm_kobj_store,
 };
 
+static ssize_t address_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%s\n", host->h_addrbuf);
+}
+
+static ssize_t source_address_show(struct nlm_host *host, char *buf)
+{
+	struct sockaddr *sap = (struct sockaddr *)(char *)&host->h_srcaddr;
+	switch (sap->sa_family) {
+	case AF_UNSPEC:
+		return snprintf(buf, PAGE_SIZE, "unspecified address\n");
+	case AF_INET:
+		return snprintf(buf, PAGE_SIZE, "%pI4\n", sap);
+	case AF_INET6:
+		return snprintf(buf, PAGE_SIZE, "%pI6c\n", sap);
+	}
+	return snprintf(buf, PAGE_SIZE, "unrecognized address\n");
+}
+
+static ssize_t name_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%s\n", host->h_name);
+}
+
+static ssize_t nlm_version_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%u\n", host->h_version);
+}
+
+static ssize_t transport_show(struct nlm_host *host, char *buf)
+{
+	switch (host->h_proto) {
+	case IPPROTO_UDP:
+		return snprintf(buf, PAGE_SIZE, "udp\n");
+	case IPPROTO_TCP:
+		return snprintf(buf, PAGE_SIZE, "tcp\n");
+	}
+	return snprintf(buf, PAGE_SIZE, "unknown\n");
+}
+
+static ssize_t reclaiming_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%s\n",
+				host->h_reclaiming ? "in progress" : "no");
+}
+
+static ssize_t peer_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%s\n",
+				host->h_server ? "client" : "server");
+}
+
+static ssize_t resvport_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%s\n",
+				host->h_noresvport ? "no" : "yes");
+}
+
+static ssize_t in_use_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%s\n",
+				host->h_inuse ? "yes" : "no");
+}
+
+static ssize_t pseudo_state_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%u\n", host->h_state);
+}
+
+static ssize_t nsm_state_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%u\n", host->h_nsmstate);
+}
+
+static ssize_t pidcount_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%u\n", host->h_pidcount);
+}
+
+static ssize_t refcount_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%d\n", atomic_read(&host->h_count));
+}
+
+static ssize_t nextrebind_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%lu seconds \n",
+				(host->h_nextrebind - jiffies) / HZ);
+}
+
+static ssize_t expires_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%lu seconds\n",
+				(host->h_expires - jiffies) / HZ);
+}
+
+#ifdef __ATTR_RO
+#undef __ATTR_RO
+#endif
+
+#define __ATTR_RO(_name) { \
+	.attr	= { .name = __stringify(_name), .mode = S_IRUSR, }, \
+ 	.show	= _name##_show,                                 \
+}
+
+static struct nlm_host_attr address_attr	= __ATTR_RO(address);
+static struct nlm_host_attr source_address_attr	= __ATTR_RO(source_address);
+static struct nlm_host_attr name_attr		= __ATTR_RO(name);
+static struct nlm_host_attr nlm_version_attr	= __ATTR_RO(nlm_version);
+static struct nlm_host_attr transport_attr	= __ATTR_RO(transport);
+static struct nlm_host_attr reclaiming_attr	= __ATTR_RO(reclaiming);
+static struct nlm_host_attr peer_attr		= __ATTR_RO(peer);
+static struct nlm_host_attr resvport_attr	= __ATTR_RO(resvport);
+static struct nlm_host_attr in_use_attr		= __ATTR_RO(in_use);
+static struct nlm_host_attr pseudo_state_attr	= __ATTR_RO(pseudo_state);
+static struct nlm_host_attr nsm_state_attr	= __ATTR_RO(nsm_state);
+static struct nlm_host_attr pidcount_attr	= __ATTR_RO(pidcount);
+static struct nlm_host_attr refcount_attr	= __ATTR_RO(refcount);
+static struct nlm_host_attr nextrebind_attr	= __ATTR_RO(nextrebind);
+static struct nlm_host_attr expires_attr	= __ATTR_RO(expires);
+
 static struct attribute *nlm_kobj_attrs[] = {
+	&address_attr.attr,
+	&source_address_attr.attr,
+	&name_attr.attr,
+	&nlm_version_attr.attr,
+	&transport_attr.attr,
+	&reclaiming_attr.attr,
+	&peer_attr.attr,
+	&resvport_attr.attr,
+	&in_use_attr.attr,
+	&pseudo_state_attr.attr,
+	&nsm_state_attr.attr,
+	&pidcount_attr.attr,
+	&refcount_attr.attr,
+	&nextrebind_attr.attr,
+	&expires_attr.attr,
+	NULL,
 };
 
 static struct kobj_type nlm_host_ktype = {

[PATCH 05/11] lockd: Add /sys/fs/lockd/mon

[Chuck Lever]

Add parent directory to contain entries representing each entry in
lockd's nsm_handles cache.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 fs/lockd/mon.c              |   27 +++++++++++++++++++++++++++
 fs/lockd/svc.c              |    8 ++++++++
 include/linux/lockd/lockd.h |    2 ++
 3 files changed, 37 insertions(+), 0 deletions(-)

diff --git a/fs/lockd/mon.c b/fs/lockd/mon.c
index f956651..77e4927 100644
--- a/fs/lockd/mon.c
+++ b/fs/lockd/mon.c
@@ -56,6 +56,8 @@ static				DEFINE_SPINLOCK(nsm_lock);
 u32	__read_mostly		nsm_local_state;
 int	__read_mostly		nsm_use_hostnames;
 
+static struct kobject		*nsm_handles_kobj;
+
 static inline struct sockaddr *nsm_addr(const struct nsm_handle *nsm)
 {
 	return (struct sockaddr *)&nsm->sm_addr;
@@ -398,6 +400,31 @@ void nsm_release(struct nsm_handle *nsm)
 	}
 }
 
+/**
+ * nsm_init - Called when lockd module is loaded
+ *
+ */
+int nsm_init(void)
+{
+	nsm_handles_kobj = kobject_create_and_add("monitor", nlm_kobj);
+	if (nsm_handles_kobj == NULL)
+		return -ENOMEM;
+	return 0;
+}
+
+/**
+ * nsm_shutdown - Called when lockd module is about to be unloaded
+ *
+ */
+void nsm_shutdown(void)
+{
+	if (!list_empty(&nsm_handles)) {
+		printk(KERN_WARNING "NSM handles remain at shutdown");
+		return;
+	}
+	kobject_put(nsm_handles_kobj);
+}
+
 /*
  * XDR functions for NSM.
  *
diff --git a/fs/lockd/svc.c b/fs/lockd/svc.c
index acfb6d7..e715ed7 100644
--- a/fs/lockd/svc.c
+++ b/fs/lockd/svc.c
@@ -526,9 +526,16 @@ static int __init init_nlm(void)
 		return -ENOMEM;
 	}
 
+	if (nsm_init()) {
+		nlm_host_shutdown();
+		kobject_put(nlm_kobj);
+		return -ENOMEM;
+	}
+
 #ifdef CONFIG_SYSCTL
 	nlm_sysctl_table = register_sysctl_table(nlm_sysctl_root);
 	if (!nlm_sysctl_table) {
+		nsm_shutdown();
 		nlm_host_shutdown();
 		kobject_put(nlm_kobj);
 		return -ENOMEM;
@@ -545,6 +552,7 @@ static void __exit exit_nlm(void)
 #ifdef CONFIG_SYSCTL
 	unregister_sysctl_table(nlm_sysctl_table);
 #endif
+	nsm_shutdown();
 	nlm_host_shutdown();
 	kobject_put(nlm_kobj);
 }
diff --git a/include/linux/lockd/lockd.h b/include/linux/lockd/lockd.h
index bdb87a6..8622a28 100644
--- a/include/linux/lockd/lockd.h
+++ b/include/linux/lockd/lockd.h
@@ -239,6 +239,8 @@ void		  nlm_host_rebooted(const struct nlm_reboot *);
 /*
  * Host monitoring
  */
+int		  nsm_init(void);
+void		  nsm_shutdown(void);
 int		  nsm_monitor(const struct nlm_host *host);
 void		  nsm_unmonitor(const struct nlm_host *host);
 

[PATCH 06/11] lockd: Add /sys/fs/lockd/nsm_handle/*

[Chuck Lever]

Populate the new nsm_handle/ directory as nsm_handles come and go.
The entries are named by their priv cookie, which is always guaranteed
to be unique.

Like the nlm_hosts cache, the kref is fixed at one when the handle
is created, and goes to zero only when the handle is ready to be
destroyed.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 fs/lockd/host.c             |    7 ++
 fs/lockd/mon.c              |  166 +++++++++++++++++++++++++++++++++++++++++--
 include/linux/lockd/lockd.h |    1 
 3 files changed, 166 insertions(+), 8 deletions(-)

diff --git a/fs/lockd/host.c b/fs/lockd/host.c
index b15579b..0a62d8d 100644
--- a/fs/lockd/host.c
+++ b/fs/lockd/host.c
@@ -279,6 +279,7 @@ int nlm_add_kobject(struct nlm_host *host)
 {
 	struct kobject *parent;
 	char *proto_name;
+	int ret;
 
 	parent = nlm_hosts_server_kobj;
 	if (host->h_server)
@@ -295,9 +296,13 @@ int nlm_add_kobject(struct nlm_host *host)
 		return -EPROTONOSUPPORT;
 	}
 
-	return kobject_init_and_add(&host->h_kobj, &nlm_host_ktype,
+	ret = kobject_init_and_add(&host->h_kobj, &nlm_host_ktype,
 				parent, "%s-%s-%u", host->h_addrbuf,
 				proto_name, host->h_version);
+	if (likely(ret == 0))
+		ret = sysfs_create_link(&host->h_kobj,
+				&host->h_nsmhandle->sm_kobj, "nsm_handle");
+	return ret;
 }
 
 /*
diff --git a/fs/lockd/mon.c b/fs/lockd/mon.c
index 77e4927..dc185e1 100644
--- a/fs/lockd/mon.c
+++ b/fs/lockd/mon.c
@@ -58,6 +58,129 @@ int	__read_mostly		nsm_use_hostnames;
 
 static struct kobject		*nsm_handles_kobj;
 
+struct nsm_handle_attr {
+	struct attribute	attr;
+	ssize_t (*show)(struct nsm_handle *nsm, char *buf);
+	ssize_t (*store)(struct nsm_handle *nsm, const char *buf, size_t count);
+};
+
+static ssize_t nsm_kobj_show(struct kobject *kobj, struct attribute *attr,
+			     char *buf)
+{
+	struct nsm_handle_attr *nattr = container_of(attr,
+						struct nsm_handle_attr, attr);
+	struct nsm_handle *nsm = container_of(kobj, struct nsm_handle, sm_kobj);
+	if (nattr->show)
+		return nattr->show(nsm, buf);
+	return 0;
+}
+
+static ssize_t nsm_kobj_store(struct kobject *kobj, struct attribute *attr,
+			      const char *buf, size_t count)
+{
+	struct nsm_handle_attr *nattr = container_of(attr,
+						struct nsm_handle_attr, attr);
+	struct nsm_handle *nsm = container_of(kobj, struct nsm_handle, sm_kobj);
+	if (nattr->store)
+		return nattr->store(nsm, buf, count);
+	return 0;
+}
+
+static void nsm_kobj_release(struct kobject *kobj)
+{
+	struct nsm_handle *nsm = container_of(kobj,
+					struct nsm_handle, sm_kobj);
+
+	dprintk("lockd: destroyed nsm_handle for %s (%s)\n",
+			nsm->sm_name, nsm->sm_addrbuf);
+	kfree(nsm);
+}
+
+static struct sysfs_ops nsm_kobj_sysfs_ops = {
+	.show		= nsm_kobj_show,
+	.store		= nsm_kobj_store,
+};
+
+static ssize_t mon_name_show(struct nsm_handle *nsm, char *buf)
+{
+	char *mon_name = nsm->sm_mon_name;
+
+	/* When this host is unmonitored, sm_mon_name is NULL,
+	 * so make up something reasonable. */
+	if (mon_name == NULL)
+		mon_name = nsm_use_hostnames ? nsm->sm_name : nsm->sm_addrbuf;
+
+	return snprintf(buf, PAGE_SIZE, "%s\n", mon_name);
+}
+
+static ssize_t my_name_show(struct nsm_handle *nsm, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%s\n", utsname()->nodename);
+}
+
+static ssize_t address_show(struct nsm_handle *nsm, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%s\n", nsm->sm_addrbuf);
+}
+
+static ssize_t rpcparms_show(struct nsm_handle *nsm, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%u %u %u\n",
+			NLM_PROGRAM, 3, NLMPROC_NSM_NOTIFY);
+}
+
+static ssize_t status_show(struct nsm_handle *nsm, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%smonitored\n",
+			nsm->sm_monitored ? "" : "un" );
+}
+
+static ssize_t sticky_show(struct nsm_handle *nsm, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%s\n",
+			nsm->sm_sticky ? "yes" : "no" );
+}
+
+static ssize_t refcount_show(struct nsm_handle *nsm, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%d\n",
+			atomic_read(&nsm->sm_count));
+}
+
+#ifdef __ATTR_RO
+#undef __ATTR_RO
+#endif
+
+#define __ATTR_RO(_name) { \
+	.attr	= { .name = __stringify(_name), .mode = S_IRUSR, }, \
+ 	.show	= _name##_show,                                 \
+}
+
+static struct nsm_handle_attr mon_name_attr	= __ATTR_RO(mon_name);
+static struct nsm_handle_attr my_name_attr	= __ATTR_RO(my_name);
+static struct nsm_handle_attr address_attr	= __ATTR_RO(address);
+static struct nsm_handle_attr rpcparms_attr	= __ATTR_RO(rpcparms);
+static struct nsm_handle_attr status_attr	= __ATTR_RO(status);
+static struct nsm_handle_attr sticky_attr	= __ATTR_RO(sticky);
+static struct nsm_handle_attr refcount_attr	= __ATTR_RO(refcount);
+
+static struct attribute *nsm_kobj_attrs[] = {
+	&mon_name_attr.attr,
+	&my_name_attr.attr,
+	&address_attr.attr,
+	&rpcparms_attr.attr,
+	&status_attr.attr,
+	&sticky_attr.attr,
+	&refcount_attr.attr,
+	NULL,
+};
+
+static struct kobj_type nsm_handle_ktype = {
+	.release		= nsm_kobj_release,
+	.sysfs_ops		= &nsm_kobj_sysfs_ops,
+	.default_attrs		= nsm_kobj_attrs,
+};
+
 static inline struct sockaddr *nsm_addr(const struct nsm_handle *nsm)
 {
 	return (struct sockaddr *)&nsm->sm_addr;
@@ -189,8 +312,10 @@ void nsm_unmonitor(const struct nlm_host *host)
 		if (status < 0)
 			printk(KERN_NOTICE "lockd: cannot unmonitor %s\n",
 					nsm->sm_name);
-		else
+		else {
 			nsm->sm_monitored = 0;
+			nsm->sm_mon_name = NULL;
+		}
 	}
 }
 
@@ -261,7 +386,10 @@ static struct nsm_handle *nsm_create_handle(const struct sockaddr *sap,
 					    const char *hostname,
 					    const size_t hostname_len)
 {
+	char *p, buf[SM_PRIV_SIZE * 2 + 1];
 	struct nsm_handle *new;
+	size_t remaining;
+	int i, len, ret;
 
 	new = kzalloc(sizeof(*new) + hostname_len + 1, GFP_KERNEL);
 	if (unlikely(new == NULL))
@@ -280,7 +408,33 @@ static struct nsm_handle *nsm_create_handle(const struct sockaddr *sap,
 	memcpy(new->sm_name, hostname, hostname_len);
 	new->sm_name[hostname_len] = '\0';
 
+	p = buf;
+	remaining = sizeof(buf);
+	for (i = 0; i < SM_PRIV_SIZE; i++) {
+		len = snprintf(p, remaining, "%02x",
+				(u32)(new->sm_priv.data[i]) & 0xff);
+		if (len != 2)
+			goto out;
+		p += len; 
+		remaining -= (size_t)len;
+	}
+	if (remaining < 1)
+		goto out;
+	strcat(buf, "\0");
+
+	/* Named after "priv" cookie, which is guaranteed unique */
+	ret = kobject_init_and_add(&new->sm_kobj, &nsm_handle_ktype,
+					nsm_handles_kobj, "%s", buf);
+	if (ret != 0)
+		goto out;
+
 	return new;
+
+out:
+	kobject_put(&new->sm_kobj);
+	dprintk("lockd: %s failed; could not add kobject\n",
+			__func__);
+	return NULL;
 }
 
 /**
@@ -323,7 +477,7 @@ retry:
 	if (cached != NULL) {
 		atomic_inc(&cached->sm_count);
 		spin_unlock(&nsm_lock);
-		kfree(new);
+		kobject_put(&new->sm_kobj);
 		dprintk("lockd: found nsm_handle for %s (%s), "
 				"cnt %d\n", cached->sm_name,
 				cached->sm_addrbuf,
@@ -392,11 +546,9 @@ struct nsm_handle *nsm_reboot_lookup(const struct nlm_reboot *info)
 void nsm_release(struct nsm_handle *nsm)
 {
 	if (atomic_dec_and_lock(&nsm->sm_count, &nsm_lock)) {
-		list_del(&nsm->sm_link);
+		list_del(&nsm->sm_link);	/* kobject_del ? */
 		spin_unlock(&nsm_lock);
-		dprintk("lockd: destroyed nsm_handle for %s (%s)\n",
-				nsm->sm_name, nsm->sm_addrbuf);
-		kfree(nsm);
+		kobject_put(&nsm->sm_kobj);
 	}
 }
 
@@ -406,7 +558,7 @@ void nsm_release(struct nsm_handle *nsm)
  */
 int nsm_init(void)
 {
-	nsm_handles_kobj = kobject_create_and_add("monitor", nlm_kobj);
+	nsm_handles_kobj = kobject_create_and_add("nsm_handles", nlm_kobj);
 	if (nsm_handles_kobj == NULL)
 		return -ENOMEM;
 	return 0;
diff --git a/include/linux/lockd/lockd.h b/include/linux/lockd/lockd.h
index 8622a28..3d1c594 100644
--- a/include/linux/lockd/lockd.h
+++ b/include/linux/lockd/lockd.h
@@ -78,6 +78,7 @@ struct nlm_host {
 #define NSM_ADDRBUF		((8 * 4 + 7) + (1 + 10) + 1)
 
 struct nsm_handle {
+	struct kobject		sm_kobj;
 	struct list_head	sm_link;
 	atomic_t		sm_count;
 	char			*sm_mon_name;

[PATCH 07/11] lockd: Refactor nlm_host_rebooted()

[Chuck Lever]

We want to be able to call the lock recovery logic directly with an
nsm_handle.  Factor that piece out of nlm_host_rebooted().

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 fs/lockd/host.c             |   35 ++++++++++++++++++++++++-----------
 include/linux/lockd/lockd.h |    2 ++
 2 files changed, 26 insertions(+), 11 deletions(-)

diff --git a/fs/lockd/host.c b/fs/lockd/host.c
index 0a62d8d..001d05b 100644
--- a/fs/lockd/host.c
+++ b/fs/lockd/host.c
@@ -669,26 +669,39 @@ void nlm_release_host(struct nlm_host *host)
  */
 void nlm_host_rebooted(const struct nlm_reboot *info)
 {
-	struct hlist_head *chain;
-	struct hlist_node *pos;
 	struct nsm_handle *nsm;
-	struct nlm_host	*host;
 
 	nsm = nsm_reboot_lookup(info);
 	if (unlikely(nsm == NULL))
 		return;
 
-	/* Mark all hosts tied to this NSM state as having rebooted.
-	 * We run the loop repeatedly, because we drop the host table
-	 * lock for this.
-	 * To avoid processing a host several times, we match the nsmstate.
-	 */
-again:	mutex_lock(&nlm_host_mutex);
+	nlm_host_rebooted_nsm(nsm, info->state);
+}
+
+/**
+ * nlm_host_rebooted_nsm - start reboot recovery on remote peer
+ * @nsm: pointer to NSM handle for rebooting peer
+ * @nsmstate: new NSM state number of rebooting peer
+ *
+ * Mark all hosts tied to this NSM state as having rebooted.  We run
+ * the loop repeatedly, because we drop the host table lock for this.
+ *
+ * To avoid processing a host several times, we match the nsmstate.
+ */
+void nlm_host_rebooted_nsm(const struct nsm_handle *nsm, const int nsmstate)
+{
+	struct hlist_head *chain;
+	struct hlist_node *pos;
+	struct nlm_host	*host;
+
+again:
+	mutex_lock(&nlm_host_mutex);
+
 	for (chain = nlm_hosts; chain < nlm_hosts + NLM_HOST_NRHASH; ++chain) {
 		hlist_for_each_entry(host, pos, chain, h_hash) {
 			if (host->h_nsmhandle == nsm
-			 && host->h_nsmstate != info->state) {
-				host->h_nsmstate = info->state;
+			 && host->h_nsmstate != nsmstate) {
+				host->h_nsmstate = nsmstate;
 				host->h_state++;
 
 				nlm_get_host(host);
diff --git a/include/linux/lockd/lockd.h b/include/linux/lockd/lockd.h
index 3d1c594..3c20114 100644
--- a/include/linux/lockd/lockd.h
+++ b/include/linux/lockd/lockd.h
@@ -235,6 +235,8 @@ void		  nlm_rebind_host(struct nlm_host *);
 struct nlm_host * nlm_get_host(struct nlm_host *);
 void		  nlm_release_host(struct nlm_host *);
 void		  nlm_shutdown_hosts(void);
+void		  nlm_host_rebooted_nsm(const struct nsm_handle *nsm,
+					const int nsmstate);
 void		  nlm_host_rebooted(const struct nlm_reboot *);
 
 /*

[PATCH 08/11] lockd: Add "reboot" attribute to nsm_handles

[Chuck Lever]

There can be multiple nlm_host cache entries associated with one
remote peer, but there is only one nsm_handle per peer.  So the
right place to add an API to trigger lock recovery for a particular
remote peer is with the nsm_handle cache entry for that remote.

Add a "reboot" attribute to each subdirectory in nsm_handles/.

Writing the new NSM state into the peer's nsm_handles "reboot"
attribute invokes the kernel's lock recovery for that peer.  A user
space tool, for example, could be constructed so an administrator
can remove stale locks on an NFS server.

Note that this new API can replace statd's NLM downcalls as well.
Instead of calling a specified RPC program/version/procedure with
the new state number and the priv cookie, statd can walk through
/sys/fs/lockd looking for a directory named after the priv cookie,
and write the new state number into its "reboot" attribute.

Using this API instead of an RPC downcall would eliminate one lockd
dependency on IPv4 loopback.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 fs/lockd/mon.c |   20 ++++++++++++++++++++
 1 files changed, 20 insertions(+), 0 deletions(-)

diff --git a/fs/lockd/mon.c b/fs/lockd/mon.c
index dc185e1..690c236 100644
--- a/fs/lockd/mon.c
+++ b/fs/lockd/mon.c
@@ -147,6 +147,23 @@ static ssize_t refcount_show(struct nsm_handle *nsm, char *buf)
 			atomic_read(&nsm->sm_count));
 }
 
+static ssize_t reboot_show(struct nsm_handle *nsm, char *buf)
+{
+	return 0;
+}
+
+static ssize_t reboot_store(struct nsm_handle *nsm, 
+				const char *buf, size_t count)
+{
+	int state;
+
+	if (sscanf(buf, "%d", &state) != 1)
+		return -EINVAL;
+
+	nlm_host_rebooted_nsm(nsm, state);
+	return count;
+}
+
 #ifdef __ATTR_RO
 #undef __ATTR_RO
 #endif
@@ -163,6 +180,8 @@ static struct nsm_handle_attr rpcparms_attr	= __ATTR_RO(rpcparms);
 static struct nsm_handle_attr status_attr	= __ATTR_RO(status);
 static struct nsm_handle_attr sticky_attr	= __ATTR_RO(sticky);
 static struct nsm_handle_attr refcount_attr	= __ATTR_RO(refcount);
+static struct nsm_handle_attr reboot_attr	= __ATTR(reboot,
+			S_IWUSR | S_IRUSR, reboot_show, reboot_store);
 
 static struct attribute *nsm_kobj_attrs[] = {
 	&mon_name_attr.attr,
@@ -172,6 +191,7 @@ static struct attribute *nsm_kobj_attrs[] = {
 	&status_attr.attr,
 	&sticky_attr.attr,
 	&refcount_attr.attr,
+	&reboot_attr.attr,
 	NULL,
 };
 

[PATCH 09/11] lockd: Keep my_name in nsm_handle

[Chuck Lever]

lockd assumes everywhere that my_name is always utsname->nodename.  If we
want to use multiple my_names (say, for multi-homed support) then we need
to get the my_name value from one place, and ensure that value is also
communicated consistently to statd.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 fs/lockd/clntproc.c         |    6 +++---
 fs/lockd/mon.c              |   20 ++++++++++++++++----
 fs/lockd/svclock.c          |    2 +-
 include/linux/lockd/lockd.h |    6 ++++++
 4 files changed, 26 insertions(+), 8 deletions(-)

diff --git a/fs/lockd/clntproc.c b/fs/lockd/clntproc.c
index c81249f..2051515 100644
--- a/fs/lockd/clntproc.c
+++ b/fs/lockd/clntproc.c
@@ -12,7 +12,6 @@
 #include <linux/errno.h>
 #include <linux/fs.h>
 #include <linux/nfs_fs.h>
-#include <linux/utsname.h>
 #include <linux/freezer.h>
 #include <linux/sunrpc/clnt.h>
 #include <linux/sunrpc/svc.h>
@@ -125,14 +124,15 @@ static void nlmclnt_setlockargs(struct nlm_rqst *req, struct file_lock *fl)
 {
 	struct nlm_args	*argp = &req->a_args;
 	struct nlm_lock	*lock = &argp->lock;
+	char *my_name = nsm_my_name(req);
 
 	nlmclnt_next_cookie(&argp->cookie);
 	memcpy(&lock->fh, NFS_FH(fl->fl_file->f_path.dentry->d_inode), sizeof(struct nfs_fh));
-	lock->caller  = utsname()->nodename;
+	lock->caller  = my_name;
 	lock->oh.data = req->a_owner;
 	lock->oh.len  = snprintf(req->a_owner, sizeof(req->a_owner), "%u@%s",
 				(unsigned int)fl->fl_u.nfs_fl.owner->pid,
-				utsname()->nodename);
+				my_name);
 	lock->svid = fl->fl_u.nfs_fl.owner->pid;
 	lock->fl.fl_start = fl->fl_start;
 	lock->fl.fl_end = fl->fl_end;
diff --git a/fs/lockd/mon.c b/fs/lockd/mon.c
index 690c236..e470038 100644
--- a/fs/lockd/mon.c
+++ b/fs/lockd/mon.c
@@ -39,6 +39,7 @@ struct nsm_args {
 	u32			proc;
 
 	char			*mon_name;
+	char			*my_name;
 };
 
 struct nsm_res {
@@ -93,6 +94,8 @@ static void nsm_kobj_release(struct kobject *kobj)
 
 	dprintk("lockd: destroyed nsm_handle for %s (%s)\n",
 			nsm->sm_name, nsm->sm_addrbuf);
+
+	kfree(nsm->sm_my_name);
 	kfree(nsm);
 }
 
@@ -115,7 +118,7 @@ static ssize_t mon_name_show(struct nsm_handle *nsm, char *buf)
 
 static ssize_t my_name_show(struct nsm_handle *nsm, char *buf)
 {
-	return snprintf(buf, PAGE_SIZE, "%s\n", utsname()->nodename);
+	return snprintf(buf, PAGE_SIZE, "%s\n", nsm->sm_my_name);
 }
 
 static ssize_t address_show(struct nsm_handle *nsm, char *buf)
@@ -236,6 +239,7 @@ static int nsm_mon_unmon(struct nsm_handle *nsm, u32 proc, struct nsm_res *res)
 		.vers		= 3,
 		.proc		= NLMPROC_NSM_NOTIFY,
 		.mon_name	= nsm->sm_mon_name,
+		.my_name	= nsm->sm_my_name,
 	};
 	struct rpc_message msg = {
 		.rpc_argp	= &args,
@@ -404,7 +408,8 @@ static void nsm_init_private(struct nsm_handle *nsm)
 static struct nsm_handle *nsm_create_handle(const struct sockaddr *sap,
 					    const size_t salen,
 					    const char *hostname,
-					    const size_t hostname_len)
+					    const size_t hostname_len,
+					    const char *my_name)
 {
 	char *p, buf[SM_PRIV_SIZE * 2 + 1];
 	struct nsm_handle *new;
@@ -415,6 +420,12 @@ static struct nsm_handle *nsm_create_handle(const struct sockaddr *sap,
 	if (unlikely(new == NULL))
 		return NULL;
 
+	new->sm_my_name = kstrdup(my_name, GFP_KERNEL);
+	if (unlikely(new->sm_my_name == NULL)) {
+		kfree(new);
+		return NULL;
+	}
+
 	atomic_set(&new->sm_count, 1);
 	new->sm_name = (char *)(new + 1);
 	memcpy(nsm_addr(new), sap, salen);
@@ -515,7 +526,8 @@ retry:
 
 	spin_unlock(&nsm_lock);
 
-	new = nsm_create_handle(sap, salen, hostname, hostname_len);
+	new = nsm_create_handle(sap, salen, hostname, hostname_len,
+						init_utsname()->nodename);
 	if (unlikely(new == NULL))
 		return NULL;
 	goto retry;
@@ -637,7 +649,7 @@ static int encode_my_id(struct xdr_stream *xdr, const struct nsm_args *argp)
 	int status;
 	__be32 *p;
 
-	status = encode_nsm_string(xdr, utsname()->nodename);
+	status = encode_nsm_string(xdr, argp->my_name);
 	if (unlikely(status != 0))
 		return status;
 	p = xdr_reserve_space(xdr, 3 * sizeof(u32));
diff --git a/fs/lockd/svclock.c b/fs/lockd/svclock.c
index d100179..938a9e4 100644
--- a/fs/lockd/svclock.c
+++ b/fs/lockd/svclock.c
@@ -304,7 +304,7 @@ static int nlmsvc_setgrantargs(struct nlm_rqst *call, struct nlm_lock *lock)
 {
 	locks_copy_lock(&call->a_args.lock.fl, &lock->fl);
 	memcpy(&call->a_args.lock.fh, &lock->fh, sizeof(call->a_args.lock.fh));
-	call->a_args.lock.caller = utsname()->nodename;
+	call->a_args.lock.caller = nsm_my_name(call);
 	call->a_args.lock.oh.len = lock->oh.len;
 
 	/* set default data area */
diff --git a/include/linux/lockd/lockd.h b/include/linux/lockd/lockd.h
index 3c20114..9e5da26 100644
--- a/include/linux/lockd/lockd.h
+++ b/include/linux/lockd/lockd.h
@@ -82,6 +82,7 @@ struct nsm_handle {
 	struct list_head	sm_link;
 	atomic_t		sm_count;
 	char			*sm_mon_name;
+	char			*sm_my_name;
 	char			*sm_name;
 	struct sockaddr_storage	sm_addr;
 	size_t			sm_addrlen;
@@ -254,6 +255,11 @@ struct nsm_handle *nsm_get_handle(const struct sockaddr *sap,
 struct nsm_handle *nsm_reboot_lookup(const struct nlm_reboot *info);
 void		  nsm_release(struct nsm_handle *nsm);
 
+static inline char *nsm_my_name(const struct nlm_rqst *req)
+{
+	return (char *)req->a_host->h_nsmhandle->sm_my_name;
+}
+
 /*
  * This is used in garbage collection and resource reclaim
  * A return value != 0 means destroy the lock/block/share

[PATCH 10/11] lockd: use sm_my_name for nsm_handle lookups

[Chuck Lever]

Have nsm_get_handle() distinguish between two nsm_handles that are the
same in every way except their sm_my_name field is different.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 fs/lockd/host.c             |    2 +-
 fs/lockd/mon.c              |   28 +++++++++++++++++++---------
 include/linux/lockd/lockd.h |    3 ++-
 3 files changed, 22 insertions(+), 11 deletions(-)

diff --git a/fs/lockd/host.c b/fs/lockd/host.c
index 001d05b..7d3cd2e 100644
--- a/fs/lockd/host.c
+++ b/fs/lockd/host.c
@@ -366,7 +366,7 @@ static struct nlm_host *nlm_lookup_host(struct nlm_lookup_host_info *ni)
 	else {
 		host = NULL;
 		nsm = nsm_get_handle(ni->sap, ni->salen,
-					ni->hostname, ni->hostname_len);
+				ni->hostname, ni->hostname_len, NULL);
 		if (!nsm) {
 			dprintk("lockd: nlm_lookup_host failed; "
 				"no nsm handle\n");
diff --git a/fs/lockd/mon.c b/fs/lockd/mon.c
index e470038..d4b9cb3 100644
--- a/fs/lockd/mon.c
+++ b/fs/lockd/mon.c
@@ -16,6 +16,7 @@
 #include <linux/sunrpc/svc.h>
 #include <linux/lockd/lockd.h>
 
+#include <asm/string.h>
 #include <asm/unaligned.h>
 
 #define NLMDBG_FACILITY		NLMDBG_MONITOR
@@ -344,23 +345,27 @@ void nsm_unmonitor(const struct nlm_host *host)
 }
 
 static struct nsm_handle *nsm_lookup_hostname(const char *hostname,
-					      const size_t len)
+					      const size_t len,
+					      const char *my_name)
 {
 	struct nsm_handle *nsm;
 
 	list_for_each_entry(nsm, &nsm_handles, sm_link)
 		if (strlen(nsm->sm_name) == len &&
-		    memcmp(nsm->sm_name, hostname, len) == 0)
+		    memcmp(nsm->sm_name, hostname, len) == 0 &&
+		    strcmp(my_name, nsm->sm_my_name) == 0)
 			return nsm;
 	return NULL;
 }
 
-static struct nsm_handle *nsm_lookup_addr(const struct sockaddr *sap)
+static struct nsm_handle *nsm_lookup_addr(const struct sockaddr *sap,
+					  const char *my_name)
 {
 	struct nsm_handle *nsm;
 
 	list_for_each_entry(nsm, &nsm_handles, sm_link)
-		if (rpc_cmp_addr(nsm_addr(nsm), sap))
+		if (rpc_cmp_addr(nsm_addr(nsm), sap) &&
+		    strcmp(my_name, nsm->sm_my_name) == 0)
 			return nsm;
 	return NULL;
 }
@@ -474,6 +479,8 @@ out:
  * @salen: length of socket address
  * @hostname: pointer to C string containing hostname to find
  * @hostname_len: length of C string
+ * @my_name: pointer to '\0'-terminated C string containing
+ *			hostname of local system, or NULL
  *
  * Behavior is modulated by the global nsm_use_hostnames variable.
  *
@@ -484,7 +491,8 @@ out:
  */
 struct nsm_handle *nsm_get_handle(const struct sockaddr *sap,
 				  const size_t salen, const char *hostname,
-				  const size_t hostname_len)
+				  const size_t hostname_len,
+				  const char *my_name)
 {
 	struct nsm_handle *cached, *new = NULL;
 
@@ -497,13 +505,16 @@ struct nsm_handle *nsm_get_handle(const struct sockaddr *sap,
 		return NULL;
 	}
 
+	if (my_name == NULL)
+		my_name = (const char *)init_utsname()->nodename;
+
 retry:
 	spin_lock(&nsm_lock);
 
 	if (nsm_use_hostnames && hostname != NULL)
-		cached = nsm_lookup_hostname(hostname, hostname_len);
+		cached = nsm_lookup_hostname(hostname, hostname_len, my_name);
 	else
-		cached = nsm_lookup_addr(sap);
+		cached = nsm_lookup_addr(sap, my_name);
 
 	if (cached != NULL) {
 		atomic_inc(&cached->sm_count);
@@ -526,8 +537,7 @@ retry:
 
 	spin_unlock(&nsm_lock);
 
-	new = nsm_create_handle(sap, salen, hostname, hostname_len,
-						init_utsname()->nodename);
+	new = nsm_create_handle(sap, salen, hostname, hostname_len, my_name);
 	if (unlikely(new == NULL))
 		return NULL;
 	goto retry;
diff --git a/include/linux/lockd/lockd.h b/include/linux/lockd/lockd.h
index 9e5da26..a19bdde 100644
--- a/include/linux/lockd/lockd.h
+++ b/include/linux/lockd/lockd.h
@@ -251,7 +251,8 @@ void		  nsm_unmonitor(const struct nlm_host *host);
 struct nsm_handle *nsm_get_handle(const struct sockaddr *sap,
 					const size_t salen,
 					const char *hostname,
-					const size_t hostname_len);
+					const size_t hostname_len,
+					const char *my_name);
 struct nsm_handle *nsm_reboot_lookup(const struct nlm_reboot *info);
 void		  nsm_release(struct nsm_handle *nsm);
 

[PATCH 11/11] lockd: Allow mount option to specify caller_name

[Chuck Lever]

NLMPROC_LOCK requests have a "caller_name" argument which is supposed
to contain the hostname the server uses to call the client back.
Linux simply stuffs the system's utsname in this field, but this is
not always the correct choice.  For example:

  o  If an unqualified hostname is used for the client's utsname,
     it could be ambiguous when the server tries to resolve it
  o  If the client's actual hostname is determined by DHCP, it may
     not match its utsname
  o  If the NFS mount was done in a network namespace, the namespace
     name won't match the client's utsname
  o  If the client has multiple network interfaces, it should provide
     a hostname that matches the source address used to contact the
     server

In all of these cases, user space can determine the correct value of
the caller_name argument at mount time.

So, add a mount option that allows user space to specify the value of
the caller_name argument of NLMPROC_LOCK requests.  If not specified,
the kernel continues to use the init utsname, as before.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 fs/lockd/clntlock.c         |    3 ++-
 fs/lockd/host.c             |    7 ++++++-
 fs/nfs/client.c             |   11 +++++++++++
 fs/nfs/internal.h           |    1 +
 fs/nfs/super.c              |    9 +++++++++
 include/linux/lockd/bind.h  |    1 +
 include/linux/lockd/lockd.h |    1 +
 include/linux/nfs_fs_sb.h   |    5 +++++
 8 files changed, 36 insertions(+), 2 deletions(-)

diff --git a/fs/lockd/clntlock.c b/fs/lockd/clntlock.c
index fc9032d..0a0d98a 100644
--- a/fs/lockd/clntlock.c
+++ b/fs/lockd/clntlock.c
@@ -61,7 +61,8 @@ struct nlm_host *nlmclnt_init(const struct nlmclnt_initdata *nlm_init)
 
 	host = nlmclnt_lookup_host(nlm_init->address, nlm_init->addrlen,
 				   nlm_init->protocol, nlm_version,
-				   nlm_init->hostname, nlm_init->noresvport);
+				   nlm_init->myname, nlm_init->hostname,
+				   nlm_init->noresvport);
 	if (host == NULL) {
 		lockd_down();
 		return ERR_PTR(-ENOLCK);
diff --git a/fs/lockd/host.c b/fs/lockd/host.c
index 7d3cd2e..c23bf0d 100644
--- a/fs/lockd/host.c
+++ b/fs/lockd/host.c
@@ -39,6 +39,7 @@ struct nlm_lookup_host_info {
 	const size_t		salen;		/* it's length */
 	const unsigned short	protocol;	/* transport to search for*/
 	const u32		version;	/* NLM version to search for */
+	const char		*my_name;	/* local hostname */
 	const char		*hostname;	/* remote's hostname */
 	const size_t		hostname_len;	/* it's length */
 	const struct sockaddr	*src_sap;	/* our address (optional) */
@@ -366,7 +367,8 @@ static struct nlm_host *nlm_lookup_host(struct nlm_lookup_host_info *ni)
 	else {
 		host = NULL;
 		nsm = nsm_get_handle(ni->sap, ni->salen,
-				ni->hostname, ni->hostname_len, NULL);
+				ni->hostname, ni->hostname_len,
+				ni->my_name);
 		if (!nsm) {
 			dprintk("lockd: nlm_lookup_host failed; "
 				"no nsm handle\n");
@@ -453,6 +455,7 @@ static void nlm_kobj_release(struct kobject *kobj)
  * @protocol: transport protocol to use
  * @version: NLM protocol version
  * @hostname: '\0'-terminated hostname of server
+ * @myname: '\0'-terminated hostname of local system
  * @noresvport: 1 if non-privileged port should be used
  *
  * Returns an nlm_host structure that matches the passed-in
@@ -465,6 +468,7 @@ struct nlm_host *nlmclnt_lookup_host(const struct sockaddr *sap,
 				     const unsigned short protocol,
 				     const u32 version,
 				     const char *hostname,
+				     const char *myname,
 				     int noresvport)
 {
 	const struct sockaddr source = {
@@ -476,6 +480,7 @@ struct nlm_host *nlmclnt_lookup_host(const struct sockaddr *sap,
 		.salen		= salen,
 		.protocol	= protocol,
 		.version	= version,
+		.my_name	= myname,
 		.hostname	= hostname,
 		.hostname_len	= strlen(hostname),
 		.src_sap	= &source,
diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index ee77713..920b842 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -98,6 +98,7 @@ struct rpc_program		nfsacl_program = {
 #endif  /* CONFIG_NFS_V3_ACL */
 
 struct nfs_client_initdata {
+	const char *myname;
 	const char *hostname;
 	const struct sockaddr *addr;
 	size_t addrlen;
@@ -129,6 +130,13 @@ static struct nfs_client *nfs_alloc_client(const struct nfs_client_initdata *cl_
 	memcpy(&clp->cl_addr, cl_init->addr, cl_init->addrlen);
 	clp->cl_addrlen = cl_init->addrlen;
 
+	if (cl_init->myname) {
+		err = -ENOMEM;
+		clp->cl_myname = kstrdup(cl_init->myname, GFP_KERNEL);
+		if (!clp->cl_myname)
+			goto error_cleanup;
+	}
+
 	if (cl_init->hostname) {
 		err = -ENOMEM;
 		clp->cl_hostname = kstrdup(cl_init->hostname, GFP_KERNEL);
@@ -159,6 +167,7 @@ static struct nfs_client *nfs_alloc_client(const struct nfs_client_initdata *cl_
 	return clp;
 
 error_cleanup:
+	kfree(clp->cl_myname);
 	kfree(clp);
 error_0:
 	return ERR_PTR(err);
@@ -645,6 +654,7 @@ static int nfs_start_lockd(struct nfs_server *server)
 	struct nlm_host *host;
 	struct nfs_client *clp = server->nfs_client;
 	struct nlmclnt_initdata nlm_init = {
+		.myname		= clp->cl_myname,
 		.hostname	= clp->cl_hostname,
 		.address	= (struct sockaddr *)&clp->cl_addr,
 		.addrlen	= clp->cl_addrlen,
@@ -780,6 +790,7 @@ static int nfs_init_server(struct nfs_server *server,
 			   const struct nfs_parsed_mount_data *data)
 {
 	struct nfs_client_initdata cl_init = {
+		.myname = data->myname,
 		.hostname = data->nfs_server.hostname,
 		.addr = (const struct sockaddr *)&data->nfs_server.address,
 		.addrlen = data->nfs_server.addrlen,
diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
index 37c37f8..ff0c906 100644
--- a/fs/nfs/internal.h
+++ b/fs/nfs/internal.h
@@ -80,6 +80,7 @@ struct nfs_parsed_mount_data {
 	unsigned int		version;
 	unsigned int		minorversion;
 	char			*fscache_uniq;
+	char			*myname;
 
 	struct {
 		struct sockaddr_storage	address;
diff --git a/fs/nfs/super.c b/fs/nfs/super.c
index 21a0dcb..50a6d82 100644
--- a/fs/nfs/super.c
+++ b/fs/nfs/super.c
@@ -99,6 +99,7 @@ enum {
 	Opt_addr, Opt_mountaddr, Opt_clientaddr,
 	Opt_lookupcache,
 	Opt_fscache_uniq,
+	Opt_caller_name,
 
 	/* Special mount options */
 	Opt_userspace, Opt_deprecated, Opt_sloppy,
@@ -170,6 +171,7 @@ static const match_table_t nfs_mount_option_tokens = {
 
 	{ Opt_lookupcache, "lookupcache=%s" },
 	{ Opt_fscache_uniq, "fsc=%s" },
+	{ Opt_caller_name, "callername=%s" },
 
 	{ Opt_err, NULL }
 };
@@ -1383,6 +1385,13 @@ static int nfs_parse_mount_options(char *raw,
 			mnt->fscache_uniq = string;
 			mnt->options |= NFS_OPTION_FSCACHE;
 			break;
+		case Opt_caller_name:
+			string = match_strdup(args);
+			if (string == NULL)
+				goto out_nomem;
+			kfree(mnt->myname);
+			mnt->myname = string;
+			break;
 
 		/*
 		 * Special options
diff --git a/include/linux/lockd/bind.h b/include/linux/lockd/bind.h
index fbc48f8..5d51b87 100644
--- a/include/linux/lockd/bind.h
+++ b/include/linux/lockd/bind.h
@@ -36,6 +36,7 @@ extern struct nlmsvc_binding *	nlmsvc_ops;
  * rpc_ops field.
  */
 struct nlmclnt_initdata {
+	const char		*myname;
 	const char		*hostname;
 	const struct sockaddr	*address;
 	size_t			addrlen;
diff --git a/include/linux/lockd/lockd.h b/include/linux/lockd/lockd.h
index a19bdde..e83b62d 100644
--- a/include/linux/lockd/lockd.h
+++ b/include/linux/lockd/lockd.h
@@ -226,6 +226,7 @@ struct nlm_host  *nlmclnt_lookup_host(const struct sockaddr *sap,
 					const size_t salen,
 					const unsigned short protocol,
 					const u32 version,
+					const char *myname,
 					const char *hostname,
 					int noresvport);
 struct nlm_host  *nlmsvc_lookup_host(const struct svc_rqst *rqstp,
diff --git a/include/linux/nfs_fs_sb.h b/include/linux/nfs_fs_sb.h
index 34fc6be..7c86c2f 100644
--- a/include/linux/nfs_fs_sb.h
+++ b/include/linux/nfs_fs_sb.h
@@ -66,6 +66,11 @@ struct nfs_client {
 	/* idmapper */
 	struct idmap *		cl_idmap;
 
+	/* Our local hostname, as a null-terminated string.
+	 * This is used to generate the caller_name for v2/v3 locking.
+	 */
+	char			*cl_myname;
+
 	/* Our own IP address, as a null-terminated string.
 	 * This is used to generate the clientid, and the callback address.
 	 */

Re: [PATCH 11/11] lockd: Allow mount option to specify caller_name

[Trond Myklebust]

On Thu, 2010-04-01 at 15:04 -0400, Chuck Lever wrote: 
> NLMPROC_LOCK requests have a "caller_name" argument which is supposed
> to contain the hostname the server uses to call the client back.
> Linux simply stuffs the system's utsname in this field, but this is
> not always the correct choice.  For example:
> 
>   o  If an unqualified hostname is used for the client's utsname,
>      it could be ambiguous when the server tries to resolve it
>   o  If the client's actual hostname is determined by DHCP, it may
>      not match its utsname
>   o  If the NFS mount was done in a network namespace, the namespace
>      name won't match the client's utsname
>   o  If the client has multiple network interfaces, it should provide
>      a hostname that matches the source address used to contact the
>      server
> 
> In all of these cases, user space can determine the correct value of
> the caller_name argument at mount time.
> 
> So, add a mount option that allows user space to specify the value of
> the caller_name argument of NLMPROC_LOCK requests.  If not specified,
> the kernel continues to use the init utsname, as before.

This argument makes no sense. Mount points do _not_ follow network
namespace boundaries, so making this hostname of yours a mount option
will make matters worse, not better.

Furthermore, even if we were to accept your argument, you are not
matching nfs_clients to your "namespace name", so if you do have more
than 1 mount to the same server but from different namespaces, then the
namespace name of the first to mount will automatically become the
default for the second mount too.

Trond

Re: [PATCH 11/11] lockd: Allow mount option to specify caller_name

[Trond Myklebust]

On Thu, 2010-04-01 at 15:45 -0400, Trond Myklebust wrote: 
> On Thu, 2010-04-01 at 15:04 -0400, Chuck Lever wrote: 
> > NLMPROC_LOCK requests have a "caller_name" argument which is supposed
> > to contain the hostname the server uses to call the client back.
> > Linux simply stuffs the system's utsname in this field, but this is
> > not always the correct choice.  For example:
> > 
> >   o  If an unqualified hostname is used for the client's utsname,
> >      it could be ambiguous when the server tries to resolve it
> >   o  If the client's actual hostname is determined by DHCP, it may
> >      not match its utsname
> >   o  If the NFS mount was done in a network namespace, the namespace
> >      name won't match the client's utsname
> >   o  If the client has multiple network interfaces, it should provide
> >      a hostname that matches the source address used to contact the
> >      server
> > 
> > In all of these cases, user space can determine the correct value of
> > the caller_name argument at mount time.
> > 
> > So, add a mount option that allows user space to specify the value of
> > the caller_name argument of NLMPROC_LOCK requests.  If not specified,
> > the kernel continues to use the init utsname, as before.
> 
> This argument makes no sense. Mount points do _not_ follow network
> namespace boundaries, so making this hostname of yours a mount option
> will make matters worse, not better.
> 
> Furthermore, even if we were to accept your argument, you are not
> matching nfs_clients to your "namespace name", so if you do have more
> than 1 mount to the same server but from different namespaces, then the
> namespace name of the first to mount will automatically become the
> default for the second mount too.

Basically, it boils down to: what does all this extra stuff give me that
the current (per-thread) utsname group namespace does not allow?

   Trond

Re: [PATCH 11/11] lockd: Allow mount option to specify caller_name

[Chuck Lever]

On 04/01/2010 04:01 PM, Trond Myklebust wrote:
> On Thu, 2010-04-01 at 15:45 -0400, Trond Myklebust wrote:
>> On Thu, 2010-04-01 at 15:04 -0400, Chuck Lever wrote:
>>> NLMPROC_LOCK requests have a "caller_name" argument which is supposed
>>> to contain the hostname the server uses to call the client back.
>>> Linux simply stuffs the system's utsname in this field, but this is
>>> not always the correct choice.  For example:
>>>
>>>    o  If an unqualified hostname is used for the client's utsname,
>>>       it could be ambiguous when the server tries to resolve it
>>>    o  If the client's actual hostname is determined by DHCP, it may
>>>       not match its utsname
>>>    o  If the NFS mount was done in a network namespace, the namespace
>>>       name won't match the client's utsname
>>>    o  If the client has multiple network interfaces, it should provide
>>>       a hostname that matches the source address used to contact the
>>>       server
>>>
>>> In all of these cases, user space can determine the correct value of
>>> the caller_name argument at mount time.
>>>
>>> So, add a mount option that allows user space to specify the value of
>>> the caller_name argument of NLMPROC_LOCK requests.  If not specified,
>>> the kernel continues to use the init utsname, as before.
>>
>> This argument makes no sense. Mount points do _not_ follow network
>> namespace boundaries, so making this hostname of yours a mount option
>> will make matters worse, not better.

Um, "this hostname of yours" is snide and unnecessary.  It's the 
caller_name string, and it's been an argument of NLMPROC_LOCK and used 
for lock recovery ever since NFSv2 was invented.  Let's keep it civil, 
please.

So, ignore the network namespace example, then, and consider the 
majority of the examples above.

The server's statd stores the client's caller_name string on the monitor 
list, and uses it as part of a heuristic to match incoming SM_NOTIFY 
requests.  If we send an accurate caller_name string in our NLMPROC_LOCK 
requests, there's a better chance that the remote statd will recognize 
us when we reboot.  Refer to Talpey's Cthon slides or _NFS_Illustrated_ 
for visual aids.

This applies to three of the four examples I provided above:

1)  It's been a best practice for a long time to ensure that your Linux 
client's nodename (ie its utsname) matches it's fully-qualified domain 
name, and for exactly this purpose (see NetApp TR-3183).  With this 
mount option, the correct caller_name can be determined automatically.

What happens if the client's utsname is unqualified, and then it 
contacts a server that is already talking to a client with the same 
unqualified hostname in a different domain?  The result is that the 
server will have to choose between these two clients when one of them 
reboots.

2)  If a client's address is assigned automatically, it won't 
necessarily match its utsname.  That's true of my laptops on wireless, 
for example.  In that case, my Dell laptop would send "SM_NOTIFY 
ellison.1015granger.net" from, say, anon-dhcp-108.1015granger.net. 
statd's DNS monname matching heuristic might fail.

Note that most contemporary Linux servers store the client's address 
rather than the caller_name string, but that just means our server won't 
recognize a client's reboot if the client is assigned a different 
address after it reboots, and that DNS configuration is especially 
critical to get lock recovery right.

If our client is operating with an automatically assigned IPv6 address, 
where a router gives an IPv6 address prefix, and the rest of the address 
is constructed from the NIC's MAC address, or, if our IPv4 address is 
DHCP-assigned by MAC address, what happens if we shut down the client, 
and then replace the NIC?  What if our client switches from wireless to 
wired?

In other words, we can't rely solely on source IP address to identify 
rebooting hosts.

3)  If the client is talking to a server on a private area network, 
there's no guarantee the server will recognize the client's caller_name 
string if it's the hostname of the client on the public side network. 
It may even attempt to contact the client via it's public side name, 
which might fail, depending on how the network is set up.

Therefore, I assert that this feature is needed to support multi-homed 
locking adequately, and to provide better lock recovery in the face of 
dynamically assigned IP addresses.

-- 
chuck[dot]lever[at]oracle[dot]com