[PATCH v4 17/18] nfsd/localio: use SRCU to dereference nn->nfsd_serv in nfsd_open_local_fh

[Mike Snitzer <snitzer@kernel.org>]

Use nfsd_serv_get to SRCU deference nn->nfsd_serv and pass the
resulting svc_serv to nfsd_local_fakerqst_create, open the file handle
and then drop the reference using nfsd_serv_put at the end of
nfsd_open_local_fh.

Verified to fix an easy to hit crash that would occur if an nfsd
instance running in a container, with a localio client mounted, is
shutdown. Upon restart of the container and associated nfsd the client
would go on to crash due to NULL pointer dereference that occuured due
to the nfs client's localio attempting to nfsd_open_local_fh(), using
nn->nfsd_serv, without having a proper reference on nn->nfsd_serv.

Signed-off-by: Mike Snitzer <snitzer@kernel.org>
---
 fs/nfsd/localio.c          |  4 +++-
 include/linux/nfslocalio.h | 24 +++++++++++++++++++++++-
 2 files changed, 26 insertions(+), 2 deletions(-)

diff --git a/fs/nfsd/localio.c b/fs/nfsd/localio.c
index cdf8e115b33e..d1d9fbaab82e 100644
--- a/fs/nfsd/localio.c
+++ b/fs/nfsd/localio.c
@@ -193,6 +193,7 @@ int nfsd_open_local_fh(struct net *net,
 	struct nfsd_file *nf;
 	int status = 0;
 	int mayflags = NFSD_MAY_LOCALIO;
+	int srcu_idx;
 	struct svc_serv *serv;
 	__be32 beres;
 
@@ -207,7 +208,7 @@ int nfsd_open_local_fh(struct net *net,
 	}
 	nn = net_generic(net, nfsd_net_id);
 
-	serv = READ_ONCE(nn->nfsd_serv);
+	serv = nfsd_serv_get(nn, &srcu_idx);
 	if (unlikely(!serv)) {
 		dprintk("%s: localio denied. Server not running\n", __func__);
 		status = -ENXIO;
@@ -247,6 +248,7 @@ int nfsd_open_local_fh(struct net *net,
 out_revertcred:
 	revert_creds(save_cred);
 out_net:
+	nfsd_serv_put(nn, srcu_idx);
 	put_net(net);
 	return status;
 }
diff --git a/include/linux/nfslocalio.h b/include/linux/nfslocalio.h
index c9592ad0afe2..face80fc80cf 100644
--- a/include/linux/nfslocalio.h
+++ b/include/linux/nfslocalio.h
@@ -19,7 +19,29 @@ extern struct list_head nfsd_uuids;
 /*
  * Each nfsd instance has an nfsd_uuid_t that is accessible through the
  * global nfsd_uuids list. Useful to allow a client to negotiate if localio
- * possible with its server.
+ * possible with its server. See fs/nfs/localio.c:nfs_local_probe() for the
+ * code that handshakes with nfsd to determine if localio is possible.
+ *
+ * The nfsd_uuids list is the basis for localio enablement, as such it has
+ * members that point to nfsd memory for direct use by the client (e.g. 'net'
+ * is the server's network namespace, through it the client can access
+ * nn->nfsd_serv with proper rcu read access). It is this client and server
+ * synchronization that enables advanced usage and lifetime of objects to
+ * span from the host kernel's nfsd to per-container knfsd instances that
+ * are connected to nfs client's running on the same local host.
+ *
+ * See fs/nfs/localio.c:nfs_local_open_fh() and
+ * fs/nfsd/localio.c:nfsd_open_local_fh() for the interface that makes
+ * focused use of nfsd_uuid_t struct to allow a client local to a server
+ * to open a file pointer without needing to go over the network.
+ *
+ * The performance advantage realized from localio's ability to bypass
+ * using XDR and RPC for reads, writes and commits can be extreme, e.g.:
+ * fio for 20 secs with 24 libaio threads, 64k directio reads, qd of 8,
+ * with localio:
+ *  read: IOPS=795k, BW=48.5GiB/s (52.1GB/s)(971GiB/20002msec)
+ * without localio:
+ *  read: IOPS=15.6k, BW=974MiB/s (1021MB/s)(19.0GiB/20013msec)
  */
 typedef struct {
 	uuid_t uuid;
-- 
2.44.0