[PATCH nfs-utils v3 07/11] mountd: give each worker its own netlink command socket

[Jeff Layton <jlayton@kernel.org>]

cache_open() opens the nfsd and sunrpc command sockets before
cache_fork_workers() forks, so every worker shares one fd, and a
copy-on-write struct nl_sock carrying the same s_seq_next/s_seq_expect.
Only the notify sockets get nl_socket_disable_seq_check(); the command
sockets keep libnl's default sequence check, which each worker will
happily pass on the other's reply.

Two workers in a command/reply round trip can therefore take each other's
ack or NLMSG_ERROR.  cache_nl_set_reqs() then reports the wrong outcome
and the caller answers the wrong client and path - marking it exported,
retryable, or negatively cached.

Reopen both command sockets in the worker child.  genl_connect() binds a
fresh port id, so each worker gets its own reply stream.  The notify
sockets stay shared: they are receive-only, have the sequence check
disabled, and are nonblocking, so whichever worker gets there first takes
the notification.

Signed-off-by: Jeff Layton <jlayton@kernel.org>
Assisted-by: LLM
---
 support/export/cache.c | 37 +++++++++++++++++++++++++++++++++++--
 1 file changed, 35 insertions(+), 2 deletions(-)

diff --git a/support/export/cache.c b/support/export/cache.c
index 6a5cdd9d9670..cbe3df83b0eb 100644
--- a/support/export/cache.c
+++ b/support/export/cache.c
@@ -3994,6 +3994,29 @@ cache_wait_for_workers(char *prog)
 	}
 }
 
+/*
+ * Replace a command socket inherited across fork().  Returns @old if a new one
+ * cannot be had; sharing it is worse than having one, but not by as much as
+ * having none.
+ */
+static struct nl_sock *nl_cmd_sock_reopen(struct nl_sock *old)
+{
+	struct nl_sock *sock;
+
+	if (!old)
+		return NULL;
+
+	sock = nl_sock_setup();
+	if (!sock) {
+		xlog(L_WARNING, "%s: cannot reopen netlink command socket,"
+		     " sharing the inherited one", __func__);
+		return old;
+	}
+
+	nl_socket_free(old);
+	return sock;
+}
+
 /* Fork num_threads worker children and wait for them */
 int
 cache_fork_workers(char *prog, int num_threads)
@@ -4015,12 +4038,22 @@ cache_fork_workers(char *prog, int num_threads)
 		if (pid == 0) {
 			/* worker child */
 
+			/*
+			 * A command socket carries a reply back to the process
+			 * that sent the request, so it cannot be shared.  Every
+			 * worker inherits the same fd and the same copied
+			 * sequence counters, and two of them mid-round-trip
+			 * will take each other's ack.
+			 */
+			nfsd_nl_cmd_sock = nl_cmd_sock_reopen(nfsd_nl_cmd_sock);
+			sunrpc_nl_cmd_sock =
+				nl_cmd_sock_reopen(sunrpc_nl_cmd_sock);
+
 			/*
 			 * cache_open() drains the netlink downcalls before we
 			 * get here, so anything it deferred is now on the retry
 			 * queues of every worker.  Let the first worker own
-			 * those, or they get answered once per worker over the
-			 * shared command socket.
+			 * those, or they get answered once per worker.
 			 */
 			if (i > 0) {
 				delayed_export_flush();

-- 
2.55.0