From: Chuck Lever <cel@kernel.org>
To: NeilBrown <neilb@ownmail.net>, Jeff Layton <jlayton@kernel.org>,
Olga Kornievskaia <okorniev@redhat.com>,
Dai Ngo <dai.ngo@oracle.com>, Tom Talpey <tom@talpey.com>,
daire@dneg.com, Mike Snitzer <snitzer@kernel.org>
Cc: <linux-nfs@vger.kernel.org>, Chuck Lever <chuck.lever@oracle.com>
Subject: [RFC PATCH 0/7] sunrpc: Reduce lock contention for NFSD TCP sockets
Date: Thu, 5 Feb 2026 10:57:22 -0500 [thread overview]
Message-ID: <20260205155729.6841-1-cel@kernel.org> (raw)
From: Chuck Lever <chuck.lever@oracle.com>
High-throughput NFSD workloads exhibit significant lock contention on
TCP connections. Worker threads compete for the socket lock during
receives and serialize on xpt_mutex during sends, limiting scalability.
This series addresses both paths:
- Receive: A dedicated kernel thread per TCP connection owns all
sock_recvmsg() calls and queues complete RPC messages for workers
via lock-free llist. This eliminates socket lock contention among
workers.
- Transmit: Flat combining allows one thread to send on behalf of
multiple waiters. Threads enqueue requests; the mutex holder
("combiner") processes the batch, amortizing lock acquisition and
enabling TCP segment coalescing via MSG_MORE.
Supporting changes include a workqueue affinity fix for single-LLC
systems, a page recycling pool for receive buffers, and explicit TCP
buffer sizing for high bandwidth-delay product networks.
Base commit: v6.19-rc8
---
Chuck Lever (7):
workqueue: Automatic affinity scope fallback for single-pod topologies
sunrpc: split svc_data_ready into protocol-specific callbacks
sunrpc: add per-transport page recycling pool
sunrpc: add dedicated TCP receiver thread
sunrpc: implement flat combining for TCP socket sends
sunrpc: unify fore and backchannel server TCP send paths
SUNRPC: Set explicit TCP socket buffer sizes for NFSD
include/linux/sunrpc/svc.h | 1 +
include/linux/sunrpc/svc_xprt.h | 32 ++
include/linux/sunrpc/svcsock.h | 40 ++
include/linux/workqueue.h | 8 +-
kernel/workqueue.c | 68 ++-
net/sunrpc/svc.c | 13 +
net/sunrpc/svc_xprt.c | 151 ++++++
net/sunrpc/svcsock.c | 797 +++++++++++++++++++++++++++++---
net/sunrpc/xprtsock.c | 60 +--
9 files changed, 1044 insertions(+), 126 deletions(-)
--
2.52.0
next reply other threads:[~2026-02-05 15:57 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-02-05 15:57 Chuck Lever [this message]
2026-02-05 15:57 ` [RFC PATCH 1/7] workqueue: Automatic affinity scope fallback for single-pod topologies Chuck Lever
2026-02-06 14:57 ` Chuck Lever
2026-02-05 15:57 ` [RFC PATCH 2/7] sunrpc: split svc_data_ready into protocol-specific callbacks Chuck Lever
2026-02-05 15:57 ` [RFC PATCH 3/7] sunrpc: add per-transport page recycling pool Chuck Lever
2026-02-05 15:57 ` [RFC PATCH 4/7] sunrpc: add dedicated TCP receiver thread Chuck Lever
2026-02-05 15:57 ` [RFC PATCH 5/7] sunrpc: implement flat combining for TCP socket sends Chuck Lever
2026-02-05 15:57 ` [RFC PATCH 6/7] sunrpc: unify fore and backchannel server TCP send paths Chuck Lever
2026-02-05 15:57 ` [RFC PATCH 7/7] SUNRPC: Set explicit TCP socket buffer sizes for NFSD Chuck Lever
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260205155729.6841-1-cel@kernel.org \
--to=cel@kernel.org \
--cc=chuck.lever@oracle.com \
--cc=dai.ngo@oracle.com \
--cc=daire@dneg.com \
--cc=jlayton@kernel.org \
--cc=linux-nfs@vger.kernel.org \
--cc=neilb@ownmail.net \
--cc=okorniev@redhat.com \
--cc=snitzer@kernel.org \
--cc=tom@talpey.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox