[PATCH v5 00/19] Allocate payload arrays dynamically

cel
2025-05-09T19:03:56Z

In order to make RPCSVC_MAXPAYLOAD larger (or variable in size), we
need to do something clever with the payload arrays embedded in
struct svc_rqst and elsewhere.

My preference is to keep these arrays allocated all the time because
allocating them on demand increases the risk of a memory allocation
failure during a large I/O. This is a quick-and-dirty approach that
might be replaced once NFSD is converted to use large folios.

The downside of this design choice is that it pins a few pages per
NFSD thread (and that's the current situation already). But note
that because RPCSVC_MAXPAGES is 259, each array is just over a page
in size, making the allocation waste quite a bit of memory beyond
the end of the array due to power-of-2 allocator round up. This gets
worse as the MAXPAGES value is doubled or quadrupled.

This series also addresses similar issues in the socket and RDMA
transports.

Changes since v4:
* Replace the use of rq_vec instead of allocating it dynamically

Changes since v3:
* Improved the rdma_rw context count estimate
* Dropped "NFSD: Remove NFSSVC_MAXBLKSIZE from .pc_xdrressize"
* Cleaned up the max size macros a bit
* Completed the implementation of adjustable max_block_size

Changes since v2:
* Address Jeff's review comments
* Address Neil's review comments
* Start removing a few uses of NFSSVC_MAXBLKSIZE

Chuck Lever (19):
svcrdma: Reduce the number of rdma_rw contexts per-QP
sunrpc: Add a helper to derive maxpages from sv_max_mesg
sunrpc: Remove backchannel check in svc_init_buffer()
sunrpc: Replace the rq_pages array with dynamically-allocated memory
sunrpc: Replace the rq_bvec array with dynamically-allocated memory
NFSD: Use rqstp->rq_bvec in nfsd_iter_read()
NFSD: De-duplicate the svc_fill_write_vector() call sites
SUNRPC: Export xdr_buf_to_bvec()
NFSD: Use rqstp->rq_bvec in nfsd_iter_write()
SUNRPC: Remove svc_fill_write_vector()
SUNRPC: Remove svc_rqst :: rq_vec
sunrpc: Adjust size of socket's receive page array dynamically
svcrdma: Adjust the number of entries in svc_rdma_recv_ctxt::rc_pages
svcrdma: Adjust the number of entries in svc_rdma_send_ctxt::sc_pages
sunrpc: Remove the RPCSVC_MAXPAGES macro
NFSD: Remove NFSD_BUFSIZE
NFSD: Remove NFSSVC_MAXBLKSIZE_V2 macro
NFSD: Add a "default" block size
SUNRPC: Bump the maximum payload size for the server

fs/nfsd/nfs3proc.c | 5 +-
fs/nfsd/nfs4proc.c | 10 +--
fs/nfsd/nfs4state.c | 2 +-
fs/nfsd/nfs4xdr.c | 2 +-
fs/nfsd/nfsd.h | 24 +++----
fs/nfsd/nfsproc.c | 13 ++--
fs/nfsd/nfssvc.c | 2 +-
fs/nfsd/nfsxdr.c | 4 +-
fs/nfsd/vfs.c | 67 ++++++++++++++------
fs/nfsd/vfs.h | 10 +--
include/linux/sunrpc/svc.h | 46 ++++++++------
include/linux/sunrpc/svc_rdma.h | 6 +-
include/linux/sunrpc/svcsock.h | 4 +-
net/sunrpc/svc.c | 80 +++++++-----------------
net/sunrpc/svc_xprt.c | 10 +--
net/sunrpc/svcsock.c | 15 +++--
net/sunrpc/xdr.c | 1 +
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 8 ++-
net/sunrpc/xprtrdma/svc_rdma_rw.c | 2 +-
net/sunrpc/xprtrdma/svc_rdma_sendto.c | 16 ++++-
net/sunrpc/xprtrdma/svc_rdma_transport.c | 14 +++--
21 files changed, 170 insertions(+), 171 deletions(-)

--
2.49.0