Re: [PATCH v2 00/10] Allocate payload arrays dynamically

[Chuck Lever <chuck.lever@oracle.com>]

On 4/19/25 1:28 PM, cel@kernel.org wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> In order to make RPCSVC_MAXPAYLOAD larger (or variable in size), we
> need to do something clever with the payload arrays embedded in
> struct svc_rqst and elsewhere.
> 
> My preference is to keep these arrays allocated all the time because
> allocating them on demand increases the risk of a memory allocation
> failure during a large I/O. This is a quick-and-dirty approach that
> might be replaced once NFSD is converted to use large folios.
> 
> The downside of this design choice is that it pins a few pages per
> NFSD thread (and that's the current situation already). But note
> that because RPCSVC_MAXPAGES is 259, each array is just over a page
> in size, making the allocation waste quite a bit of memory beyond
> the end of the array due to power-of-2 allocator round up. This gets
> worse as the MAXPAGES value is doubled or quadrupled.
> 
> This series also addresses similar issues in the socket and RDMA
> transports.

I forgot to note that this series has other benefits besides making
these arrays flexible in size. The reduction in size of struct
svc_rqst to under half a page is kind of a big deal.


> Chuck Lever (9):
>   sunrpc: Remove backchannel check in svc_init_buffer()
>   sunrpc: Add a helper to derive maxpages from sv_max_mesg
>   sunrpc: Replace the rq_pages array with dynamically-allocated memory
>   sunrpc: Replace the rq_vec array with dynamically-allocated memory
>   sunrpc: Replace the rq_bvec array with dynamically-allocated memory
>   sunrpc: Adjust size of socket's receive page array dynamically
>   svcrdma: Adjust the number of RDMA contexts per transport
>   svcrdma: Adjust the number of entries in svc_rdma_recv_ctxt::rc_pages
>   svcrdma: Adjust the number of entries in svc_rdma_send_ctxt::sc_pages
> 
>  fs/nfsd/nfs4proc.c                       |  1 -
>  fs/nfsd/vfs.c                            |  2 +-
>  include/linux/sunrpc/svc.h               | 19 +++++++--
>  include/linux/sunrpc/svc_rdma.h          |  6 ++-
>  include/linux/sunrpc/svcsock.h           |  4 +-
>  net/sunrpc/svc.c                         | 51 +++++++++++++++---------
>  net/sunrpc/svc_xprt.c                    | 10 +----
>  net/sunrpc/svcsock.c                     | 15 ++++---
>  net/sunrpc/xprtrdma/svc_rdma_recvfrom.c  |  8 +++-
>  net/sunrpc/xprtrdma/svc_rdma_rw.c        |  2 +-
>  net/sunrpc/xprtrdma/svc_rdma_sendto.c    | 16 ++++++--
>  net/sunrpc/xprtrdma/svc_rdma_transport.c |  2 +-
>  12 files changed, 88 insertions(+), 48 deletions(-)
> 


-- 
Chuck Lever