Re: [PATCH v4 08/14] svcrdma: Adjust the number of entries in svc_rdma_recv_ctxt::rc_pages

[Christoph Hellwig <hch@infradead.org>]

On Tue, May 06, 2025 at 11:20:44AM -0400, Chuck Lever wrote:
> 1. svcrdma issues RDMA Read WRs from an svc_rqst thread context. It
>    pulls the Read sink pages out of the svc_rqst's rq_pages[] array, and
>    then svc_alloc_arg() refills the rq_pages[] array before the thread
>    returns to the thread pool
> 
> 2. When the RDMA Read completes, it is picked up by an svc_rqst thread.
>    svcrdma frees the pages in the thread's rq_pages[] array, and
>    replaces them with the Read's sink pages.

...

> One idea would be for NFSD to maintain a pool of these pages. But I'm
> not convinced that we could invent anything that's less latent than the
> generic bulk page allocator: release_pages() and alloc_bulk_pages().

I think the main issue is the completion thread, which first allocates
and then frees the pages.  If you instead just have a container for
the pages (or any other future context state) you avoid that.  I.e.
lifting the concept of a receive context from the rdma transport
to common code.