Re: [PATCH v4 08/14] svcrdma: Adjust the number of entries in svc_rdma_recv_ctxt::rc_pages

Christoph Hellwig
2025-05-07T07:40:34Z

On Tue, May 06, 2025 at 11:20:44AM -0400, Chuck Lever wrote:
> 1. svcrdma issues RDMA Read WRs from an svc_rqst thread context. It
> pulls the Read sink pages out of the svc_rqst's rq_pages[] array, and
> then svc_alloc_arg() refills the rq_pages[] array before the thread
> returns to the thread pool
>
> 2. When the RDMA Read completes, it is picked up by an svc_rqst thread.
> svcrdma frees the pages in the thread's rq_pages[] array, and
> replaces them with the Read's sink pages.

...

> One idea would be for NFSD to maintain a pool of these pages. But I'm
> not convinced that we could invent anything that's less latent than the
> generic bulk page allocator: release_pages() and alloc_bulk_pages().

I think the main issue is the completion thread, which first allocates
and then frees the pages. If you instead just have a container for
the pages (or any other future context state) you avoid that. I.e.
lifting the concept of a receive context from the rdma transport
to common code.