Re: [PATCH 1/2] svcrdma: Release write chunk resources without re-queuing

[Mike Snitzer <snitzer@kernel.org>]

On Wed, May 06, 2026 at 11:26:50AM -0400, Chuck Lever wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> Each RDMA Send completion triggers a cascade of work items on the
> svcrdma_wq unbound workqueue:
> 
>   ib_cq_poll_work (on ib_comp_wq, per-CPU)
>     -> svc_rdma_send_ctxt_put -> queue_work    [work item 1]
>       -> svc_rdma_write_info_free -> queue_work [work item 2]
> 
> Every transition through queue_work contends on the unbound
> pool's spinlock. Profiling an 8KB NFSv3 read/write workload
> over RDMA shows about 4% of total CPU cycles spent on this
> lock, with the cascading re-queue of write_info release
> contributing roughly 1%.
> 
> The initial queue_work in svc_rdma_send_ctxt_put is needed to
> move release work off the CQ completion context (which runs on
> a per-CPU bound workqueue). However, once executing on
> svcrdma_wq, there is no need to re-queue for each write_info
> structure. svc_rdma_reply_chunk_release already calls
> svc_rdma_cc_release inline from the same svcrdma_wq context,
> and svc_rdma_recv_ctxt_put does the same from nfsd thread
> context.
> 
> Release write chunk resources inline in
> svc_rdma_write_info_free, removing the intermediate
> svc_rdma_write_info_free_async work item and the wi_work
> field from struct svc_rdma_write_info.
> 
> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>

You were correct: this patch alone eliminates the OOM (we tested with
both 16K and then 4K read IO from 121 clients to 8 NFS servers, no
measurable memory growth while testing).

Feel free to add:

Reviewed-by: Mike Snitzer <snitzer@kernel.org>
Tested-by: Jonathan Flynn <jonathan.flynn@hammerspace.com>

Thanks!
Mike

ps.
So you are aware, couldn't test your 2nd patch at the customer site
because the baseline kernel there is based on 6.12-stable but your
2nd patch builds on your 7.1 svcrdma changes. I think your 2nd patch
is ideal though, and will be able to pull it in to test in future, but
won't have the ability to test at this customer's scale until we can
role that newer kernel out there... might take a couple months.