Re: [RFC PATCH 0/6] Improve NFS server direct throughput with passthrough enabled

["Chuck Lever" <cel@kernel.org>]

On Tue, Oct 6, 2026, at 5:00 AM, J Louis Kaplan wrote:
> For NFSoRDMA using RoCE, and with IOMMU passthrough enabled, use of a
> 64KiB base page size was found to provide higher NFS read throughput
> than a 4KiB base page size. This patch series recovers some of that
> throughput with a 4k base page size by opportunistic use of large folios
> for the svc reply buffer.

> LLM Usage
> =========
>
> The idea of folio usage here was human-generated from observations of
> performance differences between 4KiB and 64KiB page size kernels.

Actually a similar approach has been tried before, but on x86_64:

https://lore.kernel.org/linux-nfs/20260319133610.2556826-1-cel@kernel.org/

And it had to be reverted:

- Mike Snitzer's report (4 Jun 2026):
    https://lore.kernel.org/linux-nfs/aiHlPmeZq3WgMwoJ@kernel.org/

- Jonathan Flynn's benchmark and perf data (5 Jun 2026), showing server
  CPU going from 8.54% to 76.35% with the commit present:
    https://lore.kernel.org/linux-nfs/3cb119b4b2a8aada30c0c60286778a54@mail.gmail.com/

These two are the Closes: targets in the revert, which is a39f0ce0c9da
upstream and 284d5ba931a5 in stable.

Lock contention analysis and the decision to revert

- "[PATCH] svcrdma: Cap Read sink allocations at PAGE_ALLOC_COSTLY_ORDER"
  (5 Jun 2026), whose description carries the zone->lock analysis:
    https://lore.kernel.org/linux-nfs/20260606035722.83175-1-cel@kernel.org/
- Flynn's results for that fix (25.4 GiB/s, against 30.3 regressed and
  73.9 reverted):
    https://lore.kernel.org/linux-nfs/65a2cdb132b0c28e69a29955e3bd37e7@mail.gmail.com/
- Your reply saying the two failed fixes mean 18755b8c2f24 has to be
  reverted (6 Jun 2026):
    https://lore.kernel.org/linux-nfs/096a2b91-7a19-48da-a06a-dc60e7150956@app.fastmail.com/

The earlier failed fix, "[PATCH] svcrdma: Avoid direct reclaim when
allocating Read sink buffers", is at
    https://lore.kernel.org/linux-nfs/20260605223118.75092-1-cel@kernel.org/.

So: Yes, we would like to ensure that svcrdma's RDMA Read buffers (at
least, and maybe RDMA Write) are contiguous so that the overhead of a
DMA mapping operation will be low. I'm not expert enough with the page
allocator's free path to address the lock contention problem, however.


-- 
Chuck Lever (Come to NFS bake-a-thon! https://nfsv4bat.org)