Re: need SUNRPC TCP to receive into aligned pages [was: Re: [PATCH 1/6] NFSD: add the ability to enable use of RWF_DONTCACHE for all IO]

Chuck Lever
2025-06-12T16:33:09Z

On 6/12/25 12:12 PM, Mike Snitzer wrote:

> the value of
> not requiring any RDMA hardware or client changes is too compelling to
> ignore.

I think you are vastly inflating that value.

The numbers you have presented are for big systems that are already on
a fast RDMA-capable network.

You haven't demonstrated much of an issue for low-intensity workloads
like small home directory servers. For those, unaligned WRITEs are
totally adequate and would see no real improvement if the server could
handle unaligned payloads slightly more efficiently.

I also haven't seen specific data that showed it is only the buffer
alignment issue that is slowing down NFS WRITE. IME it is actually the
per-inode i_rwsem that is the major bottleneck.

> RDMA either requires specialized hardware or software (soft-iwarp or
> soft-roce).

> Imposing those as requirements isn't going to be viable for a large
> portion of existing deployments.

I don't see clear evidence that most deployments have a buffer alignment
problem. It's easy to pick on and explain, but that doesn't mean it is
pervasive.

Some deployments have intensive performance and scalability
requirements. Those are the ones where RDMA is appropriate and feasible.

Thus IMO you're trying to solve a problem that a) is already solved and
b) does not exist for most NFS users on TCP fabrics.

There is so much low-hanging fruit here. I really don't believe it is
valuable to pursue protocol changes that will take geological amounts
of time and energy to accomplish, especially because we have a solution
now that is effective where it needs to be.

--
Chuck Lever