Re: [RFC PATCH 2/4] nfs: add NFS_CAP_P2PDMA and detect transport support

Chuck Lever
2026-04-02T13:11:26Z

On Wed, Apr 1, 2026, at 3:44 PM, Pranjal Shrivastava wrote:
> The NFS server capabilities bitmask (server->caps) is currently full,
> utilizing all 32 bits of the existing unsigned int. Expand the bitmask
> to 64 bits (u64) to allow for new feature flags.
>
> Introduce a new capability bit, NFS_CAP_P2PDMA, to indicate that the
> local mount is backed by hardware and a transport capable of PCI
> Peer-to-Peer DMA.
>
> Update nfs_server_set_init_caps() to query the underlying SunRPC
> transport for P2PDMA support during the mount process. If the transport
> (e.g., RDMA) signals support, set the NFS_CAP_P2PDMA bit in the mount's
> capabilities. This allows the high-performance Direct I/O path to
> efficiently determine if it should allow P2P memory buffers.

> diff --git a/fs/nfs/client.c b/fs/nfs/client.c
> index be02bb227741..f177cf098d44 100644
> --- a/fs/nfs/client.c
> +++ b/fs/nfs/client.c

> @@ -725,6 +727,12 @@ void nfs_server_set_init_caps(struct nfs_server *server)
> nfs4_server_set_init_caps(server);
> break;
> }
> +
> + rcu_read_lock();
> + xprt = rcu_dereference(server->client->cl_xprt);
> + if (xprt->ops->supports_p2pdma && xprt->ops->supports_p2pdma(xprt))
> + server->caps |= NFS_CAP_P2PDMA;
> + rcu_read_unlock();
> }
> EXPORT_SYMBOL_GPL(nfs_server_set_init_caps);

Is the transport even connected when the NFS client does this
test? If it isn't, xprtrdma and the RDMA core have not chosen
an underlying device yet.

Note that, even if this logic /is/ correct, if the transport
connection is lost the transport will reconnect automatically,
doing the RDMA CM dance again and possibly resolving to a
different device. The NFS client layer will be none-the-wiser
and the NFS_CAP_P2PDMA flag setting will be stale at that point,
and quite possibly incorrect if the new connection's device is
not P2P-enabled.

(Basically this is what happens when an RDMA device is removed).

So this detection has to be done as part of xprtrdma's connection
flow, and it needs to set a flag somewhere in the rpc_xprt. The
NFS direct I/O code path then has to look for that flag before
choosing the mechanism/flags it uses for each iov iter.

--
Chuck Lever