Re: [RFC PATCH 2/4] nfs: add NFS_CAP_P2PDMA and detect transport support

[Pranjal Shrivastava <praan@google.com>]

On Thu, Apr 02, 2026 at 09:11:04AM -0400, Chuck Lever wrote:
> 
> On Wed, Apr 1, 2026, at 3:44 PM, Pranjal Shrivastava wrote:
> > The NFS server capabilities bitmask (server->caps) is currently full,
> > utilizing all 32 bits of the existing unsigned int. Expand the bitmask
> > to 64 bits (u64) to allow for new feature flags.
> >
> > Introduce a new capability bit, NFS_CAP_P2PDMA, to indicate that the
> > local mount is backed by hardware and a transport capable of PCI
> > Peer-to-Peer DMA.
> >
> > Update nfs_server_set_init_caps() to query the underlying SunRPC
> > transport for P2PDMA support during the mount process. If the transport
> > (e.g., RDMA) signals support, set the NFS_CAP_P2PDMA bit in the mount's
> > capabilities. This allows the high-performance Direct I/O path to
> > efficiently determine if it should allow P2P memory buffers.
> 
> > diff --git a/fs/nfs/client.c b/fs/nfs/client.c
> > index be02bb227741..f177cf098d44 100644
> > --- a/fs/nfs/client.c
> > +++ b/fs/nfs/client.c
> 
> > @@ -725,6 +727,12 @@ void nfs_server_set_init_caps(struct nfs_server *server)
> >  		nfs4_server_set_init_caps(server);
> >  		break;
> >  	}
> > +
> > +	rcu_read_lock();
> > +	xprt = rcu_dereference(server->client->cl_xprt);
> > +	if (xprt->ops->supports_p2pdma && xprt->ops->supports_p2pdma(xprt))
> > +		server->caps |= NFS_CAP_P2PDMA;
> > +	rcu_read_unlock();
> >  }
> >  EXPORT_SYMBOL_GPL(nfs_server_set_init_caps);
> 
> Is the transport even connected when the NFS client does this
> test? If it isn't, xprtrdma and the RDMA core have not chosen
> an underlying device yet.
> 
> Note that, even if this logic /is/ correct, if the transport
> connection is lost the transport will reconnect automatically,
> doing the RDMA CM dance again and possibly resolving to a
> different device. The NFS client layer will be none-the-wiser
> and the NFS_CAP_P2PDMA flag setting will be stale at that point,
> and quite possibly incorrect if the new connection's device is
> not P2P-enabled.
> 
> (Basically this is what happens when an RDMA device is removed).
> 
> So this detection has to be done as part of xprtrdma's connection
> flow, and it needs to set a flag somewhere in the rpc_xprt. The
> NFS direct I/O code path then has to look for that flag before
> choosing the mechanism/flags it uses for each iov iter.
> 

Ack. I agree, so should we start with an inital cap and then update it 
in the event of a transport change / disconnect? Or shall we populate 
the cap only when a transport is connected?

Thanks,
Praan