Re: [PATCH] RDMA/efa: Add rdma write capability to device caps

[Leon Romanovsky <leon@kernel.org>]

On Tue, Apr 04, 2023 at 03:43:13PM +0000, ynachum@amazon.com wrote:
> From: Yonatan Nachum <ynachum@amazon.com>
> 
> Add rdma write capability that is propagated from the device to
> rdma-core.
> Enable MR creation with remote write permissions according to this
> device capability.
> 
> Reviewed-by: Firas Jahjah <firasj@amazon.com>
> Reviewed-by: Michael Margolin <mrgolin@amazon.com>
> Signed-off-by: Yonatan Nachum <ynachum@amazon.com>
> ---
>  .../infiniband/hw/efa/efa_admin_cmds_defs.h   | 12 ++++--
>  drivers/infiniband/hw/efa/efa_io_defs.h       | 42 +++++++++++++------
>  drivers/infiniband/hw/efa/efa_verbs.c         |  6 ++-
>  include/uapi/rdma/efa-abi.h                   |  1 +
>  4 files changed, 44 insertions(+), 17 deletions(-)

<...>

>  #endif /* _EFA_IO_H_ */
> diff --git a/drivers/infiniband/hw/efa/efa_verbs.c b/drivers/infiniband/hw/efa/efa_verbs.c
> index c27c36418f7f..a394011a598c 100644
> --- a/drivers/infiniband/hw/efa/efa_verbs.c
> +++ b/drivers/infiniband/hw/efa/efa_verbs.c
> @@ -253,6 +253,9 @@ int efa_query_device(struct ib_device *ibdev,
>  		if (EFA_DEV_CAP(dev, DATA_POLLING_128))
>  			resp.device_caps |= EFA_QUERY_DEVICE_CAPS_DATA_POLLING_128;
>  
> +		if (EFA_DEV_CAP(dev, RDMA_WRITE))
> +			resp.device_caps |= EFA_QUERY_DEVICE_CAPS_RDMA_WRITE;
> +
>  		if (dev->neqs)
>  			resp.device_caps |= EFA_QUERY_DEVICE_CAPS_CQ_NOTIFICATIONS;
>  
> @@ -1571,7 +1574,8 @@ static struct efa_mr *efa_alloc_mr(struct ib_pd *ibpd, int access_flags,
>  
>  	supp_access_flags =
>  		IB_ACCESS_LOCAL_WRITE |
> -		(EFA_DEV_CAP(dev, RDMA_READ) ? IB_ACCESS_REMOTE_READ : 0);
> +		(EFA_DEV_CAP(dev, RDMA_READ) ? IB_ACCESS_REMOTE_READ : 0) |
> +		(EFA_DEV_CAP(dev, RDMA_WRITE) ? IB_ACCESS_REMOTE_WRITE : 0);
>  
>  	access_flags &= ~IB_ACCESS_OPTIONAL;
>  	if (access_flags & ~supp_access_flags) {
> diff --git a/include/uapi/rdma/efa-abi.h b/include/uapi/rdma/efa-abi.h
> index 74406b4817ce..d94c32f28804 100644
> --- a/include/uapi/rdma/efa-abi.h
> +++ b/include/uapi/rdma/efa-abi.h
> @@ -121,6 +121,7 @@ enum {
>  	EFA_QUERY_DEVICE_CAPS_CQ_NOTIFICATIONS = 1 << 2,
>  	EFA_QUERY_DEVICE_CAPS_CQ_WITH_SGID     = 1 << 3,
>  	EFA_QUERY_DEVICE_CAPS_DATA_POLLING_128 = 1 << 4,
> +	EFA_QUERY_DEVICE_CAPS_RDMA_WRITE = 1 << 5,

Why do you need special device capability while all rdma-core users
set IBV_ACCESS_REMOTE_WRITE anyway without relying on anything from
providers?

Thanks

>  };
>  
>  struct efa_ibv_ex_query_device_resp {
> -- 
> 2.39.2
>