Re: [PATCH v2 4/4] RDMA/srpt: Increase max_send_sge

[Leon Romanovsky <leonro@mellanox.com>]

On Mon, May 25, 2020 at 02:51:25PM -0700, Bart Van Assche wrote:
> On 2020-05-25 12:15, Bart Van Assche wrote:
> > On 2020-05-25 10:51, Leon Romanovsky wrote:
> >> On Mon, May 25, 2020 at 10:22:12AM -0700, Bart Van Assche wrote:
> >>> The ib_srpt driver limits max_send_sge to 16. Since that is a workaround
> >>> for an mlx4 bug that has been fixed, increase max_send_sge. For mlx4, do
> >>> not use the value advertised by the driver (32) since that causes QP's
> >>> to transition to the error status.
> >>
> >> How are you avoiding mlx4 bug in this patch?
> >> Isn't "attrs->max_send_sge" come from driver as is?
> >
> > Hi Leon,
> >
> > Development of this patch started considerable time ago - before the
> > ib_srpt driver was converted to the RDMA R/W API. Before that conversion
> > the ib_srpt driver was using attrs->max_send_sge limit for both RDMA
> > writes and RDMA reads, which is wrong. Hence the need for the
> > "max_sge_delta" parameter (max_send_sge = 32 and max_sge_rd = 30 for
> > mlx4). The following code from drivers/infiniband/core/rw.c selects the
> > proper limit:
> >
> >        u32 max_sge = dir == DMA_TO_DEVICE ? qp->max_write_sge :
> >                      qp->max_read_sge;
> >
> > The following code in drivers/infiniband/core/verbs.c sets these limits:
> >
> >         qp->max_write_sge = qp_init_attr->cap.max_send_sge;
> >         qp->max_read_sge = min_t(u32, qp_init_attr->cap.max_send_sge,
> >                                  device->attrs.max_sge_rd);
>
> The commit message should be shortened to the following: "The ib_srpt
> driver limits max_send_sge to 16. Since that is a workaround
> for an mlx4 bug that has been fixed, increase max_send_sge. See also
> commit f95ccffc715b ("IB/mlx4: Use 4K pages for kernel QP's WQE buffer")."

Yes, please. The proposed commit message describes better the change.

Thanks

>
> Bart.