Re: [PATCH RFC 2/2] RDMA/isert: Support iWARP transport

[Sagi Grimberg <sagig@dev.mellanox.co.il>]

On 6/25/2015 8:06 PM, Steve Wise wrote:
>
>
>> -----Original Message-----
>> From: linux-rdma-owner@vger.kernel.org [mailto:linux-rdma-owner@vger.kernel.org] On Behalf Of Sagi Grimberg
>> Sent: Thursday, June 25, 2015 11:51 AM
>> To: Steve Wise; linux-rdma@vger.kernel.org
>> Cc: Or Gerlitz; Roi Dayan; target-devel
>> Subject: Re: [PATCH RFC 2/2] RDMA/isert: Support iWARP transport
>>
>> On 6/25/2015 6:39 PM, Steve Wise wrote:
>>> Memory regions that are the target of an iWARP RDMA READ RESPONSE need
>>> REMOTE_WRITE access rights.  So enable REMOTE_WRITE for iWARP devices.
>>>
>>> iWARP RDMA READ target sge depth is 1.  So save the max_read_sge in
>>> the target device structure and use that when creating RDMA_READ work
>>> requests.
>>
>> Hi Steve,
>>
>> CC'ing target-devel...
>>
>>>
>>> Signed-off-by: Steve Wise <swise@opengridcomputing.com>
>>> Tested-by: Vasu Dev <vasu.dev@intel.com>
>>> ---
>>>    drivers/infiniband/ulp/isert/ib_isert.c |   55 ++++++++++++++++++++++++++-----
>>>    drivers/infiniband/ulp/isert/ib_isert.h |    1 +
>>>    2 files changed, 48 insertions(+), 8 deletions(-)
>>>
>>> diff --git a/drivers/infiniband/ulp/isert/ib_isert.c b/drivers/infiniband/ulp/isert/ib_isert.c
>>> index 9e7b492..b5cde5d 100644
>>> --- a/drivers/infiniband/ulp/isert/ib_isert.c
>>> +++ b/drivers/infiniband/ulp/isert/ib_isert.c
>>> @@ -165,6 +165,11 @@ isert_create_qp(struct isert_conn *isert_conn,
>>>    	attr.cap.max_send_sge = max(2, device->dev_attr.max_sge - 2);
>>>    	isert_conn->max_sge = attr.cap.max_send_sge;
>>>
>>> +	if (rdma_cap_read_multi_sge(device->ib_device, cma_id->port_num))
>>> +		isert_conn->max_read_sge = isert_conn->max_sge;
>>> +	else
>>> +		isert_conn->max_read_sge = 1;
>>> +
>>
>> I think it would make sense to change now max_sge to max_write_sge.
>>
>
> Ok.
>
>> And, shouldn't we just use:
>> max_wr_sge = max(2, device->dev_attr.max_sge - 2);
>> max_rd_sge = device->dev_attr.max_sge_rd;
>>
>> Does the Chelsio device reports max_sge_rd != 1 ?
>>
>
> Yes, but I wasn't sure max_sge_rd is really the read target max sge.

Why not?

> I don't see it being set by the mlx* drivers.

Umm, that's a bug.
Let me send an easy fix for all mlx drivers.

> Also since we have the new rdma_cap_read_multi_sge() helper, I thought I should use it. :)

I think that reading the exact device caps max_sge, max_sge_is better
and more straight forward...