Re: mlx5 + SRP: max_qp_sz mismatch

[Bart Van Assche <bvanassche-HInyCGIudOg@public.gmane.org>]

On 08/27/14 13:28, Eli Cohen wrote:
> On 08/26/14 18:10, Sagi Grimberg wrote:
>>
>> Since I don't know how true send queue size can be computed from the 
>> device capabilities at the moment -I can suggest a fix to srpt to 
>> retry with srp_sq_size/2 (ans so on until it succeeds...)
>>
> The device capabilities provide the maximum number of send work
> requests that the device supports but the actual number of work
> requests that can be supported in a specific case depends on other
> characteristics of the work requests. For example, in the case of
> Connect-IB, the actual number depends on the number of s/g entries,
> the transport type, etc. This is in compliance with the IB spec:
> 
> 11.2.1.2 QUERY HCA
> Description:
> Returns the attributes for the specified HCA.
> The maximum values defined in this section are guaranteed
> not-to-exceed values. It is possible for an implementation to allocate
> some HCA resources from the same space. In that case, the maximum
> values returned are not guaranteed for all of those resources
> simultaneously.
> 
> So, a well written application should try smaller values if it fails
> with ENOMEM.
 
Hello Mark,

It would help if you could test the patch below. Sorry but I don't
have access to a ConnectIB setup myself.

Thanks,

Bart.

Reported-by: Mark Lehrer <lehrer-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Signed-off-by: Bart Van Assche <bvanassche-HInyCGIudOg@public.gmane.org>
---
 drivers/infiniband/ulp/srpt/ib_srpt.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c
index fe09f27..3ffaf4e 100644
--- a/drivers/infiniband/ulp/srpt/ib_srpt.c
+++ b/drivers/infiniband/ulp/srpt/ib_srpt.c
@@ -2091,6 +2091,7 @@ static int srpt_create_ch_ib(struct srpt_rdma_ch *ch)
 	if (!qp_init)
 		goto out;
 
+retry:
 	ch->cq = ib_create_cq(sdev->device, srpt_completion, NULL, ch,
 			      ch->rq_size + srp_sq_size, 0);
 	if (IS_ERR(ch->cq)) {
@@ -2114,6 +2115,13 @@ static int srpt_create_ch_ib(struct srpt_rdma_ch *ch)
 	ch->qp = ib_create_qp(sdev->pd, qp_init);
 	if (IS_ERR(ch->qp)) {
 		ret = PTR_ERR(ch->qp);
+		if (ret == -ENOMEM) {
+			srp_sq_size /= 2;
+			if (srp_sq_size >= MIN_SRPT_SQ_SIZE) {
+				ib_destroy_cq(ch->cq);
+				goto retry;
+			}
+		}
 		printk(KERN_ERR "failed to create_qp ret= %d\n", ret);
 		goto err_destroy_cq;
 	}
-- 
1.8.4.5


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html