maximum QP size

maximum QP size

[Dimitris Dimitropoulos]

Hi,

I'm running the ibv_rc_pingpong example and I notice as as rx-depth
increases the QP create fails, eg at 50k. My understanding is this
variable controls the number of RECV WR that will be posted in advance
before SEND WRs are posted.

Is there a limitation on the queue size (the size is unit32_t) and if
so why ? Also is there a way around it ?

Regards
Dimitris

Re: maximum QP size

[Leon Romanovsky]

On Thu, Feb 06, 2020 at 09:18:00PM -0800, Dimitris Dimitropoulos wrote:
> Hi,
>
> I'm running the ibv_rc_pingpong example and I notice as as rx-depth
> increases the QP create fails, eg at 50k. My understanding is this
> variable controls the number of RECV WR that will be posted in advance
> before SEND WRs are posted.
>
> Is there a limitation on the queue size (the size is unit32_t) and if
> so why ? Also is there a way around it ?

The rx-depth is translated to the size of completion queue buffer.
That buffer used by hardware to post completions - writing completion
queue elements (CQEs) and it is allocated when creating the CQ.

Maximum number of CQEs can be retrieved by the ibv_devinfo -v, see max_cqe field.

Thanks

>
> Regards
> Dimitris

Re: maximum QP size

[Dimitris Dimitropoulos]

Thank you, there is a wealth of information with the "-v" switch. For
the Mellanox Connect-5 that I'm using I see:

max_cq:                         16777216
max_cqe:                        4194303

I'm encountering a smaller limit on the maximum entries, so I presume
this is the max total number of cqe for all queues in the system.

Regards
Dimitris

> > Hi,
> >
> > I'm running the ibv_rc_pingpong example and I notice as as rx-depth
> > increases the QP create fails, eg at 50k. My understanding is this
> > variable controls the number of RECV WR that will be posted in advance
> > before SEND WRs are posted.
> >
> > Is there a limitation on the queue size (the size is unit32_t) and if
> > so why ? Also is there a way around it ?
>
> The rx-depth is translated to the size of completion queue buffer.
> That buffer used by hardware to post completions - writing completion
> queue elements (CQEs) and it is allocated when creating the CQ.
>
> Maximum number of CQEs can be retrieved by the ibv_devinfo -v, see max_cqe field.