* Re: CQ creation for RDMA use case
[not found] <BL1PR21MB32831892955F3493715EEE33D6869@BL1PR21MB3283.namprd21.prod.outlook.com>
@ 2022-07-21 18:58 ` Jason Gunthorpe
0 siblings, 0 replies; only message in thread
From: Jason Gunthorpe @ 2022-07-21 18:58 UTC (permalink / raw)
To: Ajay Sharma; +Cc: linux-rdma@vger.kernel.org, Long Li, Mahmoud Elhaddad
On Tue, Jul 12, 2022 at 07:55:21PM +0000, Ajay Sharma wrote:
> Hello,
>
> We have a situation where the hardware treats cq creation differently
> when attaching it to RAW_QP use case vs the RC_QP use case. The
> situation is as follows :
>
>
> When creating the cq for RAW_QP_TYPE use case , the hw requires that
> memory be registered with the hw but it does not yet attaches it to the
> qp meaning it does not yet allocate any hw side resources for the cq.
>
> The hardware will allocate resources for this qp type only when it
> see’s the work queue creation request and that’s when it allocates
> resources for both cq and wq.
>
>
> When creating cq for the RC_QP_TYPE use case the hardware creates
> resources immediately on cq creation request.
>
>
> The problem is from the user mode side there is no way to tell whether
> this cq is intended for RAW QP use case or RC QP use case. Can we add a
> flag in the ibv_cq_init_attr_ex to differentiate the use case we are
> trying to address.
>
> We have explored the option of customizing in the firmware side and at
> this point it seems a very expensive operation and would like to avoid
> that.
I don't have a good advice for you, this is very non-compliant if CQs
can only be used in certain configurations. The entire verbs is
designed around the idea that the CQ is a sharable object - having a
device that creates a CQ for every queue is very far away.
Why can you not correct this?
A flag is not so sufficient, it is an entire set of bad behavior that you
have to opt into.
You may consider a dv for creating a QP/CQ pair if that models your
HW.
Jason
^ permalink raw reply [flat|nested] only message in thread