Seeking clarification on the XRC Annex wrt. a TGT QP having a SQ

Seeking clarification on the XRC Annex wrt. a TGT QP having a SQ

[Haakon Bugge]

Hey,


I have a question to XRC TGT QPs and whether they do have a set of requestor resources or not.


The XRC Annex (March 2, 2009 Revision 1.0) (*1) boldly states:

	XRC TGT QPs are similar to RD EECs but do not have a requester side.


Nevertheless, in Table 9, page 36, it is stated that "Local ACK Timeout" and "SQ PSN" are required attributes during an RTR -> RTS transition for an XRC Target QP. This seems to be an incorrect requirement, subject to the XRC Target QP not having a send queue?

Further, looking at a vendor's creation of an XRC TGT QP, we see:

	MLX5_SET(qpc, qpc, no_sq, 1);

in the function create_xrc_tgt().

If the interpretation that an XRC TGT does _not_ have a send queue is correct, we cannot simply remove "Local ACK Timeout" and "SQ PSN" as mandatory attributes during the state transition, because that will break all current software. Is it an idea to move those to optional attributes in the qp_state_table[]? Then remove the IB_QPT_XRC_TGT label in cm_init_qp_rts_attr()?


Thxs, Håkon

*1: IBTA Spec Release 1.6 is equal to the XRC Annex in this respect

Re: Seeking clarification on the XRC Annex wrt. a TGT QP having a SQ

[Leon Romanovsky]

On Mon, Feb 07, 2022 at 02:55:35PM +0000, Haakon Bugge wrote:
> Hey,
> 
> 
> I have a question to XRC TGT QPs and whether they do have a set of requestor resources or not.
> 
> 
> The XRC Annex (March 2, 2009 Revision 1.0) (*1) boldly states:
> 
> 	XRC TGT QPs are similar to RD EECs but do not have a requester side.
> 
> 
> Nevertheless, in Table 9, page 36, it is stated that "Local ACK Timeout" and "SQ PSN" are required attributes during an RTR -> RTS transition for an XRC Target QP. This seems to be an incorrect requirement, subject to the XRC Target QP not having a send queue?
> 
> Further, looking at a vendor's creation of an XRC TGT QP, we see:
> 
> 	MLX5_SET(qpc, qpc, no_sq, 1);
> 
> in the function create_xrc_tgt().
> 
> If the interpretation that an XRC TGT does _not_ have a send queue is correct, we cannot simply remove "Local ACK Timeout" and "SQ PSN" as mandatory attributes during the state transition, because that will break all current software. Is it an idea to move those to optional attributes in the qp_state_table[]? Then remove the IB_QPT_XRC_TGT label in cm_init_qp_rts_attr()?

I think that your interpretation of spec is correct, but why do you want
to remove these attributes? The device ignores them anyway.

Thanks

> 
> 
> Thxs, Håkon
> 
> *1: IBTA Spec Release 1.6 is equal to the XRC Annex in this respect
> 
> 

Re: Seeking clarification on the XRC Annex wrt. a TGT QP having a SQ

[Haakon Bugge]

> On 7 Feb 2022, at 19:59, Leon Romanovsky <leon@kernel.org> wrote:
> 
> On Mon, Feb 07, 2022 at 02:55:35PM +0000, Haakon Bugge wrote:
>> Hey,
>> 
>> 
>> I have a question to XRC TGT QPs and whether they do have a set of requestor resources or not.
>> 
>> 
>> The XRC Annex (March 2, 2009 Revision 1.0) (*1) boldly states:
>> 
>> 	XRC TGT QPs are similar to RD EECs but do not have a requester side.
>> 
>> 
>> Nevertheless, in Table 9, page 36, it is stated that "Local ACK Timeout" and "SQ PSN" are required attributes during an RTR -> RTS transition for an XRC Target QP. This seems to be an incorrect requirement, subject to the XRC Target QP not having a send queue?
>> 
>> Further, looking at a vendor's creation of an XRC TGT QP, we see:
>> 
>> 	MLX5_SET(qpc, qpc, no_sq, 1);
>> 
>> in the function create_xrc_tgt().
>> 
>> If the interpretation that an XRC TGT does _not_ have a send queue is correct, we cannot simply remove "Local ACK Timeout" and "SQ PSN" as mandatory attributes during the state transition, because that will break all current software. Is it an idea to move those to optional attributes in the qp_state_table[]? Then remove the IB_QPT_XRC_TGT label in cm_init_qp_rts_attr()?
> 
> I think that your interpretation of spec is correct, but why do you want
> to remove these attributes? The device ignores them anyway.

:-)

Or, rephrasing your question, why have these attributes as mandatory? The device ignores them anyway ;-)

To me, it is confusing. For example, in rdma_set_min_rnr_timer(), the test for XRC TGT is correct, since an XRG INI QP does not have any responder part. But this is then inconsistent with other parts of CM/CMA.


Thxs, Håkon

> Thanks
> 
>> 
>> 
>> Thxs, Håkon
>> 
>> *1: IBTA Spec Release 1.6 is equal to the XRC Annex in this respect

Re: Seeking clarification on the XRC Annex wrt. a TGT QP having a SQ

[Leon Romanovsky]

On Tue, Feb 08, 2022 at 10:53:15AM +0000, Haakon Bugge wrote:
> 
> 
> > On 7 Feb 2022, at 19:59, Leon Romanovsky <leon@kernel.org> wrote:
> > 
> > On Mon, Feb 07, 2022 at 02:55:35PM +0000, Haakon Bugge wrote:
> >> Hey,
> >> 
> >> 
> >> I have a question to XRC TGT QPs and whether they do have a set of requestor resources or not.
> >> 
> >> 
> >> The XRC Annex (March 2, 2009 Revision 1.0) (*1) boldly states:
> >> 
> >> 	XRC TGT QPs are similar to RD EECs but do not have a requester side.
> >> 
> >> 
> >> Nevertheless, in Table 9, page 36, it is stated that "Local ACK Timeout" and "SQ PSN" are required attributes during an RTR -> RTS transition for an XRC Target QP. This seems to be an incorrect requirement, subject to the XRC Target QP not having a send queue?
> >> 
> >> Further, looking at a vendor's creation of an XRC TGT QP, we see:
> >> 
> >> 	MLX5_SET(qpc, qpc, no_sq, 1);
> >> 
> >> in the function create_xrc_tgt().
> >> 
> >> If the interpretation that an XRC TGT does _not_ have a send queue is correct, we cannot simply remove "Local ACK Timeout" and "SQ PSN" as mandatory attributes during the state transition, because that will break all current software. Is it an idea to move those to optional attributes in the qp_state_table[]? Then remove the IB_QPT_XRC_TGT label in cm_init_qp_rts_attr()?
> > 
> > I think that your interpretation of spec is correct, but why do you want
> > to remove these attributes? The device ignores them anyway.
> 
> :-)
> 
> Or, rephrasing your question, why have these attributes as mandatory? The device ignores them anyway ;-)

I can only guess. The spec is written by HW people, together with
statement than XRC TGT QP same as RC, most likely, the decision was
to take RC block and copy/paste for XRC TGT.

Thanks

> 
> To me, it is confusing. For example, in rdma_set_min_rnr_timer(), the test for XRC TGT is correct, since an XRG INI QP does not have any responder part. But this is then inconsistent with other parts of CM/CMA.



> 
> 
> Thxs, Håkon
> 
> > Thanks
> > 
> >> 
> >> 
> >> Thxs, Håkon
> >> 
> >> *1: IBTA Spec Release 1.6 is equal to the XRC Annex in this respect
>