XRC Questions for Mellanox and All

XRC Questions for Mellanox and All

[Amrani, Ram]

Hi Mellanox, All,
I've been reading XRC code, currently implemented only by Mellanox.
I have a few questions regarding specifications vs. implementation.

 (1) The protocol specifies:
11.6.3.2 AFFILIATED ASYNCHRONOUS ERRORS
...
The following describes the new Affiliated Asynchronous Errors for
XRC TGT QPs:
* XRC Domain Violation - Responder's Receive Queue detected an
XRC Domain that does not match the XRC Domain of the XRC SRQ.
* Invalid XRCETH - Responder detected that the XRC SRQ does not
exist or is not in the right state or wire protocol violation.

I don't see any dedicated entries in the enum ib_event_type.
Why? How, do you currently treat these errors?

(2) The protocol specifies:
11.4.2.1 POLL FOR COMPLETION
...
Output Modifiers:
...
* Local XRC TGT QP Number. Returned only for completions
of WRs posted to XRC SRQs.

I don't see any dedicated field in the struct ib_wc.
Why? How, do you currently return this value, if at all?

(3) The protocol specifies:
11.4.2.1 POLL FOR COMPLETION 
A new "XRC violation error" is returned for requests that caused the responder
to return a "NAK-Invalid RD Request" NAK. This could have been
caused by either a Remote XRC Domain Violation or an XRCETH Violation
as detailed in the transport section.

What entry from the enum ib_wc_status do you use for this?

Thanks,
Ram

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: XRC Questions for Mellanox and All

[Amrani, Ram]

Hi Leon, Mellanox,
Can you help get a comment on this e-mail?

(Added Sean and Roland as by git log they seem related too)

Thanks,
Ram


> -----Original Message-----
> [This sender failed our fraud detection checks and may not be who they appear to be. Learn about spoofing at
> http://aka.ms/LearnAboutSpoofing]
> 
> Hi Mellanox, All,
> I've been reading XRC code, currently implemented only by Mellanox.
> I have a few questions regarding specifications vs. implementation.
> 
>  (1) The protocol specifies:
> 11.6.3.2 AFFILIATED ASYNCHRONOUS ERRORS
> ...
> The following describes the new Affiliated Asynchronous Errors for
> XRC TGT QPs:
> * XRC Domain Violation - Responder's Receive Queue detected an
> XRC Domain that does not match the XRC Domain of the XRC SRQ.
> * Invalid XRCETH - Responder detected that the XRC SRQ does not
> exist or is not in the right state or wire protocol violation.
> 
> I don't see any dedicated entries in the enum ib_event_type.
> Why? How, do you currently treat these errors?
> 
> (2) The protocol specifies:
> 11.4.2.1 POLL FOR COMPLETION
> ...
> Output Modifiers:
> ...
> * Local XRC TGT QP Number. Returned only for completions
> of WRs posted to XRC SRQs.
> 
> I don't see any dedicated field in the struct ib_wc.
> Why? How, do you currently return this value, if at all?
> 
> (3) The protocol specifies:
> 11.4.2.1 POLL FOR COMPLETION
> A new "XRC violation error" is returned for requests that caused the responder
> to return a "NAK-Invalid RD Request" NAK. This could have been
> caused by either a Remote XRC Domain Violation or an XRCETH Violation
> as detailed in the transport section.
> 
> What entry from the enum ib_wc_status do you use for this?
> 
> Thanks,
> Ram
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: XRC Questions for Mellanox and All

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 159 bytes --]

On Sun, Nov 19, 2017 at 12:02:10PM +0000, Amrani, Ram wrote:
> Hi Leon, Mellanox,
> Can you help get a comment on this e-mail?

Jack is working on it,

Thanks

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

RE: XRC Questions for Mellanox and All

[Amrani, Ram]

> Jack is working on it,
> 

I'll patiently wait for a response then.

Thanks,
Ram

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: XRC Questions for Mellanox and All

[Amrani, Ram]

> > Jack is working on it,
> >
> 
> I'll patiently wait for a response then.
> 
> Thanks,
> Ram
> 

Any news on this?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: XRC Questions for Mellanox and All

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 329 bytes --]

On Tue, Dec 12, 2017 at 09:49:31AM +0000, Amrani, Ram wrote:
> > > Jack is working on it,
> > >
> >
> > I'll patiently wait for a response then.
> >
> > Thanks,
> > Ram
> >
>
> Any news on this?

Ram,

I know that it doesn't look like this, but we (Jack and me) didn't
forget it. It was supposed to be answered by Liran.

Thanks

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: XRC Questions for Mellanox and All

[jackm]

On Sun, 19 Nov 2017 12:02:10 +0000
"Amrani, Ram" <Ram.Amrani-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org> wrote:


Hi Ram,
I apologize for the delay in responding.

The Linux XRC implementation does not provide XRC specific errors.
Rather, these errors are perceived as local access violation errors at
the responder. Consequently, at the responder side, for both XRCETH and
XRC domain violation errors, a local access violation asynchronous
error is reported (IB_EVENT_QP_ACCESS_ERR). The remote QP then
transitions to the ERR state.

At the requester side, these errors are perceived as Remote Invalid
Request errors, and generate IB_WC_REM_INV_REQ_ERR completions. The
local QP then transitions to the ERR state.

Operationally, then, the resources behave as they should upon XRC
errors.

On the responder side, XRC receive completions report the XRC_TGT_QP in
the “qp” field of the ib_wc structure;  the XRC SRQ number is returned
in field “src_qpn”.

(The src_qp field was previously used only for UD qps; there was no
reason not to use it to return the XRC SRQ number, especially as doing
this meant no need to add a new field to struct ib_wc).

-Jack

> Hi Leon, Mellanox,
> Can you help get a comment on this e-mail?
> 
> (Added Sean and Roland as by git log they seem related too)
> 
> Thanks,
> Ram
> 
> 
> > -----Original Message-----
> > [This sender failed our fraud detection checks and may not be who
> > they appear to be. Learn about spoofing at
> > http://aka.ms/LearnAboutSpoofing]
> > 
> > Hi Mellanox, All,
> > I've been reading XRC code, currently implemented only by Mellanox.
> > I have a few questions regarding specifications vs. implementation.
> > 
> >  (1) The protocol specifies:
> > 11.6.3.2 AFFILIATED ASYNCHRONOUS ERRORS
> > ...
> > The following describes the new Affiliated Asynchronous Errors for
> > XRC TGT QPs:
> > * XRC Domain Violation - Responder's Receive Queue detected an
> > XRC Domain that does not match the XRC Domain of the XRC SRQ.
> > * Invalid XRCETH - Responder detected that the XRC SRQ does not
> > exist or is not in the right state or wire protocol violation.
> > 
> > I don't see any dedicated entries in the enum ib_event_type.
> > Why? How, do you currently treat these errors?
> > 
> > (2) The protocol specifies:
> > 11.4.2.1 POLL FOR COMPLETION
> > ...
> > Output Modifiers:
> > ...
> > * Local XRC TGT QP Number. Returned only for completions
> > of WRs posted to XRC SRQs.
> > 
> > I don't see any dedicated field in the struct ib_wc.
> > Why? How, do you currently return this value, if at all?
> > 
> > (3) The protocol specifies:
> > 11.4.2.1 POLL FOR COMPLETION
> > A new "XRC violation error" is returned for requests that caused
> > the responder to return a "NAK-Invalid RD Request" NAK. This could
> > have been caused by either a Remote XRC Domain Violation or an
> > XRCETH Violation as detailed in the transport section.
> > 
> > What entry from the enum ib_wc_status do you use for this?
> > 
> > Thanks,
> > Ram
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe
> > linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html  
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma"
> in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: XRC Questions for Mellanox and All

[Amrani, Ram]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 1327 bytes --]

> Hi Ram,
> I apologize for the delay in responding.
> 
> The Linux XRC implementation does not provide XRC specific errors.
> Rather, these errors are perceived as local access violation errors at
> the responder. Consequently, at the responder side, for both XRCETH and
> XRC domain violation errors, a local access violation asynchronous
> error is reported (IB_EVENT_QP_ACCESS_ERR). The remote QP then
> transitions to the ERR state.
> 
> At the requester side, these errors are perceived as Remote Invalid
> Request errors, and generate IB_WC_REM_INV_REQ_ERR completions. The
> local QP then transitions to the ERR state.
> 
> Operationally, then, the resources behave as they should upon XRC
> errors.
> 
> On the responder side, XRC receive completions report the XRC_TGT_QP in
> the “qp” field of the ib_wc structure;  the XRC SRQ number is returned
> in field “src_qpn”.
> 
> (The src_qp field was previously used only for UD qps; there was no
> reason not to use it to return the XRC SRQ number, especially as doing
> this meant no need to add a new field to struct ib_wc).
> 
> -Jack
> 

OK. We'll implement in the same manner.

Thanks,
Ram

N‹§²æìr¸›yúèšØb²X¬¶Ç§vØ^–)Þº{.nÇ+‰·¥Š{±­ÙšŠ{ayº\x1dʇڙë,j\a­¢f£¢·hš‹»öì\x17/oSc¾™Ú³9˜uÀ¦æå‰È&jw¨®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿïêäz¹Þ–Šàþf£¢·hšˆ§~ˆmš