out of order packets when querying all ports in a switch through OpenSM

out of order packets when querying all ports in a switch through OpenSM

[Jesus Camacho Villanueva]

Hello,

I am trying to send a port info request to all the ports in a switch,
one after the other.

I am sending all requests to the same switch in sequence through the
same directed routing path:

Request port number 1
Request port number 2
Request port number 3
Request port number 4

But sometimes I get responses out of order:

Response port number 2
Response port number 1
Response port number 3
Response port number 4

Is it possible that the switch is processing port requests out of
order (or maybe I am doing something wrong)?

Best regards,
Jesus
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: out of order packets when querying all ports in a switch through OpenSM

[Hal Rosenstock]

Hi Jesus,

On 5/27/2015 8:56 AM, Jesus Camacho Villanueva wrote:
> Hello,
> 
> I am trying to send a port info request to all the ports in a switch,
> one after the other.
> 
> I am sending all requests to the same switch in sequence through the
> same directed routing path:
> 
> Request port number 1
> Request port number 2
> Request port number 3
> Request port number 4
> 
> But sometimes I get responses out of order:
> 
> Response port number 2
> Response port number 1
> Response port number 3
> Response port number 4
> 
> Is it possible that the switch is processing port requests out of
> order (or maybe I am doing something wrong)?

One way this could occur is that request to switch for port number 1
either does not make it to switch or response does not make it back and
it times out and is retransmitted by kernel. This depends on how umad is
setup in terms of retries and timeout.

Is it always the same switch or switch type ? What switch is it ?

-- Hal

> Best regards,
> Jesus
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: out of order packets when querying all ports in a switch through OpenSM

[Jesus Camacho Villanueva]

Hi Hal,

Thanks for your quick response :)

What you say here makes sense. I have seen this behaviour in one of
the switches, but I don't discard that this problem can arise in other
switches.

The switch is: Infiniscale-IV Mellanox Technologies 4xQDR

Jesus

On Wed, May 27, 2015 at 3:55 PM, Hal Rosenstock <hal-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> wrote:
> Hi Jesus,
>
> On 5/27/2015 8:56 AM, Jesus Camacho Villanueva wrote:
>> Hello,
>>
>> I am trying to send a port info request to all the ports in a switch,
>> one after the other.
>>
>> I am sending all requests to the same switch in sequence through the
>> same directed routing path:
>>
>> Request port number 1
>> Request port number 2
>> Request port number 3
>> Request port number 4
>>
>> But sometimes I get responses out of order:
>>
>> Response port number 2
>> Response port number 1
>> Response port number 3
>> Response port number 4
>>
>> Is it possible that the switch is processing port requests out of
>> order (or maybe I am doing something wrong)?
>
> One way this could occur is that request to switch for port number 1
> either does not make it to switch or response does not make it back and
> it times out and is retransmitted by kernel. This depends on how umad is
> setup in terms of retries and timeout.
>
> Is it always the same switch or switch type ? What switch is it ?
>
> -- Hal
>
>> Best regards,
>> Jesus
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: out of order packets when querying all ports in a switch through OpenSM

[Hal Rosenstock]

On 5/27/2015 10:16 AM, Jesus Camacho Villanueva wrote:
> Hi Hal,
> 
> Thanks for your quick response :)
> 
> What you say here makes sense. I have seen this behaviour in one of
> the switches, but I don't discard that this problem can arise in other
> switches.

Note that for MADs, transaction ID is "looser" than traditional
transaction ID semantics in that "the combination of TID, SGID, and
MgmtClass is different from that of any currently executing operation"
and does not imply ordering. If there is ordering required, it is the
responsibility of the manager to enforce that.

> The switch is: Infiniscale-IV Mellanox Technologies 4xQDR

Is it a managed or unmanaged switch ?

-- Hal

> 
> Jesus
> 
> On Wed, May 27, 2015 at 3:55 PM, Hal Rosenstock <hal-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> wrote:
>> Hi Jesus,
>>
>> On 5/27/2015 8:56 AM, Jesus Camacho Villanueva wrote:
>>> Hello,
>>>
>>> I am trying to send a port info request to all the ports in a switch,
>>> one after the other.
>>>
>>> I am sending all requests to the same switch in sequence through the
>>> same directed routing path:
>>>
>>> Request port number 1
>>> Request port number 2
>>> Request port number 3
>>> Request port number 4
>>>
>>> But sometimes I get responses out of order:
>>>
>>> Response port number 2
>>> Response port number 1
>>> Response port number 3
>>> Response port number 4
>>>
>>> Is it possible that the switch is processing port requests out of
>>> order (or maybe I am doing something wrong)?
>>
>> One way this could occur is that request to switch for port number 1
>> either does not make it to switch or response does not make it back and
>> it times out and is retransmitted by kernel. This depends on how umad is
>> setup in terms of retries and timeout.
>>
>> Is it always the same switch or switch type ? What switch is it ?
>>
>> -- Hal
>>
>>> Best regards,
>>> Jesus
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
>>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: out of order packets when querying all ports in a switch through OpenSM

[Jesus Camacho Villanueva]

The switches are unmanaged.
Is it possible to ensure in-order packets with managed switches?

Jesus

On Wed, May 27, 2015 at 4:30 PM, Hal Rosenstock <hal-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> wrote:
> On 5/27/2015 10:16 AM, Jesus Camacho Villanueva wrote:
>> Hi Hal,
>>
>> Thanks for your quick response :)
>>
>> What you say here makes sense. I have seen this behaviour in one of
>> the switches, but I don't discard that this problem can arise in other
>> switches.
>
> Note that for MADs, transaction ID is "looser" than traditional
> transaction ID semantics in that "the combination of TID, SGID, and
> MgmtClass is different from that of any currently executing operation"
> and does not imply ordering. If there is ordering required, it is the
> responsibility of the manager to enforce that.
>
>> The switch is: Infiniscale-IV Mellanox Technologies 4xQDR
>
> Is it a managed or unmanaged switch ?
>
> -- Hal
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: out of order packets when querying all ports in a switch through OpenSM

[Hal Rosenstock]

On 5/27/2015 11:31 AM, Jesus Camacho Villanueva wrote:
> The switches are unmanaged.
> Is it possible to ensure in-order packets with managed switches?

No; same issue as I previously mentioned exists with both.

Unmanaged switches just use firmware whereas managed switches have CPU
and kernel in addition to firmware so the system is more complex.

-- Hal

> Jesus
> 
> On Wed, May 27, 2015 at 4:30 PM, Hal Rosenstock <hal-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> wrote:
>> On 5/27/2015 10:16 AM, Jesus Camacho Villanueva wrote:
>>> Hi Hal,
>>>
>>> Thanks for your quick response :)
>>>
>>> What you say here makes sense. I have seen this behaviour in one of
>>> the switches, but I don't discard that this problem can arise in other
>>> switches.
>>
>> Note that for MADs, transaction ID is "looser" than traditional
>> transaction ID semantics in that "the combination of TID, SGID, and
>> MgmtClass is different from that of any currently executing operation"
>> and does not imply ordering. If there is ordering required, it is the
>> responsibility of the manager to enforce that.
>>
>>> The switch is: Infiniscale-IV Mellanox Technologies 4xQDR
>>
>> Is it a managed or unmanaged switch ?
>>
>> -- Hal
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html