Re: Problems with link, opensm complains IB_SA_MAD_STATUS_REQ_INVALID

[Gerben Roest <g.roest-99SnrGqf+M9mR6Xm/wNWPw@public.gmane.org>]

On 16-12-2011 16:43, Hal Rosenstock wrote:
> On 12/16/2011 10:37 AM, Gerben Roest wrote:
>> On 16-12-2011 14:10, Hal Rosenstock wrote:
>>
>>>> They should be DDR cards. Does Rate=2 mean DDR?
>>>
>>> No; it means 1x SDR (lowest speed/width). 4x DDR would be rate 6 (20
>>> Gbps). See IBA 1.2.1 vol 1 PathRecord SA attribute Rate component.
>>>
>>> By default, OpenSM sets the rate for the IPoIB broadcast groups when not
>>> explicitly specified is rate 3 (10 Gbps) which is 4x SDR.
>>
>> I have a similar twin node that does work correctly (has DDR IB) and it
>> says at ibstat: "Rate: 20"
>> whereas the two that are having problems say "Rate: 2".
>>
>> Testing with openmpi osu_bw show:
>>
>> Rate=2:	max bw: 245 MB/s
>> Rate=20: max bw: 1970 MB/s
> 
> Yes, that's consistent.
> 
> Can you temporarily try the cable that is known to work (for rate 20)
> between the ports that come up at rate 2 and see if they come up
> properly (at rate 20) ?

Yes, I'll try that but that will be next week or so. I'll get back to
you on that.

thanks for your help,

Gerben
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html