From mboxrd@z Thu Jan 1 00:00:00 1970 From: Gerben Roest Subject: Re: Problems with link, opensm complains IB_SA_MAD_STATUS_REQ_INVALID Date: Fri, 16 Dec 2011 16:37:52 +0100 Message-ID: <4EEB65D0.8040802@grepit.nl> References: <4EEA8004.4060103@grepit.nl> <20111215160600.ebccb033.weiny2@llnl.gov> <4EEB07C3.90803@grepit.nl> <20111216091416.GA3448@calypso> <4EEB216D.2010407@grepit.nl> <4EEB39E8.5030601@dev.mellanox.co.il> <4EEB3FD3.3080409@grepit.nl> <4EEB4362.1050505@dev.mellanox.co.il> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <4EEB4362.1050505-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> Sender: linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org Cc: "linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" List-Id: linux-rdma@vger.kernel.org On 16-12-2011 14:10, Hal Rosenstock wrote: >> They should be DDR cards. Does Rate=2 mean DDR? > > No; it means 1x SDR (lowest speed/width). 4x DDR would be rate 6 (20 > Gbps). See IBA 1.2.1 vol 1 PathRecord SA attribute Rate component. > > By default, OpenSM sets the rate for the IPoIB broadcast groups when not > explicitly specified is rate 3 (10 Gbps) which is 4x SDR. I have a similar twin node that does work correctly (has DDR IB) and it says at ibstat: "Rate: 20" whereas the two that are having problems say "Rate: 2". Testing with openmpi osu_bw show: Rate=2: max bw: 245 MB/s Rate=20: max bw: 1970 MB/s greetings, Gerben -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html