From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Jim Schutt" Subject: Re: [PATCH] mlx4: allow for 4K mtu configuration of IB ports Date: Thu, 26 May 2011 08:30:17 -0600 Message-ID: <4DDE63F9.8060502@sandia.gov> References: <055701cc1b29$c8fcc6c0$5af65440$@systemfabricworks.com> <4DDDF0B1.6090305@mellanox.com> <17495_1306419903_p4QEMqSb018281_4DDE6291.1090809@sandia.gov> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <17495_1306419903_p4QEMqSb018281_4DDE6291.1090809-4OHPYypu0djtX7QSmKvirg@public.gmane.org> Sender: linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: Jim Schutt Cc: Or Gerlitz , Roland Dreier , Bob Pearson , Or Gerlitz , linux-rdma , Vladimir Sokolovsky , Alex Netes List-Id: linux-rdma@vger.kernel.org Jim Schutt wrote: > Or Gerlitz wrote: >> Roland Dreier wrote: >>> Bob Pearson wrote: >>>> With lash+mesh redsky required 6-7 VLs to wire up without deadlocks. >>>> I think >>>> that Jim's version uses 8 SLs but only 2VLs to work. >>>> If someone was using a torus and also wanted to support QOS and also >>>> wanted >>>> to separate multicast and management on a separate VL to be >>>> absolutely sure >>>> that there is no possibility of a deadlock you might end up with >>>> #QOS * 2 + >>>> 1 which would be 5 using the current algorithm. >> >>> But again you don't need all those VLs on the HCAs' links, do you? >> >> Jason Gunthorpe wrote: >>> Routing algorithms only need VLs on interswitch links, not on HCA to >>> switch links. The only use of the HCA to switch VLs is for QoS. Mesh >>> topologies can usually be routed with only two VLs, but you need alot >>> of SLs to make that work. >> >> Bob, Jim, Alex >> >> I wasn't sure if the SL-to-VL mapping done by open SM is dictated by >> the directives @ the user config file or if the routing algorithm is >> "VL aware" but the routing engine? if the latter, do interswitch links >> use different mapping vs. HCA - switch links? > > FWIW, the torus-2QoS routing engine uses VL bit 0 for torus deadlock > avoidance, VL bit 1 to route around a missing switch without deadlocks, > and VL bit 2 to provide two QoS levels. It needs the port dependence > of the SL2VL maps to do this in switches. > > The interswitch and HCAs use the same mapping, but only VL bit 2 > is needed on HCAs, to provide the QoS levels. It occurred to me as soon I sent the above that there's no good reason to insist that the VL usage is the same for both interswitch links, and switch-CA links. Do I need to change this? -- Jim > > I chose that bit usage because it seemed the proper ordering of > capabilities if there are fewer than 8 data VLs available - basic > deadlock avoidance is most important; some QoS is nice to have but > not that useful if the fabric can deadlock. > > Is that what you were asking, at least WRT. torus-2QoS? > > -- Jim > >> >> Or. >> >> > > > -- > To unsubscribe from this list: send the line "unsubscribe linux-rdma" in > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html