Re: opensm configuration question

[Ira Weiny <weiny2-i2BcT+NCU+M@public.gmane.org>]

On Tue, 14 Dec 2010 14:42:32 -0800
Tom Ammon <tom.ammon-wbocuHtxKic@public.gmane.org> wrote:

> Hi,
> 
> I've got an opensm question.
> 
> With opensm running, I run opensm --help, and it tells me what I am 
> expecting from the configuration I have given it. I configured it for 
> updn routing with /etc/opensm/rootguids.conf as the root guids file:
> 
> 
> -------------------------------------------------
> OpenSM 3.3.7
>   Reading Cached Option File: /etc/opensm/opensm.conf
>   Loading Cached Option:guid = 0x0002c9030009714d
>   Loading Cached Option:routing_engine = updn
>   Loading Cached Option:root_guid_file = /etc/opensm/rootguids.conf
>   Loading Cached Option:sm_priority = 15
>   Loading Cached Option:log_flags = 4
>   Loading Cached Option:console = local
> Command Line Arguments:
> 
> 
> However, when I do an ibtracert between two nodes on different leaf 
> switches, it appears to be using minhop routing (the root guids I 
> specified are not in the path):

Up/Down does not mean that everything will be routed through your spine
switches.  Up/Down only requires that your path can't go "up" after starting
"down".  From the trace below it appears that the path from em001 to em056
only required 3 hops through the line board.  A node farther away might
require routing through the spine board.

Here's an example:

     S
    / \
  L1   L2
 / \  / \
E1  E2  E3
|    |   |
N1  N2   N3

Min Hop == Up/Down when routing from N1 to N2 (3 hops).

But when routing N1 to N3 Min Hop can choose either E1,L1,S,L2,E3 OR
E1,L1,E2,L2,E3 (both 5 hops) but Up/Down can only chose the former through the
Spine.

> 
> [root@sm2 ~]# ibtracert 24 69
>  From ca {0x0002c903000bb5d8} portnum 1 lid 24-24 "em001 HCA-1"
> [1] -> switch port {0x0008f10500201d8c}[1] lid 15-15 "Voltaire 4036 # 
> komas-is-a1-1"
> [36] -> switch port {0x0008f105006515ca}[19] lid 9-9 "Voltaire sLB-4018 
>     Line 8  Chip 1 4200 #komas-is-a6-1"
> [20] -> switch port {0x0008f10500201b32}[36] lid 18-18 "Voltaire 4036 # 
> komas-is-a2-1"
> [28] -> ca port {0x0002c903000bb5f9}[1] lid 69-69 "em056 HCA-1"
> To ca {0x0002c903000bb5f8} portnum 1 lid 69-69 "em056 HCA-1"
> 
> Looking in the opensm log file, I see this over and over:
> 
> OpenSM $ OpenSM $ OpenSM $ Dec : Command not found
> 
> Supported commands and syntax:
> help [<command>]
> quit (not valid in local mode; use ctl-c)
> loglevel [<log-level>]
> priority [<sm-priority>]
> resweep [heavy|light]
> reroute
> sweep [on|off]
> status [loop]
> logflush -- flush the opensm.log file
> querylid lid -- print internal information about the lid specified
> portstatus [ca|switch|router]
> switchbalance [verbose] [guid]
> lidbalance [switchguid]
> dump_conf
> update_desc
> version -- print the OSM version
> dump_portguid [file filename] regexp1 [regexp2 [regexp3 ...]] -- Dump 
> port GUID matching a regexp
> OpenSM $
> 
> 
> Any ideas?

In the log file???  Not sure.

> 
> Also, how can I use the console? I noticed a couple of config options 
> for it, but I don't understand how to actually attach to the console to 
> try things out.

What options do you have set for the console?

16:08:54 > opensm -h | grep console
--console, -q [off|local|socket|loopback]
          This option activates the OpenSM console (default off).
--console-port, -C <port>
          Specify an alternate telnet port for the console (default 10000).

Ira

> 
> Thanks,
> 
> Tom
> 
> 
> 
> -- 
> Tom Ammon
> Network Engineer
> Office: 801.587.0976
> Mobile: 801.674.9273
> 
> Center for High Performance Computing
> University of Utah
> http://www.chpc.utah.edu
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


-- 
Ira Weiny
Math Programmer/Computer Scientist
Lawrence Livermore National Lab
925-423-8008
weiny2-i2BcT+NCU+M@public.gmane.org
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html