Re: [PATCH v2] opensm: Modify connect_roots to allow connectivity between all switches

[Yevgeny Kliteynik <kliteyn-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>]

On 15-Jun-10 10:09 AM, Eli Dorfman (Voltaire) wrote:
> Yevgeny Kliteynik wrote:
>> Eli,
>>
>> On 13-Jun-10 6:10 PM, Eli Dorfman (Voltaire) wrote:
>>> After a second thought and in order not to break current configuration,
>>> I send this modified patch that does not change connect_roots option but
>>> changes its functionality in up-down (I think that in fat-tree it is
>>> already implemented)
>>>
>>> Modify connect_roots option to allow connectivity between
>>> all switches in up-down routing algorithm and in this way be
>>> fully IBA compliant
>>>
>>> Signed-off-by: Eli Dorfman<elid-smomgflXvOZWk0Htik3J/w@public.gmane.org>
>>> ---
>>>    opensm/man/opensm.8.in         |    2 +-
>>>    opensm/opensm/main.c           |    2 +-
>>>    opensm/opensm/osm_ucast_updn.c |    4 +---
>>>    3 files changed, 3 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/opensm/man/opensm.8.in b/opensm/man/opensm.8.in
>>> index 9053611..c67126e 100644
>>> --- a/opensm/man/opensm.8.in
>>> +++ b/opensm/man/opensm.8.in
>>> @@ -174,7 +174,7 @@ the host comes back online.
>>>    .TP
>>>    \fB\-z\fR, \fB\-\-connect_roots\fR
>>>    This option enforces routing engines (up/down and
>>> -fat-tree) to make connectivity between root switches and in
>>> +fat-tree) to make connectivity between all switches and in
>>>    this way to be fully IBA complaint. In many cases this can
>>>    violate "pure" deadlock free algorithm, so use it carefully.
>>>    .TP
>>> diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c
>>> index 0093aa7..82ca78f 100644
>>> --- a/opensm/opensm/main.c
>>> +++ b/opensm/opensm/main.c
>>> @@ -187,7 +187,7 @@ static void show_usage(void)
>>>               "          Sets the SL to use to communicate with the
>>> SM/SA. Defaults to 0.\n\n");
>>>        printf("--connect_roots, -z\n"
>>>               "          This option enforces routing engines (up/down
>>> and \n"
>>> -           "          fat-tree) to make connectivity between root
>>> switches\n"
>>> +           "          fat-tree) to make connectivity between all
>>> switches\n"
>>>               "          and in this way be IBA compliant. In many
>>> cases,\n"
>>>               "          this can violate \"pure\" deadlock free
>>> algorithm, so\n"
>>>               "          use it carefully.\n\n");
>>> diff --git a/opensm/opensm/osm_ucast_updn.c
>>> b/opensm/opensm/osm_ucast_updn.c
>>> index 164c6f4..f44ca24 100644
>>> --- a/opensm/opensm/osm_ucast_updn.c
>>> +++ b/opensm/opensm/osm_ucast_updn.c
>>> @@ -314,9 +314,7 @@ static int updn_set_min_hop_table(IN updn_t * p_updn)
>>>             item = cl_qmap_next(item)) {
>>>            p_sw = (osm_switch_t *)item;
>>>            /* Clear Min Hop Table */
>>> -        if (p_subn->opt.connect_roots)
>>> -            updn_clear_non_root_hops(p_updn, p_sw);
>>> -        else
>>> +        if (!p_subn->opt.connect_roots)
>>>                osm_switch_clear_hops(p_sw);
>>>        }
>>>
>>
>> What kind of testing was done for this?
>> I have a strong feeling that it will break up/down.
>> If the connect_roots option is on, you will not clear
>> the lid matrix at all, and it will contain also the
>> down/up routes.
>
> that is the idea.
> instead of leaving only up/dpwn routes we want to keep routes between all switches.
> the routes between host nodes will still be up/down.

Lid matrix might contain down/up routes between many
kinds of switches, not only between roots. This means
that you might have down/up paths between leaf switches,
even though you could have found up/down path for them.
This also means that you might end up with down/up routes
between HCAs that are connected to these leafs.

-- Yevgeny

> Eli
>
>>
>> -- Yevgeny
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html