Re: [PATCHv2] opensm: Add a rate based mechanism for SMP transactions

[Sasha Khapyorsky <sashak-smomgflXvOZWk0Htik3J/w@public.gmane.org>]

On 06:27 Wed 09 Jun     , Hal Rosenstock wrote:
> Hi Sasha,
> 
> On Wed, Jun 9, 2010 at 2:05 AM, Sasha Khapyorsky <sashak-smomgflXvOZWk0Htik3J/w@public.gmane.org> wrote:
> > Hi Hal,
> >
> > On 08:40 Mon 07 Jun     , Hal Rosenstock wrote:
> >>
> >> In order to better handle non responsive SMAs (when link is physically up
> >> but the SMA does not respond), a rate based mechanism for SMPs is added
> >> to better enable forward progress in a more timely fashion. So rather than
> >> wait for timeouts and outstanding wire SMPs to drop below some configured
> >> value, there is also a periodic rate for transaction based SMPs. These
> >> rate based SMPs are capped at a configured maximum value.
> >>
> >> Two new options are added for this:
> >> rate_based_smp_usecs indicates the number of microseconds between rate
> >> based SMPs.
> >> max_rate_based_smps indicates the maximum number of rate based SMPs
> >> supported. When this limit is reached, rate based SMPs are no longer
> >> sent (until the number of outstanding ones drops below this limit).
> >>
> >> The rate based SMP mechanism can be disabled by setting rate_based_smp_usecs
> >> to 0. This is equivalent to the (current) algorithm prior to this change.
> >>
> >> By default, this mechanism is disabled.
> >
> > I would strongly suggest to rework the terminology here to make things
> > clearer.
> >
> > Actually we don't have here any "rate based" SMPs,
> 
> Doesn't the timeout value pace the second limit of SMPs ? If so, in my
> mind, that is rate based.

All those SMPs are equivalent in processing, etc., the only difference
is in MADs injection mechanism. So trying to differentiate MADs itself
seems confused for me.

> > but instead two mad
> > injection limits (regular and timedout) and timeout value (which BTW
> > likely should be a function of --timeout parameter). Isn't it?
> 
> The separate timeout for this provides finer control over pacing the
> higher SMP limit rather than basing it on the transaction timeout. If
> it is a function of the transaction timeout as you propose above, is
> there admin control over it ? If there is, then there is another
> config param to express this anyhow.

No problem to have this configurable for finer control, but in case
when requested smps_on_wire_limit_low < smps_on_wire_limit_high we could
want to have some reasonable default value for the timeout.

> If there isn't, then what hard
> coded function do you think is appropriate ?

timeout * retries ?

Sasha

> 
> -- Hal
> 
> > Sasha
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html