Re: VST and Sched Load Balance

[Nick Piggin <nickpiggin@yahoo.com.au>]

Srivatsa Vaddagiri wrote:

> I think a potential area which VST may need to address is 
> scheduler load balance. If idle CPUs stop taking local timer ticks for 
> some time, then during that period it could cause the various runqueues to 
> go out of balance, since the idle CPUs will no longer pull tasks from 
> non-idle CPUs. 
> 

Yep.

> Do we care about this imbalance? Especially considering that most 
> implementations will let the idle CPUs sleep only for some max duration
> (~900 ms in case of x86).
> 

I think we do care, yes. It could be pretty harmful to sleep for
even a few 10s of ms on a regular basis for some workloads. Although
I guess many of those will be covered by try_to_wake_up events...

Not sure in practice, I would imagine it will hurt some multiprocessor
workloads.

> If we do care about this imbalance, then we could hope that the balance logic
> present in try_to_wake_up and sched_exec may avoid this imbalance, but can we 
> bank upon these events to restore the runqueue balance?
> 
> If we cannot, then I had something in mind on these lines:
> 
> 1. A non-idle CPU (having nr_running > 1) can wakeup a idle sleeping CPU if it 
>    finds that the sleeping CPU has not balanced itself for it's 
>    "balance_interval" period.
> 
> 2. It would be nice to minimize the "cross-domain" wakeups. For ex: we may want 
>    to avoid a non-idle CPU in node B sending a wakeup to a idle sleeping CPU in 
>    another node A, when this wakeup could have been sent from another non-idle
>    CPU in node A itself. 
>  

3. This is exactly one of the situations that the balancing backoff code
    was designed for. Can you just schedule interrupts to fire when the
    next balance interval has passed? This may require some adjustments to
    the backoff code in order to get good powersaving, but it would be the
    cleanest approach from the scheduler's point of view.

Nick

-- 
SUSE Labs, Novell Inc.