Re: VST and Sched Load Balance

[Ingo Molnar <mingo@elte.hu>]

* Srivatsa Vaddagiri <vatsa@in.ibm.com> wrote:

> Hi,
> 	VST patch (http://lwn.net/Articles/118693/) attempts to avoid useless 
> regular (local) timer ticks when a CPU is idle.
> 
> I think a potential area which VST may need to address is scheduler 
> load balance. If idle CPUs stop taking local timer ticks for some 
> time, then during that period it could cause the various runqueues to 
> go out of balance, since the idle CPUs will no longer pull tasks from 
> non-idle CPUs.
> 
> Do we care about this imbalance? Especially considering that most 
> implementations will let the idle CPUs sleep only for some max 
> duration (~900 ms in case of x86).

yeah, we care about this imbalance, it would materially change the 
scheduling logic, which side-effect we dont want. Interaction with VST 
is not a big issue right now because this only matters on SMP boxes 
which is a rare (but not unprecedented) target for embedded platforms.  

One solution would be to add an exponential backoff would be ok (as Nick 
suggested too), not an unconditional 'we wont fire a timer interrupt for 
the next 10 seconds' logic. It still impacts scheduling though.

Another, more effective, less intrusive but also more complex approach 
would be to make a distinction between 'totally idle' and 'partially 
idle or busy' system states. When all CPUs are idle then all timer irqs 
may be stopped and full VST logic applies. When at least one CPU is 
busy, all the other CPUs may still be put to sleep completely and 
immediately, but the busy CPU(s) have to take over a 'watchdog' role, 
and need to run the 'do the idle CPUs need new tasks' balancing 
functions. I.e. the scheduling function of other CPUs is migrated to 
busy CPUs. If there are no busy CPUs then there's no work, so this ought 
to be simple on the VST side. This needs some reorganization on the 
scheduler side but ought to be doable as well.

	Ingo