Re: [PATCH] sched: move enough load to balance average load per task

[Peter Williams <pwil3058@bigpond.net.au>]

Siddha, Suresh B wrote:
> On Mon, Apr 10, 2006 at 04:45:32PM +1000, Peter Williams wrote:
>> Problem:
>>
>> The current implementation of find_busiest_group() recognizes that 
>> approximately equal average loads per task for each group/queue are 
>> desirable (e.g. this condition will increase the probability that the 
>> top N highest priority tasks on an N CPU system will be on different 
>> CPUs) by being slightly more aggressive when *imbalance is small but the 
>> average load per task in "busiest" group is more than that in "this" 
>> group.  Unfortunately, the amount moved from "busiest" to "this" is too 
>> small to reduce the average load per task on "busiest" (at best there 
>> will be no change and at worst it will get bigger).
> 
> Peter, We don't need to reduce the average load per task on "busiest"
> always. By moving a "busiest_load_per_task", we will increase the 
> average load per task of lesser busy cpu (there by trying to achieve
> the equality with busiest...)
> 
> Can you give an example scenario where this patch helps? And doesn't
> the normal imabalance calculations capture those issues?

Yes, I think that the normal imbalance calculations (in 
find_busiest_queue()) will generally capture the aim of having 
approximately equal average loads per task on run queues.  But this bit 
of code is a special case in that the extra aggression being taken by 
the load balancer (in response to a scenario raised by you) is being 
justified by the imbalance in the average loads per task so it behooves 
us to do the best we can to ensure that that imbalance is addressed.

I don't think this is true for try_to_wake_up() and some changes may be 
desirable there.  However, any such changes would interact with the RT 
load balancing that Ingo is working on and would need to be considered 
in conjunction with that.

Why I think "approximately equal average loads per task" is worthwhile 
secondary aim for the load balancer is because it helps restore an 
implicit aim (approximately equal numbers of tasks per run queue) that 
was present in the original version.  This in turn means that the 
distribution of priorities within the queues will be similar and this 
increases the chances that (on an N CPU system) the N highest priority 
tasks will be on different CPUs.  This is a desirable state of affairs.

Peter
-- 
Peter Williams                                   pwil3058@bigpond.net.au

"Learning, n. The kind of ignorance distinguishing the studious."
  -- Ambrose Bierce