Re: [patch] smpnice: don't consider sched groups which are lightly loaded for balancing

[Peter Williams <pwil3058@bigpond.net.au>]

Siddha, Suresh B wrote:
> updated patch appended. thanks.
> 
> --
> with smpnice, sched groups with highest priority tasks can mask the
> imbalance between the other sched groups with in the same domain.
> This patch fixes some of the listed down scenarios by not considering
> the sched groups which are lightly loaded.
> 
> a) on a simple 4-way MP system, if we have one high priority and 4 normal
> priority tasks, with smpnice we would like to see the high priority task
> scheduled on one cpu, two other cpus getting one normal task each and the
> fourth cpu getting the remaining two normal tasks. but with current smpnice
> extra normal priority task keeps jumping from one cpu to another cpu having
> the normal priority task.  This is because of the busiest_has_loaded_cpus,
> nr_loaded_cpus logic.. We are not including the cpu with high priority
> task in max_load calculations but including that in total and avg_load
> calcuations.. leading to max_load < avg_load and load balance between
> cpus running normal priority tasks(2 Vs 1) will always show imbalanace
> as one normal priority and the extra normal priority task will keep moving
> from one cpu to another cpu having normal priority task..
> 
> b) 4-way system with HT (8 logical processors). Package-P0 T0 has a highest
> priority task, T1 is idle. Package-P1 Both T0 and T1 have 1 normal priority
> task each..  P2 and P3 are idle.  With this patch, one of the normal priority
> tasks on P1 will be moved to P2 or P3..
> 
> c) With the current weighted smp nice calculations, it doesn't always make
> sense to look at the highest weighted runqueue in the busy group..
> Consider a load balance scenario on a DP with HT system, with Package-0
> containing one high priority and one low priority, Package-1 containing
> one low priority(with other thread being idle)..  Package-1 thinks that it
> need to take the low priority thread from Package-0. And find_busiest_queue()
> returns the cpu thread with highest priority task.. And ultimately(with help
> of active load balance) we move high priority task to Package-1. And same
> continues with Package-0 now, moving high priority task from package-1 to
> package-0..  Even without the presence of active load balance, load balance
> will fail to balance the above scenario..  Fix find_busiest_queue to use
> "imbalance" when it is lightly loaded.
> 
> Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>

Acked-by: Peter Williams <pwil3058@bigpond.com.au>

-- 
Peter Williams                                   pwil3058@bigpond.net.au

"Learning, n. The kind of ignorance distinguishing the studious."
  -- Ambrose Bierce