Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs

[Preeti U Murthy <preeti@linux.vnet.ibm.com>]

On 03/30/2015 05:33 PM, Morten Rasmussen wrote:
> On Mon, Mar 30, 2015 at 12:06:32PM +0100, Peter Zijlstra wrote:
>> On Fri, Mar 27, 2015 at 05:56:51PM +0000, Morten Rasmussen wrote:
>>
>>> I agree that it is hard to predict how many additional cpus you need,
>>> but I don't think you necessarily need that information as long as you
>>> start by filling up the cpu that was kicked to do the
>>> nohz_idle_balance() first.
>>
>>> Reducing unnecessary wakeups is quite important for energy consumption
>>> and something a lot of effort is put into. You really don't want to wake
>>> up another cluster/package unnecessarily just because there was only one
>>> nohz-idle cpu left in the previous one which could have handled the
>>> additional load. It gets even worse if the other cluster is less
>>> energy-efficient (big.LITTLE).
>>
>> So the only way to get tasks to cross your cluster is by balancing that
>> domain. At this point we'll compute sg stats for either group
>> (=cluster).
>>
>> The only thing we need to ensure is that it doesn't view the small
>> cluster as overloaded (as long as it really isn't of course), as long as
>> its not viewed as overloaded it will not pull tasks from it into the big
>> cluster, no matter how many ILBs we run before the ILB duty cpu's
>> rebalance_domains() call.
>>
>> I'm really not seeing the problem here.
> 
> I see. The group_classify() should take care of it in all cases of
> balancing across clusters. You would be iterating over all cpus in the
> other cluster running rebalance_domains() if the balancer cpu happens to
> be the last one in the little cluster though. However, within the
> cluster (in case you have 2 or more nohz-idle cpus) you still take a
> double hit. No?

Morten,

I am a bit confused about the problem you are pointing to.
nohz_idle_balance() does not kick the idle CPUs into action unless there
is work to be done. So there are no redundant wakeups. Hence I see no
problem here.

The ILB CPU is woken up to do the nohz idle balancing, but with this
patch, may end up with no work for itself at the end of
nohz_idle_balance() and return to sleep. That is one wakeup for merely
doing idle load balancing, but this wakeup is needed and worthwhile for
spreading the load. So I find no problem here too.

I am unable to see the issue. What is it that I am missing ?

Regards
Preeti U Murthy


>