Re: [PATCH v2] sched/fair: Introduce priority load balance to reduce interference from IDLE tasks

[Abel Wu <wuyun.abel@bytedance.com>]

On 8/17/22 8:58 PM, Vincent Guittot Wrote:
> On Tue, 16 Aug 2022 at 04:53, zhangsong (J) <zhangsong34@huawei.com> wrote:
>>
>> For co-location with NORMAL and IDLE tasks, when CFS trigger load balance,
>> it is reasonable to prefer migrating NORMAL(Latency Sensitive) tasks from
>> the busy src CPU to dst CPU, and migrating IDLE tasks lastly.
>>
>>
>> Considering the large weight difference between normal and idle tasks,
>> does the re-ordering really change things? It would be helpful if you
>> can offer more detailed info.
>>
>> Please consider the situation that CPU A has several normal tasks and hundreds of idle tasks
>> while CPU B is idle, and CPU B needs to pull some tasks from CPU A, but the cfs_tasks in CPU A
>> are not in order of priority, and the max number of pulling tasks depends on env->loop_max,
>> which value is sysctl_sched_nr_migrate, i.e. 32.
>>
>>
>> The case you elaborated above is really rare, the only possibility I
>> can imagine is that all these tasks are affined to one single cpu and
>> suddenly remove the affinity constrain. Otherwise, the load balancing
>> including wakeup cpu selection logic will make things right.
>>
>>
>> Yes, this is usually a corner case, but suppose that some non-idle tasks bounds to CPU 1-2
>>
>> and idle tasks bounds to CPU 0-1, so CPU 1 may has many idle tasks and some non-idle
>>
>> tasks while idle tasks on CPU 1 can not be pulled to CPU 2, when trigger load balance if
>>
>> CPU 2 should pull some tasks from CPU 1, the bad result is idle tasks of CPU 1 cannot be
>>
>> migrated and non-idle tasks also cannot be migrated in case of env->loop_max constraint.
> 
> env->loop_max adds a break but load_balance will continue with next
> tasks so it also tries to pull your non idle task at the end after
> several breaks.

Loop will be terminated without LBF_NEED_BREAK if exceeds loop_max :)

> 
>>
>> This will cause non-idle  tasks cannot achieve  more CPU utilization.
> 
> Your problem is not linked to IDLE vs NORMAL tasks but to the large
> number of pinned tasks that can't migrate on CPU2. You can end with
> the same behavior without using IDLE tasks but only NORMAL tasks.

I feel the same thing.

Best,
Abel