Looping more in detach_tasks() when RT and CFS tasks are present

Looping more in detach_tasks() when RT and CFS tasks are present

[Pavan Kondeti]

Hi Vincent/Peter,

in load_balance(), we derive env->loop_max based on rq->nr_running.
When the busiest rq has both RT and CFS tasks, we do more loops in
detach_tasks(). Is there any reason for not using
rq->cfs.h_nr_running?

Lei Wen attempted to fix this before.
https://lore.kernel.org/lkml/1376814322-7320-2-git-send-email-leiwen@marvell.com/

Thanks,
Pavan

-- 
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a
Linux Foundation Collaborative Project

Re: Looping more in detach_tasks() when RT and CFS tasks are present

[Vincent Guittot]

Hi Pavan,

On Wed, 24 Jun 2020 at 13:42, Pavan Kondeti <pkondeti@codeaurora.org> wrote:
>
> Hi Vincent/Peter,
>
> in load_balance(), we derive env->loop_max based on rq->nr_running.
> When the busiest rq has both RT and CFS tasks, we do more loops in
> detach_tasks(). Is there any reason for not using
> rq->cfs.h_nr_running?

Using cfs.h_nr_running seems fine for loop_max

>
> Lei Wen attempted to fix this before.
> https://lore.kernel.org/lkml/1376814322-7320-2-git-send-email-leiwen@marvell.com/

The 1st part of the patch is wrong because even if h_nr_running == 1
but nr_running > 1, we can pull the cfs thread without using active
balance

>
> Thanks,
> Pavan
>
> --
> Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc.
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a
> Linux Foundation Collaborative Project

Re: Looping more in detach_tasks() when RT and CFS tasks are present

[Pavan Kondeti]

Hi Vincent,

On Wed, Jun 24, 2020 at 02:39:25PM +0200, Vincent Guittot wrote:
> Hi Pavan,
> 
> On Wed, 24 Jun 2020 at 13:42, Pavan Kondeti <pkondeti@codeaurora.org> wrote:
> >
> > Hi Vincent/Peter,
> >
> > in load_balance(), we derive env->loop_max based on rq->nr_running.
> > When the busiest rq has both RT and CFS tasks, we do more loops in
> > detach_tasks(). Is there any reason for not using
> > rq->cfs.h_nr_running?
> 
> Using cfs.h_nr_running seems fine for loop_max
> 

Thanks for taking a look.

> >
> > Lei Wen attempted to fix this before.
> > https://lore.kernel.org/lkml/1376814322-7320-2-git-send-email-leiwen@marvell.com/
> 
> The 1st part of the patch is wrong because even if h_nr_running == 1
> but nr_running > 1, we can pull the cfs thread without using active
> balance
> 

Right. When a RT and CFS tasks are packed, I have seen CFS task getting pulled
via load balancer without waking migration/X.

I was using the below patch along with some prints in detach_tasks() loop.
I will update Lei Wen's patch and resend it.

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 02f323b..f042016 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -9509,7 +9509,7 @@ static int load_balance(int this_cpu, struct rq *this_rq,
 		 * correctly treated as an imbalance.
 		 */
 		env.flags |= LBF_ALL_PINNED;
-		env.loop_max  = min(sysctl_sched_nr_migrate, busiest->nr_running);
+		env.loop_max  = min(sysctl_sched_nr_migrate, busiest->cfs.h_nr_running);
 
 more_balance:
 		rq_lock_irqsave(busiest, &rf);

Thanks,
Pavan
-- 
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project.