[PATCH v7 0/1] sched: update the rq->avg_idle when a task is moved to an idle CPU

[PATCH v7 0/1] sched: update the rq->avg_idle when a task is moved to an idle CPU

[Huang Shijie]

In the newidle balance, the rq->idle_stamp may set to a non-zero value
if it cannot pull any task.

In the wakeup, it will detect the rq->idle_stamp, and updates
the rq->avg_idle, then ends the CPU idle status by setting rq->idle_stamp
to zero.

Besides the wakeup, current code does not end the CPU idle status
when a task is moved to the idle CPU, such as fork/clone, execve,
or other cases.

This patch set tries to resolve it.


v6--> v7:
    Call the update_rq_avg_idle() in the put_prev_task_idle().
    Remove the patch 1 in the original patch set.
   --v6:https://lkml.org/lkml/2025/12/9/377

v5--> v6:
    Remove "this_rq->idle_stamp = 0;" in patch 1.
    Update the test result with Specjbb.
   --v5:https://lkml.org/lkml/2025/12/3/179

v4--> v5:
    Modify the changelog.

   --v4:https://lkml.org/lkml/2025/11/28/300

v3--> v4:
     Remove the code for delayed task.

   --v3: https://lkml.org/lkml/2025/11/27/456

v2--> v3:
  -- merge patch 3 into patch 2:
      move update_rq_avg_idle() to enqueue_task().

   --v2: https://lkml.org/lkml/2025/11/27/214

v1--> v2:
  -- Put update_rq_avg_idle() to activate_task()
  -- Add Delay-dequeue task check.

   --v1: https://lkml.org/lkml/2025/11/24/97

Huang Shijie (1):
  sched: update the rq->avg_idle when a task is moved to an idle CPU

 kernel/sched/core.c  | 27 +++++++++++++++------------
 kernel/sched/idle.c  |  1 +
 kernel/sched/sched.h |  1 +
 3 files changed, 17 insertions(+), 12 deletions(-)

-- 
2.43.0

[PATCH v7 1/1] sched: update the rq->avg_idle when a task is moved to an idle CPU

[Huang Shijie]

In the newidle balance, the rq->idle_stamp may set to a non-zero value
if it cannot pull any task.

In the wakeup, it will detect the rq->idle_stamp, and updates
the rq->avg_idle, then ends the CPU idle status by setting rq->idle_stamp
to zero.

Besides the wakeup, current code does not end the CPU idle status
when a task is moved to the idle CPU, such as fork/clone, execve,
or other cases. In order to get more accurate rq->avg_idle,
we need to update it at more places(not only the wakeup).

This patch introduces a helper: update_rq_avg_idle().
And uses it in put_prev_task_idle(), so it will update the rq->avg_idle
when a task is moved to an idle CPU at:
   -- wakeup
   -- fork/clone
   -- execve
   -- idle balance
   -- other cases

Signed-off-by: Huang Shijie <shijie8@gmail.com>
---
 kernel/sched/core.c  | 27 +++++++++++++++------------
 kernel/sched/idle.c  |  1 +
 kernel/sched/sched.h |  1 +
 3 files changed, 17 insertions(+), 12 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 5b17d8e3cb55..ad52f3a3c6bf 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3609,6 +3609,21 @@ static inline void ttwu_do_wakeup(struct task_struct *p)
 	trace_sched_wakeup(p);
 }
 
+void update_rq_avg_idle(struct rq *rq)
+{
+	if (rq->idle_stamp) {
+		u64 delta = rq_clock(rq) - rq->idle_stamp;
+		u64 max = 2*rq->max_idle_balance_cost;
+
+		update_avg(&rq->avg_idle, delta);
+
+		if (rq->avg_idle > max)
+			rq->avg_idle = max;
+
+		rq->idle_stamp = 0;
+	}
+}
+
 static void
 ttwu_do_activate(struct rq *rq, struct task_struct *p, int wake_flags,
 		 struct rq_flags *rf)
@@ -3644,18 +3659,6 @@ ttwu_do_activate(struct rq *rq, struct task_struct *p, int wake_flags,
 		p->sched_class->task_woken(rq, p);
 		rq_repin_lock(rq, rf);
 	}
-
-	if (rq->idle_stamp) {
-		u64 delta = rq_clock(rq) - rq->idle_stamp;
-		u64 max = 2*rq->max_idle_balance_cost;
-
-		update_avg(&rq->avg_idle, delta);
-
-		if (rq->avg_idle > max)
-			rq->avg_idle = max;
-
-		rq->idle_stamp = 0;
-	}
 }
 
 /*
diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c
index 65eb8f8c1a5d..aba5ad53c07d 100644
--- a/kernel/sched/idle.c
+++ b/kernel/sched/idle.c
@@ -460,6 +460,7 @@ static void put_prev_task_idle(struct rq *rq, struct task_struct *prev, struct t
 {
 	update_curr_idle(rq);
 	scx_update_idle(rq, false, true);
+	update_rq_avg_idle(rq);
 }
 
 static void set_next_task_idle(struct rq *rq, struct task_struct *next, bool first)
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 3ceaa9dc9a9e..6e3dd8c975e0 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -1651,6 +1651,7 @@ static inline struct cfs_rq *group_cfs_rq(struct sched_entity *grp)
 
 #endif /* !CONFIG_FAIR_GROUP_SCHED */
 
+extern void update_rq_avg_idle(struct rq *rq);
 extern void update_rq_clock(struct rq *rq);
 
 /*
-- 
2.43.0

Re: [PATCH v7 1/1] sched: update the rq->avg_idle when a task is moved to an idle CPU

[Valentin Schneider]

On 26/12/25 14:32, Huang Shijie wrote:
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -3609,6 +3609,21 @@ static inline void ttwu_do_wakeup(struct task_struct *p)
>       trace_sched_wakeup(p);
>  }
>
> +void update_rq_avg_idle(struct rq *rq)
> +{
> +	if (rq->idle_stamp) {
> +		u64 delta = rq_clock(rq) - rq->idle_stamp;
> +		u64 max = 2*rq->max_idle_balance_cost;
> +
> +		update_avg(&rq->avg_idle, delta);
> +
> +		if (rq->avg_idle > max)
> +			rq->avg_idle = max;
> +
> +		rq->idle_stamp = 0;
> +	}
> +}
> +

So if we have this invoked every time we switch to the idle task via
put_prev_task_idle(), do we want to move sched_balance_newidle()'s update
of rq->idle_stamp() to set_next_task_idle()?

That does change the behaviour as we'd now record any idle duration as
opposed to only idle-from-fair duration, but that would mean we'd
unconditionally record a rq->idle_stamp and could thus ditch the if{} clause.

>  static void
>  ttwu_do_activate(struct rq *rq, struct task_struct *p, int wake_flags,
>                struct rq_flags *rf)
> @@ -3644,18 +3659,6 @@ ttwu_do_activate(struct rq *rq, struct task_struct *p, int wake_flags,
>               p->sched_class->task_woken(rq, p);
>               rq_repin_lock(rq, rf);
>       }
> -
> -	if (rq->idle_stamp) {
> -		u64 delta = rq_clock(rq) - rq->idle_stamp;
> -		u64 max = 2*rq->max_idle_balance_cost;
> -
> -		update_avg(&rq->avg_idle, delta);
> -
> -		if (rq->avg_idle > max)
> -			rq->avg_idle = max;
> -
> -		rq->idle_stamp = 0;
> -	}
>  }
>
>  /*
> diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c
> index 65eb8f8c1a5d..aba5ad53c07d 100644
> --- a/kernel/sched/idle.c
> +++ b/kernel/sched/idle.c
> @@ -460,6 +460,7 @@ static void put_prev_task_idle(struct rq *rq, struct task_struct *prev, struct t
>  {
>       update_curr_idle(rq);
>       scx_update_idle(rq, false, true);
> +	update_rq_avg_idle(rq);

AFAICT we can't have put_prev_task_idle() immediately followed by
set_next_task_idle(); put_prev_set_next_task() especially already handles
this, so I think we're good, but maybe worth mentioning in the changelog?

>  }
>
>  static void set_next_task_idle(struct rq *rq, struct task_struct *next, bool first)
> diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
> index 3ceaa9dc9a9e..6e3dd8c975e0 100644
> --- a/kernel/sched/sched.h
> +++ b/kernel/sched/sched.h
> @@ -1651,6 +1651,7 @@ static inline struct cfs_rq *group_cfs_rq(struct sched_entity *grp)
>
>  #endif /* !CONFIG_FAIR_GROUP_SCHED */
>
> +extern void update_rq_avg_idle(struct rq *rq);
>  extern void update_rq_clock(struct rq *rq);
>
>  /*
> --
> 2.43.0

Re: [PATCH v7 1/1] sched: update the rq->avg_idle when a task is moved to an idle CPU

[K Prateek Nayak]

Hello Valentin,

On 1/9/2026 2:42 PM, Valentin Schneider wrote:
> On 26/12/25 14:32, Huang Shijie wrote:
>> --- a/kernel/sched/core.c
>> +++ b/kernel/sched/core.c
>> @@ -3609,6 +3609,21 @@ static inline void ttwu_do_wakeup(struct task_struct *p)
>>       trace_sched_wakeup(p);
>>  }
>>
>> +void update_rq_avg_idle(struct rq *rq)
>> +{
>> +	if (rq->idle_stamp) {
>> +		u64 delta = rq_clock(rq) - rq->idle_stamp;
>> +		u64 max = 2*rq->max_idle_balance_cost;
>> +
>> +		update_avg(&rq->avg_idle, delta);
>> +
>> +		if (rq->avg_idle > max)
>> +			rq->avg_idle = max;
>> +
>> +		rq->idle_stamp = 0;
>> +	}
>> +}
>> +
> 
> So if we have this invoked every time we switch to the idle task via
> put_prev_task_idle(), do we want to move sched_balance_newidle()'s update
> of rq->idle_stamp() to set_next_task_idle()?
> > That does change the behaviour as we'd now record any idle duration as
> opposed to only idle-from-fair duration, but that would mean we'd
> unconditionally record a rq->idle_stamp and could thus ditch the if{} clause.

So I'm a wee bit skeptical of this - the avg_idle also serves as a
bailout for newidle_balance(). If a tasks keeps waking up during newidle
balance, we would like to discourage further attempts of newidle balance
for a while to avoid CPU being stuck doing newidle balance while having
runnable tasks waken up on it.

There is no bailout past should_we_balance(), and for large domains, it
can take a while to get out of balancing.

If we move this to {put_prev,set_next}_task_idle(), we'll completely
fail to capture that part of newidle balance bailout and I'm afraid
we'll start doing newidle balance more aggressively.

I'll get some data over the weekend for the different variants being
discussed here - if it doesn't reveal anything drastic, we can
consider moving this accounting to idle task's switch.

-- 
Thanks and Regards,
Prateek

Re: [PATCH v7 1/1] sched: update the rq->avg_idle when a task is moved to an idle CPU

[Vincent Guittot]

On Fri, 9 Jan 2026 at 10:12, Valentin Schneider <vschneid@redhat.com> wrote:
>
> On 26/12/25 14:32, Huang Shijie wrote:
> > --- a/kernel/sched/core.c
> > +++ b/kernel/sched/core.c
> > @@ -3609,6 +3609,21 @@ static inline void ttwu_do_wakeup(struct task_struct *p)
> >       trace_sched_wakeup(p);
> >  }
> >
> > +void update_rq_avg_idle(struct rq *rq)
> > +{
> > +     if (rq->idle_stamp) {
> > +             u64 delta = rq_clock(rq) - rq->idle_stamp;
> > +             u64 max = 2*rq->max_idle_balance_cost;
> > +
> > +             update_avg(&rq->avg_idle, delta);
> > +
> > +             if (rq->avg_idle > max)
> > +                     rq->avg_idle = max;
> > +
> > +             rq->idle_stamp = 0;
> > +     }
> > +}
> > +
>
> So if we have this invoked every time we switch to the idle task via
> put_prev_task_idle(), do we want to move sched_balance_newidle()'s update
> of rq->idle_stamp() to set_next_task_idle()?

I don't think that this is necessary. In worst case we will set
idle_stamp in sched_balance_newidle() but a sched_ext task will be
picked instead of going idle and the idle_stamp will not be used and
will be overwritten next time we try to pick next task

>
> That does change the behaviour as we'd now record any idle duration as
> opposed to only idle-from-fair duration, but that would mean we'd
> unconditionally record a rq->idle_stamp and could thus ditch the if{} clause.

yes the if test is probably not necessary anymore

>
> >  static void
> >  ttwu_do_activate(struct rq *rq, struct task_struct *p, int wake_flags,
> >                struct rq_flags *rf)
> > @@ -3644,18 +3659,6 @@ ttwu_do_activate(struct rq *rq, struct task_struct *p, int wake_flags,
> >               p->sched_class->task_woken(rq, p);
> >               rq_repin_lock(rq, rf);
> >       }
> > -
> > -     if (rq->idle_stamp) {
> > -             u64 delta = rq_clock(rq) - rq->idle_stamp;
> > -             u64 max = 2*rq->max_idle_balance_cost;
> > -
> > -             update_avg(&rq->avg_idle, delta);
> > -
> > -             if (rq->avg_idle > max)
> > -                     rq->avg_idle = max;
> > -
> > -             rq->idle_stamp = 0;
> > -     }
> >  }
> >
> >  /*
> > diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c
> > index 65eb8f8c1a5d..aba5ad53c07d 100644
> > --- a/kernel/sched/idle.c
> > +++ b/kernel/sched/idle.c
> > @@ -460,6 +460,7 @@ static void put_prev_task_idle(struct rq *rq, struct task_struct *prev, struct t
> >  {
> >       update_curr_idle(rq);
> >       scx_update_idle(rq, false, true);
> > +     update_rq_avg_idle(rq);
>
> AFAICT we can't have put_prev_task_idle() immediately followed by
> set_next_task_idle(); put_prev_set_next_task() especially already handles
> this, so I think we're good, but maybe worth mentioning in the changelog?
>
> >  }
> >
> >  static void set_next_task_idle(struct rq *rq, struct task_struct *next, bool first)
> > diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
> > index 3ceaa9dc9a9e..6e3dd8c975e0 100644
> > --- a/kernel/sched/sched.h
> > +++ b/kernel/sched/sched.h
> > @@ -1651,6 +1651,7 @@ static inline struct cfs_rq *group_cfs_rq(struct sched_entity *grp)
> >
> >  #endif /* !CONFIG_FAIR_GROUP_SCHED */
> >
> > +extern void update_rq_avg_idle(struct rq *rq);
> >  extern void update_rq_clock(struct rq *rq);
> >
> >  /*
> > --
> > 2.43.0
>

Re: [PATCH v7 1/1] sched: update the rq->avg_idle when a task is moved to an idle CPU

[Valentin Schneider]

On 09/01/26 16:06, K Prateek Nayak wrote:
> Hello Valentin,
>
> On 1/9/2026 2:42 PM, Valentin Schneider wrote:
>> On 26/12/25 14:32, Huang Shijie wrote:
>>> --- a/kernel/sched/core.c
>>> +++ b/kernel/sched/core.c
>>> @@ -3609,6 +3609,21 @@ static inline void ttwu_do_wakeup(struct task_struct *p)
>>>       trace_sched_wakeup(p);
>>>  }
>>>
>>> +void update_rq_avg_idle(struct rq *rq)
>>> +{
>>> +	if (rq->idle_stamp) {
>>> +		u64 delta = rq_clock(rq) - rq->idle_stamp;
>>> +		u64 max = 2*rq->max_idle_balance_cost;
>>> +
>>> +		update_avg(&rq->avg_idle, delta);
>>> +
>>> +		if (rq->avg_idle > max)
>>> +			rq->avg_idle = max;
>>> +
>>> +		rq->idle_stamp = 0;
>>> +	}
>>> +}
>>> +
>>
>> So if we have this invoked every time we switch to the idle task via
>> put_prev_task_idle(), do we want to move sched_balance_newidle()'s update
>> of rq->idle_stamp() to set_next_task_idle()?
>> > That does change the behaviour as we'd now record any idle duration as
>> opposed to only idle-from-fair duration, but that would mean we'd
>> unconditionally record a rq->idle_stamp and could thus ditch the if{} clause.
>
> So I'm a wee bit skeptical of this - the avg_idle also serves as a
> bailout for newidle_balance(). If a tasks keeps waking up during newidle
> balance, we would like to discourage further attempts of newidle balance
> for a while to avoid CPU being stuck doing newidle balance while having
> runnable tasks waken up on it.
>
> There is no bailout past should_we_balance(), and for large domains, it
> can take a while to get out of balancing.
>
> If we move this to {put_prev,set_next}_task_idle(), we'll completely
> fail to capture that part of newidle balance bailout and I'm afraid
> we'll start doing newidle balance more aggressively.
>

Ah you're right, I'd forgotten about this, there's even a comment above the
idle_stamp update pointing this out... Although AFAICT that means we'd end
up with smaller rq->avg_idle and thus would newidle_balance() less often;
regardless that is a behaviour change.

> I'll get some data over the weekend for the different variants being
> discussed here - if it doesn't reveal anything drastic, we can
> consider moving this accounting to idle task's switch.
>
> --
> Thanks and Regards,
> Prateek

Re: [PATCH v7 1/1] sched: update the rq->avg_idle when a task is moved to an idle CPU

[Vincent Guittot]

On Fri, 9 Jan 2026 at 14:01, Valentin Schneider <vschneid@redhat.com> wrote:
>
> On 09/01/26 16:06, K Prateek Nayak wrote:
> > Hello Valentin,
> >
> > On 1/9/2026 2:42 PM, Valentin Schneider wrote:
> >> On 26/12/25 14:32, Huang Shijie wrote:
> >>> --- a/kernel/sched/core.c
> >>> +++ b/kernel/sched/core.c
> >>> @@ -3609,6 +3609,21 @@ static inline void ttwu_do_wakeup(struct task_struct *p)
> >>>       trace_sched_wakeup(p);
> >>>  }
> >>>
> >>> +void update_rq_avg_idle(struct rq *rq)
> >>> +{
> >>> +   if (rq->idle_stamp) {
> >>> +           u64 delta = rq_clock(rq) - rq->idle_stamp;
> >>> +           u64 max = 2*rq->max_idle_balance_cost;
> >>> +
> >>> +           update_avg(&rq->avg_idle, delta);
> >>> +
> >>> +           if (rq->avg_idle > max)
> >>> +                   rq->avg_idle = max;
> >>> +
> >>> +           rq->idle_stamp = 0;
> >>> +   }
> >>> +}
> >>> +
> >>
> >> So if we have this invoked every time we switch to the idle task via
> >> put_prev_task_idle(), do we want to move sched_balance_newidle()'s update
> >> of rq->idle_stamp() to set_next_task_idle()?
> >> > That does change the behaviour as we'd now record any idle duration as
> >> opposed to only idle-from-fair duration, but that would mean we'd
> >> unconditionally record a rq->idle_stamp and could thus ditch the if{} clause.
> >
> > So I'm a wee bit skeptical of this - the avg_idle also serves as a
> > bailout for newidle_balance(). If a tasks keeps waking up during newidle
> > balance, we would like to discourage further attempts of newidle balance
> > for a while to avoid CPU being stuck doing newidle balance while having
> > runnable tasks waken up on it.
> >
> > There is no bailout past should_we_balance(), and for large domains, it
> > can take a while to get out of balancing.
> >
> > If we move this to {put_prev,set_next}_task_idle(), we'll completely
> > fail to capture that part of newidle balance bailout and I'm afraid
> > we'll start doing newidle balance more aggressively.

That would be the opposite; We will do less newidle balance because
avg_idle will be shorter because the time spent in newidle balance
will not be accounted as idle time whereas it should

> >
>
> Ah you're right, I'd forgotten about this, there's even a comment above the
> idle_stamp update pointing this out... Although AFAICT that means we'd end
> up with smaller rq->avg_idle and thus would newidle_balance() less often;
> regardless that is a behaviour change.
>
> > I'll get some data over the weekend for the different variants being
> > discussed here - if it doesn't reveal anything drastic, we can
> > consider moving this accounting to idle task's switch.
> >
> > --
> > Thanks and Regards,
> > Prateek
>