The Linux Kernel Mailing List
 help / color / mirror / Atom feed
* [PATCH v3] sched/rt: Do not account migrate-disabled tasks as pushable
@ 2026-11-13 16:06 Mel Gorman
  2026-11-17 14:27 ` Mel Gorman
  2026-11-17 15:20 ` Peter Zijlstra
  0 siblings, 2 replies; 8+ messages in thread
From: Mel Gorman @ 2026-11-13 16:06 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Ingo Molnar, Vincent Guittot, Steven Rostedt, Seiji Nishikawa,
	Mian Yousaf Kaukab, LKML, Mel Gorman

RT_PUSH_IPI is intended to minimise latency and rq lock contention when
migrating tasks to balance RT task distribution. The problem is that
if a CPU is rt_overloaded but has no tasks that can be pushed then IPIs
can be fired continually at the same CPU but fail to move any task due
to migrate_disable() on PREEMPT_RT. A specific case was "pgrep" running
at RT priority, its CPU being the target of a continuous stream of IPIs
without any task moving.  While the pgrep can be bound to another CPU by
the application, the underlying problem was that pgrep is spinlock-intensive
which disables migration on PREEMPT_RT.

Commit dae5c0292080 ("sched/rt,dl: Skip migrate-disabled tasks when
picking a push candidate") stops pick_next_pushable_task() returning a
migrate-disabled task which fixes the balancer trying to push pinned tasks
off the CPU. It notes

    The self-IPI path is closed by the rto_next_cpu() fix above, and that
    part works. But the runqueue is still marked overloaded, because the
    pinned task is still advertised as pushable. Other CPUs now send the
    push-IPIs during their own RT balancing, and the same loop runs again.
    Closing the self-IPI path did not stop a pinned task from triggering
    push balancing.

As above, the runqueue is still marked overloaded by enqueue_task_rt()
and put_prev_task_rt() even if the task is migrate-disabled. A
migrate-disabled task reaches the pushable list through enqueue_task_rt()
or put_prev_task_rt() when it is preempted, or when it wakes while
still migrate-disabled. Either path calls enqueue_pushable_task() to set
rq->rt.overloaded and rto_mask if unset. Once set, every pull from another
CPU that fires an IPI may only find migrate-disabled tasks rendering the IPI
useless for a condition that was detectable when placed on the pushable list.

Stop adding migrate-disabled tasks to the pushable list if RT_PUSH_IPI is
enabled. This is not a heuristic as migrate-disabled tasks cannot move. Even
when migration is enabled, the rq still has no pushable candidates until the
task schedules. The !RT_PUSH_IPI behaviour is preserved and migrate-disabled
tasks are added to the pushable list. Without IPI, the alternative path in
pull_rt_task() must find runqueues with off-CPU migrate-disabled tasks. It
can then push the running task away allowing the migrate-disabled task
to run.

A test case creates a "victim" task with short work/sleep cycles allowed
to run on CPU1 and CPU2 that optionally disables migration (md=1) while
running.  A second task mostly does work pinned on CPU1 with short sleeps,
a third task mostly sleeps on CPU2 so CPU2 pulls frequently.  The results are

| Kernel (7.4-rc2)      |     CPU1-md=1 |   CPU2-md=1 |    CPU1-md=0 |    CPU2-md=0 |
|-----------------------|---------------|-------------|--------------|--------------|
| vanilla-rt-push-ipi   | 41474.7 ±0.1% |           0 | 4784.4 ±0.2% |   85.0 ±9.9% |
| patch-rt-push-ipi     |             0 | 0.3 ±380.6% | 4792.1 ±0.3% |  87.1 ±10.8% |
| patch-no-rt-push-ipi  |             0 |           0 |            0 |            0 |
(RT push IPIs per CPU per 5 s interval, mean ±CoV over 30 intervals,
 md=1 means the "victim" with cpumask for CPUs 1-2 disables migration)

The vanilla 7.4-rc2 kernel fired 40k+ IPIs every 5 seconds with
low variance over 30 samples with a migrate-disabled task sitting on
CPU1. Separate testing showed that the IPIs were not resulting in tasks
moving and therefore spurious. The patch stops the spurious IPIs (second
row). Disabling RT_PUSH_IPI also showed no IPIs but it was confirmed that
tasks were still moved.

Note that altering the RT class only is deliberate and deadline is left
alone other than moving the "p->nr_cpus_allowed > 1" check for consistency.
The justification for RT special-casing migrate-disabled is eliminating
ineffective RT_PUSH_IPIs. Deadline tries to push the task running on the rq
that holds an off-CPU, migrate-disabled task, so that the migrate-disabled
task can run there instead of waiting while another CPU runs later-deadline
work. The cost of that push is bounded, unlike arbitrary IPIs, and
justifiable because deadline ordering is otherwise violated.

For deadline the same two decision sites on pushable exist, and any change
must update both together.

In the specific context of put_prev_task_dl(), the task is going off-CPU
with migration disabled, so pushing the running task may take longer
than letting the running task use its budget. The problem is that the
time to migrate the running task can only be estimated but ignoring EDF
requirements is a root-domain-wide violation of deadline principles.
That is out of scope for this patch.

In the specific context of enqueue_task_dl(), it could be justified on
the basis that the task is only enqueued, not yet running. This is the
same question as put_prev_task_dl() -- honour EDF ordering or ignore it.
That is also out of scope for this patch.

Updating deadline with a matching is_migration_disabled() check is not
justified by the RT IPI problem. Deadline has different requirements that
merit its own patch and separate justification.

[ykaukab@suse.de: authored fix for spurious IPIs when RT_PUSH_IPI enabled]
[mgorman@techsingularity.net: deadline, changelog, preserve !RT_PUSH_IPI]
Signed-off-by: Yousaf Kaukab <ykaukab@suse.de>
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
---
 kernel/sched/deadline.c |  7 +++++--
 kernel/sched/rt.c       | 11 +++++++++--
 2 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index dfae08625342..9f58da0014a5 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -584,6 +584,9 @@ static void enqueue_pushable_dl_task(struct rq *rq, struct task_struct *p)
 
 	WARN_ON_ONCE(!RB_EMPTY_NODE(&p->pushable_dl_tasks));
 
+	if (p->nr_cpus_allowed <= 1)
+		return;
+
 	leftmost = rb_add_cached(&p->pushable_dl_tasks,
 				 &rq->dl.pushable_dl_tasks_root,
 				 __pushable_less);
@@ -2545,7 +2548,7 @@ static void enqueue_task_dl(struct rq *rq, struct task_struct *p, int flags)
 	if (dl_rq->curr == dl_se)
 		return;
 
-	if (!task_current(rq, p) && !dl_se->dl_throttled && p->nr_cpus_allowed > 1)
+	if (!task_current(rq, p) && !dl_se->dl_throttled)
 		enqueue_pushable_dl_task(rq, p);
 }
 
@@ -2867,7 +2870,7 @@ static void put_prev_task_dl(struct rq *rq, struct task_struct *p, struct task_s
 	if (task_is_blocked(p))
 		return;
 
-	if (on_dl_rq(dl_se) && p->nr_cpus_allowed > 1)
+	if (on_dl_rq(dl_se))
 		enqueue_pushable_dl_task(rq, p);
 }
 
diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
index 1535046a23ff..bb640278d30e 100644
--- a/kernel/sched/rt.c
+++ b/kernel/sched/rt.c
@@ -396,6 +396,13 @@ static inline void rt_queue_pull_task(struct rq *rq)
 
 static void enqueue_pushable_task(struct rq *rq, struct task_struct *p)
 {
+	if (p->nr_cpus_allowed <= 1)
+		return;
+#ifdef HAVE_RT_PUSH_IPI
+	if (sched_feat(RT_PUSH_IPI) && is_migration_disabled(p))
+		return;
+#endif
+
 	plist_del(&p->pushable_tasks, &rq->rt.pushable_tasks);
 	plist_node_init(&p->pushable_tasks, p->prio);
 	plist_add(&p->pushable_tasks, &rq->rt.pushable_tasks);
@@ -1448,7 +1455,7 @@ enqueue_task_rt(struct rq *rq, struct task_struct *p, int flags)
 	if (task_is_blocked(p))
 		return;
 
-	if (!task_current(rq, p) && p->nr_cpus_allowed > 1)
+	if (!task_current(rq, p))
 		enqueue_pushable_task(rq, p);
 }
 
@@ -1746,7 +1753,7 @@ static void put_prev_task_rt(struct rq *rq, struct task_struct *p, struct task_s
 	 * The previous task needs to be made eligible for pushing
 	 * if it is still active
 	 */
-	if (on_rt_rq(&p->rt) && p->nr_cpus_allowed > 1)
+	if (on_rt_rq(&p->rt))
 		enqueue_pushable_task(rq, p);
 }
 
-- 
2.55.0


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [PATCH v3] sched/rt: Do not account migrate-disabled tasks as pushable
  2026-11-13 16:06 [PATCH v3] sched/rt: Do not account migrate-disabled tasks as pushable Mel Gorman
@ 2026-11-17 14:27 ` Mel Gorman
  2026-11-17 15:10   ` Peter Zijlstra
  2026-11-17 15:20 ` Peter Zijlstra
  1 sibling, 1 reply; 8+ messages in thread
From: Mel Gorman @ 2026-11-17 14:27 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Ingo Molnar, Vincent Guittot, Steven Rostedt, Seiji Nishikawa,
	Mian Yousaf Kaukab, LKML

On Fri, Nov 13, 2026 at 04:06:55PM +0000, Mel Gorman wrote:
> RT_PUSH_IPI is intended to minimise latency and rq lock contention when
> migrating tasks to balance RT task distribution. The problem is that
> if a CPU is rt_overloaded but has no tasks that can be pushed then IPIs
> can be fired continually at the same CPU but fail to move any task due
> to migrate_disable() on PREEMPT_RT. A specific case was "pgrep" running
> at RT priority, its CPU being the target of a continuous stream of IPIs
> without any task moving.  While the pgrep can be bound to another CPU by
> the application, the underlying problem was that pgrep is spinlock-intensive
> which disables migration on PREEMPT_RT.
> 

I know the rate of scheduler patches at the moment is completely insane
but is there anything else I should do with this patch?

-- 
Mel Gorman
SUSE Labs

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH v3] sched/rt: Do not account migrate-disabled tasks as pushable
  2026-11-17 14:27 ` Mel Gorman
@ 2026-11-17 15:10   ` Peter Zijlstra
  0 siblings, 0 replies; 8+ messages in thread
From: Peter Zijlstra @ 2026-11-17 15:10 UTC (permalink / raw)
  To: Mel Gorman
  Cc: Ingo Molnar, Vincent Guittot, Steven Rostedt, Seiji Nishikawa,
	Mian Yousaf Kaukab, LKML

On Tue, Nov 17, 2026 at 02:27:38PM +0000, Mel Gorman wrote:
> On Fri, Nov 13, 2026 at 04:06:55PM +0000, Mel Gorman wrote:
> > RT_PUSH_IPI is intended to minimise latency and rq lock contention when
> > migrating tasks to balance RT task distribution. The problem is that
> > if a CPU is rt_overloaded but has no tasks that can be pushed then IPIs
> > can be fired continually at the same CPU but fail to move any task due
> > to migrate_disable() on PREEMPT_RT. A specific case was "pgrep" running
> > at RT priority, its CPU being the target of a continuous stream of IPIs
> > without any task moving.  While the pgrep can be bound to another CPU by
> > the application, the underlying problem was that pgrep is spinlock-intensive
> > which disables migration on PREEMPT_RT.
> > 
> 
> I know the rate of scheduler patches at the moment is completely insane
> but is there anything else I should do with this patch?

IKR :/ Let me go have a look.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH v3] sched/rt: Do not account migrate-disabled tasks as pushable
  2026-11-13 16:06 [PATCH v3] sched/rt: Do not account migrate-disabled tasks as pushable Mel Gorman
  2026-11-17 14:27 ` Mel Gorman
@ 2026-11-17 15:20 ` Peter Zijlstra
  2026-11-17 15:24   ` Peter Zijlstra
  2026-11-17 15:25   ` Steven Rostedt
  1 sibling, 2 replies; 8+ messages in thread
From: Peter Zijlstra @ 2026-11-17 15:20 UTC (permalink / raw)
  To: Mel Gorman
  Cc: Ingo Molnar, Vincent Guittot, Steven Rostedt, Seiji Nishikawa,
	Mian Yousaf Kaukab, LKML

On Fri, Nov 13, 2026 at 04:06:55PM +0000, Mel Gorman wrote:

> As above, the runqueue is still marked overloaded by enqueue_task_rt()
> and put_prev_task_rt() even if the task is migrate-disabled. A
> migrate-disabled task reaches the pushable list through enqueue_task_rt()
> or put_prev_task_rt() when it is preempted, or when it wakes while
> still migrate-disabled. Either path calls enqueue_pushable_task() to set
> rq->rt.overloaded and rto_mask if unset. Once set, every pull from another
> CPU that fires an IPI may only find migrate-disabled tasks rendering the IPI
> useless for a condition that was detectable when placed on the pushable list.
> 
> Stop adding migrate-disabled tasks to the pushable list if RT_PUSH_IPI is
> enabled. This is not a heuristic as migrate-disabled tasks cannot move. Even
> when migration is enabled, the rq still has no pushable candidates until the
> task schedules. The !RT_PUSH_IPI behaviour is preserved and migrate-disabled
> tasks are added to the pushable list. Without IPI, the alternative path in
> pull_rt_task() must find runqueues with off-CPU migrate-disabled tasks. It
> can then push the running task away allowing the migrate-disabled task
> to run.

Last time I either didn't notice, or I've gone dumb from staring at perf
issues all day, but I'm not entirely sure I understand how we get here.

Suppose:

  migrate_disable();
  ...
  schedule()
    migrate_disable_switch()
      // switches p->cpus_ptr and p->nr_cpus_allowed
    pick_next_task()
      p = class->pick_task();
      put_prev_set_next_task(rq, rq->donor, p);
      return p;

So any task that ends up in put_prev_task_rt() wlll have passed through
migrate_disable_switch(), which have updated nr_cpus_allowed.

Or so my current thinking goes. You seeing this patch make a difference,
seems to suggest I'm missing something, but I couldn't spot it in that
wall of text.

Anyway, I'm still not sure why we can't just write:

> diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
> index 1535046a23ff..bb640278d30e 100644
> --- a/kernel/sched/rt.c
> +++ b/kernel/sched/rt.c
> @@ -396,6 +396,13 @@ static inline void rt_queue_pull_task(struct rq *rq)
>  
>  static void enqueue_pushable_task(struct rq *rq, struct task_struct *p)
>  {
> +	if (p->nr_cpus_allowed <= 1)
> +		return;
> +#ifdef HAVE_RT_PUSH_IPI
> +	if (sched_feat(RT_PUSH_IPI) && is_migration_disabled(p))
> +		return;
> +#endif
> + 

	if (p->nr_cpus_allowed <= 1 || is_migration_disabled(p))
		return;

for both RT and DL. Sure, we might get an extra load, but is that really
going to be noticable? Lots of scheduler code already looks at the
migrate disable thing anyway.

I don't see why it should depend on RT_PUSH_IPI, it never is actually
migratable, and so it never makes sense to have on the pushable list.

But perhaps I'm missing something; the above seems to suggest that is a
theme today.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH v3] sched/rt: Do not account migrate-disabled tasks as pushable
  2026-11-17 15:20 ` Peter Zijlstra
@ 2026-11-17 15:24   ` Peter Zijlstra
  2026-11-18 10:05     ` Mel Gorman
  2026-11-17 15:25   ` Steven Rostedt
  1 sibling, 1 reply; 8+ messages in thread
From: Peter Zijlstra @ 2026-11-17 15:24 UTC (permalink / raw)
  To: Mel Gorman
  Cc: Ingo Molnar, Vincent Guittot, Steven Rostedt, Seiji Nishikawa,
	Mian Yousaf Kaukab, LKML

On Tue, Nov 17, 2026 at 04:20:26PM +0100, Peter Zijlstra wrote:
> On Fri, Nov 13, 2026 at 04:06:55PM +0000, Mel Gorman wrote:
> 
> > As above, the runqueue is still marked overloaded by enqueue_task_rt()
> > and put_prev_task_rt() even if the task is migrate-disabled. A
> > migrate-disabled task reaches the pushable list through enqueue_task_rt()
> > or put_prev_task_rt() when it is preempted, or when it wakes while
> > still migrate-disabled. Either path calls enqueue_pushable_task() to set
> > rq->rt.overloaded and rto_mask if unset. Once set, every pull from another
> > CPU that fires an IPI may only find migrate-disabled tasks rendering the IPI
> > useless for a condition that was detectable when placed on the pushable list.
> > 
> > Stop adding migrate-disabled tasks to the pushable list if RT_PUSH_IPI is
> > enabled. This is not a heuristic as migrate-disabled tasks cannot move. Even
> > when migration is enabled, the rq still has no pushable candidates until the
> > task schedules. The !RT_PUSH_IPI behaviour is preserved and migrate-disabled
> > tasks are added to the pushable list. Without IPI, the alternative path in
> > pull_rt_task() must find runqueues with off-CPU migrate-disabled tasks. It
> > can then push the running task away allowing the migrate-disabled task
> > to run.
> 
> Last time I either didn't notice, or I've gone dumb from staring at perf
> issues all day, but I'm not entirely sure I understand how we get here.
> 
> Suppose:
> 
>   migrate_disable();
>   ...
>   schedule()
>     migrate_disable_switch()
>       // switches p->cpus_ptr and p->nr_cpus_allowed
>     pick_next_task()
>       p = class->pick_task();
>       put_prev_set_next_task(rq, rq->donor, p);
>       return p;
> 
> So any task that ends up in put_prev_task_rt() wlll have passed through
> migrate_disable_switch(), which have updated nr_cpus_allowed.
> 
> Or so my current thinking goes. You seeing this patch make a difference,
> seems to suggest I'm missing something, but I couldn't spot it in that
> wall of text.

Ah, is the sched_change pattern the clue? I'm thinking that RT might hit
that more than usual due to the PI path.

In that case we do indeed end up in put_prev_task() / set_next_task()
without going through schedule(). In that case the task would be on the
pushable list during the change, HOWEVER we should hold rq->lock (which
implies have IRQs disabled) over the entire thing, so again, I'm not
sure how this is then observable.

> Anyway, I'm still not sure why we can't just write:
> 
> > diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
> > index 1535046a23ff..bb640278d30e 100644
> > --- a/kernel/sched/rt.c
> > +++ b/kernel/sched/rt.c
> > @@ -396,6 +396,13 @@ static inline void rt_queue_pull_task(struct rq *rq)
> >  
> >  static void enqueue_pushable_task(struct rq *rq, struct task_struct *p)
> >  {
> > +	if (p->nr_cpus_allowed <= 1)
> > +		return;
> > +#ifdef HAVE_RT_PUSH_IPI
> > +	if (sched_feat(RT_PUSH_IPI) && is_migration_disabled(p))
> > +		return;
> > +#endif
> > + 
> 
> 	if (p->nr_cpus_allowed <= 1 || is_migration_disabled(p))
> 		return;
> 
> for both RT and DL. Sure, we might get an extra load, but is that really
> going to be noticable? Lots of scheduler code already looks at the
> migrate disable thing anyway.
> 
> I don't see why it should depend on RT_PUSH_IPI, it never is actually
> migratable, and so it never makes sense to have on the pushable list.
> 
> But perhaps I'm missing something; the above seems to suggest that is a
> theme today.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH v3] sched/rt: Do not account migrate-disabled tasks as pushable
  2026-11-17 15:20 ` Peter Zijlstra
  2026-11-17 15:24   ` Peter Zijlstra
@ 2026-11-17 15:25   ` Steven Rostedt
  1 sibling, 0 replies; 8+ messages in thread
From: Steven Rostedt @ 2026-11-17 15:25 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Mel Gorman, Ingo Molnar, Vincent Guittot, Seiji Nishikawa,
	Mian Yousaf Kaukab, LKML

On Tue, 17 Nov 2026 16:20:26 +0100
Peter Zijlstra <peterz@infradead.org> wrote:

> >  
> >  static void enqueue_pushable_task(struct rq *rq, struct task_struct *p)
> >  {
> > +	if (p->nr_cpus_allowed <= 1)
> > +		return;
> > +#ifdef HAVE_RT_PUSH_IPI
> > +	if (sched_feat(RT_PUSH_IPI) && is_migration_disabled(p))
> > +		return;
> > +#endif
> > +   
> 
> 	if (p->nr_cpus_allowed <= 1 || is_migration_disabled(p))
> 		return;
> 
> for both RT and DL. Sure, we might get an extra load, but is that really
> going to be noticable? Lots of scheduler code already looks at the
> migrate disable thing anyway.
> 
> I don't see why it should depend on RT_PUSH_IPI, it never is actually
> migratable, and so it never makes sense to have on the pushable list.

When I took a look at this, I was thinking the same thing. That there's no
need to have the HAVE_RT_PUSH_IPI check here and always do the test.

> 
> But perhaps I'm missing something; the above seems to suggest that is a
> theme today.

I'm still trying to dig myself out of the tracing patches backlog to spend
time on looking at this :-p

-- Steve

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH v3] sched/rt: Do not account migrate-disabled tasks as pushable
  2026-11-17 15:24   ` Peter Zijlstra
@ 2026-11-18 10:05     ` Mel Gorman
  2026-11-19 12:06       ` Mel Gorman
  0 siblings, 1 reply; 8+ messages in thread
From: Mel Gorman @ 2026-11-18 10:05 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Ingo Molnar, Vincent Guittot, Steven Rostedt, Seiji Nishikawa,
	Mian Yousaf Kaukab, LKML

On Tue, Nov 17, 2026 at 04:24:14PM +0100, Peter Zijlstra wrote:
> On Tue, Nov 17, 2026 at 04:20:26PM +0100, Peter Zijlstra wrote:
> > On Fri, Nov 13, 2026 at 04:06:55PM +0000, Mel Gorman wrote:
> > 
> > > As above, the runqueue is still marked overloaded by enqueue_task_rt()
> > > and put_prev_task_rt() even if the task is migrate-disabled. A
> > > migrate-disabled task reaches the pushable list through enqueue_task_rt()
> > > or put_prev_task_rt() when it is preempted, or when it wakes while
> > > still migrate-disabled. Either path calls enqueue_pushable_task() to set
> > > rq->rt.overloaded and rto_mask if unset. Once set, every pull from another
> > > CPU that fires an IPI may only find migrate-disabled tasks rendering the IPI
> > > useless for a condition that was detectable when placed on the pushable list.
> > > 
> > > Stop adding migrate-disabled tasks to the pushable list if RT_PUSH_IPI is
> > > enabled. This is not a heuristic as migrate-disabled tasks cannot move. Even
> > > when migration is enabled, the rq still has no pushable candidates until the
> > > task schedules. The !RT_PUSH_IPI behaviour is preserved and migrate-disabled
> > > tasks are added to the pushable list. Without IPI, the alternative path in
> > > pull_rt_task() must find runqueues with off-CPU migrate-disabled tasks. It
> > > can then push the running task away allowing the migrate-disabled task
> > > to run.
> > 
> > Last time I either didn't notice, or I've gone dumb from staring at perf
> > issues all day, but I'm not entirely sure I understand how we get here.
> > 
> > Suppose:
> > 
> >   migrate_disable();
> >   ...
> >   schedule()
> >     migrate_disable_switch()
> >       // switches p->cpus_ptr and p->nr_cpus_allowed
> >     pick_next_task()
> >       p = class->pick_task();
> >       put_prev_set_next_task(rq, rq->donor, p);
> >       return p;
> > 
> > So any task that ends up in put_prev_task_rt() wlll have passed through
> > migrate_disable_switch(), which have updated nr_cpus_allowed.
> > 

I did not check in this direction but that does not mean I think you're
right but I also am struggling to prove it.

An earlier version of the patch was like what you suggested (check both
conditionally in rt.c and copy to deadline.c), hen I thought "deadline had
no RTO IPI" and the copy felt like cargo culting which definitely would get
a kicking and I'd be asked "Why did you copy that" and my answer would be
"Uhhh.. cargo cult?" and then feel like a right idiot.

Without the push IPI, deadline.c needed an explanation why the same check
is necessary so looked closer at the !PUSH_IPI handling.  The !IPI path
in both assumes that such tasks can be visible so "why look up?"

In rt.c

--8<--
 static void pull_rt_task(struct rq *this_rq)
	....
	#ifdef HAVE_RT_PUSH_IPI
		if (sched_feat(RT_PUSH_IPI)) {
			tell_cpu_to_push(this_rq);
			return;
		}
	#endif

	(!RT_PUSH_IPI path, default in !PREEMPT_RT or sched_debug toggle)

	for_each_cpu(cpu, this_rq->rd->rto_mask) {
		...

/*****
  ****
  **** This bit
  ****
  ****

		/*
		 * Do we have an RT task that preempts
		 * the to-be-scheduled task?
		 */
		if (p && (p->prio < this_rq->rt.highest_prio.curr)) {
			...
			if (is_migration_disabled(p)) {
				push_task = get_push_task(src_rq);
			} else {
				move_queued_task_locked(src_rq, this_rq, p);
				resched = true;
			}
			...
		}
	}
--8<--

The !IPI path assumes that a migrate_disabled(p) can exist on the rto_mask
and it's been like that for a while. Compounding the problem that kept me
looking down was that PREEMPT_RT tree used to have a patch e18de5e2e3e9
("sched/rt: Don't try push tasks if there are none.")  that tangentially
affected the bug (the PREEMPT_RT patch was visible in the distro kernel
the bug was reported against). The PREEMPT_RT patch was removed around 6.7
because "it's much better now" and while it was not stated why, I assumed
it was due to 612f769edd06 ("sched/rt: Make rt_rq->pushable_tasks updates
drive rto_mask")

I'm not saying this is right, I am saying that blocks of code in both
rt.c and deadline.c assume this condition can occur.

> > Or so my current thinking goes. You seeing this patch make a difference,
> > seems to suggest I'm missing something, but I couldn't spot it in that
> > wall of text.
> 

It is definitely the case, and I've measured it, that the patch removes
spurious IPIs and it's verified on a mainline kernel. Spurious IPI in
this case is that a CPU that has no task that can migrate is considered
pushable and receives a *lot* of IPIs per second that never move a task
because there is no task that *can* move. I didn't try quantifing it much
beyond that because reducing thousands of IPIs/sec seemed like suffficient
justification to me.

> Ah, is the sched_change pattern the clue? I'm thinking that RT might hit
> that more than usual due to the PI path.
> 

You give me too much credit, I started with a cargo cult copy of rt.c to
deadline.c but couldn't put an obviously correct comment or changelog
on it so a short paragraph of a changelog turned into a "wall of text"
while I talked to myself for longer than is considered healthy even by
the rubber duck.

> In that case we do indeed end up in put_prev_task() / set_next_task()
> without going through schedule(). In that case the task would be on the
> pushable list during the change, HOWEVER we should hold rq->lock (which
> implies have IRQs disabled) over the entire thing, so again, I'm not
> sure how this is then observable.
> 
> > Anyway, I'm still not sure why we can't just write:
> > 
> > > diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
> > > index 1535046a23ff..bb640278d30e 100644
> > > --- a/kernel/sched/rt.c
> > > +++ b/kernel/sched/rt.c
> > > @@ -396,6 +396,13 @@ static inline void rt_queue_pull_task(struct rq *rq)
> > >  
> > >  static void enqueue_pushable_task(struct rq *rq, struct task_struct *p)
> > >  {
> > > +	if (p->nr_cpus_allowed <= 1)
> > > +		return;
> > > +#ifdef HAVE_RT_PUSH_IPI
> > > +	if (sched_feat(RT_PUSH_IPI) && is_migration_disabled(p))
> > > +		return;
> > > +#endif
> > > + 
> > 
> > 	if (p->nr_cpus_allowed <= 1 || is_migration_disabled(p))
> > 		return;
> > 

Yeah, I had that and couldn't copy it to deadline.c because the bug is
about IPIs, IPIs of the same type don't exist in deadline.c, so the copy
is "obviously bogus" and on rt.c the same check makes a bunch of
long-lived code unreachable. That is obviously shit logic to use for
justifying a kernel change.

I have to look closer at why you think it's not reachable and Steven
thinks it's plausible. All I've got so far is that

o migrate_disable_switch checks does nothing if
  "p->cpus_ptr != &p->cpus_mask"
  and so do_set_cpus_allowed is not guaranteed to be called and
  therefore the statement "migrate_disable_switch(), which have updated
  nr_cpus_allowed" might have a counter-example.

o pick_next_pushable_task() has BUG_ON(p->nr_cpus_allowed <= 1) and this
  thing certainly seems to believe that migrate_disabled tasks can reach
  it
                        if (unlikely(is_migration_disabled(task) ||
                                     !cpumask_test_cpu(lowest_rq->cpu, &task->cpus_mask) ||
                                     task != pick_next_pushable_task(rq))

  which means nothing at all yet, other than it's an example that is
  very difficult to reconcile with your version (which matches my old
  version) and state "this is ok".

> > But perhaps I'm missing something; the above seems to suggest that is a
> > theme today.

Not sure yet, all I know is that I cannot find so far a way of writing
a comment or changelog on that version that shows it's safe and doesn't
degrade rt.c, deadline.c at minimum or break it at worst.

-- 
Mel Gorman
SUSE Labs

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH v3] sched/rt: Do not account migrate-disabled tasks as pushable
  2026-11-18 10:05     ` Mel Gorman
@ 2026-11-19 12:06       ` Mel Gorman
  0 siblings, 0 replies; 8+ messages in thread
From: Mel Gorman @ 2026-11-19 12:06 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Ingo Molnar, Vincent Guittot, Steven Rostedt, Seiji Nishikawa,
	Mian Yousaf Kaukab, LKML

On Wed, Nov 18, 2026 at 10:05:31AM +0000, Mel Gorman wrote:
> On Tue, Nov 17, 2026 at 04:24:14PM +0100, Peter Zijlstra wrote:
> > On Tue, Nov 17, 2026 at 04:20:26PM +0100, Peter Zijlstra wrote:
> > > On Fri, Nov 13, 2026 at 04:06:55PM +0000, Mel Gorman wrote:
> > > 
> > > > As above, the runqueue is still marked overloaded by enqueue_task_rt()
> > > > and put_prev_task_rt() even if the task is migrate-disabled. A
> > > > migrate-disabled task reaches the pushable list through enqueue_task_rt()
> > > > or put_prev_task_rt() when it is preempted, or when it wakes while
> > > > still migrate-disabled. Either path calls enqueue_pushable_task() to set
> > > > rq->rt.overloaded and rto_mask if unset. Once set, every pull from another
> > > > CPU that fires an IPI may only find migrate-disabled tasks rendering the IPI
> > > > useless for a condition that was detectable when placed on the pushable list.
> > > > 
> > > > Stop adding migrate-disabled tasks to the pushable list if RT_PUSH_IPI is
> > > > enabled. This is not a heuristic as migrate-disabled tasks cannot move. Even
> > > > when migration is enabled, the rq still has no pushable candidates until the
> > > > task schedules. The !RT_PUSH_IPI behaviour is preserved and migrate-disabled
> > > > tasks are added to the pushable list. Without IPI, the alternative path in
> > > > pull_rt_task() must find runqueues with off-CPU migrate-disabled tasks. It
> > > > can then push the running task away allowing the migrate-disabled task
> > > > to run.
> > > 
> > > Last time I either didn't notice, or I've gone dumb from staring at perf
> > > issues all day, but I'm not entirely sure I understand how we get here.
> > > 
> > > Suppose:
> > > 
> > >   migrate_disable();
> > >   ...
> > >   schedule()
> > >     migrate_disable_switch()
> > >       // switches p->cpus_ptr and p->nr_cpus_allowed
> > >     pick_next_task()
> > >       p = class->pick_task();
> > >       put_prev_set_next_task(rq, rq->donor, p);
> > >       return p;
> > > 
> > > So any task that ends up in put_prev_task_rt() wlll have passed through
> > > migrate_disable_switch(), which have updated nr_cpus_allowed.
> > > 

Short summary, this assumption is wrong, nr_cpus_allowed is not updated
in that path.

> 
> I did not check in this direction but that does not mean I think you're
> right but I also am struggling to prove it.
> 
> An earlier version of the patch was like what you suggested (check both
> conditionally in rt.c and copy to deadline.c), hen I thought "deadline had
> no RTO IPI" and the copy felt like cargo culting which definitely would get
> a kicking and I'd be asked "Why did you copy that" and my answer would be
> "Uhhh.. cargo cult?" and then feel like a right idiot.
> 
> <SNIP query on why the necessary conditions can occur>
>
> I have to look closer at why you think it's not reachable and Steven
> thinks it's plausible. All I've got so far is that
> 
> o migrate_disable_switch checks does nothing if
>   "p->cpus_ptr != &p->cpus_mask"
>   and so do_set_cpus_allowed is not guaranteed to be called and
>   therefore the statement "migrate_disable_switch(), which have updated
>   nr_cpus_allowed" might have a counter-example.
> 
> o pick_next_pushable_task() has BUG_ON(p->nr_cpus_allowed <= 1) and this
>   thing certainly seems to believe that migrate_disabled tasks can reach
>   it
>                         if (unlikely(is_migration_disabled(task) ||
>                                      !cpumask_test_cpu(lowest_rq->cpu, &task->cpus_mask) ||
>                                      task != pick_next_pushable_task(rq))
> 
>   which means nothing at all yet, other than it's an example that is
>   very difficult to reconcile with your version (which matches my old
>   version) and state "this is ok".
> 

This was a sad excuse so I cheated and put a WARN_ONCE_ONCE() if
migrate_disabled tasks are encountered on the !RT_PUSH_IPI path like this

diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
index bb640278d30e..e8384f90bf4c 100644
--- a/kernel/sched/rt.c
+++ b/kernel/sched/rt.c
@@ -2347,7 +2347,13 @@ static void pull_rt_task(struct rq *this_rq)
 			if (p->prio < src_rq->donor->prio)
 				goto skip;
 
-			if (is_migration_disabled(p)) {
+			if (WARN_ON_ONCE(is_migration_disabled(p))) {
+				static int t_printed = 0;
+				if (!t_printed) {
+					t_printed++;
+					printk("found migrate_disabled on CPU[%d->%d] (%d:%s)", this_cpu, cpu, task_pid_nr(p), p->comm);
+				}
+				trace_printk("found migrate_disabled on CPU[%d->%d] (%d:%s)", this_cpu, cpu, task_pid_nr(p), p->comm);
 				push_task = get_push_task(src_rq);
 			} else {
 				move_queued_task_locked(src_rq, this_rq, p);

and it fires when a) running the test case on PREEMPT_RT and b) echo
NO_RT_PUSH_IPI > /sys/kernel/debug/sched/features. The warning below is
trimmed for noise

[  359.731411][ T2181] ------------[ cut here ]------------
[  359.731413][ T2181] WARNING: kernel/sched/rt.c:2350 at pull_rt_task+0x3ed/0x4a0, CPU#2: rpt_pinger/2181
[  359.959501][ T2181] RIP: 0010:pull_rt_task+0x3ed/0x4a0
[  360.068397][ T2181] Call Trace:
[  360.072005][ T2181]  <TASK>
[  360.075197][ T2181]  balance_rt+0x58/0xf0
[  360.079802][ T2181]  __pick_next_task+0xf4/0x1e0
[  360.085098][ T2181]  __schedule+0x1f1/0x1900

If I remove the RT_PUSH_IPI check and replace it with yours, which matches
early fix that was not posted, the warning no longer fires because the
code is no longer reachable for the test case. This shows the check is
necessary as stated in the changelog

	The !RT_PUSH_IPI behaviour is preserved and migrate-disabled tasks
	are added to the pushable list. Without IPI, the alternative path in
	pull_rt_task() must find runqueues with off-CPU migrate-disabled
	tasks. It can then push the running task away allowing the
	migrate-disabled task to run.

The reason for stating it in the changelog was shoddy logic. I assumed
the code must be doing something or it would not be there so stuck a
reason on it.

The warning firing or not firing is still not great reasoning though,
it's still shoddy with extra sprinkles on top so I started back with
this assumption on your patch

> > > So any task that ends up in put_prev_task_rt() wlll have passed through
> > > migrate_disable_switch(), which have updated nr_cpus_allowed.

That sounded plausible and while Steven did not explicitly say so, I
think he agreed with you and it sounded believable to me but looked
anyway

	(disable side)
	migrate_disable_switch
	  do_set_cpus_allowed(p, &ac);
	    p->sched_class->set_cpus_allowed(p, ctx);
    
	(enable side)
	___migrate_enable
	  __set_cpus_allowed_ptr(p, &ac);

Other than the assymetry, it looks ok so disable side for rt.c

DEFINE_SCHED_CLASS(rt) = {
	.set_cpus_allowed       = set_cpus_allowed_common,
}

Nothing special, back to core.c

void set_cpus_allowed_common(struct task_struct *p, struct affinity_context *ctx)
{
        if (ctx->flags & (SCA_MIGRATE_ENABLE | SCA_MIGRATE_DISABLE)) {
                p->cpus_ptr = ctx->new_mask;
                return;
        }

        cpumask_copy(&p->cpus_mask, ctx->new_mask);
        p->nr_cpus_allowed = cpumask_weight(ctx->new_mask);
        mm_update_cpus_allowed(p->mm, ctx->new_mask);


set_cpus_allowed_common only sets cpus_ptr for SCA_MIGRATE_ENABLE|SCA_MIGRATE_DISABLE
and **nr_cpus_allowed is not changed** which is the assumption the
straight-forward patch relies on.

I do not recall looking too closely at migrate_disable or forgot why it's
like this so I kept looking to see where SCA_MIGRATE_ENABLE originated
(presumably Set Cpus Allowed (SCA) instead of a typo of Ska) leading to
af449901b84c ("sched: Add migrate_disable()") which was no help explaining
the why. It's there with a tonne of warnings about locking violations
but does not explicitly state why. Best is this bit

-       if (p->nr_cpus_allowed > 1)
+       if (p->nr_cpus_allowed > 1 && !is_migration_disabled(p))

That sortof suggests the patch relied on the behaviour to distinguish
between nr_cpus_allowed == 1 and nr_cpus_allowed is equivalent to 1
because if migrate_disabled even though p->migration_enabled existed. I was
under time pressure at this point so had to draw the line and stop. It's
not particularly clear why set_cpus_allowed_common() is like this but
get_push_task() still checks

	if (p->nr_cpus_allowed == 1)
		return NULL;

        if (p->migration_disabled)
                return NULL;

It's not clear at all to me why both are needed. I expect if I kept digging
that I would find something clever but the obvious fix does not work,
or at least if it does work, it's not very obvious why it's ok and has
no side-effects.

-- 
Mel Gorman
SUSE Labs

^ permalink raw reply related	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2026-11-19 12:06 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-13 16:06 [PATCH v3] sched/rt: Do not account migrate-disabled tasks as pushable Mel Gorman
2026-11-17 14:27 ` Mel Gorman
2026-11-17 15:10   ` Peter Zijlstra
2026-11-17 15:20 ` Peter Zijlstra
2026-11-17 15:24   ` Peter Zijlstra
2026-11-18 10:05     ` Mel Gorman
2026-11-19 12:06       ` Mel Gorman
2026-11-17 15:25   ` Steven Rostedt

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox