The Linux Kernel Mailing List
 help / color / mirror / Atom feed
From: Mel Gorman <mgorman@techsingularity.net>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@redhat.com>,
	 Vincent Guittot <vincent.guittot@linaro.org>,
	Steven Rostedt <rostedt@goodmis.org>,
	 Seiji Nishikawa <snishika@redhat.com>,
	Mian Yousaf Kaukab <ykaukab@suse.de>,
	 LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v3] sched/rt: Do not account migrate-disabled tasks as pushable
Date: Thu, 19 Nov 2026 23:34:18 +0000	[thread overview]
Message-ID: <av9x3iMaKhtByVzM@techsingularity.net> (raw)
In-Reply-To: <20261119164050.GJ1922751@noisy.programming.kicks-ass.net>

On Thu, Nov 19, 2026 at 05:40:50PM +0100, Peter Zijlstra wrote:
> On Thu, Nov 19, 2026 at 04:28:27PM +0100, Peter Zijlstra wrote:
> > On Thu, Nov 19, 2026 at 04:17:40PM +0100, Peter Zijlstra wrote:
> > > On Thu, Nov 19, 2026 at 04:08:12PM +0100, Peter Zijlstra wrote:
> > > > On Thu, Nov 19, 2026 at 12:06:21PM +0000, Mel Gorman wrote:
> > > > 
> > > > > > > > Suppose:
> > > > > > > > 
> > > > > > > >   migrate_disable();
> > > > > > > >   ...
> > > > > > > >   schedule()
> > > > > > > >     migrate_disable_switch()
> > > > > > > >       // switches p->cpus_ptr and p->nr_cpus_allowed
> > > > > > > >     pick_next_task()
> > > > > > > >       p = class->pick_task();
> > > > > > > >       put_prev_set_next_task(rq, rq->donor, p);
> > > > > > > >       return p;
> > > > > > > > 
> > > > > > > > So any task that ends up in put_prev_task_rt() wlll have passed through
> > > > > > > > migrate_disable_switch(), which have updated nr_cpus_allowed.
> > > > > > > > 
> > > > > 
> > > > > Short summary, this assumption is wrong, nr_cpus_allowed is not updated
> > > > > in that path.
> > > 
> > > > Bah, and I cannot remember either. Let me have a poke, see if something
> > > > bubbles up.
> > > 
> > > So this version did update nr_cpus_allowed:
> > > 
> > > https://patch.msgid.link/20200921163845.769861942%40infradead.org
> > > 
> > > And the next version it did not:
> > > 
> > > https://patch.msgid.link/20201005150921.859981475%40infradead.org
> > > 
> > > But I can't seem to locate quite yet why it changed.
> > 
> > Tada!
> > 
> > https://patch.msgid.link/20200924123527.GJ2628%40hirez.programming.kicks-ass.net
> > 
> > I'm leaning towards "we got it wrong" on this.
> 

I'm assuming you mean the nr_cpus_allowed matching rather than this patch on
spurious IPIs.  I'm not even sure the decision was "wrong" though because
I doubt it was much fun back then either.

> One issue seems to be concurrent set_cpus_allowed_ptr(); the current
> rule is that nr_cpus_allowed is the number of bits in p->cpus_mask.
> 

I'm not sure why set_cpus_allowed_ptr is a problem precisely but if it
is, it's one of many.

> I'm not sure changing this is going to be 'trivial', so perhaps

I'm fairly sure it's non-trivial because the audit requirements alone would
be a slog. Even simple stuff like this may or may not break and it's hard
to tell at a glance.

void mm_init_cid(struct mm_struct *mm, struct task_struct *p)
{
        mm->mm_cid.max_cids = 0;
        mm->mm_cid.mode = 0;
        mm->mm_cid.nr_cpus_allowed = p->nr_cpus_allowed;

It that always correct and without side-effects? It's mm-scope, not
task-scope so it's extra fun. I'm going to guess "no" because thread
something something races with migrate_disable on another task something
else.

update_misfit_status probably breaks because it's using nr_cpus_allowed
== 1 to detect pinning in a non-obvious way. I don't *think* it breaks,
degrades in some cases.

is_per_cpu_kthread() breaks although the impact is degraded performance
in corner cases.

A number of places seem to use nr_cpus_allowed to detect being pinned
so each would have to be audited to determine if they care about a CPU
affinity pin or a migration-disabled pin with extra double plus fun that
it only matters on PREEMPT_RT. If they care about the affinity pin, does
it matter if it changes in the near future due to migrate_disable? An
ironic example is that I think the task pushing a running
task away to allow a migrate_disabled task on the same CPU might break
just because a bunch of its helpers make assumptions on nr_cpus_allowed
vs task is migrate-disabled

I'm sure sched_ext has some fun but not going going to guess what.

It probably can be done with enough time and pain but it's not easy and
it's a disproportionate level of effort for a relatively simple bug.

> sprinkling more is_migration_disabled() is the simple solution for now.
> Dunno.

For the spurious IPI thing sure, but I still think the RT_PUSH_IPI
check is necessary as well just because deadline doesn't even have a
PUSH_IPI. Introducing one would probably get rejected because it endangers
tasks meeting their deadline. Applying the check anyway degrades deadline
itself. sched-rt can tolerate the PUSH_IPI without breaking its own rules
(priority over all else) and it solves a real problem. Introducing the IPI
in deadline just to match RT probably breaks or degrades deadline but it
still needs to find CPUs with migrate-disabled tasks. Someone might say
it won't break anything because the cost of the IPI, the migration and
the remaining budget is known and they'll get shutdown because the IPI
and migration costs are unknown. Same person or another will say the IPI
and migration costs can be estimated and they'll get kicked in the heuristic.

-- 
Mel Gorman
SUSE Labs

  reply	other threads:[~2026-11-19 23:34 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-13 16:06 [PATCH v3] sched/rt: Do not account migrate-disabled tasks as pushable Mel Gorman
2026-11-17 14:27 ` Mel Gorman
2026-11-17 15:10   ` Peter Zijlstra
2026-11-17 15:20 ` Peter Zijlstra
2026-11-17 15:24   ` Peter Zijlstra
2026-11-18 10:05     ` Mel Gorman
2026-11-19 12:06       ` Mel Gorman
2026-11-19 15:08         ` Peter Zijlstra
2026-11-19 15:17           ` Peter Zijlstra
2026-11-19 15:28             ` Peter Zijlstra
2026-11-19 16:40               ` Peter Zijlstra
2026-11-19 23:34                 ` Mel Gorman [this message]
2026-11-19 16:53           ` Mel Gorman
2026-11-17 15:25   ` Steven Rostedt

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=av9x3iMaKhtByVzM@techsingularity.net \
    --to=mgorman@techsingularity.net \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@redhat.com \
    --cc=peterz@infradead.org \
    --cc=rostedt@goodmis.org \
    --cc=snishika@redhat.com \
    --cc=vincent.guittot@linaro.org \
    --cc=ykaukab@suse.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox