From: Waiman Long <longman@redhat.com>
To: Frederic Weisbecker <frederic@kernel.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
Juri Lelli <juri.lelli@redhat.com>,
Vincent Guittot <vincent.guittot@linaro.org>,
Michal Hocko <mhocko@suse.com>, Tejun Heo <tj@kernel.org>,
Dietmar Eggemann <dietmar.eggemann@arm.com>,
Steven Rostedt <rostedt@goodmis.org>,
Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
Valentin Schneider <vschneid@redhat.com>,
K Prateek Nayak <kprateek.nayak@amd.com>,
linux-kernel@vger.kernel.org
Subject: Re: [BUG] sched: BUG() in select_fallback_rq() at boot with isolcpus=domain on the boot CPU and CONFIG_CPUSETS=n
Date: Fri, 20 Nov 2026 12:31:45 -0500 [thread overview]
Message-ID: <d09eae60-9fb5-4e10-86c4-50914207c355@redhat.com> (raw)
In-Reply-To: <5a16dda3-f128-4109-a52b-f51e01309b35@redhat.com>
On 11/20/26 11:52 AM, Waiman Long wrote:
> On 11/19/26 11:03 AM, Frederic Weisbecker wrote:
>> Le Wed, Nov 18, 2026 at 10:55:35AM -0500, Waiman Long a écrit :
>>> On 11/18/26 9:49 AM, Frederic Weisbecker wrote:
>>>> Le Tue, Nov 17, 2026 at 10:13:02AM -0500, Waiman Long a écrit :
>>>>> On 11/17/26 9:48 AM, Mathieu Desnoyers wrote:
>>>>>> Hi,
>>>>>>
>>>>>> With CONFIG_CPUSETS=n, a kernel booted with isolcpus=domain,0
>>>>>> hits the
>>>>>> BUG() in select_fallback_rq() during boot. CPU 0 is the boot CPU. I
>>>>>> found it while testing a SCHED_DEADLINE series [1]. I then
>>>>>> reproduced
>>>>>> it on v7.4-rc2 alone. tip/master of 2026-11-17 and next-20261116
>>>>>> have
>>>>>> the same code. A proposed fix follows the scissors line.
>>>>>>
>>>>>> Trigger
>>>>>> =======
>>>>>>
>>>>>> Build x86-64 with CONFIG_CPUSETS=n. Boot a VM of 4 CPUs with
>>>>>> isolcpus=domain,0 or isolcpus=domain,0,1. The kernel panics before
>>>>>> init runs, in every boot.
>>>>>>
>>>>>> These boot fine:
>>>>>> - the same kernel with isolcpus=domain,1,2, or without isolcpus;
>>>>>> - a kernel with CONFIG_CPUSETS=y, with isolcpus=domain,0,1.
>>>>> I am now thinking about disallowing the boot CPU from being domain
>>>>> isolated.
>>>>> That will include using isolcpus=domain boot time isolation
>>>>> parameter as
>>>>> well as in setting up isolated cpuset partition. Will that be a
>>>>> simpler
>>>>> solution than fixing all these places where isolating the boot CPU
>>>>> will
>>>>> cause various problems that need to be fixed?
>>>> You can try that, hopefully nobody relies on that. nohz_full
>>>> supports the
>>>> boot CPU to be isolated but it's a heresy.
>>> The upcoming runtime modification of nohz_full CPU will not support
>>> the boot
>>> CPU
>> Agreed.
>>
>>> at least for x86, as the boot CPU cannot be made offline.
>> We should forbid that on all archs.
>>
>>> I would not
>>> recommend putting the boot CPU into nohz_full boot parameter, though
>>> we are
>>> not going to forbid it.
>> It's already supported and unfortunately we can't remove that as that
>> would
>> break kernel ABI.
>
> I just found that that CPU 0 will be automatically cleared from the
> nohz_full list in tick_nohz_init() for CONFIG_PM_SLEEP_SMP &&
> !CONFIG_PM_SLEEP_SMP_NONZERO_CPU kernels. IOW, that means most kernel
> will do CPU 0 cleariing. Although we don't explicitly say that CPU 0
> cannot be a nohz_full CPU, most kernels will not allow that.
Thinking about it, the clearing is done on the tick_nohz_full_mask which
is used by the timekeeping code and in
/sys/devices/system/cpu/nohz_full. So from a user point of view CPU 0
isn't in the nohz_full list. However, the HK_TYPE_KERNEL_NOISE cpumask
will also have CPU 0 missing. This is inconsistent. Should we fix that
by setting the bit 0 of HK_TYPE_KERNEL_NOISE cpumask when it is cleared
from the tick_nohz_full_mask?
Regards,
Longman
prev parent reply other threads:[~2026-11-20 17:31 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-17 14:48 [BUG] sched: BUG() in select_fallback_rq() at boot with isolcpus=domain on the boot CPU and CONFIG_CPUSETS=n Mathieu Desnoyers
2026-11-17 15:02 ` Peter Zijlstra
2026-11-18 17:35 ` Mathieu Desnoyers
2026-11-17 15:13 ` Waiman Long
2026-11-18 14:49 ` Frederic Weisbecker
2026-11-18 15:55 ` Waiman Long
2026-11-19 16:03 ` Frederic Weisbecker
2026-11-20 16:52 ` Waiman Long
2026-11-20 17:31 ` Waiman Long [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=d09eae60-9fb5-4e10-86c4-50914207c355@redhat.com \
--to=longman@redhat.com \
--cc=bsegall@google.com \
--cc=dietmar.eggemann@arm.com \
--cc=frederic@kernel.org \
--cc=juri.lelli@redhat.com \
--cc=kprateek.nayak@amd.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=mgorman@suse.de \
--cc=mhocko@suse.com \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tj@kernel.org \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.