[PATCH] cpu/hotplug: Prevent offlining the last domain housekeeping CPU

[PATCH] cpu/hotplug: Prevent offlining the last domain housekeeping CPU

[Guopeng Zhang]

From: Guopeng Zhang <zhangguopeng@kylinos.cn>

_cpu_down() is supposed to leave at least one HK_TYPE_DOMAIN CPU online.
However, cpumask_any_and() includes the CPU being offlined. The target
therefore satisfies the check when it is the last online CPU in that
housekeeping mask.

Offlining it leaves no online CPU in the HK_TYPE_DOMAIN mask. The next
scheduler-domain rebuild passes an empty span to build_sched_domains()
and triggers its WARN_ON().

Use cpumask_any_and_but() to look for another online HK_TYPE_DOMAIN CPU
and reject the offline request with -EBUSY if none exists.

Testing:
Tested on a 32-vCPU Ubuntu 24.04 guest with:

    isolcpus=domain,0,3-31

On the unpatched kernel, offlining the last HK_TYPE_DOMAIN CPU triggered
a kernel panic.

With this patch, the same operation was rejected with -EBUSY (errno 16),
the target CPU remained online, and no scheduler-domain warning was
observed.

Fixes: de715325cc47 ("cpu: Revert "cpu/hotplug: Prevent self deadlock on CPU hot-unplug"")
Signed-off-by: Guopeng Zhang <zhangguopeng@kylinos.cn>
---
 kernel/cpu.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/kernel/cpu.c b/kernel/cpu.c
index b3c8553d7bd6..81e961964e59 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -1418,8 +1418,9 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,
 	 * Keep at least one housekeeping cpu onlined to avoid generating
 	 * an empty sched_domain span.
 	 */
-	if (cpumask_any_and(cpu_online_mask,
-			    housekeeping_cpumask(HK_TYPE_DOMAIN)) >= nr_cpu_ids) {
+	if (cpumask_any_and_but(cpu_online_mask,
+				housekeeping_cpumask(HK_TYPE_DOMAIN),
+				cpu) >= nr_cpu_ids) {
 		ret = -EBUSY;
 		goto out;
 	}
-- 
2.43.0

Re: [PATCH] cpu/hotplug: Prevent offlining the last domain housekeeping CPU

[Bradley Morgan]

Hi Guopeng,

On Fri, 18 Jul 2026, Guopeng Zhang wrote:
> _cpu_down() is supposed to leave at least one HK_TYPE_DOMAIN CPU online.
> However, cpumask_any_and() includes the CPU being offlined. The target
> therefore satisfies the check when it is the last online CPU in that
> housekeeping mask.

The bug is real. Note the code before the revert got this right,
38685e2a0476
("cpu/hotplug: Don't offline the last non-isolated CPU") had

	for_each_cpu_and(cpu, cpu_online_mask, housekeeping_cpumask(HK_TYPE_DOMAIN)) {
		if (cpu != work.cpu)

and the exclusion got lost when de715325cc47 moved the check into
_cpu_down(). please say that in the changelog, it makes the Fixes: tag
obvious.

> Offlining it leaves no online CPU in the HK_TYPE_DOMAIN mask. The next
> scheduler-domain rebuild passes an empty span to build_sched_domains()
> and triggers its WARN_ON().

[...]

> On the unpatched kernel, offlining the last HK_TYPE_DOMAIN CPU triggered
> a kernel panic.

WARN_ON() or panic, pick one. if that was panic_on_warn, say so. if it
kept going and crashed later, paste the splat. The changelog should say
what actually happens on a default config.

> -	if (cpumask_any_and(cpu_online_mask,
> -			    housekeeping_cpumask(HK_TYPE_DOMAIN)) >= nr_cpu_ids) {
> +	if (cpumask_any_and_but(cpu_online_mask,
> +				housekeeping_cpumask(HK_TYPE_DOMAIN),
> +				cpu) >= nr_cpu_ids) {

The check itself is correct for the sysfs path. cpumask_any_and_but()
returns >= nr_cpu_ids when nothing is left, and excluding a cpu that is
not in the housekeeping mask is harmless. Placement is fine too,
de715325cc47 put this under cpus_write_lock() specifically to serialize
against cpuset partition updates, so it should stay where it is.

but this breaks suspend. _cpu_down() is also the frozen path,
freeze_secondary_cpus() does

	error = _cpu_down(cpu, 1, CPUHP_OFFLINE);

for every CPU except the boot CPU. With your own cmdline
(HK_TYPE_DOMAIN = {1,2}, primary = CPU0) suspend now dies at the last
housekeeping CPU:

	Error taking CPU1 down: -16
	Non-boot CPUs are not disabled

The old check passed there because it counted the outgoing CPU. I did not
check whether the frozen path can hit the same empty span WARN today, but
turning a suspend that used to work into -EBUSY is a regression either
way. do e.g:

	if (!tasks_frozen &&
	    cpumask_any_and_but(cpu_online_mask,
				housekeeping_cpumask(HK_TYPE_DOMAIN),
				cpu) >= nr_cpu_ids) {

and test suspend/resume with the same isolcpus cmdline for v2, not just
the sysfs offline.

The Fixes: tag LGTM, the revert is where the exclusion went missing. If
de715325cc47 is in a released kernel by now, CC stable. iirc cpu hotplug
is root only and this needs isolcpus on the cmdline, so put that in the
impact line and let the stable folks judge.
Thanks!