[PATCH v2] cpu/hotplug: Prevent offlining the last domain housekeeping CPU

[PATCH v2] cpu/hotplug: Prevent offlining the last domain housekeeping CPU

[Guopeng Zhang]

From: Guopeng Zhang <zhangguopeng@kylinos.cn>

Commit 38685e2a0476 ("cpu/hotplug: Don't offline the last
non-isolated CPU") kept at least one HK_TYPE_DOMAIN CPU online by
selecting a CPU other than the one being offlined. Commit de715325cc47
("cpu: Revert "cpu/hotplug: Prevent self deadlock on CPU hot-unplug"")
moved the check into _cpu_down(), but lost that exclusion.

The replacement cpumask_any_and() check still sees the outgoing CPU in
cpu_online_mask. The target therefore satisfies the check when it is
the last online CPU in the HK_TYPE_DOMAIN mask.

On a system booted with domain isolation, a root-initiated attempt to
offline the last HK_TYPE_DOMAIN CPU leaves scheduler-domain rebuilds
with an empty span. With panic_on_warn=0 and warn_limit=0, this first
triggered the WARN_ON() in build_sched_domains(). The kernel then
crashed with a general protection fault in build_perf_domains():

 WARNING: ... at build_sched_domains+0x443/0xa60
 ...
 Oops: general protection fault ...
 RIP: 0010:build_perf_domains+0x40/0x230
 Call Trace:
  <TASK>
  partition_sched_domains_locked+0x3d9/0x710
  partition_sched_domains+0x30/0x40
  cpuset_reset_sched_domains+0x25/0x40
  sched_cpu_deactivate+0x2f2/0x300
  cpuhp_invoke_callback+0x1a4/0x780
  cpuhp_thread_fun+0x1b4/0x230
  </TASK>

Use cpumask_any_and_but() to exclude the outgoing CPU and return -EBUSY
when no other online HK_TYPE_DOMAIN CPU remains.

Skip this check when tasks_frozen is set because CPU freeze deliberately
takes all non-primary CPUs offline.

The sysfs CPU-offline path was tested on a 32-vCPU Ubuntu 24.04 guest
booted with:

    isolcpus=domain,0,3-31

The unpatched kernel warned in build_sched_domains() and then crashed
with a general protection fault in build_perf_domains(). With this
patch, the request was rejected with -EBUSY, the target CPU remained
online, and no scheduler-domain warning was observed.

Fixes: de715325cc47 ("cpu: Revert "cpu/hotplug: Prevent self deadlock on CPU hot-unplug"")
Suggested-by: Bradley Morgan <include@grrlz.net>
Signed-off-by: Guopeng Zhang <zhangguopeng@kylinos.cn>
---
Changes in v2:
- Explain that 38685e2a0476 excluded the outgoing CPU and that
  de715325cc47 lost that exclusion when moving the check.
- Report the observed WARN_ON() followed by a GPF in
  build_perf_domains(), and include the relevant splat.
- Skip the regular-hotplug guard while tasks_frozen is set, as suggested
  by Bradley.
- State the root and domain-isolation requirements.
- Retest the frozen path with the same isolcpus setting. The new guard no
  longer returns -EBUSY.

v1: https://lore.kernel.org/all/20260718095249.1271870-1-guopeng.zhang@linux.dev/

 kernel/cpu.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/kernel/cpu.c b/kernel/cpu.c
index b3c8553d7bd6..8ffbfc20b8fe 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -1415,11 +1415,13 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,
 	cpus_write_lock();
 
 	/*
-	 * Keep at least one housekeeping cpu onlined to avoid generating
-	 * an empty sched_domain span.
+	 * Keep at least one HK_TYPE_DOMAIN CPU online during regular hotplug
+	 * to avoid generating an empty sched_domain span.
 	 */
-	if (cpumask_any_and(cpu_online_mask,
-			    housekeeping_cpumask(HK_TYPE_DOMAIN)) >= nr_cpu_ids) {
+	if (!tasks_frozen &&
+	    cpumask_any_and_but(cpu_online_mask,
+				housekeeping_cpumask(HK_TYPE_DOMAIN),
+				cpu) >= nr_cpu_ids) {
 		ret = -EBUSY;
 		goto out;
 	}
-- 
2.43.0

Re: [PATCH v2] cpu/hotplug: Prevent offlining the last domain housekeeping CPU

[Bradley Morgan]

On 11 August 2026 13:13:07 BST, Guopeng Zhang <guopeng.zhang@linux.dev>
wrote:
>From: Guopeng Zhang <zhangguopeng@kylinos.cn>
>
>Commit 38685e2a0476 ("cpu/hotplug: Don't offline the last
>non-isolated CPU") kept at least one HK_TYPE_DOMAIN CPU online by
>selecting a CPU other than the one being offlined. Commit de715325cc47
>("cpu: Revert "cpu/hotplug: Prevent self deadlock on CPU hot-unplug"")
>moved the check into _cpu_down(), but lost that exclusion.
>
>The replacement cpumask_any_and() check still sees the outgoing CPU in
>cpu_online_mask. The target therefore satisfies the check when it is
>the last online CPU in the HK_TYPE_DOMAIN mask.
>
>On a system booted with domain isolation, a root-initiated attempt to
>offline the last HK_TYPE_DOMAIN CPU leaves scheduler-domain rebuilds
>with an empty span. With panic_on_warn=0 and warn_limit=0, this first
>triggered the WARN_ON() in build_sched_domains(). The kernel then
>crashed with a general protection fault in build_perf_domains():
>
> WARNING: ... at build_sched_domains+0x443/0xa60
> ...
> Oops: general protection fault ...
> RIP: 0010:build_perf_domains+0x40/0x230
> Call Trace:
>  <TASK>
>  partition_sched_domains_locked+0x3d9/0x710
>  partition_sched_domains+0x30/0x40
>  cpuset_reset_sched_domains+0x25/0x40
>  sched_cpu_deactivate+0x2f2/0x300
>  cpuhp_invoke_callback+0x1a4/0x780
>  cpuhp_thread_fun+0x1b4/0x230
>  </TASK>
>
>Use cpumask_any_and_but() to exclude the outgoing CPU and return -EBUSY
>when no other online HK_TYPE_DOMAIN CPU remains.
>
>Skip this check when tasks_frozen is set because CPU freeze deliberately
>takes all non-primary CPUs offline.
>
>The sysfs CPU-offline path was tested on a 32-vCPU Ubuntu 24.04 guest
>booted with:
>
>    isolcpus=domain,0,3-31
>
>The unpatched kernel warned in build_sched_domains() and then crashed
>with a general protection fault in build_perf_domains(). With this
>patch, the request was rejected with -EBUSY, the target CPU remained
>online, and no scheduler-domain warning was observed.
>
>Fixes: de715325cc47 ("cpu: Revert "cpu/hotplug: Prevent self deadlock on CPU hot-unplug"")
>Suggested-by: Bradley Morgan <include@grrlz.net>

Reviewed-by: Bradley Morgan <include@grrlz.net>


>Signed-off-by: Guopeng Zhang <zhangguopeng@kylinos.cn>
>---
>Changes in v2:
>- Explain that 38685e2a0476 excluded the outgoing CPU and that
>  de715325cc47 lost that exclusion when moving the check.
>- Report the observed WARN_ON() followed by a GPF in
>  build_perf_domains(), and include the relevant splat.
>- Skip the regular-hotplug guard while tasks_frozen is set, as suggested
>  by Bradley.
>- State the root and domain-isolation requirements.
>- Retest the frozen path with the same isolcpus setting. The new guard no
>  longer returns -EBUSY.
>
>v1: https://lore.kernel.org/all/20260718095249.1271870-1-guopeng.zhang@linux.dev/
>
> kernel/cpu.c | 10 ++++++----
> 1 file changed, 6 insertions(+), 4 deletions(-)
>
>diff --git a/kernel/cpu.c b/kernel/cpu.c
>index b3c8553d7bd6..8ffbfc20b8fe 100644
>--- a/kernel/cpu.c
>+++ b/kernel/cpu.c
>@@ -1415,11 +1415,13 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,
> 	cpus_write_lock();
> 
> 	/*
>-	 * Keep at least one housekeeping cpu onlined to avoid generating
>-	 * an empty sched_domain span.
>+	 * Keep at least one HK_TYPE_DOMAIN CPU online during regular hotplug
>+	 * to avoid generating an empty sched_domain span.
> 	 */
>-	if (cpumask_any_and(cpu_online_mask,
>-			    housekeeping_cpumask(HK_TYPE_DOMAIN)) >= nr_cpu_ids) {
>+	if (!tasks_frozen &&
>+	    cpumask_any_and_but(cpu_online_mask,
>+				housekeeping_cpumask(HK_TYPE_DOMAIN),
>+				cpu) >= nr_cpu_ids) {
> 		ret = -EBUSY;
> 		goto out;
> 	}
>

Thanks!