[RFC PATCH 0/3] sched: Handle CPU freeze without active domain housekeeping CPUs

[Guopeng Zhang <guopeng.zhang@linux.dev>]

From: Guopeng Zhang <zhangguopeng@kylinos.cn>

During suspend, freeze_secondary_cpus() takes every CPU except the
primary offline. With domain isolation, the primary CPU can be excluded
from HK_TYPE_DOMAIN. Offlining the last domain housekeeping CPU then
leaves the frozen hotplug path without an active CPU in that mask.

This exposes three related problems. The frozen cpuset callback asks for
one fallback scheduler domain even though its span is empty. A user task
restricted to the CPUs being frozen can exhaust select_fallback_rq()
because the normal fallback mask has no active CPU. Finally, forcing such
a task onto a temporary CPU clears user_cpus_ptr, so the temporary
affinity can survive after the CPUs are brought back online.

This series preserves and restores user affinity across the frozen CPU
fallback, permits an isolated but task-capable active CPU as a temporary
last resort, and explicitly tears down scheduler domains while no active
HK_TYPE_DOMAIN CPU exists.

I am sending this as an RFC because the first patch adds scheduler state
for the freeze transaction and collects affected tasks during thaw. In
particular, feedback would be useful on the task tracking and restore
scheme, the use of task_cpu_possible_mask() as the temporary fallback,
and the explicit zero-domain request passed to
partition_sched_domains().

Reproduction
============

The problem was reproduced on a 32-CPU x86 physical machine booted with:

    isolcpus=domain,0,3-31

This leaves CPUs 1 and 2 in HK_TYPE_DOMAIN while the suspend primary is
CPU0. On an unpatched 7.2.0-rc3-next-20260716 kernel, with
panic_on_warn=0 and warn_limit=0, the following processor-stage suspend
test crashed the machine:

    echo deep > /sys/power/mem_sleep
    echo processors > /sys/power/pm_test
    echo mem > /sys/power/state

After CPU2 had been taken offline, rebuilding the scheduler domains for
CPU1 produced an empty span warning, followed by a general protection
fault in build_perf_domains():

    smpboot: CPU 2 is now offline
    ------------[ cut here ]------------
    WARNING: kernel/sched/topology.c:3093 at build_sched_domains+0x443/0xa60
    ...
    Call Trace:
     partition_sched_domains_locked+0x2dd/0x710
     partition_sched_domains+0x30/0x40
     cpuset_reset_sched_domains+0x25/0x40
     sched_cpu_deactivate+0x2f2/0x300
    ...
    Oops: general protection fault, probably for non-canonical address ...
    RIP: 0010:build_perf_domains+0x40/0x230
    Call Trace:
     partition_sched_domains_locked+0x3d9/0x710
     partition_sched_domains+0x30/0x40
     cpuset_reset_sched_domains+0x25/0x40
     sched_cpu_deactivate+0x2f2/0x300

Kdump captured a vmcore after the fault. This was not a panic_on_warn
failure.

Testing
=======

With this series applied, the processor-stage test was repeated while
the shell was restricted to the two domain housekeeping CPUs:

    taskset -pc 1,2 $$
    echo deep > /sys/power/mem_sleep
    echo processors > /sys/power/pm_test
    echo mem > /sys/power/state
    ret=$?
    echo none > /sys/power/pm_test
    echo "return=$ret"
    taskset -pc $$
    cat /sys/devices/system/cpu/online

The fallback path was exercised:

    process 7271 (bash) no longer affine to cpu1

The test completed successfully and the requested affinity and online
CPU mask were restored:

    return=0
    pid 7271's current affinity list: 1,2
    0-31

A real S3 cycle was then tested with:

    echo deep > /sys/power/mem_sleep
    echo none > /sys/power/pm_test
    rtcwake -m no -s 10
    echo mem > /sys/power/state
    ret=$?
    echo "return=$ret"
    taskset -pc $$
    cat /sys/devices/system/cpu/online

The machine entered S3, resumed, and brought CPUs 1-31 back online. The
result was again:

    return=0
    pid 7271's current affinity list: 1,2
    0-31

Neither test produced a scheduler-domain warning, GPF, BUG, Oops, or
panic.

Each patch was build-tested in sequence with x86_64_defconfig. The final
series completed a full x86_64 build. The affected objects were also
build-tested with CONFIG_CPUSETS=n and CONFIG_CPU_ISOLATION=y.

Guopeng Zhang (3):
  sched: Preserve user affinity across frozen CPU fallback
  sched: Allow isolated CPUs as a last resort during CPU freeze
  sched/topology: Tear down domains without active domain housekeeping
    CPUs

 include/linux/cpuset.h        |   7 +-
 include/linux/sched/hotplug.h |   2 +
 kernel/cgroup/cpuset.c        |   6 +-
 kernel/cpu.c                  |   2 +
 kernel/sched/core.c           | 191 ++++++++++++++++++++++++++++++++--
 kernel/sched/sched.h          |   4 +-
 kernel/sched/topology.c       |   9 +-
 7 files changed, 207 insertions(+), 14 deletions(-)


base-commit: 0718283ab28bc3907e10b61a6b4be6fefa1cbb2f
-- 
2.43.0