[PATCH v2 0/3] cgroup/cpuset: Keep the top scheduling domain populated

[PATCH v2 0/3] cgroup/cpuset: Keep the top scheduling domain populated

[Guopeng Zhang]

From: Guopeng Zhang <zhangguopeng@kylinos.cn>

Boot-isolated CPUs stay in top_cpuset.effective_cpus, but are excluded
from its scheduling domain. Allocating all boot domain housekeeping CPUs
to partitions can therefore leave the top scheduling domain empty even
though effective_cpus is nonempty, leading to warnings and an OOPS in
build_perf_domains(). CPU hotplug can reach the same state.

The patches are ordered as follows:

  1. Check local and remote partition creation and CPU mask updates.
  2. Check the hotplug paths without crediting CPUs that remain allocated
     to child partitions, and invalidate local or remote partitions as
     needed to restore an active boot domain housekeeping CPU.
  3. Release an isolated child's CPUs when its parent is already invalid.

Changes since v1:
  Rework the fix following Ridong's suggestion and add test cases.
  Add patch 2 for CPU hotplug and patch 3 for isolated CPU cleanup
  when the parent partition becomes invalid.

  v1: https://lore.kernel.org/all/20261104071600.474587-1-guopeng.zhang@linux.dev/

Guopeng Zhang (3):
  cgroup/cpuset: Reject updates that empty the top sched domain
  cgroup/cpuset: Keep the top sched domain non-empty on CPU hotplug
  cgroup/cpuset: Release isolated CPUs when the parent is invalid

 kernel/cgroup/cpuset.c                        | 59 ++++++++++++++++++-
 .../selftests/cgroup/test_cpuset_prs.sh       | 55 +++++++++++++++++
 2 files changed, 111 insertions(+), 3 deletions(-)

-- 
2.43.0

[PATCH v2 1/3] cgroup/cpuset: Reject updates that empty the top sched domain

[Guopeng Zhang]

From: Guopeng Zhang <zhangguopeng@kylinos.cn>

Boot-isolated CPUs remain in top_cpuset.effective_cpus, but are excluded
from its scheduling domain by generate_sched_domains(). Checking only
effective_cpus when distributing CPUs to partitions can therefore allow
all boot domain housekeeping CPUs to be taken from the top cpuset.

The resulting empty domain triggers a warning in build_sched_domains()
and can cause an out-of-bounds CPU access in build_perf_domains(). Both
root and isolated partitions can trigger this, whether local or remote.
For example, on a 32-CPU system booted with isolcpus=domain,10-31:

  echo 0-9 > A/cpuset.cpus
  echo root > A/cpuset.cpus.partition
  Oops: general protection fault
  RIP: 0010:build_perf_domains+...

Require local and remote partition creation and CPU mask updates to
leave an active boot domain housekeeping CPU in the top cpuset,
accounting for CPUs returned by the same update.

Add test cases to test_cpuset_prs.sh to verify that every partition
update leaves an active boot domain housekeeping CPU in the top cpuset.

Fixes: 9b496a8bbed9 ("cgroup/cpuset: Prevent leakage of isolated CPUs into sched domains")
Co-developed-by: Ridong Chen <ridong.chen@linux.dev>
Signed-off-by: Ridong Chen <ridong.chen@linux.dev>
Signed-off-by: Guopeng Zhang <zhangguopeng@kylinos.cn>
---
 kernel/cgroup/cpuset.c                        | 48 ++++++++++++++++++-
 .../selftests/cgroup/test_cpuset_prs.sh       | 47 ++++++++++++++++++
 2 files changed, 93 insertions(+), 2 deletions(-)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 3052413e41e7..c405719ccd12 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -1180,12 +1180,56 @@ static void update_partition_sd_lb(struct cpuset *cs, int old_prs)
 		cpuset_force_rebuild();
 }
 
+/*
+ * top_cpuset_sd_nocpu - Check if top_cpuset will have no sched domain CPU
+ * @add_cpus: CPUs to be taken away from top_cpuset
+ * @del_cpus: CPUs to be given back to top_cpuset, can be NULL
+ * Return: true if no active CPU will be left in the top_cpuset sched domain
+ *
+ * Note that @del_cpus may include CPUs that are also in @add_cpus and
+ * will stay in the partition. They are skipped by the @add_cpus check,
+ * so callers can pass the current effective_xcpus of a partition
+ * without masking it first.
+ *
+ * Boot time isolated CPUs stay in top_cpuset.effective_cpus, but they are
+ * excluded from the sched domain of top_cpuset in generate_sched_domains().
+ * So checking top_cpuset.effective_cpus alone is not enough to ensure that
+ * the top_cpuset sched domain won't become empty after @add_cpus are taken
+ * away from it. The sched domain of top_cpuset must keep at least one
+ * active boot domain housekeeping CPU.
+ */
+static bool top_cpuset_sd_nocpu(const struct cpumask *add_cpus,
+				const struct cpumask *del_cpus)
+{
+	const struct cpumask *hk_mask = housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT);
+	int cpu;
+
+	for_each_cpu_and(cpu, hk_mask, cpu_active_mask) {
+		if (cpumask_test_cpu(cpu, add_cpus))
+			continue;
+		if (cpumask_test_cpu(cpu, top_cpuset.effective_cpus) ||
+		    (del_cpus && cpumask_test_cpu(cpu, del_cpus)))
+			return false;
+	}
+	return true;
+}
+
 /*
  * tasks_nocpu_error - Return true if tasks will have no effective_cpus
  */
 static bool tasks_nocpu_error(struct cpuset *parent, struct cpuset *cs,
 			      struct cpumask *xcpus)
 {
+	/*
+	 * The top_cpuset always has tasks (kthreads at least), so its sched
+	 * domain can't be empty. The current effective_xcpus of a valid
+	 * partition will be given back to top_cpuset if not in @xcpus.
+	 */
+	if (parent == &top_cpuset &&
+	    top_cpuset_sd_nocpu(xcpus, is_partition_valid(cs)
+				     ? cs->effective_xcpus : NULL))
+		return true;
+
 	/*
 	 * A populated partition (cs or parent) can't have empty effective_cpus
 	 */
@@ -1724,7 +1768,7 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs,
 	 */
 	compute_excpus(cs, tmp->new_cpus);
 	if (!cpumask_intersects(tmp->new_cpus, cpu_active_mask) ||
-	    cpumask_subset(top_cpuset.effective_cpus, tmp->new_cpus))
+	    top_cpuset_sd_nocpu(tmp->new_cpus, NULL))
 		return PERR_INVCPUS;
 	if (cpumask_intersects(tmp->new_cpus, subpartitions_cpus))
 		return PERR_NOCPUS;
@@ -1828,7 +1872,7 @@ static void remote_cpus_update(struct cpuset *cs, struct cpumask *xcpus,
 		if (!capable(CAP_SYS_ADMIN))
 			WRITE_ONCE(cs->prs_err, PERR_ACCESS);
 		else if (cpumask_intersects(tmp->addmask, subpartitions_cpus) ||
-			 cpumask_subset(top_cpuset.effective_cpus, tmp->addmask))
+			 top_cpuset_sd_nocpu(tmp->addmask, tmp->delmask))
 			WRITE_ONCE(cs->prs_err, PERR_NOCPUS);
 		else if (prstate_housekeeping_conflict(prs, PRS_ROOT,
 						       excpus, tmp->delmask))
diff --git a/tools/testing/selftests/cgroup/test_cpuset_prs.sh b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
index bdca6c471f87..e813ea9959f4 100755
--- a/tools/testing/selftests/cgroup/test_cpuset_prs.sh
+++ b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
@@ -133,6 +133,15 @@ LAST_PRESENT_CPU=$(cat /sys/devices/system/cpu/present | sed -e "s/.*[,-]//")
 [[ -n "$FIRST_NOHZ_FULL_CPU" && "$FIRST_NOHZ_FULL_CPU" = 8 &&
    "$LAST_NOHZ_FULL_CPU" = $LAST_PRESENT_CPU ]] && RUN_NOHZ_FULL_TESTS=1
 
+# These tests require boot domain housekeeping CPUs to be exactly 0-9.
+# Check the actual masks to cover isolcpus= with or without explicit flags.
+DOMAIN_ISOLCPUS="10-$LAST_PRESENT_CPU"
+[[ "$LAST_PRESENT_CPU" = 10 ]] && DOMAIN_ISOLCPUS=10
+RUN_ISOLCPUS_DOMAIN_TESTS=
+[[ -z "$NOHZ_FULL_CPUS" &&
+	"$(cat /sys/devices/system/cpu/isolated)" = "$DOMAIN_ISOLCPUS" ]] &&
+	RUN_ISOLCPUS_DOMAIN_TESTS=1
+
 #
 # If isolated CPUs have been reserved at boot time (as shown in
 # /sys/devices/system/cpu/isolated), these isolated CPUs should be outside of
@@ -602,6 +611,43 @@ ISOLCPUS_TEST_MATRIX=(
 	" CX1-2,I:X1,I:P1 . .      .     X1,2,I  .      .      .     0 A1:1-2,I|XA1:1-2,I A1:P-1 ."
 )
 
+# Test matrix for isolcpus=domain,10-<last> or isolcpus=10-<last> boots.
+# The top cpuset must retain an active boot domain housekeeping CPU.
+ISOLCPUS_DOMAIN_TEST_MATRIX=(
+	#  old-A1 old-A2 old-A3 old-B1 new-A1 new-A2 new-A3 new-B1 fail ECPUs Pstate ISOLCPUS
+	#  ------ ------ ------ ------ ------ ------ ------ ------ ---- ----- ------ --------
+	# Neither a root nor an isolated partition may take all boot HK CPUs.
+	" C0-9      .      .      .     P1      .      .      .     0 A1:0-9 A1:P-1 ."
+	" C0-9      .      .      .     P2      .      .      .     0 A1:0-9 A1:P-2 ."
+
+	# Growing cpuset.cpus must not take the last top-level HK CPU.
+	" C0-8:P1   .      .      .    C0-9     .      .      .     0 A1:0-9 A1:P-1 ."
+
+	# CPUs returned by the same update can keep the top sched domain alive.
+	" C2-9:P2   .      .      .    C0-4     .      .      .     0 A1:0-4 A1:P2 0-4"
+
+	# Root and isolated partitions together must leave a top-level HK CPU.
+	" C0-4:P1   .      .     C5-9    .      .      .     P2    0 A1:0-4|B1:5-9 A1:P1|B1:P-2 ."
+
+	# Remote partition creation must enforce the same constraint.
+	" X0-9    CX0-9    .      .      .     P1      .      .     0 A2:0-9 A2:P-1 ."
+
+	# Growing a remote partition through cpuset.cpus.exclusive is rejected.
+	" X0-9 C0-9:X0-8:P1 .     .      .    X0-9     .      .     0 A2:0-9 A2:P-1 ."
+
+	# Remote CPU replacements must account for the CPUs being returned.
+	" X0-9 C0-9:X2-9:P2 .     .      .    X0-4     .      .     0 A2:0-4 A2:P2 0-4"
+
+	# Local exclusive CPU updates must not consume the last top-level HK CPU.
+	" C0-9:X0-8:P2 .   .      .    X0-9     .      .      .     0 A1:0-9 A1:P-2 ."
+
+	# Ancestor updates must not grow a remote partition over the last HK CPU.
+	" X0-8 CX0-9:P1    .      .    X0-9     .      .      .     0 A2:0-9 A2:P-1 ."
+
+	# An invalid partition can recover once it leaves a top-level HK CPU.
+	" C0-9:P1   .      .      .    C0-8     .      .      .     0 A1:0-8 A1:P1 ."
+)
+
 # Test matrix to be run when boot time nohz_full CPUs cover all CPUs except CPUs 0-7
 # This also means that CPU 8 is available and is a nohz_full CPU.
 NOHZ_FULL_TEST_MATRIX=(
@@ -1472,6 +1518,7 @@ test_inotify()
 run_state_test TEST_MATRIX
 run_remote_state_test REMOTE_TEST_MATRIX
 TEST_ISOLCPUS=1 run_state_test ISOLCPUS_TEST_MATRIX
+[[ -n "$RUN_ISOLCPUS_DOMAIN_TESTS" ]] && run_state_test ISOLCPUS_DOMAIN_TEST_MATRIX
 [[ -n "$RUN_NOHZ_FULL_TESTS" ]] && run_state_test NOHZ_FULL_TEST_MATRIX
 test_isolated
 test_boot_isolated
-- 
2.43.0

[PATCH v2 2/3] cgroup/cpuset: Keep the top sched domain non-empty on CPU hotplug

[Guopeng Zhang]

From: Guopeng Zhang <zhangguopeng@kylinos.cn>

Configuration updates must leave an active boot domain housekeeping CPU
in the top cpuset, but CPU hotplug can still empty its scheduling domain.

For a local partition with child partitions, cpuset_hotplug_update_tasks()
passes the partition's own effective CPUs to tasks_nocpu_error(). The
embedded top_cpuset_sd_nocpu() check credits the old exclusive CPUs not
in that mask as CPUs being returned to the top cpuset. Those CPUs may
still belong to child partitions, so no such return takes place.
For example, with isolcpus=domain,10-31 on a 32-CPU system:

  echo 0,2-9 > A/cpuset.cpus
  echo root > A/cpuset.cpus.partition
  echo 5-9 > A/B/cpuset.cpus
  echo root > A/B/cpuset.cpus.partition
  echo 0 > /sys/devices/system/cpu/cpu1/online
  Oops: general protection fault
  RIP: 0010:build_perf_domains+...

Remote partition hotplug checks also miss the empty top scheduling
domain because they only examine the remote partition's own CPUs.

Check the local partition's full exclusive mask without crediting any
returned CPUs. For remote partitions, check the top scheduling domain
with all partition grants still excluded. Invalidate a partition through
its existing path to return its CPUs. Once an active boot domain
housekeeping CPU is returned, subsequent partitions no longer fail this
check.

Add hotplug test cases to test_cpuset_prs.sh to verify that the top
cpuset sched domain stays non-empty when the last housekeeping CPU
goes away.

Fixes: 9b496a8bbed9 ("cgroup/cpuset: Prevent leakage of isolated CPUs into sched domains")
Signed-off-by: Guopeng Zhang <zhangguopeng@kylinos.cn>
---
 kernel/cgroup/cpuset.c                            | 4 ++++
 tools/testing/selftests/cgroup/test_cpuset_prs.sh | 8 ++++++++
 2 files changed, 12 insertions(+)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index c405719ccd12..fc5454c2740f 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -4053,6 +4053,7 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp)
 		goto update_tasks;
 
 	if (is_remote_partition(cs) && (cpumask_empty(subpartitions_cpus) ||
+				       top_cpuset_sd_nocpu(subpartitions_cpus, NULL) ||
 				       (cpumask_empty(&new_cpus) &&
 					partition_is_populated(cs, NULL)))) {
 		WRITE_ONCE(cs->prs_err, PERR_HOTPLUG);
@@ -4067,9 +4068,12 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp)
 	 * 2) parent is invalid or doesn't grant any cpus to child
 	 *    partitions.
 	 * 3) subpartitions_cpus is empty.
+	 * 4) top_cpuset has no active boot domain housekeeping CPU left.
 	 */
 	if (is_local_partition(cs) &&
 	    (!is_partition_valid(parent) ||
+	     (parent == &top_cpuset &&
+	      top_cpuset_sd_nocpu(cs->effective_xcpus, NULL)) ||
 	     tasks_nocpu_error(parent, cs, &new_cpus) ||
 	     cpumask_empty(subpartitions_cpus)))
 		partcmd = partcmd_invalidate;
diff --git a/tools/testing/selftests/cgroup/test_cpuset_prs.sh b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
index e813ea9959f4..f03fffeab034 100755
--- a/tools/testing/selftests/cgroup/test_cpuset_prs.sh
+++ b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
@@ -646,6 +646,14 @@ ISOLCPUS_DOMAIN_TEST_MATRIX=(
 
 	# An invalid partition can recover once it leaves a top-level HK CPU.
 	" C0-9:P1   .      .      .    C0-8     .      .      .     0 A1:0-8 A1:P1 ."
+
+	# Offlining CPU 1 leaves only boot-isolated CPUs in the top cpuset.
+	# A local root must not count its child's CPUs as already returned.
+	# In 1|, offline CPU 1 has no sched domain but is not cpuset-isolated.
+	" C0,2-9:P1 C5-9:P1 . . O1=0 . . . 0 A1:0,2-9|A2:5-9 A1:P-1|A2:P-1 1|"
+
+	# A remote root must return CPUs when the top domain loses its last HK CPU.
+	" X0,2-9 CX0,2-9:P1 . . O1=0 . . . 0 A2:0,2-9 A2:P-1 1|"
 )
 
 # Test matrix to be run when boot time nohz_full CPUs cover all CPUs except CPUs 0-7
-- 
2.43.0

[PATCH v2 3/3] cgroup/cpuset: Release isolated CPUs when the parent is invalid

[Guopeng Zhang]

From: Guopeng Zhang <zhangguopeng@kylinos.cn>

An isolated partition can be invalidated after its parent has already
become invalid. update_parent_effective_cpumask() then skips returning
CPUs to the parent, but also skips removing them from isolated_cpus.
The invalid child continues to isolate CPUs that no longer belong to
any valid isolated partition.

For example, on a 32-CPU system with all CPUs online and without
isolcpus= or nohz_full= boot parameters:

  cd /sys/fs/cgroup
  echo +cpuset > cgroup.subtree_control
  mkdir A
  echo 0,2-31 > A/cpuset.cpus
  echo root > A/cpuset.cpus.partition
  echo +cpuset > A/cgroup.subtree_control
  mkdir A/B
  echo 2-3 > A/B/cpuset.cpus
  echo isolated > A/B/cpuset.cpus.partition
  cat cpuset.cpus.isolated
  2-3
  echo 0 > /sys/devices/system/cpu/cpu1/online
  cat A/B/cpuset.cpus.partition
  isolated invalid
  cat cpuset.cpus.isolated
  2-3

Release the invalidated child's isolated CPUs when its parent is no
longer a valid partition. Use isolated_cpus_release() to preserve CPUs
still covered by a valid isolated ancestor and by boot isolation.

After this fix, cpuset.cpus.isolated is empty for this reproducer.

Fixes: 11e5f407b64a ("cgroup/cpuset: Keep track of CPUs in isolated partitions")
Signed-off-by: Guopeng Zhang <zhangguopeng@kylinos.cn>
---
 kernel/cgroup/cpuset.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index fc5454c2740f..f3b808cea51c 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -1976,10 +1976,15 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
 		/*
 		 * Make the current partition invalid.
 		 */
-		if (is_partition_valid(parent))
+		if (is_partition_valid(parent)) {
 			adding = cpumask_and(tmp->addmask,
 					     cs->effective_xcpus,
 					     parent->effective_xcpus);
+		} else if (old_prs == PRS_ISOLATED) {
+			/* No valid parent will release these isolated CPUs. */
+			cpumask_copy(tmp->addmask, cs->effective_xcpus);
+			isolated_cpus_release(cs, tmp->addmask);
+		}
 		if (old_prs > 0)
 			new_prs = -old_prs;
 
-- 
2.43.0