cgroups.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH-next v4 0/6] cgroup/cpuset: More fix patches
@ 2026-11-16  1:23 Waiman Long
  2026-11-16  1:23 ` [PATCH-next v4 1/6] cgroup/cpuset: Fix incorrect partition invalidation logic in cpuset_hotplug_update_tasks() Waiman Long
                   ` (5 more replies)
  0 siblings, 6 replies; 18+ messages in thread
From: Waiman Long @ 2026-11-16  1:23 UTC (permalink / raw)
  To: Ridong Chen, Tejun Heo, Johannes Weiner, Michal Koutný,
	Shuah Khan
  Cc: cgroups, linux-kernel, linux-kselftest, Guopeng Zhang,
	Waiman Long

[v4]
  - Drop v3 patch 1 as it is fixed in a different way in patch 5.
  - Add a new patch 2 fixing a minor bug reported by sashiko.
  - Add patch 4/5 to fix a remote partition hotplug problem reported
    by Guopeng.
  - Add patch 6 to fix a bug in lockdep_is_cpuset_held().

[v3]
  - Drop patch 1, 4 & 5 of v2 as they are merged.
  - Update the new patch 1 as suggested by Chen Ridong
  - Add a new fix patch 3.

[v2]
  - Reorder the series by putting the original patch 4 before patch 2.
  - Update the Fixes tags in patch 1 and 5 as suggested by sashiko.
  - Update the new patch 3 to remove obsolete WARN_ON_ONCE() and update
    comment as reported by sashiko.
  - Minor fix to the test case in patch 5 as reported by sashiko.

This patch series contains an additional set of cpuset fix patches.

Waiman Long (6):
  cgroup/cpuset: Fix incorrect partition invalidation logic in
    cpuset_hotplug_update_tasks()
  cgroup/cpuset: excpus parameter of remote_cpus_update() must have at
    least 1 active CPU
  cgroup/cpuset: Check remote partition underneath before making an
    invalid partition valid
  cgroup/cpuset: Revert "remove global remote_children list"
  cgroup/cpuset: Handle remote partitions before local partitions in
    cpuset_handle_hotplug()
  cgroup/cpuset: Fix lockdep_is_cpuset_held()

 kernel/cgroup/cpuset-internal.h               |  10 +-
 kernel/cgroup/cpuset-v1.c                     |  16 +-
 kernel/cgroup/cpuset.c                        | 178 +++++++++++++-----
 .../selftests/cgroup/test_cpuset_prs.sh       |   6 +-
 4 files changed, 149 insertions(+), 61 deletions(-)

-- 
2.56.0


^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH-next v4 1/6] cgroup/cpuset: Fix incorrect partition invalidation logic in cpuset_hotplug_update_tasks()
  2026-11-16  1:23 [PATCH-next v4 0/6] cgroup/cpuset: More fix patches Waiman Long
@ 2026-11-16  1:23 ` Waiman Long
  2026-11-16 10:19   ` Guopeng Zhang
  2026-11-16  1:23 ` [PATCH-next v4 2/6] cgroup/cpuset: excpus parameter of remote_cpus_update() must have at least 1 active CPU Waiman Long
                   ` (4 subsequent siblings)
  5 siblings, 1 reply; 18+ messages in thread
From: Waiman Long @ 2026-11-16  1:23 UTC (permalink / raw)
  To: Ridong Chen, Tejun Heo, Johannes Weiner, Michal Koutný,
	Shuah Khan
  Cc: cgroups, linux-kernel, linux-kselftest, Guopeng Zhang,
	Waiman Long, Sashiko

When most of the active CPUs of a system are allocated to
sub-partitions below the top_cpuset partition, it is possible that a
CPU offline operation may remove the last active CPU in the top_cpuset
partition. When that really happens, cpuset_handle_hotplug() will clear
the subpartitions_cpus to make sure active CPUs are present in the top
partition and let the other sub-partitions get invalidated and then
competes for their exclusive CPUs again.

In cpuset_hotplug_update_tasks(), the current code use
"cpumask_empty(subpartitions_cpus)" to determine if that partition
should be invalidated. That is fine if there is only one sub-partition.
As reported by sashiko that when there are more than one sub-partitions
and the first sub-partition will get invalidated and then it will
try to acquire the exclusive CPUs and become valid again. When this
happens, the subpartitions_cpus will no longer be empty. The remaining
sub-partitions will assume their exclusive CPUs are safe and proceed
as normal. As a result, active CPUs owned by those sub-partitions
remain in both top_cpuset.effective_cpus and the sub-partitions'
effective_cpus while missing from subpartitions_cpus, breaking partition
CPU exclusivity and triggering WARN_ON_ONCE() in generate_sched_domains()
and remote_partition_disable().

Fix that by checking if the sub-partitions' effective_xcpus are subset
of subpartitions_cpus to determine if they should be invalidated. Also
remove the obsolete WARN_ON_ONCE() call in remote_partition_disable().

Fixes: 0c7f293efc87 ("cgroup/cpuset: Add cpuset.cpus.exclusive.effective for v2")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Link: https://sashiko.dev/#/bug/linux-bf69cd3c-833a-4599-a1f9-8dd9e044cd7c
Reviewed-by: Ridong Chen <ridong.chen@linux.dev>
Signed-off-by: Waiman Long <longman@redhat.com>
---
 kernel/cgroup/cpuset.c | 19 +++++--------------
 1 file changed, 5 insertions(+), 14 deletions(-)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 1b88a93b3db4..631fb449cef8 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -1768,15 +1768,6 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs,
 static void remote_partition_disable(struct cpuset *cs, struct tmpmasks *tmp)
 {
 	WARN_ON_ONCE(!is_remote_partition(cs));
-	/*
-	 * When a CPU is offlined, top_cpuset may end up with no available CPUs,
-	 * which should clear subpartitions_cpus. We should not emit a warning for this
-	 * scenario: the hierarchy is updated from top to bottom, so subpartitions_cpus
-	 * may already be cleared when disabling the partition.
-	 */
-	WARN_ON_ONCE(!cpumask_subset(cs->effective_xcpus, subpartitions_cpus) &&
-		     !cpumask_empty(subpartitions_cpus));
-
 	spin_lock_irq(&callback_lock);
 	cs->remote_partition = false;
 	partition_xcpus_del(cs->partition_root_state, NULL, cs->effective_xcpus);
@@ -3983,9 +3974,9 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp)
 	if (!tmp || !cs->partition_root_state)
 		goto update_tasks;
 
-	if (is_remote_partition(cs) && (cpumask_empty(subpartitions_cpus) ||
-				       (cpumask_empty(&new_cpus) &&
-					partition_is_populated(cs, NULL)))) {
+	if (is_remote_partition(cs) &&
+	    (!cpumask_subset(cs->effective_xcpus, subpartitions_cpus) ||
+	     (cpumask_empty(&new_cpus) && partition_is_populated(cs, NULL)))) {
 		WRITE_ONCE(cs->prs_err, PERR_HOTPLUG);
 		remote_partition_disable(cs, tmp);
 		compute_effective_cpumask(&new_cpus, cs, parent);
@@ -3997,12 +3988,12 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp)
 	 * 1) empty effective cpus but not valid empty partition.
 	 * 2) parent is invalid or doesn't grant any cpus to child
 	 *    partitions.
-	 * 3) subpartitions_cpus is empty.
+	 * 3) effective_xcpus isn't a subset of subpartitions_cpus.
 	 */
 	if (is_local_partition(cs) &&
 	    (!is_partition_valid(parent) ||
 	     tasks_nocpu_error(parent, cs, &new_cpus) ||
-	     cpumask_empty(subpartitions_cpus)))
+	     !cpumask_subset(cs->effective_xcpus, subpartitions_cpus)))
 		partcmd = partcmd_invalidate;
 	/*
 	 * On the other hand, an invalid partition root may be transitioned
-- 
2.56.0


^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [PATCH-next v4 2/6] cgroup/cpuset: excpus parameter of remote_cpus_update() must have at least 1 active CPU
  2026-11-16  1:23 [PATCH-next v4 0/6] cgroup/cpuset: More fix patches Waiman Long
  2026-11-16  1:23 ` [PATCH-next v4 1/6] cgroup/cpuset: Fix incorrect partition invalidation logic in cpuset_hotplug_update_tasks() Waiman Long
@ 2026-11-16  1:23 ` Waiman Long
  2026-11-16  3:07   ` Ridong Chen
  2026-11-16  1:23 ` [PATCH-next v4 3/6] cgroup/cpuset: Check remote partition underneath before making an invalid partition valid Waiman Long
                   ` (3 subsequent siblings)
  5 siblings, 1 reply; 18+ messages in thread
From: Waiman Long @ 2026-11-16  1:23 UTC (permalink / raw)
  To: Ridong Chen, Tejun Heo, Johannes Weiner, Michal Koutný,
	Shuah Khan
  Cc: cgroups, linux-kernel, linux-kselftest, Guopeng Zhang,
	Waiman Long, Sashiko

It is reported that remote_cpus_update() fails to invalidate a
populated remote partition when an ancestor cgroup update reduces its
effective_xcpus mask to only offline CPUs.

When update_cpumasks_hier() updates a descendant remote partition, it
passes the new effective_xcpus mask (excpus) to remote_cpus_update().
Because excpus still contains offline CPUs, cpumask_empty(excpus) is false
and remote_cpus_update() only checks tasks_nocpu_error() when adding CPUs.

With the partition left valid, update_cpumasks_hier() computes an empty
effective_cpus mask and jumps to get_css. The populated partition stays
valid with no effective CPUs instead of inheriting its parent's CPUs,
and its tasks remain affine to the online CPUs returned to top_cpuset.

Fix it by invalidating the remote partition if there is no active CPU
available in the given excpus parameter.

Reported-by: Sashiko <sashiko-bot@kernel.org>
Link: https://sashiko.dev/#/bug/linux-936b7155-f652-4146-a4ab-ec48fd392a1d
Signed-off-by: Waiman Long <longman@redhat.com>
---
 kernel/cgroup/cpuset.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 631fb449cef8..195d4e3a283c 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -1810,7 +1810,8 @@ static void remote_cpus_update(struct cpuset *cs, struct cpumask *xcpus,
 
 	WARN_ON_ONCE(!cpumask_subset(cs->effective_xcpus, subpartitions_cpus));
 
-	if (cpumask_empty(excpus)) {
+	/* New effective_xcpus must have at least one active CPU */
+	if (!cpumask_intersects(excpus, cpu_active_mask)) {
 		WRITE_ONCE(cs->prs_err, PERR_CPUSEMPTY);
 		goto invalidate;
 	}
-- 
2.56.0


^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [PATCH-next v4 3/6] cgroup/cpuset: Check remote partition underneath before making an invalid partition valid
  2026-11-16  1:23 [PATCH-next v4 0/6] cgroup/cpuset: More fix patches Waiman Long
  2026-11-16  1:23 ` [PATCH-next v4 1/6] cgroup/cpuset: Fix incorrect partition invalidation logic in cpuset_hotplug_update_tasks() Waiman Long
  2026-11-16  1:23 ` [PATCH-next v4 2/6] cgroup/cpuset: excpus parameter of remote_cpus_update() must have at least 1 active CPU Waiman Long
@ 2026-11-16  1:23 ` Waiman Long
  2026-11-16  3:47   ` Ridong Chen
  2026-11-16  1:23 ` [PATCH-next v4 4/6] cgroup/cpuset: Revert "remove global remote_children list" Waiman Long
                   ` (2 subsequent siblings)
  5 siblings, 1 reply; 18+ messages in thread
From: Waiman Long @ 2026-11-16  1:23 UTC (permalink / raw)
  To: Ridong Chen, Tejun Heo, Johannes Weiner, Michal Koutný,
	Shuah Khan
  Cc: cgroups, linux-kernel, linux-kselftest, Guopeng Zhang,
	Waiman Long, Sashiko

When re-evaluating an invalid local partition via partcmd_update,
update_parent_effective_cpumask() does not check whether a descendant
remote partition is using its exclusive CPUs, promoting the cpuset to a
valid partition root over the remote partition. This condition should only
be possible if parent is the top_cpuset and exclusive_cpus isn't empty.

This leaves an active remote partition underneath a valid
local partition root with overlapping exclusive CPUs, breaking
scheduler domain partitioning and triggering warnings in
compute_partition_effective_cpumask() and generate_sched_domains().

Fix that by checking the effective_xcpus against subpartitions_cpus
when an underneath remote partition is possible to make sure that all
the exclusive CPUs are available before allowing the invalid partition
to be switched back to valid.

Also make sure that effective_xcpus is non-empty before swtiching an
invalid partition back to valid.

Add a test case in test_cpuset_prs.sh to verify that an invalid partition
with a remote partition underneath cannot be switched back to valid
with a hotplug event.

In the special case when the last active CPU in the top_cpuset partition
is offlined, the hotplug code will empty subpartitions_cpus and let the
existing local partition to compete for the exclusive CPUs again. If
there is an invalid local partition above a remote partition and that
invalid local partition is able to switch to a valid partition, the remote
partition below will now share some the of exclusive CPUs violating the
CPU exclusivity rule. This special case will be handled in a later patch.

Fixes: 181c8e091aae ("cgroup/cpuset: Introduce remote partition")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Link: https://sashiko.dev/#/bug/linux-47715b89-d5fb-4731-b7c9-070773c6ff75
Signed-off-by: Waiman Long <longman@redhat.com>
---
 kernel/cgroup/cpuset.c                        | 42 ++++++++++++-------
 .../selftests/cgroup/test_cpuset_prs.sh       |  4 ++
 2 files changed, 30 insertions(+), 16 deletions(-)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 195d4e3a283c..81481fa62cfe 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -2029,11 +2029,7 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
 		 * tasks_nocpu_error() with this very mask, and a failing
 		 * write is routed to partcmd_invalidate instead.
 		 */
-		if (is_partition_invalid(cs)) {
-			adding = false;
-			deleting = cpumask_and(tmp->delmask,
-					newmask, parent->effective_xcpus);
-		} else {
+		if (is_partition_valid(cs)) {
 			cpumask_andnot(tmp->addmask, cs->effective_xcpus, newmask);
 			adding = cpumask_and(tmp->addmask, tmp->addmask,
 					     parent->effective_xcpus);
@@ -2041,6 +2037,17 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
 			cpumask_andnot(tmp->delmask, newmask, cs->effective_xcpus);
 			deleting = cpumask_and(tmp->delmask, tmp->delmask,
 					       parent->effective_xcpus);
+		} else {
+			deleting = cpumask_and(tmp->delmask,
+					newmask, parent->effective_xcpus);
+
+			/* Check if a remote partition may be underneath */
+			if ((parent == &top_cpuset) &&
+			    !cpumask_empty(cs->exclusive_cpus) &&
+			    cpumask_intersects(tmp->delmask, subpartitions_cpus)) {
+				part_error = PERR_REMOTE;
+				deleting = false;
+			}
 		}
 
 		/*
@@ -2107,34 +2114,37 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
 		} else if (is_partition_invalid(cs)) {
 			struct cgroup_subsys_state *css;
 			struct cpuset *child;
-			bool exclusive = true;
 
 			/*
 			 * Convert invalid partition to valid has to pass the
-			 * cpu exclusivity test when cpuset.cpus.exclusive is
-			 * empty as well as the housekeeping check.
+			 * following tests:
+			 * 1) housekeeping CPU conflict check
+			 * 2) cpu exclusivity test when cpuset.cpus.exclusive is
+			 *    empty.
+			 * 3) if parent is top_cpuset, none of the effective_xcpus
+			 *    (tmp->delmask) are in subpartitions_cpus, if not a
+			 *    remote partition may be underneath.
 			 */
 			if (prstate_housekeeping_conflict(cs->partition_root_state,
 							  parent_prs, tmp->delmask, NULL)) {
 				part_error = PERR_HKEEPING;
-				goto write_error;
-			}
-			if (cpumask_empty(cs->exclusive_cpus)) {
+			} else if (cpumask_empty(cs->exclusive_cpus)) {
 				rcu_read_lock();
 				cpuset_for_each_child(child, css, parent) {
 					if (child == cs)
 						continue;
 					if (!cpusets_are_exclusive(cs, child)) {
-						exclusive = false;
+						part_error = PERR_NOTEXCL;
 						break;
 					}
 				}
 				rcu_read_unlock();
+			} else if ((parent == &top_cpuset) &&
+				   cpumask_intersects(tmp->delmask, subpartitions_cpus)) {
+				part_error = PERR_REMOTE;
 			}
-			if (exclusive)
+			if (!part_error)
 				deleting = !cpumask_empty(tmp->delmask);
-			else
-				part_error = PERR_NOTEXCL;
 		}
 	}
 
@@ -2155,7 +2165,7 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
 			break;
 		case PRS_INVALID_ROOT:
 		case PRS_INVALID_ISOLATED:
-			if (!part_error)
+			if (!part_error && deleting)
 				new_prs = -old_prs;
 			break;
 		}
diff --git a/tools/testing/selftests/cgroup/test_cpuset_prs.sh b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
index 1397ec428c46..81f0eecaca80 100755
--- a/tools/testing/selftests/cgroup/test_cpuset_prs.sh
+++ b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
@@ -563,6 +563,10 @@ TEST_MATRIX=(
 	" C1-6:P1 C3-6:P2 C5-6:P2  .       .    P1      .      .     0 A1:1-2|A2:3-4|A3:5-6 \
 								       A1:P1|A2:P1|A3:P2 5-6"
 
+	# A hotplug event will not incorrectly activate an invalid partition with
+	# a remote partition underneath.
+	"  CX1-4  C3-4:P1   .      .      P1   O1=0     .      .     0 A1:2|A2:3-4 A1:P-1|A2:P1"
+
 	#
 	#  old-A1 old-A2 old-A3 old-B1 new-A1 new-A2 new-A3 new-B1 fail ECPUs Pstate ISOLCPUS
 	#  ------ ------ ------ ------ ------ ------ ------ ------ ---- ----- ------ --------
-- 
2.56.0


^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [PATCH-next v4 4/6] cgroup/cpuset: Revert "remove global remote_children list"
  2026-11-16  1:23 [PATCH-next v4 0/6] cgroup/cpuset: More fix patches Waiman Long
                   ` (2 preceding siblings ...)
  2026-11-16  1:23 ` [PATCH-next v4 3/6] cgroup/cpuset: Check remote partition underneath before making an invalid partition valid Waiman Long
@ 2026-11-16  1:23 ` Waiman Long
  2026-11-16  1:23 ` [PATCH-next v4 5/6] cgroup/cpuset: Handle remote partitions before local partitions in cpuset_handle_hotplug() Waiman Long
  2026-11-16  1:23 ` [PATCH-next v4 6/6] cgroup/cpuset: Fix lockdep_is_cpuset_held() Waiman Long
  5 siblings, 0 replies; 18+ messages in thread
From: Waiman Long @ 2026-11-16  1:23 UTC (permalink / raw)
  To: Ridong Chen, Tejun Heo, Johannes Weiner, Michal Koutný,
	Shuah Khan
  Cc: cgroups, linux-kernel, linux-kselftest, Guopeng Zhang,
	Waiman Long

Commit 648d43da64f0 ("cpuset: remove global remote_children list") is being
reverted as there is now a need to iterate the list of remote partition
roots available.

Signed-off-by: Waiman Long <longman@redhat.com>
---
 kernel/cgroup/cpuset-internal.h | 10 +++-------
 kernel/cgroup/cpuset.c          | 14 +++++++++++---
 2 files changed, 14 insertions(+), 10 deletions(-)

diff --git a/kernel/cgroup/cpuset-internal.h b/kernel/cgroup/cpuset-internal.h
index 1ba94ffaff9d..f3fed906e7ee 100644
--- a/kernel/cgroup/cpuset-internal.h
+++ b/kernel/cgroup/cpuset-internal.h
@@ -154,13 +154,6 @@ struct cpuset {
 	/* partition root state */
 	int partition_root_state;
 
-	/*
-	 * Whether cpuset is a remote partition.
-	 * It used to be a list anchoring all remote partitions — we can switch back
-	 * to a list if we need to iterate over the remote partitions.
-	 */
-	bool remote_partition;
-
 	/*
 	 * Set to true when a remote partition is undergoing update.
 	 */
@@ -186,6 +179,9 @@ struct cpuset {
 	/* Handle for cpuset.cpus.partition */
 	struct cgroup_file partition_file;
 
+	/* Remote partition silbling list anchored at remote_children */
+	struct list_head remote_sibling;
+
 #ifdef CONFIG_CPUSETS_V1
 	struct fmeter fmeter;		/* memory_pressure filter */
 
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 81481fa62cfe..0fe9f2731fee 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -148,6 +148,11 @@ static cpumask_var_t	subpartitions_cpus;	/* RWCS */
  */
 static cpumask_var_t	isolated_cpus;		/* CSCB */
 
+/*
+ * List of remote partition root children
+ */
+static struct list_head remote_children;	/* RWCS */
+
 /*
  * Set if housekeeping cpumasks are to be updated.
  */
@@ -286,6 +291,7 @@ struct cpuset top_cpuset = {
 	.flags = BIT(CS_CPU_EXCLUSIVE) |
 		 BIT(CS_MEM_EXCLUSIVE) | BIT(CS_SCHED_LOAD_BALANCE),
 	.partition_root_state = PRS_ROOT,
+	.remote_sibling = LIST_HEAD_INIT(top_cpuset.remote_sibling),
 	.dl_bw_cpu = -1,
 };
 
@@ -1213,7 +1219,7 @@ static void reset_partition_data(struct cpuset *cs)
 
 static inline bool is_remote_partition(struct cpuset *cs)
 {
-	return cs->remote_partition;
+	return !list_empty(&cs->remote_sibling);
 }
 
 static inline bool is_local_partition(struct cpuset *cs)
@@ -1742,7 +1748,7 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs,
 
 	spin_lock_irq(&callback_lock);
 	partition_xcpus_add(new_prs, NULL, tmp->new_cpus);
-	cs->remote_partition = true;
+	list_add(&cs->remote_sibling, &remote_children);
 	cpumask_copy(cs->effective_xcpus, tmp->new_cpus);
 	spin_unlock_irq(&callback_lock);
 	cpuset_force_rebuild();
@@ -1769,7 +1775,7 @@ static void remote_partition_disable(struct cpuset *cs, struct tmpmasks *tmp)
 {
 	WARN_ON_ONCE(!is_remote_partition(cs));
 	spin_lock_irq(&callback_lock);
-	cs->remote_partition = false;
+	list_del_init(&cs->remote_sibling);
 	partition_xcpus_del(cs->partition_root_state, NULL, cs->effective_xcpus);
 	if (cs->prs_err)
 		cs->partition_root_state = -cs->partition_root_state;
@@ -3677,6 +3683,7 @@ cpuset_css_alloc(struct cgroup_subsys_state *parent_css)
 		return ERR_PTR(-ENOMEM);
 
 	__set_bit(CS_SCHED_LOAD_BALANCE, &cs->flags);
+	INIT_LIST_HEAD(&cs->remote_sibling);
 	cpuset1_init(cs);
 
 	/* Set CS_MEMORY_MIGRATE for default hierarchy */
@@ -3920,6 +3927,7 @@ int __init cpuset_init(void)
 	cpumask_setall(top_cpuset.exclusive_cpus);
 	nodes_setall(top_cpuset.effective_mems);
 	init_llist_node(&top_cpuset.attach_node);
+	INIT_LIST_HEAD(&remote_children);
 
 	cpuset1_init(&top_cpuset);
 
-- 
2.56.0


^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [PATCH-next v4 5/6] cgroup/cpuset: Handle remote partitions before local partitions in cpuset_handle_hotplug()
  2026-11-16  1:23 [PATCH-next v4 0/6] cgroup/cpuset: More fix patches Waiman Long
                   ` (3 preceding siblings ...)
  2026-11-16  1:23 ` [PATCH-next v4 4/6] cgroup/cpuset: Revert "remove global remote_children list" Waiman Long
@ 2026-11-16  1:23 ` Waiman Long
  2026-11-16  7:08   ` Ridong Chen
                     ` (2 more replies)
  2026-11-16  1:23 ` [PATCH-next v4 6/6] cgroup/cpuset: Fix lockdep_is_cpuset_held() Waiman Long
  5 siblings, 3 replies; 18+ messages in thread
From: Waiman Long @ 2026-11-16  1:23 UTC (permalink / raw)
  To: Ridong Chen, Tejun Heo, Johannes Weiner, Michal Koutný,
	Shuah Khan
  Cc: cgroups, linux-kernel, linux-kselftest, Guopeng Zhang,
	Waiman Long, Sashiko

When there is a valid remote partition underneath an invalid local
partition with subpartitions_cpus cleared because of missing active
CPUs in the top_cpuset partition, the current iteration logic in
cpuset_handle_hotplug() will handle the invalid local partition first
and potentially make the local partition valid. The remote partition
will be handled later with the potential that both the local and remote
partitions will be valid with overlapping exclusive CPUs violating the
cpu exclusivity rule.

Fix this problem by adding a new remote_partition_scan() helper to
iterate the existing remote partitions first before iterating down
the hierarchy to handle the local partitions. That also means that
when subpartitions_cpus is cleared, remote partitions have precedence
to acquire the exclusive CPUs first before local partitions. It also
gives remote partition the ability to recover from the clearing of
subpartitions_cpus as all the remote partitions will be disabled before
this commit.

A remote partition can only be created if there is at least one active
CPU in its effective_xcpus. So remote_partition_scan() will also
invalidate the remote partition if there is no active CPU left in its
effective_xcpus. This requires a minor change in a test_cpuset_prs.sh
test case where a remote partition with no active CPU left will now
be invalidated.

With the addition of the remote_partition_scan() call in
cpuset_handle_hotplug(), there is no longer a need to do a remote
partition check in cpuset_hotplug_update_tasks() and the corresponding
code can be removed.

While at it, the subpartition_cpus clearing logic is enhanced to
clear it when no domain housekeeping CPUs are left in the top_cpuset
partition. When the remote partition is disabled, the effective_cpus
is modified in a such a way that cpus_updated will be set true later
in cpuset_hotplug_update_tasks() so that the CPU affinity of the
tasks within the partition will be updated. This will also fix a
bug reported by sashiko that CPU affinity will not be updated when
remote_partition_disable() is called in such circumstance.

When efffective_cpus is read during this interim period, the content
will be invalid before and after this commit.

Fixes: 181c8e091aae ("cgroup/cpuset: Introduce remote partition")
Reported-by: Guopeng Zhang <guopeng.zhang@linux.dev>
Link: https://lore.kernel.org/lkml/fe7aa630-d323-4981-9f89-0958cb6204a2@linux.dev
Reported-by: Sashiko <sashiko-bot@kernel.org>
Link: https://sashiko.dev/#/bug/linux-ab90f7e8-0d60-40ca-83e9-e405075b24d2
Signed-off-by: Waiman Long <longman@redhat.com>
---
 kernel/cgroup/cpuset.c                        | 97 ++++++++++++++++---
 .../selftests/cgroup/test_cpuset_prs.sh       |  2 +-
 2 files changed, 84 insertions(+), 15 deletions(-)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 0fe9f2731fee..64cbcbdbf6da 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -153,6 +153,12 @@ static cpumask_var_t	isolated_cpus;		/* CSCB */
  */
 static struct list_head remote_children;	/* RWCS */
 
+/*
+ * Temporary cpumask for local cpumask computation
+ * The cpumask may be changed when a cpuset helper is being called.
+ */
+static cpumask_var_t	tmp_cpus;		/* RWCS */
+
 /*
  * Set if housekeeping cpumasks are to be updated.
  */
@@ -1873,6 +1879,67 @@ static void remote_cpus_update(struct cpuset *cs, struct cpumask *xcpus,
 	remote_partition_disable(cs, tmp);
 }
 
+/**
+ * remote_partition_scan: scan remote partitions to check exclusive CPUs
+ * @tmpmask: ptr to tmpmasks structure
+ *
+ * This function is called in cpuset_handle_hotplug() after the update on
+ * top_cpuset.effective_cpus when there is cpu online/offline operation.
+ * As subpartitions_cpus may have been cleared, we need to check if the remote
+ * partition should acquire the exclusive CPUs again without impacting
+ * effective_cpus or it needs to be disabled when one or more of the following
+ * conditions is true.
+ * 1) All the domain housekeeping CPUs in the top_cpuset will be taken away
+ * 2) A populated partition with no active CPU or no active CPU in
+ *    effective_xcpus
+ */
+static void remote_partition_scan(struct tmpmasks *tmp)
+{
+	struct cpuset *cs, *next;
+
+	rcu_read_lock();
+	list_for_each_entry_safe(cs, next, &remote_children, remote_sibling) {
+		bool noxcpus = !cpumask_subset(cs->effective_xcpus, subpartitions_cpus);
+		bool nocpus = !cpumask_intersects(cs->effective_xcpus, cpu_active_mask) ||
+			      (!cpumask_intersects(cs->effective_cpus, cpu_active_mask) &&
+			       partition_is_populated(cs, NULL));
+		int cpu;
+
+		cpumask_and(tmp_cpus, top_cpuset.effective_cpus,
+			    housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT));
+		if (noxcpus && !nocpus && !cpumask_subset(tmp_cpus, cs->effective_xcpus)) {
+			/*
+			 * Reacquire exclusive CPUs & update top_cpuset.effective_cpus
+			 */
+			spin_lock_irq(&callback_lock);
+			cpumask_andnot(subpartitions_cpus, subpartitions_cpus,
+				       cs->effective_xcpus);
+			cpumask_andnot(top_cpuset.effective_cpus, top_cpuset.effective_xcpus,
+				       cs->effective_xcpus);
+			spin_unlock_irq(&callback_lock);
+			continue;
+		}
+
+		if (noxcpus || nocpus || cpumask_empty(tmp_cpus)) {
+			/*
+			 * Disable remote partition & modifiy effective_cpus
+			 * so that cpus_updated will be set true in
+			 * cpuset_hotplug_update_tasks().
+			 */
+			WRITE_ONCE(cs->prs_err, PERR_HOTPLUG);
+			remote_partition_disable(cs, tmp);
+			cpu = cpumask_first(cs->effective_cpus);
+			spin_lock_irq(&callback_lock);
+			if (cpu < nr_cpu_ids)
+				cpumask_clear_cpu(cpu, cs->effective_cpus);
+			else
+				cpumask_set_cpu(0, cs->effective_cpus);
+			spin_unlock_irq(&callback_lock);
+		}
+	}
+	rcu_read_unlock();
+}
+
 /**
  * update_parent_effective_cpumask - update effective_cpus mask of parent cpuset
  * @cs:      The cpuset that requests change in partition root state
@@ -3919,6 +3986,7 @@ int __init cpuset_init(void)
 	BUG_ON(!zalloc_cpumask_var(&subpartitions_cpus, GFP_KERNEL));
 	BUG_ON(!zalloc_cpumask_var(&isolated_cpus, GFP_KERNEL));
 	BUG_ON(!zalloc_cpumask_var(&isolated_hk_cpus, GFP_KERNEL));
+	BUG_ON(!alloc_cpumask_var(&tmp_cpus, GFP_KERNEL));
 
 	cpumask_setall(top_cpuset.cpus_allowed);
 	nodes_setall(top_cpuset.mems_allowed);
@@ -3993,14 +4061,6 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp)
 	if (!tmp || !cs->partition_root_state)
 		goto update_tasks;
 
-	if (is_remote_partition(cs) &&
-	    (!cpumask_subset(cs->effective_xcpus, subpartitions_cpus) ||
-	     (cpumask_empty(&new_cpus) && partition_is_populated(cs, NULL)))) {
-		WRITE_ONCE(cs->prs_err, PERR_HOTPLUG);
-		remote_partition_disable(cs, tmp);
-		compute_effective_cpumask(&new_cpus, cs, parent);
-	}
-
 	/*
 	 * Force the partition to become invalid if either one of
 	 * the following conditions hold:
@@ -4104,22 +4164,31 @@ static void cpuset_handle_hotplug(void)
 		if (!on_dfl)
 			cpumask_copy(top_cpuset.cpus_allowed, &new_cpus);
 		/*
-		 * Make sure that CPUs allocated to child partitions
-		 * do not show up in effective_cpus. If no CPU is left,
-		 * we clear the subpartitions_cpus & let the child partitions
-		 * fight for the CPUs again.
+		 * Make sure that CPUs allocated to child partitions do not
+		 * show up in effective_cpus. If no housekeeping CPU is left,
+		 * we clear the subpartitions_cpus, scan the remote partitions
+		 * to determine if they can remain valid & let the descendant
+		 * local partitions fight for the CPUs again.
 		 */
 		if (!cpumask_empty(subpartitions_cpus)) {
+			cpumask_andnot(&new_cpus, cpu_active_mask,
+				       housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT));
 			if (cpumask_subset(&new_cpus, subpartitions_cpus)) {
 				cpumask_clear(subpartitions_cpus);
+				cpumask_copy(&new_cpus, cpu_active_mask);
 			} else {
-				cpumask_andnot(&new_cpus, &new_cpus,
+				cpumask_andnot(&new_cpus, cpu_active_mask,
 					       subpartitions_cpus);
 			}
 		}
 		cpumask_copy(top_cpuset.effective_cpus, &new_cpus);
 		spin_unlock_irq(&callback_lock);
-		/* we don't mess with cpumasks of tasks in top_cpuset */
+		/*
+		 * We don't mess with cpumasks of tasks in top_cpuset, but we
+		 * have to examine impact on remote partition roots if present.
+		 */
+		if (cpuset_v2() && ptmp)
+			remote_partition_scan(ptmp);
 	}
 
 	/* synchronize mems_allowed to N_MEMORY */
diff --git a/tools/testing/selftests/cgroup/test_cpuset_prs.sh b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
index 81f0eecaca80..5c7b583cba18 100755
--- a/tools/testing/selftests/cgroup/test_cpuset_prs.sh
+++ b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
@@ -378,7 +378,7 @@ TEST_MATRIX=(
 	# Remote partition offline tests
 	"   C0-3    C1-3  C2-3     .    X2-3   X2-3 X2-3:P2:O2=0 .   0 A1:0-1|A2:1|A3:3 A1:P0|A3:P2 2-3"
 	"   C0-3    C1-3  C2-3     .    X2-3   X2-3 X2-3:P2:O2=0 O2=1 0 A1:0-1|A2:1|A3:2-3 A1:P0|A3:P2 2-3"
-	"   C0-3    C1-3   C3      .    X2-3   X2-3    P2:O3=0   .   0 A1:0-2|A2:1-2|A3: A1:P0|A3:P2 3"
+	"   C0-3    C1-3   C3      .    X2-3   X2-3    P2:O3=0   .   0 A1:0-2|A2:1-2|A3:1-2 A1:P0|A3:P-2 3|"
 	"   C0-3    C1-3   C3      .    X2-3   X2-3   T:P2:O3=0  .   0 A1:0-2|A2:1-2|A3:1-2 A1:P0|A3:P-2 3|"
 
 	# An invalidated remote partition cannot self-recover from hotplug
-- 
2.56.0


^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [PATCH-next v4 6/6] cgroup/cpuset: Fix lockdep_is_cpuset_held()
  2026-11-16  1:23 [PATCH-next v4 0/6] cgroup/cpuset: More fix patches Waiman Long
                   ` (4 preceding siblings ...)
  2026-11-16  1:23 ` [PATCH-next v4 5/6] cgroup/cpuset: Handle remote partitions before local partitions in cpuset_handle_hotplug() Waiman Long
@ 2026-11-16  1:23 ` Waiman Long
  2026-11-16  7:24   ` Ridong Chen
  2026-11-16 10:34   ` Guopeng Zhang
  5 siblings, 2 replies; 18+ messages in thread
From: Waiman Long @ 2026-11-16  1:23 UTC (permalink / raw)
  To: Ridong Chen, Tejun Heo, Johannes Weiner, Michal Koutný,
	Shuah Khan
  Cc: cgroups, linux-kernel, linux-kselftest, Guopeng Zhang,
	Waiman Long

Commit a84097e625f2 ("cgroup/cpuset: Call housekeeping_update()
without holding cpus_read_lock") changes lockdep_is_cpuset_held()
to return true if either cpuset_mutex or cpuset_top_mutex is held.
lockdep_is_cpuset_held() is used by housekeeping_dereference_check()
as one of the conditions that rcu_read_lock() is not needed to
access HK_TYPE_DOMAIN cpumask that can be changed at run time.
In reality, it should return true only if cpuset_top_mutex is held as
housekeeping_update() is called with only cpuset_top_mutex held.

If only cpuset_mutex is held, it is possible that there can be a parallel
housekeeping_update() call leading to a potential use-after-free situation
when HK_TYPE_DOMAIN housekeeping cpumask is being accessed. It also
means that rcu_read_lock will be required if only cpuset_mutex is being
held like when calling from hotplug, scheduler or cgroup.

Fixes: a84097e625f2 ("cgroup/cpuset: Call housekeeping_update() without holding cpus_read_lock")
Signed-off-by: Waiman Long <longman@redhat.com>
---
 kernel/cgroup/cpuset-v1.c | 16 ++++++++++------
 kernel/cgroup/cpuset.c    |  9 +++++++--
 2 files changed, 17 insertions(+), 8 deletions(-)

diff --git a/kernel/cgroup/cpuset-v1.c b/kernel/cgroup/cpuset-v1.c
index 7c7f2887dfaf..6bf6da639fec 100644
--- a/kernel/cgroup/cpuset-v1.c
+++ b/kernel/cgroup/cpuset-v1.c
@@ -686,6 +686,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,
 	int csn;		/* how many cpuset ptrs in csa so far */
 	int i, j;		/* indices for partition finding loops */
 	cpumask_var_t *doms;	/* resulting partition; i.e. sched domains */
+	cpumask_var_t hk_cpus;	/* HK_TYPE_DOMAIN housekeeping CPUs */
 	struct sched_domain_attr *dattr;  /* attributes for custom domains */
 	int ndoms = 0;		/* number of sched domains in result */
 	int nslot;		/* next empty doms[] struct cpumask slot */
@@ -698,6 +699,11 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,
 	dattr = NULL;
 	csa = NULL;
 
+	alloc_cpumask_var(&hk_cpus, GFP_KERNEL);
+	scoped_guard(rcu) {
+		cpumask_copy(hk_cpus, housekeeping_cpumask(HK_TYPE_DOMAIN));
+	}
+
 	/* Special case for the 99% of systems with one, full, sched domain */
 	if (is_sched_load_balance(&top_cpuset)) {
 		ndoms = 1;
@@ -710,9 +716,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,
 			*dattr = SD_ATTR_INIT;
 			update_domain_attr_tree(dattr, &top_cpuset);
 		}
-		cpumask_and(doms[0], top_cpuset.effective_cpus,
-			    housekeeping_cpumask(HK_TYPE_DOMAIN));
-
+		cpumask_and(doms[0], top_cpuset.effective_cpus, hk_cpus);
 		goto done;
 	}
 
@@ -736,8 +740,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,
 		 */
 		if (!cpumask_empty(cp->cpus_allowed) &&
 		    !(is_sched_load_balance(cp) &&
-		      cpumask_intersects(cp->cpus_allowed,
-					 housekeeping_cpumask(HK_TYPE_DOMAIN))))
+		      cpumask_intersects(cp->cpus_allowed, hk_cpus)))
 			continue;
 
 		if (is_sched_load_balance(cp) &&
@@ -794,7 +797,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,
 						*(dattr + nslot) = SD_ATTR_INIT;
 				}
 				cpumask_or(dp, dp, csa[j]->effective_cpus);
-				cpumask_and(dp, dp, housekeeping_cpumask(HK_TYPE_DOMAIN));
+				cpumask_and(dp, dp, hk_cpus);
 				if (dattr)
 					update_domain_attr_tree(dattr + nslot, csa[j]);
 			}
@@ -806,6 +809,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,
 
 done:
 	kfree(csa);
+	free_cpumask_var(hk_cpus);
 
 	/*
 	 * Fallback to the default domain if kmalloc() failed.
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 64cbcbdbf6da..d18439e13bce 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -343,10 +343,13 @@ void cpuset_full_unlock(void)
 }
 
 #ifdef CONFIG_LOCKDEP
+/*
+ * Holding cpuset_top_mutex will guarantee that a parallel
+ * housekeeping_update() call modifying housekeeping cpumasks is not possible.
+ */
 bool lockdep_is_cpuset_held(void)
 {
-	return lockdep_is_held(&cpuset_mutex) ||
-	       lockdep_is_held(&cpuset_top_mutex);
+	return lockdep_is_held(&cpuset_top_mutex);
 }
 #endif
 
@@ -1416,8 +1419,10 @@ static bool prstate_housekeeping_conflict(int new_prs, int parent_prs,
 		}
 	}
 
+	rcu_read_lock();
 	cpumask_and(full_hk_cpus, housekeeping_cpumask(HK_TYPE_KERNEL_NOISE),
 		    housekeeping_cpumask(HK_TYPE_DOMAIN));
+	rcu_read_unlock();
 	cpumask_andnot(full_hk_cpus, full_hk_cpus, isolated_cpus);
 	cpumask_and(full_hk_cpus, full_hk_cpus, cpu_active_mask);
 	res = !cpumask_weight_andnot(full_hk_cpus, new_cpus);
-- 
2.56.0


^ permalink raw reply related	[flat|nested] 18+ messages in thread

* Re: [PATCH-next v4 2/6] cgroup/cpuset: excpus parameter of remote_cpus_update() must have at least 1 active CPU
  2026-11-16  1:23 ` [PATCH-next v4 2/6] cgroup/cpuset: excpus parameter of remote_cpus_update() must have at least 1 active CPU Waiman Long
@ 2026-11-16  3:07   ` Ridong Chen
  2026-11-16  3:34     ` Waiman Long
  0 siblings, 1 reply; 18+ messages in thread
From: Ridong Chen @ 2026-11-16  3:07 UTC (permalink / raw)
  To: Waiman Long, Tejun Heo, Johannes Weiner, Michal Koutný,
	Shuah Khan
  Cc: cgroups, linux-kernel, linux-kselftest, Guopeng Zhang, Sashiko



On 11/16/2026 9:23 AM, Waiman Long wrote:
> It is reported that remote_cpus_update() fails to invalidate a
> populated remote partition when an ancestor cgroup update reduces its
> effective_xcpus mask to only offline CPUs.
> 
> When update_cpumasks_hier() updates a descendant remote partition, it
> passes the new effective_xcpus mask (excpus) to remote_cpus_update().
> Because excpus still contains offline CPUs, cpumask_empty(excpus) is false
> and remote_cpus_update() only checks tasks_nocpu_error() when adding CPUs.
> 
> With the partition left valid, update_cpumasks_hier() computes an empty
> effective_cpus mask and jumps to get_css. The populated partition stays
> valid with no effective CPUs instead of inheriting its parent's CPUs,
> and its tasks remain affine to the online CPUs returned to top_cpuset.
> 
> Fix it by invalidating the remote partition if there is no active CPU
> available in the given excpus parameter.
> 
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Link: https://sashiko.dev/#/bug/linux-936b7155-f652-4146-a4ab-ec48fd392a1d
> Signed-off-by: Waiman Long <longman@redhat.com>
> ---
>   kernel/cgroup/cpuset.c | 3 ++-
>   1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
> index 631fb449cef8..195d4e3a283c 100644
> --- a/kernel/cgroup/cpuset.c
> +++ b/kernel/cgroup/cpuset.c
> @@ -1810,7 +1810,8 @@ static void remote_cpus_update(struct cpuset *cs, struct cpumask *xcpus,
>   
>   	WARN_ON_ONCE(!cpumask_subset(cs->effective_xcpus, subpartitions_cpus));
>   
> -	if (cpumask_empty(excpus)) {
> +	/* New effective_xcpus must have at least one active CPU */
> +	if (!cpumask_intersects(excpus, cpu_active_mask)) {
>   		WRITE_ONCE(cs->prs_err, PERR_CPUSEMPTY);
>   		goto invalidate;
>   	}

This also invalidates a remote partition with no task whose excpus
only has offline CPUs. According to cgroup-v2.rst, such a partition
is valid: cpuset.cpus.exclusive.effective may contain offline CPUs,
and cpuset.cpus.effective can be empty if there is no task in the
partition. The hotplug path and tasks_nocpu_error() follow the same
rule. How about only invalidating it when it is populated?

	  if (cpumask_empty(excpus) ||
		  (!cpumask_intersects(excpus, cpu_active_mask) &&
		   partition_is_populated(cs, NULL))) {
		
Also, it probably needs a Fixes tag, maybe:

Fixes: f62a5d39368e ("cgroup/cpuset: Remove remote_partition_check() & make 
update_cpumasks_hier() handle remote partition")
-- 
Best regards
Ridong


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH-next v4 2/6] cgroup/cpuset: excpus parameter of remote_cpus_update() must have at least 1 active CPU
  2026-11-16  3:07   ` Ridong Chen
@ 2026-11-16  3:34     ` Waiman Long
  2026-11-16  7:37       ` Ridong Chen
  0 siblings, 1 reply; 18+ messages in thread
From: Waiman Long @ 2026-11-16  3:34 UTC (permalink / raw)
  To: Ridong Chen, Tejun Heo, Johannes Weiner, Michal Koutný,
	Shuah Khan
  Cc: cgroups, linux-kernel, linux-kselftest, Guopeng Zhang, Sashiko

On 11/15/26 10:07 PM, Ridong Chen wrote:
>
>
> On 11/16/2026 9:23 AM, Waiman Long wrote:
>> It is reported that remote_cpus_update() fails to invalidate a
>> populated remote partition when an ancestor cgroup update reduces its
>> effective_xcpus mask to only offline CPUs.
>>
>> When update_cpumasks_hier() updates a descendant remote partition, it
>> passes the new effective_xcpus mask (excpus) to remote_cpus_update().
>> Because excpus still contains offline CPUs, cpumask_empty(excpus) is 
>> false
>> and remote_cpus_update() only checks tasks_nocpu_error() when adding 
>> CPUs.
>>
>> With the partition left valid, update_cpumasks_hier() computes an empty
>> effective_cpus mask and jumps to get_css. The populated partition stays
>> valid with no effective CPUs instead of inheriting its parent's CPUs,
>> and its tasks remain affine to the online CPUs returned to top_cpuset.
>>
>> Fix it by invalidating the remote partition if there is no active CPU
>> available in the given excpus parameter.
>>
>> Reported-by: Sashiko <sashiko-bot@kernel.org>
>> Link: 
>> https://sashiko.dev/#/bug/linux-936b7155-f652-4146-a4ab-ec48fd392a1d
>> Signed-off-by: Waiman Long <longman@redhat.com>
>> ---
>>   kernel/cgroup/cpuset.c | 3 ++-
>>   1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
>> index 631fb449cef8..195d4e3a283c 100644
>> --- a/kernel/cgroup/cpuset.c
>> +++ b/kernel/cgroup/cpuset.c
>> @@ -1810,7 +1810,8 @@ static void remote_cpus_update(struct cpuset 
>> *cs, struct cpumask *xcpus,
>>         WARN_ON_ONCE(!cpumask_subset(cs->effective_xcpus, 
>> subpartitions_cpus));
>>   -    if (cpumask_empty(excpus)) {
>> +    /* New effective_xcpus must have at least one active CPU */
>> +    if (!cpumask_intersects(excpus, cpu_active_mask)) {
>>           WRITE_ONCE(cs->prs_err, PERR_CPUSEMPTY);
>>           goto invalidate;
>>       }
>
> This also invalidates a remote partition with no task whose excpus
> only has offline CPUs. According to cgroup-v2.rst, such a partition
> is valid: cpuset.cpus.exclusive.effective may contain offline CPUs,
> and cpuset.cpus.effective can be empty if there is no task in the
> partition. The hotplug path and tasks_nocpu_error() follow the same
> rule. How about only invalidating it when it is populated?
>
>       if (cpumask_empty(excpus) ||
>           (!cpumask_intersects(excpus, cpu_active_mask) &&
>            partition_is_populated(cs, NULL))) {

A partition can only be created if there are active CPUs in the 
effective_xcpus. However, such a partition root can distribute its 
exclusive CPUs down to child local partitions with the results that 
effective_cpus can be empty if there is no task there. IOW, 
effective_xcpus must have active CPUs while effective_cpus does not need 
to have active CPUs if there are child partitions underneath. I am going 
to clarify that in the doc. I will check the hotplug path and 
task_nocpu_error() to make sure that it follows this rule. Does that 
rule make sense to you?


>
> Also, it probably needs a Fixes tag, maybe:
>
> Fixes: f62a5d39368e ("cgroup/cpuset: Remove remote_partition_check() & 
> make update_cpumasks_hier() handle remote partition")

Yes, I miss the fix tag. Will update that in the next version.

Cheers,
Longman


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH-next v4 3/6] cgroup/cpuset: Check remote partition underneath before making an invalid partition valid
  2026-11-16  1:23 ` [PATCH-next v4 3/6] cgroup/cpuset: Check remote partition underneath before making an invalid partition valid Waiman Long
@ 2026-11-16  3:47   ` Ridong Chen
  2026-11-16  4:21     ` Waiman Long
  0 siblings, 1 reply; 18+ messages in thread
From: Ridong Chen @ 2026-11-16  3:47 UTC (permalink / raw)
  To: Waiman Long, Tejun Heo, Johannes Weiner, Michal Koutný,
	Shuah Khan
  Cc: cgroups, linux-kernel, linux-kselftest, Guopeng Zhang, Sashiko



On 11/16/2026 9:23 AM, Waiman Long wrote:
> When re-evaluating an invalid local partition via partcmd_update,
> update_parent_effective_cpumask() does not check whether a descendant
> remote partition is using its exclusive CPUs, promoting the cpuset to a
> valid partition root over the remote partition. This condition should only
> be possible if parent is the top_cpuset and exclusive_cpus isn't empty.
> 
> This leaves an active remote partition underneath a valid
> local partition root with overlapping exclusive CPUs, breaking
> scheduler domain partitioning and triggering warnings in
> compute_partition_effective_cpumask() and generate_sched_domains().
> 
> Fix that by checking the effective_xcpus against subpartitions_cpus
> when an underneath remote partition is possible to make sure that all
> the exclusive CPUs are available before allowing the invalid partition
> to be switched back to valid.
> 
> Also make sure that effective_xcpus is non-empty before swtiching an
> invalid partition back to valid.
> 
> Add a test case in test_cpuset_prs.sh to verify that an invalid partition
> with a remote partition underneath cannot be switched back to valid
> with a hotplug event.
> 
> In the special case when the last active CPU in the top_cpuset partition
> is offlined, the hotplug code will empty subpartitions_cpus and let the
> existing local partition to compete for the exclusive CPUs again. If
> there is an invalid local partition above a remote partition and that
> invalid local partition is able to switch to a valid partition, the remote
> partition below will now share some the of exclusive CPUs violating the
> CPU exclusivity rule. This special case will be handled in a later patch.
> 
> Fixes: 181c8e091aae ("cgroup/cpuset: Introduce remote partition")
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Link: https://sashiko.dev/#/bug/linux-47715b89-d5fb-4731-b7c9-070773c6ff75
> Signed-off-by: Waiman Long <longman@redhat.com>
> ---
>   kernel/cgroup/cpuset.c                        | 42 ++++++++++++-------
>   .../selftests/cgroup/test_cpuset_prs.sh       |  4 ++
>   2 files changed, 30 insertions(+), 16 deletions(-)
> 
> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
> index 195d4e3a283c..81481fa62cfe 100644
> --- a/kernel/cgroup/cpuset.c
> +++ b/kernel/cgroup/cpuset.c
> @@ -2029,11 +2029,7 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
>   		 * tasks_nocpu_error() with this very mask, and a failing
>   		 * write is routed to partcmd_invalidate instead.
>   		 */
> -		if (is_partition_invalid(cs)) {
> -			adding = false;
> -			deleting = cpumask_and(tmp->delmask,
> -					newmask, parent->effective_xcpus);
> -		} else {
> +		if (is_partition_valid(cs)) {
>   			cpumask_andnot(tmp->addmask, cs->effective_xcpus, newmask);
>   			adding = cpumask_and(tmp->addmask, tmp->addmask,
>   					     parent->effective_xcpus);
> @@ -2041,6 +2037,17 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
>   			cpumask_andnot(tmp->delmask, newmask, cs->effective_xcpus);
>   			deleting = cpumask_and(tmp->delmask, tmp->delmask,
>   					       parent->effective_xcpus);
> +		} else {
> +			deleting = cpumask_and(tmp->delmask,
> +					newmask, parent->effective_xcpus);
> +
> +			/* Check if a remote partition may be underneath */
> +			if ((parent == &top_cpuset) &&
> +			    !cpumask_empty(cs->exclusive_cpus) &&
> +			    cpumask_intersects(tmp->delmask, subpartitions_cpus)) {
> +				part_error = PERR_REMOTE;
> +				deleting = false;
> +			}
>   		}
>   
>   		/*
> @@ -2107,34 +2114,37 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
>   		} else if (is_partition_invalid(cs)) {
>   			struct cgroup_subsys_state *css;
>   			struct cpuset *child;
> -			bool exclusive = true;
>   
>   			/*
>   			 * Convert invalid partition to valid has to pass the
> -			 * cpu exclusivity test when cpuset.cpus.exclusive is
> -			 * empty as well as the housekeeping check.
> +			 * following tests:
> +			 * 1) housekeeping CPU conflict check
> +			 * 2) cpu exclusivity test when cpuset.cpus.exclusive is
> +			 *    empty.
> +			 * 3) if parent is top_cpuset, none of the effective_xcpus
> +			 *    (tmp->delmask) are in subpartitions_cpus, if not a
> +			 *    remote partition may be underneath.
>   			 */
>   			if (prstate_housekeeping_conflict(cs->partition_root_state,
>   							  parent_prs, tmp->delmask, NULL)) {
>   				part_error = PERR_HKEEPING;
> -				goto write_error;
> -			}
> -			if (cpumask_empty(cs->exclusive_cpus)) {
> +			} else if (cpumask_empty(cs->exclusive_cpus)) {
>   				rcu_read_lock();
>   				cpuset_for_each_child(child, css, parent) {
>   					if (child == cs)
>   						continue;
>   					if (!cpusets_are_exclusive(cs, child)) {
> -						exclusive = false;
> +						part_error = PERR_NOTEXCL;
>   						break;
>   					}
>   				}
>   				rcu_read_unlock();
> +			} else if ((parent == &top_cpuset) &&
> +				   cpumask_intersects(tmp->delmask, subpartitions_cpus)) {
> +				part_error = PERR_REMOTE;
>   			}
> -			if (exclusive)
> +			if (!part_error)
>   				deleting = !cpumask_empty(tmp->delmask);
> -			else
> -				part_error = PERR_NOTEXCL;
>   		}
>   	}
>   
> @@ -2155,7 +2165,7 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
>   			break;
>   		case PRS_INVALID_ROOT:
>   		case PRS_INVALID_ISOLATED:
> -			if (!part_error)
> +			if (!part_error && deleting)
>   				new_prs = -old_prs;
>   			break;
>   		}

Nit: deleting is always true when part_error is 0, so this check
doesn't change anything. It's fine to keep it as a safeguard, but
then the changelog shouldn't describe it as a fix.

Typos in the changelog: "swtiching", "share some the of".

Otherwise,

Reviewed-by: Ridong Chen <ridong.chen@linux.dev>


> diff --git a/tools/testing/selftests/cgroup/test_cpuset_prs.sh b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
> index 1397ec428c46..81f0eecaca80 100755
> --- a/tools/testing/selftests/cgroup/test_cpuset_prs.sh
> +++ b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
> @@ -563,6 +563,10 @@ TEST_MATRIX=(
>   	" C1-6:P1 C3-6:P2 C5-6:P2  .       .    P1      .      .     0 A1:1-2|A2:3-4|A3:5-6 \
>   								       A1:P1|A2:P1|A3:P2 5-6"
>   
> +	# A hotplug event will not incorrectly activate an invalid partition with
> +	# a remote partition underneath.
> +	"  CX1-4  C3-4:P1   .      .      P1   O1=0     .      .     0 A1:2|A2:3-4 A1:P-1|A2:P1"
> +
>   	#
>   	#  old-A1 old-A2 old-A3 old-B1 new-A1 new-A2 new-A3 new-B1 fail ECPUs Pstate ISOLCPUS
>   	#  ------ ------ ------ ------ ------ ------ ------ ------ ---- ----- ------ --------

-- 
Best regards
Ridong


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH-next v4 3/6] cgroup/cpuset: Check remote partition underneath before making an invalid partition valid
  2026-11-16  3:47   ` Ridong Chen
@ 2026-11-16  4:21     ` Waiman Long
  0 siblings, 0 replies; 18+ messages in thread
From: Waiman Long @ 2026-11-16  4:21 UTC (permalink / raw)
  To: Ridong Chen, Tejun Heo, Johannes Weiner, Michal Koutný,
	Shuah Khan
  Cc: cgroups, linux-kernel, linux-kselftest, Guopeng Zhang, Sashiko

On 11/15/26 10:47 PM, Ridong Chen wrote:
>
>
> On 11/16/2026 9:23 AM, Waiman Long wrote:
>> When re-evaluating an invalid local partition via partcmd_update,
>> update_parent_effective_cpumask() does not check whether a descendant
>> remote partition is using its exclusive CPUs, promoting the cpuset to a
>> valid partition root over the remote partition. This condition should 
>> only
>> be possible if parent is the top_cpuset and exclusive_cpus isn't empty.
>>
>> This leaves an active remote partition underneath a valid
>> local partition root with overlapping exclusive CPUs, breaking
>> scheduler domain partitioning and triggering warnings in
>> compute_partition_effective_cpumask() and generate_sched_domains().
>>
>> Fix that by checking the effective_xcpus against subpartitions_cpus
>> when an underneath remote partition is possible to make sure that all
>> the exclusive CPUs are available before allowing the invalid partition
>> to be switched back to valid.
>>
>> Also make sure that effective_xcpus is non-empty before swtiching an
>> invalid partition back to valid.
>>
>> Add a test case in test_cpuset_prs.sh to verify that an invalid 
>> partition
>> with a remote partition underneath cannot be switched back to valid
>> with a hotplug event.
>>
>> In the special case when the last active CPU in the top_cpuset partition
>> is offlined, the hotplug code will empty subpartitions_cpus and let the
>> existing local partition to compete for the exclusive CPUs again. If
>> there is an invalid local partition above a remote partition and that
>> invalid local partition is able to switch to a valid partition, the 
>> remote
>> partition below will now share some the of exclusive CPUs violating the
>> CPU exclusivity rule. This special case will be handled in a later 
>> patch.
>>
>> Fixes: 181c8e091aae ("cgroup/cpuset: Introduce remote partition")
>> Reported-by: Sashiko <sashiko-bot@kernel.org>
>> Link: 
>> https://sashiko.dev/#/bug/linux-47715b89-d5fb-4731-b7c9-070773c6ff75
>> Signed-off-by: Waiman Long <longman@redhat.com>
>> ---
>>   kernel/cgroup/cpuset.c                        | 42 ++++++++++++-------
>>   .../selftests/cgroup/test_cpuset_prs.sh       |  4 ++
>>   2 files changed, 30 insertions(+), 16 deletions(-)
>>
>> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
>> index 195d4e3a283c..81481fa62cfe 100644
>> --- a/kernel/cgroup/cpuset.c
>> +++ b/kernel/cgroup/cpuset.c
>> @@ -2029,11 +2029,7 @@ static int 
>> update_parent_effective_cpumask(struct cpuset *cs, int cmd,
>>            * tasks_nocpu_error() with this very mask, and a failing
>>            * write is routed to partcmd_invalidate instead.
>>            */
>> -        if (is_partition_invalid(cs)) {
>> -            adding = false;
>> -            deleting = cpumask_and(tmp->delmask,
>> -                    newmask, parent->effective_xcpus);
>> -        } else {
>> +        if (is_partition_valid(cs)) {
>>               cpumask_andnot(tmp->addmask, cs->effective_xcpus, 
>> newmask);
>>               adding = cpumask_and(tmp->addmask, tmp->addmask,
>>                            parent->effective_xcpus);
>> @@ -2041,6 +2037,17 @@ static int 
>> update_parent_effective_cpumask(struct cpuset *cs, int cmd,
>>               cpumask_andnot(tmp->delmask, newmask, 
>> cs->effective_xcpus);
>>               deleting = cpumask_and(tmp->delmask, tmp->delmask,
>>                              parent->effective_xcpus);
>> +        } else {
>> +            deleting = cpumask_and(tmp->delmask,
>> +                    newmask, parent->effective_xcpus);
>> +
>> +            /* Check if a remote partition may be underneath */
>> +            if ((parent == &top_cpuset) &&
>> +                !cpumask_empty(cs->exclusive_cpus) &&
>> +                cpumask_intersects(tmp->delmask, subpartitions_cpus)) {
>> +                part_error = PERR_REMOTE;
>> +                deleting = false;
>> +            }
>>           }
>>             /*
>> @@ -2107,34 +2114,37 @@ static int 
>> update_parent_effective_cpumask(struct cpuset *cs, int cmd,
>>           } else if (is_partition_invalid(cs)) {
>>               struct cgroup_subsys_state *css;
>>               struct cpuset *child;
>> -            bool exclusive = true;
>>                 /*
>>                * Convert invalid partition to valid has to pass the
>> -             * cpu exclusivity test when cpuset.cpus.exclusive is
>> -             * empty as well as the housekeeping check.
>> +             * following tests:
>> +             * 1) housekeeping CPU conflict check
>> +             * 2) cpu exclusivity test when cpuset.cpus.exclusive is
>> +             *    empty.
>> +             * 3) if parent is top_cpuset, none of the effective_xcpus
>> +             *    (tmp->delmask) are in subpartitions_cpus, if not a
>> +             *    remote partition may be underneath.
>>                */
>>               if 
>> (prstate_housekeeping_conflict(cs->partition_root_state,
>>                                 parent_prs, tmp->delmask, NULL)) {
>>                   part_error = PERR_HKEEPING;
>> -                goto write_error;
>> -            }
>> -            if (cpumask_empty(cs->exclusive_cpus)) {
>> +            } else if (cpumask_empty(cs->exclusive_cpus)) {
>>                   rcu_read_lock();
>>                   cpuset_for_each_child(child, css, parent) {
>>                       if (child == cs)
>>                           continue;
>>                       if (!cpusets_are_exclusive(cs, child)) {
>> -                        exclusive = false;
>> +                        part_error = PERR_NOTEXCL;
>>                           break;
>>                       }
>>                   }
>>                   rcu_read_unlock();
>> +            } else if ((parent == &top_cpuset) &&
>> +                   cpumask_intersects(tmp->delmask, 
>> subpartitions_cpus)) {
>> +                part_error = PERR_REMOTE;
>>               }
>> -            if (exclusive)
>> +            if (!part_error)
>>                   deleting = !cpumask_empty(tmp->delmask);
>> -            else
>> -                part_error = PERR_NOTEXCL;
>>           }
>>       }
>>   @@ -2155,7 +2165,7 @@ static int 
>> update_parent_effective_cpumask(struct cpuset *cs, int cmd,
>>               break;
>>           case PRS_INVALID_ROOT:
>>           case PRS_INVALID_ISOLATED:
>> -            if (!part_error)
>> +            if (!part_error && deleting)
>>                   new_prs = -old_prs;
>>               break;
>>           }
>
> Nit: deleting is always true when part_error is 0, so this check
> doesn't change anything. It's fine to keep it as a safeguard, but
> then the changelog shouldn't describe it as a fix.

tmp->delmask can be empty and deleting will be set to false in this 
case. Of course, we should not activate an invalid partition root with 
empty effective_xcpus.


>
> Typos in the changelog: "swtiching", "share some the of".
>
Will fix that in the next version.

Thanks,
Longman

> Otherwise,
>
> Reviewed-by: Ridong Chen <ridong.chen@linux.dev>
>
>
>> diff --git a/tools/testing/selftests/cgroup/test_cpuset_prs.sh 
>> b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
>> index 1397ec428c46..81f0eecaca80 100755
>> --- a/tools/testing/selftests/cgroup/test_cpuset_prs.sh
>> +++ b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
>> @@ -563,6 +563,10 @@ TEST_MATRIX=(
>>       " C1-6:P1 C3-6:P2 C5-6:P2  .       .    P1      . .     0 
>> A1:1-2|A2:3-4|A3:5-6 \
>>                                          A1:P1|A2:P1|A3:P2 5-6"
>>   +    # A hotplug event will not incorrectly activate an invalid 
>> partition with
>> +    # a remote partition underneath.
>> +    "  CX1-4  C3-4:P1   .      .      P1   O1=0     . .     0 
>> A1:2|A2:3-4 A1:P-1|A2:P1"
>> +
>>       #
>>       #  old-A1 old-A2 old-A3 old-B1 new-A1 new-A2 new-A3 new-B1 fail 
>> ECPUs Pstate ISOLCPUS
>>       #  ------ ------ ------ ------ ------ ------ ------ ------ ---- 
>> ----- ------ --------
>


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH-next v4 5/6] cgroup/cpuset: Handle remote partitions before local partitions in cpuset_handle_hotplug()
  2026-11-16  1:23 ` [PATCH-next v4 5/6] cgroup/cpuset: Handle remote partitions before local partitions in cpuset_handle_hotplug() Waiman Long
@ 2026-11-16  7:08   ` Ridong Chen
  2026-11-16  8:56   ` kernel test robot
  2026-11-16 10:33   ` Guopeng Zhang
  2 siblings, 0 replies; 18+ messages in thread
From: Ridong Chen @ 2026-11-16  7:08 UTC (permalink / raw)
  To: Waiman Long, Tejun Heo, Johannes Weiner, Michal Koutný,
	Shuah Khan, Guopeng Zhang
  Cc: cgroups, linux-kernel, linux-kselftest, Guopeng Zhang, Sashiko



On 11/16/2026 9:23 AM, Waiman Long wrote:
> When there is a valid remote partition underneath an invalid local
> partition with subpartitions_cpus cleared because of missing active
> CPUs in the top_cpuset partition, the current iteration logic in
> cpuset_handle_hotplug() will handle the invalid local partition first
> and potentially make the local partition valid. The remote partition
> will be handled later with the potential that both the local and remote
> partitions will be valid with overlapping exclusive CPUs violating the
> cpu exclusivity rule.
> 

Hi Longman,

Patch 5 and Guopeng's patch [1] handle the same case: after hotplug,
top_cpuset has no usable CPU left (no active CPU, or no active boot
domain housekeeping CPU). But they take different approaches:

- Current code: clear subpartitions_cpus and invalidate all the
  partitions.
- [1]: only invalidate the partitions that can give an active
  housekeeping CPU back to top_cpuset.
- Patch 5: invalidate all the local partitions but keep the remote
  ones valid if possible.

I think we should agree on the expected behavior first. My preference
is to keep it simple and invalidate all the partitions as the current
code does. The problem is that in the same hotplug pass, after valid
partition A is invalidated and its CPUs are given back to top_cpuset,
partition B, which was invalid before, may become valid again. B puts
its CPUs back into subpartitions_cpus, so a valid partition visited
after B is not invalidated (patch 1), or B overlaps with a remote
partition underneath it (patch 5). If invalid partitions are not
allowed to become valid again in that pass, both issues go away.
Do we really need to keep some partitions valid in this case?

[1] https://lore.kernel.org/cgroups/20261111134250.16852-2-guopeng.zhang@linux.dev/

-- 
Best regards
Ridong


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH-next v4 6/6] cgroup/cpuset: Fix lockdep_is_cpuset_held()
  2026-11-16  1:23 ` [PATCH-next v4 6/6] cgroup/cpuset: Fix lockdep_is_cpuset_held() Waiman Long
@ 2026-11-16  7:24   ` Ridong Chen
  2026-11-16 10:34   ` Guopeng Zhang
  1 sibling, 0 replies; 18+ messages in thread
From: Ridong Chen @ 2026-11-16  7:24 UTC (permalink / raw)
  To: Waiman Long, Tejun Heo, Johannes Weiner, Michal Koutný,
	Shuah Khan
  Cc: cgroups, linux-kernel, linux-kselftest, Guopeng Zhang



On 11/16/2026 9:23 AM, Waiman Long wrote:
> Commit a84097e625f2 ("cgroup/cpuset: Call housekeeping_update()
> without holding cpus_read_lock") changes lockdep_is_cpuset_held()
> to return true if either cpuset_mutex or cpuset_top_mutex is held.
> lockdep_is_cpuset_held() is used by housekeeping_dereference_check()
> as one of the conditions that rcu_read_lock() is not needed to
> access HK_TYPE_DOMAIN cpumask that can be changed at run time.
> In reality, it should return true only if cpuset_top_mutex is held as
> housekeeping_update() is called with only cpuset_top_mutex held.
> 
> If only cpuset_mutex is held, it is possible that there can be a parallel
> housekeeping_update() call leading to a potential use-after-free situation
> when HK_TYPE_DOMAIN housekeeping cpumask is being accessed. It also
> means that rcu_read_lock will be required if only cpuset_mutex is being
> held like when calling from hotplug, scheduler or cgroup.
> 
> Fixes: a84097e625f2 ("cgroup/cpuset: Call housekeeping_update() without holding cpus_read_lock")
> Signed-off-by: Waiman Long <longman@redhat.com>
> ---
>   kernel/cgroup/cpuset-v1.c | 16 ++++++++++------
>   kernel/cgroup/cpuset.c    |  9 +++++++--
>   2 files changed, 17 insertions(+), 8 deletions(-)
> 
> diff --git a/kernel/cgroup/cpuset-v1.c b/kernel/cgroup/cpuset-v1.c
> index 7c7f2887dfaf..6bf6da639fec 100644
> --- a/kernel/cgroup/cpuset-v1.c
> +++ b/kernel/cgroup/cpuset-v1.c
> @@ -686,6 +686,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,
>   	int csn;		/* how many cpuset ptrs in csa so far */
>   	int i, j;		/* indices for partition finding loops */
>   	cpumask_var_t *doms;	/* resulting partition; i.e. sched domains */
> +	cpumask_var_t hk_cpus;	/* HK_TYPE_DOMAIN housekeeping CPUs */
>   	struct sched_domain_attr *dattr;  /* attributes for custom domains */
>   	int ndoms = 0;		/* number of sched domains in result */
>   	int nslot;		/* next empty doms[] struct cpumask slot */
> @@ -698,6 +699,11 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,
>   	dattr = NULL;
>   	csa = NULL;
>   
> +	alloc_cpumask_var(&hk_cpus, GFP_KERNEL);
> +	scoped_guard(rcu) {
> +		cpumask_copy(hk_cpus, housekeeping_cpumask(HK_TYPE_DOMAIN));
> +	}
> +

The return value of alloc_cpumask_var() isn't checked. With
CONFIG_CPUMASK_OFFSTACK=y, hk_cpus is NULL on failure. Maybe goto done
like the other allocation failures here?

Also, rebuild_sched_domains() can be called from outside cpuset, e.g.
arch_topology and the energy model, with only cpus_read_lock and
cpuset_mutex held. partition_sched_domains_locked() reads
HK_TYPE_DOMAIN in asym_cpu_capacity_scan() and in its fallback path.
Will these trigger a lockdep warning after this change? If so, do they
need an RCU read lock as well?


>   	/* Special case for the 99% of systems with one, full, sched domain */
>   	if (is_sched_load_balance(&top_cpuset)) {
>   		ndoms = 1;
> @@ -710,9 +716,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,
>   			*dattr = SD_ATTR_INIT;
>   			update_domain_attr_tree(dattr, &top_cpuset);
>   		}
> -		cpumask_and(doms[0], top_cpuset.effective_cpus,
> -			    housekeeping_cpumask(HK_TYPE_DOMAIN));
> -
> +		cpumask_and(doms[0], top_cpuset.effective_cpus, hk_cpus);
>   		goto done;
>   	}
>   
> @@ -736,8 +740,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,
>   		 */
>   		if (!cpumask_empty(cp->cpus_allowed) &&
>   		    !(is_sched_load_balance(cp) &&
> -		      cpumask_intersects(cp->cpus_allowed,
> -					 housekeeping_cpumask(HK_TYPE_DOMAIN))))
> +		      cpumask_intersects(cp->cpus_allowed, hk_cpus)))
>   			continue;
>   
>   		if (is_sched_load_balance(cp) &&
> @@ -794,7 +797,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,
>   						*(dattr + nslot) = SD_ATTR_INIT;
>   				}
>   				cpumask_or(dp, dp, csa[j]->effective_cpus);
> -				cpumask_and(dp, dp, housekeeping_cpumask(HK_TYPE_DOMAIN));
> +				cpumask_and(dp, dp, hk_cpus);
>   				if (dattr)
>   					update_domain_attr_tree(dattr + nslot, csa[j]);
>   			}
> @@ -806,6 +809,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,
>   
>   done:
>   	kfree(csa);
> +	free_cpumask_var(hk_cpus);
>   
>   	/*
>   	 * Fallback to the default domain if kmalloc() failed.
> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
> index 64cbcbdbf6da..d18439e13bce 100644
> --- a/kernel/cgroup/cpuset.c
> +++ b/kernel/cgroup/cpuset.c
> @@ -343,10 +343,13 @@ void cpuset_full_unlock(void)
>   }
>   
>   #ifdef CONFIG_LOCKDEP
> +/*
> + * Holding cpuset_top_mutex will guarantee that a parallel
> + * housekeeping_update() call modifying housekeeping cpumasks is not possible.
> + */
>   bool lockdep_is_cpuset_held(void)
>   {
> -	return lockdep_is_held(&cpuset_mutex) ||
> -	       lockdep_is_held(&cpuset_top_mutex);
> +	return lockdep_is_held(&cpuset_top_mutex);
>   }
>   #endif
>   
> @@ -1416,8 +1419,10 @@ static bool prstate_housekeeping_conflict(int new_prs, int parent_prs,
>   		}
>   	}
>   
> +	rcu_read_lock();
>   	cpumask_and(full_hk_cpus, housekeeping_cpumask(HK_TYPE_KERNEL_NOISE),
>   		    housekeeping_cpumask(HK_TYPE_DOMAIN));
> +	rcu_read_unlock();
>   	cpumask_andnot(full_hk_cpus, full_hk_cpus, isolated_cpus);
>   	cpumask_and(full_hk_cpus, full_hk_cpus, cpu_active_mask);
>   	res = !cpumask_weight_andnot(full_hk_cpus, new_cpus);

-- 
Best regards
Ridong


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH-next v4 2/6] cgroup/cpuset: excpus parameter of remote_cpus_update() must have at least 1 active CPU
  2026-11-16  3:34     ` Waiman Long
@ 2026-11-16  7:37       ` Ridong Chen
  0 siblings, 0 replies; 18+ messages in thread
From: Ridong Chen @ 2026-11-16  7:37 UTC (permalink / raw)
  To: Waiman Long, Tejun Heo, Johannes Weiner, Michal Koutný,
	Shuah Khan
  Cc: cgroups, linux-kernel, linux-kselftest, Guopeng Zhang, Sashiko



On 11/16/2026 11:34 AM, Waiman Long wrote:
> On 11/15/26 10:07 PM, Ridong Chen wrote:
>>
>>
>> On 11/16/2026 9:23 AM, Waiman Long wrote:
>>> It is reported that remote_cpus_update() fails to invalidate a
>>> populated remote partition when an ancestor cgroup update reduces its
>>> effective_xcpus mask to only offline CPUs.
>>>
>>> When update_cpumasks_hier() updates a descendant remote partition, it
>>> passes the new effective_xcpus mask (excpus) to remote_cpus_update().
>>> Because excpus still contains offline CPUs, cpumask_empty(excpus) is false
>>> and remote_cpus_update() only checks tasks_nocpu_error() when adding CPUs.
>>>
>>> With the partition left valid, update_cpumasks_hier() computes an empty
>>> effective_cpus mask and jumps to get_css. The populated partition stays
>>> valid with no effective CPUs instead of inheriting its parent's CPUs,
>>> and its tasks remain affine to the online CPUs returned to top_cpuset.
>>>
>>> Fix it by invalidating the remote partition if there is no active CPU
>>> available in the given excpus parameter.
>>>
>>> Reported-by: Sashiko <sashiko-bot@kernel.org>
>>> Link: https://sashiko.dev/#/bug/linux-936b7155-f652-4146-a4ab-ec48fd392a1d
>>> Signed-off-by: Waiman Long <longman@redhat.com>
>>> ---
>>>   kernel/cgroup/cpuset.c | 3 ++-
>>>   1 file changed, 2 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
>>> index 631fb449cef8..195d4e3a283c 100644
>>> --- a/kernel/cgroup/cpuset.c
>>> +++ b/kernel/cgroup/cpuset.c
>>> @@ -1810,7 +1810,8 @@ static void remote_cpus_update(struct cpuset *cs, 
>>> struct cpumask *xcpus,
>>>         WARN_ON_ONCE(!cpumask_subset(cs->effective_xcpus, subpartitions_cpus));
>>>   -    if (cpumask_empty(excpus)) {
>>> +    /* New effective_xcpus must have at least one active CPU */
>>> +    if (!cpumask_intersects(excpus, cpu_active_mask)) {
>>>           WRITE_ONCE(cs->prs_err, PERR_CPUSEMPTY);
>>>           goto invalidate;
>>>       }
>>
>> This also invalidates a remote partition with no task whose excpus
>> only has offline CPUs. According to cgroup-v2.rst, such a partition
>> is valid: cpuset.cpus.exclusive.effective may contain offline CPUs,
>> and cpuset.cpus.effective can be empty if there is no task in the
>> partition. The hotplug path and tasks_nocpu_error() follow the same
>> rule. How about only invalidating it when it is populated?
>>
>>       if (cpumask_empty(excpus) ||
>>           (!cpumask_intersects(excpus, cpu_active_mask) &&
>>            partition_is_populated(cs, NULL))) {
> 
> A partition can only be created if there are active CPUs in the effective_xcpus. 
> However, such a partition root can distribute its exclusive CPUs down to child 
> local partitions with the results that effective_cpus can be empty if there is 
> no task there. IOW, effective_xcpus must have active CPUs while effective_cpus 
> does not need to have active CPUs if there are child partitions underneath. I am 
> going to clarify that in the doc. I will check the hotplug path and 
> task_nocpu_error() to make sure that it follows this rule. Does that rule make 
> sense to you?
> 

Yes, the rule makes sense to me, and it is clearer.

Note that the hotplug path currently keeps a partition with no task
valid after all its CPUs are offlined, e.g. these test cases in
test_cpuset_prs.sh:

"  C2-3:P1  C3:P1   .      .     O3=0    .      .      .     0 A1:2|A2: A1:P1|A2:P1"
"$SETUP_A123_PARTITIONS    .     O3=0    .      .      .     0 A1:1|A2:2|A3: 
A1:P1|A2:P1|A3:P1"

So the hotplug path and these test cases need to be updated together
with this patch to keep the behavior consistent?

> 
>>
>> Also, it probably needs a Fixes tag, maybe:
>>
>> Fixes: f62a5d39368e ("cgroup/cpuset: Remove remote_partition_check() & make 
>> update_cpumasks_hier() handle remote partition")
> 
> Yes, I miss the fix tag. Will update that in the next version.
> 
> Cheers,
> Longman

-- 
Best regards
Ridong


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH-next v4 5/6] cgroup/cpuset: Handle remote partitions before local partitions in cpuset_handle_hotplug()
  2026-11-16  1:23 ` [PATCH-next v4 5/6] cgroup/cpuset: Handle remote partitions before local partitions in cpuset_handle_hotplug() Waiman Long
  2026-11-16  7:08   ` Ridong Chen
@ 2026-11-16  8:56   ` kernel test robot
  2026-11-16 10:33   ` Guopeng Zhang
  2 siblings, 0 replies; 18+ messages in thread
From: kernel test robot @ 2026-11-16  8:56 UTC (permalink / raw)
  To: Waiman Long, Ridong Chen, Tejun Heo, Johannes Weiner,
	Michal Koutný, Shuah Khan
  Cc: llvm, oe-kbuild-all, cgroups, linux-kernel, linux-kselftest,
	Guopeng Zhang, Waiman Long, Sashiko

Hi Waiman,

kernel test robot noticed the following build warnings:

[auto build test WARNING on tj-cgroup/for-next]
[also build test WARNING on next-20261113]
[cannot apply to linus/master v7.4-rc2]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Waiman-Long/cgroup-cpuset-Fix-incorrect-partition-invalidation-logic-in-cpuset_hotplug_update_tasks/20261115-202304
base:   https://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git for-next
patch link:    https://lore.kernel.org/r/20261116012330.2271010-6-longman%40redhat.com
patch subject: [PATCH-next v4 5/6] cgroup/cpuset: Handle remote partitions before local partitions in cpuset_handle_hotplug()
config: riscv-defconfig (https://download.01.org/0day-ci/archive/20261116/202611161625.POIP7bQr-lkp@intel.com/config)
compiler: clang version 24.0.0git (https://github.com/llvm/llvm-project ca14e3d05c4e63d6e8103945c86b958ed2f55d5f)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20261116/202611161625.POIP7bQr-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202611161625.POIP7bQr-lkp@intel.com/

All warnings (new ones prefixed by >>):

>> Warning: kernel/cgroup/cpuset.c:1895 function parameter 'tmp' not described in 'remote_partition_scan'
>> Warning: kernel/cgroup/cpuset.c:1895 Excess function parameter 'tmpmask' description in 'remote_partition_scan' (did you mean one of: 'tmp')
>> Warning: kernel/cgroup/cpuset.c:1895 function parameter 'tmp' not described in 'remote_partition_scan'
>> Warning: kernel/cgroup/cpuset.c:1895 Excess function parameter 'tmpmask' description in 'remote_partition_scan' (did you mean one of: 'tmp')

--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH-next v4 1/6] cgroup/cpuset: Fix incorrect partition invalidation logic in cpuset_hotplug_update_tasks()
  2026-11-16  1:23 ` [PATCH-next v4 1/6] cgroup/cpuset: Fix incorrect partition invalidation logic in cpuset_hotplug_update_tasks() Waiman Long
@ 2026-11-16 10:19   ` Guopeng Zhang
  0 siblings, 0 replies; 18+ messages in thread
From: Guopeng Zhang @ 2026-11-16 10:19 UTC (permalink / raw)
  To: Waiman Long, Ridong Chen, Tejun Heo, Johannes Weiner,
	Michal Koutný, Shuah Khan
  Cc: cgroups, linux-kernel, linux-kselftest, Sashiko



在 2026/11/16 09:23, Waiman Long 写道:
> When most of the active CPUs of a system are allocated to
> sub-partitions below the top_cpuset partition, it is possible that a
> CPU offline operation may remove the last active CPU in the top_cpuset
> partition. When that really happens, cpuset_handle_hotplug() will clear
> the subpartitions_cpus to make sure active CPUs are present in the top
> partition and let the other sub-partitions get invalidated and then
> competes for their exclusive CPUs again.
> 
> In cpuset_hotplug_update_tasks(), the current code use
> "cpumask_empty(subpartitions_cpus)" to determine if that partition
> should be invalidated. That is fine if there is only one sub-partition.
> As reported by sashiko that when there are more than one sub-partitions
> and the first sub-partition will get invalidated and then it will
> try to acquire the exclusive CPUs and become valid again. When this
> happens, the subpartitions_cpus will no longer be empty. The remaining
> sub-partitions will assume their exclusive CPUs are safe and proceed
> as normal. As a result, active CPUs owned by those sub-partitions
> remain in both top_cpuset.effective_cpus and the sub-partitions'
> effective_cpus while missing from subpartitions_cpus, breaking partition
> CPU exclusivity and triggering WARN_ON_ONCE() in generate_sched_domains()
> and remote_partition_disable().
> 
> Fix that by checking if the sub-partitions' effective_xcpus are subset
> of subpartitions_cpus to determine if they should be invalidated. Also
> remove the obsolete WARN_ON_ONCE() call in remote_partition_disable().
> 
> Fixes: 0c7f293efc87 ("cgroup/cpuset: Add cpuset.cpus.exclusive.effective for v2")
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Link: https://sashiko.dev/#/bug/linux-bf69cd3c-833a-4599-a1f9-8dd9e044cd7c
> Reviewed-by: Ridong Chen <ridong.chen@linux.dev>
> Signed-off-by: Waiman Long <longman@redhat.com>
> ---
>  kernel/cgroup/cpuset.c | 19 +++++--------------
>  1 file changed, 5 insertions(+), 14 deletions(-)
> 
> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
> index 1b88a93b3db4..631fb449cef8 100644
> --- a/kernel/cgroup/cpuset.c
> +++ b/kernel/cgroup/cpuset.c
> @@ -1768,15 +1768,6 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs,
>  static void remote_partition_disable(struct cpuset *cs, struct tmpmasks *tmp)
>  {
>  	WARN_ON_ONCE(!is_remote_partition(cs));
> -	/*
> -	 * When a CPU is offlined, top_cpuset may end up with no available CPUs,
> -	 * which should clear subpartitions_cpus. We should not emit a warning for this
> -	 * scenario: the hierarchy is updated from top to bottom, so subpartitions_cpus
> -	 * may already be cleared when disabling the partition.
> -	 */
> -	WARN_ON_ONCE(!cpumask_subset(cs->effective_xcpus, subpartitions_cpus) &&
> -		     !cpumask_empty(subpartitions_cpus));
> -
>  	spin_lock_irq(&callback_lock);
>  	cs->remote_partition = false;
>  	partition_xcpus_del(cs->partition_root_state, NULL, cs->effective_xcpus);
> @@ -3983,9 +3974,9 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp)
>  	if (!tmp || !cs->partition_root_state)
>  		goto update_tasks;
>  
> -	if (is_remote_partition(cs) && (cpumask_empty(subpartitions_cpus) ||
> -				       (cpumask_empty(&new_cpus) &&
> -					partition_is_populated(cs, NULL)))) {
> +	if (is_remote_partition(cs) &&
> +	    (!cpumask_subset(cs->effective_xcpus, subpartitions_cpus) ||
> +	     (cpumask_empty(&new_cpus) && partition_is_populated(cs, NULL)))) {
>  		WRITE_ONCE(cs->prs_err, PERR_HOTPLUG);
>  		remote_partition_disable(cs, tmp);
>  		compute_effective_cpumask(&new_cpus, cs, parent);
> @@ -3997,12 +3988,12 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp)
>  	 * 1) empty effective cpus but not valid empty partition.
>  	 * 2) parent is invalid or doesn't grant any cpus to child
>  	 *    partitions.
> -	 * 3) subpartitions_cpus is empty.
> +	 * 3) effective_xcpus isn't a subset of subpartitions_cpus.
>  	 */
>  	if (is_local_partition(cs) &&
>  	    (!is_partition_valid(parent) ||
>  	     tasks_nocpu_error(parent, cs, &new_cpus) ||
> -	     cpumask_empty(subpartitions_cpus)))
> +	     !cpumask_subset(cs->effective_xcpus, subpartitions_cpus)))
>  		partcmd = partcmd_invalidate;
>  	/*
>  	 * On the other hand, an invalid partition root may be transitioned

Reviewed-by: Guopeng Zhang <zhangguopeng@kylinos.cn>

Thanks,
Guopeng

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH-next v4 5/6] cgroup/cpuset: Handle remote partitions before local partitions in cpuset_handle_hotplug()
  2026-11-16  1:23 ` [PATCH-next v4 5/6] cgroup/cpuset: Handle remote partitions before local partitions in cpuset_handle_hotplug() Waiman Long
  2026-11-16  7:08   ` Ridong Chen
  2026-11-16  8:56   ` kernel test robot
@ 2026-11-16 10:33   ` Guopeng Zhang
  2 siblings, 0 replies; 18+ messages in thread
From: Guopeng Zhang @ 2026-11-16 10:33 UTC (permalink / raw)
  To: Waiman Long, Ridong Chen, Tejun Heo, Johannes Weiner,
	Michal Koutný, Shuah Khan
  Cc: cgroups, linux-kernel, linux-kselftest, Sashiko



在 2026/11/16 09:23, Waiman Long 写道:
> When there is a valid remote partition underneath an invalid local
> partition with subpartitions_cpus cleared because of missing active
> CPUs in the top_cpuset partition, the current iteration logic in
> cpuset_handle_hotplug() will handle the invalid local partition first
> and potentially make the local partition valid. The remote partition
> will be handled later with the potential that both the local and remote
> partitions will be valid with overlapping exclusive CPUs violating the
> cpu exclusivity rule.
> 
> Fix this problem by adding a new remote_partition_scan() helper to
> iterate the existing remote partitions first before iterating down
> the hierarchy to handle the local partitions. That also means that
> when subpartitions_cpus is cleared, remote partitions have precedence
> to acquire the exclusive CPUs first before local partitions. It also
> gives remote partition the ability to recover from the clearing of
> subpartitions_cpus as all the remote partitions will be disabled before
> this commit.
> 
> A remote partition can only be created if there is at least one active
> CPU in its effective_xcpus. So remote_partition_scan() will also
> invalidate the remote partition if there is no active CPU left in its
> effective_xcpus. This requires a minor change in a test_cpuset_prs.sh
> test case where a remote partition with no active CPU left will now
> be invalidated.
> 
> With the addition of the remote_partition_scan() call in
> cpuset_handle_hotplug(), there is no longer a need to do a remote
> partition check in cpuset_hotplug_update_tasks() and the corresponding
> code can be removed.
> 
> While at it, the subpartition_cpus clearing logic is enhanced to
> clear it when no domain housekeeping CPUs are left in the top_cpuset
> partition. When the remote partition is disabled, the effective_cpus
> is modified in a such a way that cpus_updated will be set true later
> in cpuset_hotplug_update_tasks() so that the CPU affinity of the
> tasks within the partition will be updated. This will also fix a
> bug reported by sashiko that CPU affinity will not be updated when
> remote_partition_disable() is called in such circumstance.
> 
> When efffective_cpus is read during this interim period, the content
> will be invalid before and after this commit.
> 
> Fixes: 181c8e091aae ("cgroup/cpuset: Introduce remote partition")
> Reported-by: Guopeng Zhang <guopeng.zhang@linux.dev>
> Link: https://lore.kernel.org/lkml/fe7aa630-d323-4981-9f89-0958cb6204a2@linux.dev
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Link: https://sashiko.dev/#/bug/linux-ab90f7e8-0d60-40ca-83e9-e405075b24d2
> Signed-off-by: Waiman Long <longman@redhat.com>

Hi,Longman

Tested all six patches on next-20261113 (32-CPU VM, cgroup v2, no boot
isolation, all CPUs initially online). I can still reproduce an invalid
local partition becoming valid while its remote descendant remains
valid:

  cd /sys/fs/cgroup
  echo +cpuset > cgroup.subtree_control
  echo 0 > /sys/devices/system/cpu/cpu31/online
  mkdir A
  echo 0-30 > A/cpuset.cpus
  echo 0-30 > A/cpuset.cpus.exclusive
  echo root > A/cpuset.cpus.partition          # invalid local root
  echo +cpuset > A/cgroup.subtree_control
  mkdir A/B
  echo 20-25 > A/B/cpuset.cpus
  echo 20-25 > A/B/cpuset.cpus.exclusive
  echo root > A/B/cpuset.cpus.partition        # valid remote partition
  echo 1 > /sys/devices/system/cpu/cpu31/online

After CPU31 comes online, A is a valid root (exclusive.effective 0-30),
A/B is still a valid remote (exclusive.effective 20-25), and the top
cpuset's effective CPUs are 31. The kernel reports:

  WARNING: cpuset.c:1654 compute_partition_effective_cpumask()
           WARN_ON_ONCE(is_remote_partition(child))

Two more results:
1. Local partitions break too, no remote involved.  test_cpuset_prs.sh
   fails at TEST_MATRIX[19] (C0-3:P1 C2-3:P1 + O2=0): with cpu2
   offlined, both partitions become "root invalid" instead of remaining
   valid. A1's effective_cpus then falls back to 0-1,3 instead of 0-1.
2. Booted with isolcpus=31, the same kernel passes the whole suite.

Thanks,
Guopeng> ---
>  kernel/cgroup/cpuset.c                        | 97 ++++++++++++++++---
>  .../selftests/cgroup/test_cpuset_prs.sh       |  2 +-
>  2 files changed, 84 insertions(+), 15 deletions(-)
> 
> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
> index 0fe9f2731fee..64cbcbdbf6da 100644
> --- a/kernel/cgroup/cpuset.c
> +++ b/kernel/cgroup/cpuset.c
> @@ -153,6 +153,12 @@ static cpumask_var_t	isolated_cpus;		/* CSCB */
>   */
>  static struct list_head remote_children;	/* RWCS */
>  
> +/*
> + * Temporary cpumask for local cpumask computation
> + * The cpumask may be changed when a cpuset helper is being called.
> + */
> +static cpumask_var_t	tmp_cpus;		/* RWCS */
> +
>  /*
>   * Set if housekeeping cpumasks are to be updated.
>   */
> @@ -1873,6 +1879,67 @@ static void remote_cpus_update(struct cpuset *cs, struct cpumask *xcpus,
>  	remote_partition_disable(cs, tmp);
>  }
>  
> +/**
> + * remote_partition_scan: scan remote partitions to check exclusive CPUs
> + * @tmpmask: ptr to tmpmasks structure
> + *
> + * This function is called in cpuset_handle_hotplug() after the update on
> + * top_cpuset.effective_cpus when there is cpu online/offline operation.
> + * As subpartitions_cpus may have been cleared, we need to check if the remote
> + * partition should acquire the exclusive CPUs again without impacting
> + * effective_cpus or it needs to be disabled when one or more of the following
> + * conditions is true.
> + * 1) All the domain housekeeping CPUs in the top_cpuset will be taken away
> + * 2) A populated partition with no active CPU or no active CPU in
> + *    effective_xcpus
> + */
> +static void remote_partition_scan(struct tmpmasks *tmp)
> +{
> +	struct cpuset *cs, *next;
> +
> +	rcu_read_lock();
> +	list_for_each_entry_safe(cs, next, &remote_children, remote_sibling) {
> +		bool noxcpus = !cpumask_subset(cs->effective_xcpus, subpartitions_cpus);
> +		bool nocpus = !cpumask_intersects(cs->effective_xcpus, cpu_active_mask) ||
> +			      (!cpumask_intersects(cs->effective_cpus, cpu_active_mask) &&
> +			       partition_is_populated(cs, NULL));
> +		int cpu;
> +
> +		cpumask_and(tmp_cpus, top_cpuset.effective_cpus,
> +			    housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT));
> +		if (noxcpus && !nocpus && !cpumask_subset(tmp_cpus, cs->effective_xcpus)) {
> +			/*
> +			 * Reacquire exclusive CPUs & update top_cpuset.effective_cpus
> +			 */
> +			spin_lock_irq(&callback_lock);
> +			cpumask_andnot(subpartitions_cpus, subpartitions_cpus,
> +				       cs->effective_xcpus);
> +			cpumask_andnot(top_cpuset.effective_cpus, top_cpuset.effective_xcpus,
> +				       cs->effective_xcpus);
> +			spin_unlock_irq(&callback_lock);
> +			continue;
> +		}
> +
> +		if (noxcpus || nocpus || cpumask_empty(tmp_cpus)) {
> +			/*
> +			 * Disable remote partition & modifiy effective_cpus
> +			 * so that cpus_updated will be set true in
> +			 * cpuset_hotplug_update_tasks().
> +			 */
> +			WRITE_ONCE(cs->prs_err, PERR_HOTPLUG);
> +			remote_partition_disable(cs, tmp);
> +			cpu = cpumask_first(cs->effective_cpus);
> +			spin_lock_irq(&callback_lock);
> +			if (cpu < nr_cpu_ids)
> +				cpumask_clear_cpu(cpu, cs->effective_cpus);
> +			else
> +				cpumask_set_cpu(0, cs->effective_cpus);
> +			spin_unlock_irq(&callback_lock);
> +		}
> +	}
> +	rcu_read_unlock();
> +}
> +
>  /**
>   * update_parent_effective_cpumask - update effective_cpus mask of parent cpuset
>   * @cs:      The cpuset that requests change in partition root state
> @@ -3919,6 +3986,7 @@ int __init cpuset_init(void)
>  	BUG_ON(!zalloc_cpumask_var(&subpartitions_cpus, GFP_KERNEL));
>  	BUG_ON(!zalloc_cpumask_var(&isolated_cpus, GFP_KERNEL));
>  	BUG_ON(!zalloc_cpumask_var(&isolated_hk_cpus, GFP_KERNEL));
> +	BUG_ON(!alloc_cpumask_var(&tmp_cpus, GFP_KERNEL));
>  
>  	cpumask_setall(top_cpuset.cpus_allowed);
>  	nodes_setall(top_cpuset.mems_allowed);
> @@ -3993,14 +4061,6 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp)
>  	if (!tmp || !cs->partition_root_state)
>  		goto update_tasks;
>  
> -	if (is_remote_partition(cs) &&
> -	    (!cpumask_subset(cs->effective_xcpus, subpartitions_cpus) ||
> -	     (cpumask_empty(&new_cpus) && partition_is_populated(cs, NULL)))) {
> -		WRITE_ONCE(cs->prs_err, PERR_HOTPLUG);
> -		remote_partition_disable(cs, tmp);
> -		compute_effective_cpumask(&new_cpus, cs, parent);
> -	}
> -
>  	/*
>  	 * Force the partition to become invalid if either one of
>  	 * the following conditions hold:
> @@ -4104,22 +4164,31 @@ static void cpuset_handle_hotplug(void)
>  		if (!on_dfl)
>  			cpumask_copy(top_cpuset.cpus_allowed, &new_cpus);
>  		/*
> -		 * Make sure that CPUs allocated to child partitions
> -		 * do not show up in effective_cpus. If no CPU is left,
> -		 * we clear the subpartitions_cpus & let the child partitions
> -		 * fight for the CPUs again.
> +		 * Make sure that CPUs allocated to child partitions do not
> +		 * show up in effective_cpus. If no housekeeping CPU is left,
> +		 * we clear the subpartitions_cpus, scan the remote partitions
> +		 * to determine if they can remain valid & let the descendant
> +		 * local partitions fight for the CPUs again.
>  		 */
>  		if (!cpumask_empty(subpartitions_cpus)) {
> +			cpumask_andnot(&new_cpus, cpu_active_mask,
> +				       housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT));
>  			if (cpumask_subset(&new_cpus, subpartitions_cpus)) {
>  				cpumask_clear(subpartitions_cpus);
> +				cpumask_copy(&new_cpus, cpu_active_mask);
>  			} else {
> -				cpumask_andnot(&new_cpus, &new_cpus,
> +				cpumask_andnot(&new_cpus, cpu_active_mask,
>  					       subpartitions_cpus);
>  			}
>  		}
>  		cpumask_copy(top_cpuset.effective_cpus, &new_cpus);
>  		spin_unlock_irq(&callback_lock);
> -		/* we don't mess with cpumasks of tasks in top_cpuset */
> +		/*
> +		 * We don't mess with cpumasks of tasks in top_cpuset, but we
> +		 * have to examine impact on remote partition roots if present.
> +		 */
> +		if (cpuset_v2() && ptmp)
> +			remote_partition_scan(ptmp);
>  	}
>  
>  	/* synchronize mems_allowed to N_MEMORY */
> diff --git a/tools/testing/selftests/cgroup/test_cpuset_prs.sh b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
> index 81f0eecaca80..5c7b583cba18 100755
> --- a/tools/testing/selftests/cgroup/test_cpuset_prs.sh
> +++ b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
> @@ -378,7 +378,7 @@ TEST_MATRIX=(
>  	# Remote partition offline tests
>  	"   C0-3    C1-3  C2-3     .    X2-3   X2-3 X2-3:P2:O2=0 .   0 A1:0-1|A2:1|A3:3 A1:P0|A3:P2 2-3"
>  	"   C0-3    C1-3  C2-3     .    X2-3   X2-3 X2-3:P2:O2=0 O2=1 0 A1:0-1|A2:1|A3:2-3 A1:P0|A3:P2 2-3"
> -	"   C0-3    C1-3   C3      .    X2-3   X2-3    P2:O3=0   .   0 A1:0-2|A2:1-2|A3: A1:P0|A3:P2 3"
> +	"   C0-3    C1-3   C3      .    X2-3   X2-3    P2:O3=0   .   0 A1:0-2|A2:1-2|A3:1-2 A1:P0|A3:P-2 3|"
>  	"   C0-3    C1-3   C3      .    X2-3   X2-3   T:P2:O3=0  .   0 A1:0-2|A2:1-2|A3:1-2 A1:P0|A3:P-2 3|"
>  
>  	# An invalidated remote partition cannot self-recover from hotplug


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH-next v4 6/6] cgroup/cpuset: Fix lockdep_is_cpuset_held()
  2026-11-16  1:23 ` [PATCH-next v4 6/6] cgroup/cpuset: Fix lockdep_is_cpuset_held() Waiman Long
  2026-11-16  7:24   ` Ridong Chen
@ 2026-11-16 10:34   ` Guopeng Zhang
  1 sibling, 0 replies; 18+ messages in thread
From: Guopeng Zhang @ 2026-11-16 10:34 UTC (permalink / raw)
  To: Waiman Long, Ridong Chen, Tejun Heo, Johannes Weiner,
	Michal Koutný, Shuah Khan
  Cc: cgroups, linux-kernel, linux-kselftest



在 2026/11/16 09:23, Waiman Long 写道:
> Commit a84097e625f2 ("cgroup/cpuset: Call housekeeping_update()
> without holding cpus_read_lock") changes lockdep_is_cpuset_held()
> to return true if either cpuset_mutex or cpuset_top_mutex is held.
> lockdep_is_cpuset_held() is used by housekeeping_dereference_check()
> as one of the conditions that rcu_read_lock() is not needed to
> access HK_TYPE_DOMAIN cpumask that can be changed at run time.
> In reality, it should return true only if cpuset_top_mutex is held as
> housekeeping_update() is called with only cpuset_top_mutex held.
> 
> If only cpuset_mutex is held, it is possible that there can be a parallel
> housekeeping_update() call leading to a potential use-after-free situation
> when HK_TYPE_DOMAIN housekeeping cpumask is being accessed. It also
> means that rcu_read_lock will be required if only cpuset_mutex is being
> held like when calling from hotplug, scheduler or cgroup.
> 
> Fixes: a84097e625f2 ("cgroup/cpuset: Call housekeeping_update() without holding cpus_read_lock")
> Signed-off-by: Waiman Long <longman@redhat.com>
> ---
>  kernel/cgroup/cpuset-v1.c | 16 ++++++++++------
>  kernel/cgroup/cpuset.c    |  9 +++++++--
>  2 files changed, 17 insertions(+), 8 deletions(-)
> 
> diff --git a/kernel/cgroup/cpuset-v1.c b/kernel/cgroup/cpuset-v1.c
> index 7c7f2887dfaf..6bf6da639fec 100644
> --- a/kernel/cgroup/cpuset-v1.c
> +++ b/kernel/cgroup/cpuset-v1.c
> @@ -686,6 +686,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,
>  	int csn;		/* how many cpuset ptrs in csa so far */
>  	int i, j;		/* indices for partition finding loops */
>  	cpumask_var_t *doms;	/* resulting partition; i.e. sched domains */
> +	cpumask_var_t hk_cpus;	/* HK_TYPE_DOMAIN housekeeping CPUs */
>  	struct sched_domain_attr *dattr;  /* attributes for custom domains */
>  	int ndoms = 0;		/* number of sched domains in result */
>  	int nslot;		/* next empty doms[] struct cpumask slot */
> @@ -698,6 +699,11 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,
>  	dattr = NULL;
>  	csa = NULL;
>  
> +	alloc_cpumask_var(&hk_cpus, GFP_KERNEL);
> +	scoped_guard(rcu) {
> +		cpumask_copy(hk_cpus, housekeeping_cpumask(HK_TYPE_DOMAIN));
> +	}
> +

Can alloc_cpumask_var() fail? Should we add the following check?

	if (!alloc_cpumask_var(&hk_cpus, GFP_KERNEL))
		goto done;

Thanks,
Guopeng

>  	/* Special case for the 99% of systems with one, full, sched domain */
>  	if (is_sched_load_balance(&top_cpuset)) {
>  		ndoms = 1;
> @@ -710,9 +716,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,
>  			*dattr = SD_ATTR_INIT;
>  			update_domain_attr_tree(dattr, &top_cpuset);
>  		}
> -		cpumask_and(doms[0], top_cpuset.effective_cpus,
> -			    housekeeping_cpumask(HK_TYPE_DOMAIN));
> -
> +		cpumask_and(doms[0], top_cpuset.effective_cpus, hk_cpus);
>  		goto done;
>  	}
>  
> @@ -736,8 +740,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,
>  		 */
>  		if (!cpumask_empty(cp->cpus_allowed) &&
>  		    !(is_sched_load_balance(cp) &&
> -		      cpumask_intersects(cp->cpus_allowed,
> -					 housekeeping_cpumask(HK_TYPE_DOMAIN))))
> +		      cpumask_intersects(cp->cpus_allowed, hk_cpus)))
>  			continue;
>  
>  		if (is_sched_load_balance(cp) &&
> @@ -794,7 +797,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,
>  						*(dattr + nslot) = SD_ATTR_INIT;
>  				}
>  				cpumask_or(dp, dp, csa[j]->effective_cpus);
> -				cpumask_and(dp, dp, housekeeping_cpumask(HK_TYPE_DOMAIN));
> +				cpumask_and(dp, dp, hk_cpus);
>  				if (dattr)
>  					update_domain_attr_tree(dattr + nslot, csa[j]);
>  			}
> @@ -806,6 +809,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,
>  
>  done:
>  	kfree(csa);
> +	free_cpumask_var(hk_cpus);
>  
>  	/*
>  	 * Fallback to the default domain if kmalloc() failed.
> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
> index 64cbcbdbf6da..d18439e13bce 100644
> --- a/kernel/cgroup/cpuset.c
> +++ b/kernel/cgroup/cpuset.c
> @@ -343,10 +343,13 @@ void cpuset_full_unlock(void)
>  }
>  
>  #ifdef CONFIG_LOCKDEP
> +/*
> + * Holding cpuset_top_mutex will guarantee that a parallel
> + * housekeeping_update() call modifying housekeeping cpumasks is not possible.
> + */
>  bool lockdep_is_cpuset_held(void)
>  {
> -	return lockdep_is_held(&cpuset_mutex) ||
> -	       lockdep_is_held(&cpuset_top_mutex);
> +	return lockdep_is_held(&cpuset_top_mutex);
>  }
>  #endif
>  
> @@ -1416,8 +1419,10 @@ static bool prstate_housekeeping_conflict(int new_prs, int parent_prs,
>  		}
>  	}
>  
> +	rcu_read_lock();
>  	cpumask_and(full_hk_cpus, housekeeping_cpumask(HK_TYPE_KERNEL_NOISE),
>  		    housekeeping_cpumask(HK_TYPE_DOMAIN));
> +	rcu_read_unlock();
>  	cpumask_andnot(full_hk_cpus, full_hk_cpus, isolated_cpus);
>  	cpumask_and(full_hk_cpus, full_hk_cpus, cpu_active_mask);
>  	res = !cpumask_weight_andnot(full_hk_cpus, new_cpus);


^ permalink raw reply	[flat|nested] 18+ messages in thread

end of thread, other threads:[~2026-11-16 10:34 UTC | newest]

Thread overview: 18+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-16  1:23 [PATCH-next v4 0/6] cgroup/cpuset: More fix patches Waiman Long
2026-11-16  1:23 ` [PATCH-next v4 1/6] cgroup/cpuset: Fix incorrect partition invalidation logic in cpuset_hotplug_update_tasks() Waiman Long
2026-11-16 10:19   ` Guopeng Zhang
2026-11-16  1:23 ` [PATCH-next v4 2/6] cgroup/cpuset: excpus parameter of remote_cpus_update() must have at least 1 active CPU Waiman Long
2026-11-16  3:07   ` Ridong Chen
2026-11-16  3:34     ` Waiman Long
2026-11-16  7:37       ` Ridong Chen
2026-11-16  1:23 ` [PATCH-next v4 3/6] cgroup/cpuset: Check remote partition underneath before making an invalid partition valid Waiman Long
2026-11-16  3:47   ` Ridong Chen
2026-11-16  4:21     ` Waiman Long
2026-11-16  1:23 ` [PATCH-next v4 4/6] cgroup/cpuset: Revert "remove global remote_children list" Waiman Long
2026-11-16  1:23 ` [PATCH-next v4 5/6] cgroup/cpuset: Handle remote partitions before local partitions in cpuset_handle_hotplug() Waiman Long
2026-11-16  7:08   ` Ridong Chen
2026-11-16  8:56   ` kernel test robot
2026-11-16 10:33   ` Guopeng Zhang
2026-11-16  1:23 ` [PATCH-next v4 6/6] cgroup/cpuset: Fix lockdep_is_cpuset_held() Waiman Long
2026-11-16  7:24   ` Ridong Chen
2026-11-16 10:34   ` Guopeng Zhang

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).