From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 73A502E7398 for ; Sat, 11 Jul 2026 02:06:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783735598; cv=none; b=mxloD7VRU13aKjV8jiUSnBY0A+FOQCNnuBWIyEQcMfqaqjUXzUpRGnFnWSRl2nVJYYAHvA+TiiIFa5XbrqwQs2DZP+PgRaMQOo4c4lINo7yXHIRwH5doVQUW/GWfu1qkMfY2kXJ5WTCib0Vnsbgr0C1z5mpRXvKFw2tggPmQf6U= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783735598; c=relaxed/simple; bh=d5x5hxSCNVRHC/vROq1y9apSYTx0rHEbfnGJ+dEkbas=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=imokH7knML2RwgW9LbJwiuF1JzZmHbcIOKNXZYKfOZ4GYhjClOsmNlIhjBRj9S8L+7Z4+VHonYNSOxlitd6UURyO42f492m+pICeRE7Vsoz9hmuzwqbYxtk1YcT9YaWPyGF+rrPhM5ysH1/Gs6Zq1RHIJCj7+kdc10xlrgcRulI= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=AZctTN90; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="AZctTN90" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783735595; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=w7oN7C4SdS1DdzK7LkW8aF1xWEBreX2hjWsU9u4vHF0=; b=AZctTN906c3GXIf6g4lELlSXiFBlFl86CEZsPP2SdMMsJggBSywxINiU2qJW0e9bBh8GJd OQBLTOi11D+zz0eizC7xU5E0JHjKc9anmzPvoCUFjRASMxj1Mm03YMmlobc5Rl+w5fHkts WSjroraRtV01NwBaA3CL+351deb/G48= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-48-0Id7R01XOZiJ9jJf86e2XA-1; Fri, 10 Jul 2026 22:06:32 -0400 X-MC-Unique: 0Id7R01XOZiJ9jJf86e2XA-1 X-Mimecast-MFC-AGG-ID: 0Id7R01XOZiJ9jJf86e2XA_1783735590 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 65FD1180025F; Sat, 11 Jul 2026 02:06:30 +0000 (UTC) Received: from llong-thinkpadp1gen5.rmtusnh.csb (unknown [10.22.88.253]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 03CF7195419F; Sat, 11 Jul 2026 02:06:28 +0000 (UTC) From: Waiman Long To: Ridong Chen , Tejun Heo , Johannes Weiner , =?UTF-8?q?Michal=20Koutn=C3=BD?= , Shuah Khan Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Waiman Long Subject: [PATCH-next 2/3] cgroup/cpuset: Support multiple destination cpusets for cpuset_*attach() Date: Fri, 10 Jul 2026 22:05:39 -0400 Message-ID: <20260711020540.176740-3-longman@redhat.com> In-Reply-To: <20260711020540.176740-1-longman@redhat.com> References: <20260711020540.176740-1-longman@redhat.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 The only case where the cgroup_taskset structure requires task migration to multiple cpusets is when enabling a cpuset controller in cgroup v2 where the newly created child cpusets inherits the same effective CPUs and memory nodes from the parent. In that case, task migration can happen directly with no update to tasks' CPU and memory nodes assignment and no further work needed from the cpuset side except updating nr_deadline_tasks when DL tasks are involved and setting old_mems_allowed in the child cpusets. Do that by tracking all the destination cpusets with a new dst_cs_head singly linked list. The reset_migrate_dl_data() function is integrated into clear_attach_data() so that it can be used for both source and destination cpusets. A warning will be printed if there are multiple destination cpusets but it is not on default hierarchy or when the CPUs or memory nodes change. Signed-off-by: Waiman Long --- kernel/cgroup/cpuset.c | 99 ++++++++++++++++++++++++------------------ 1 file changed, 56 insertions(+), 43 deletions(-) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index d99184ec60b5..68943671d97d 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -366,10 +366,12 @@ static struct { bool cpus_updated; bool mems_updated; bool task_work_queued; + bool many_dest_cs; /* Have many destination cpusets */ struct cpuset *old_cs; /* Source cpuset */ nodemask_t nodemask_to; } attach_ctx; static LLIST_HEAD(src_cs_head); +static LLIST_HEAD(dst_cs_head); /* * Wait if task attach is in progress until it is done and then acquire @@ -3052,6 +3054,9 @@ static int cpuset_can_attach_check(struct cpuset *cs, struct cpuset *oldcs, if (!llist_on_list(&oldcs->attach_node)) llist_add(&oldcs->attach_node, &src_cs_head); + if (!llist_on_list(&cs->attach_node)) + llist_add(&cs->attach_node, &dst_cs_head); + cpus_updated = !cpumask_equal(cs->effective_cpus, oldcs->effective_cpus); mems_updated = !nodes_equal(cs->effective_mems, oldcs->effective_mems); @@ -3080,35 +3085,31 @@ static int cpuset_can_attach_check(struct cpuset *cs, struct cpuset *oldcs, return 0; } -static int cpuset_reserve_dl_bw(struct cpuset *cs) +static int cpuset_reserve_dl_bw(void) { + struct cpuset *cs; int cpu, ret; - if (!cs->sum_migrate_dl_bw) - return 0; + llist_for_each_entry(cs, dst_cs_head.first, attach_node) { + if (!cs->sum_migrate_dl_bw) + continue; - cpu = cpumask_any_and(cpu_active_mask, cs->effective_cpus); - if (unlikely(cpu >= nr_cpu_ids)) - return -EINVAL; + cpu = cpumask_any_and(cpu_active_mask, cs->effective_cpus); + if (unlikely(cpu >= nr_cpu_ids)) + return -EINVAL; - ret = dl_bw_alloc(cpu, cs->sum_migrate_dl_bw); - if (ret) - return ret; + ret = dl_bw_alloc(cpu, cs->sum_migrate_dl_bw); + if (ret) + return ret; - cs->dl_bw_cpu = cpu; + cs->dl_bw_cpu = cpu; + } return 0; } -static void reset_migrate_dl_data(struct cpuset *cs) -{ - cs->nr_migrate_dl_tasks = 0; - cs->sum_migrate_dl_bw = 0; - cs->dl_bw_cpu = -1; -} - /* * Clear and optionally apply (@cancel is false) the attach related data in the - * source cpusets. + * source or destination cpuset. */ static void clear_attach_data(struct llist_head *head, bool cancel) { @@ -3120,7 +3121,11 @@ static void clear_attach_data(struct llist_head *head, bool cancel) if (cs->nr_migrate_dl_tasks) { if (!cancel) atomic_add(cs->nr_migrate_dl_tasks, &cs->nr_deadline_tasks); + else if (cs->dl_bw_cpu >= 0) /* && cacnel */ + dl_bw_free(cs->dl_bw_cpu, cs->sum_migrate_dl_bw); cs->nr_migrate_dl_tasks = 0; + cs->sum_migrate_dl_bw = 0; + cs->dl_bw_cpu = -1; } } } @@ -3142,6 +3147,7 @@ static int cpuset_can_attach(struct cgroup_taskset *tset) mutex_lock(&cpuset_mutex); attach_ctx.cpus_updated = false; attach_ctx.mems_updated = false; + attach_ctx.many_dest_cs = false; /* * The attach_ctx.old_cs is used mainly by cpuset_migrate_mm() to get @@ -3166,9 +3172,13 @@ static int cpuset_can_attach(struct cgroup_taskset *tset) * actual migration wrt cpuset. */ cgroup_taskset_for_each(task, css, tset) { + struct cpuset *new_cs = css_cs(css); struct cpuset *new_oldcs = task_cs(task); - if (new_oldcs != oldcs) { + if ((new_oldcs != oldcs) || (new_cs != cs)) { + if (new_cs != cs) + attach_ctx.many_dest_cs = true; + cs = new_cs; oldcs = new_oldcs; if (oldcs == cs) continue; @@ -3207,12 +3217,28 @@ static int cpuset_can_attach(struct cgroup_taskset *tset) } } - ret = cpuset_reserve_dl_bw(cs); + /* + * The only case where there are multiple destination cpusets for + * task migration is when enabling a v2 cpuset controllers where + * tasks will be migrated to multiple child cpusets from a parent + * cpuset with the same effective CPUs and memory nodes. IOW, + * both attach_cpus_updated and attach_mems_updated should be false. + * If not, it is a condition that the current code cannot handle. + * Print a warning and abort the attach operation as further code + * change may be needed. + */ + if (WARN_ON_ONCE(attach_ctx.many_dest_cs && (!cpuset_v2() || + attach_ctx.cpus_updated || attach_ctx.mems_updated))) { + ret = -EINVAL; + goto out_unlock; + } + + ret = cpuset_reserve_dl_bw(); out_unlock: if (ret) { - reset_migrate_dl_data(cs); /* Destination cpuset only */ clear_attach_data(&src_cs_head, true); + clear_attach_data(&dst_cs_head, true); } else { attach_ctx.in_progress++; } @@ -3223,22 +3249,10 @@ static int cpuset_can_attach(struct cgroup_taskset *tset) static void cpuset_cancel_attach(struct cgroup_taskset *tset) { - struct cgroup_subsys_state *css; - struct cpuset *cs; - - cgroup_taskset_first(tset, &css); - cs = css_cs(css); - mutex_lock(&cpuset_mutex); dec_attach_in_progress_locked(); clear_attach_data(&src_cs_head, true); - - if (cs->dl_bw_cpu >= 0) - dl_bw_free(cs->dl_bw_cpu, cs->sum_migrate_dl_bw); - - if (cs->nr_migrate_dl_tasks) - reset_migrate_dl_data(cs); - + clear_attach_data(&dst_cs_head, true); mutex_unlock(&cpuset_mutex); } @@ -3321,25 +3335,24 @@ static void cpuset_attach(struct cgroup_taskset *tset) * In the default hierarchy, enabling cpuset in the child cgroups * will trigger a cpuset_attach() call with no change in effective cpus * and mems. In that case, we can optimize out by skipping the task - * iteration and update. + * iteration and the destination cpuset list is iterated to set + * old_mems_allowed. */ - if (cpuset_v2() && !attach_ctx.cpus_updated && !attach_ctx.mems_updated) + if (cpuset_v2() && !attach_ctx.cpus_updated && !attach_ctx.mems_updated) { + llist_for_each_entry(cs, dst_cs_head.first, attach_node) + cs->old_mems_allowed = attach_ctx.nodemask_to; goto out; + } cgroup_taskset_for_each(task, css, tset) cpuset_attach_task(cs, task); -out: if (attach_ctx.task_work_queued) schedule_flush_migrate_mm(); cs->old_mems_allowed = attach_ctx.nodemask_to; - - if (cs->nr_migrate_dl_tasks) { - atomic_add(cs->nr_migrate_dl_tasks, &cs->nr_deadline_tasks); - reset_migrate_dl_data(cs); - } - +out: clear_attach_data(&src_cs_head, false); + clear_attach_data(&dst_cs_head, false); dec_attach_in_progress_locked(); mutex_unlock(&cpuset_mutex); -- 2.55.0