The Linux Kernel Mailing List
 help / color / mirror / Atom feed
From: sashiko-bot@kernel.org
To: "Mathieu Desnoyers" <mathieu.desnoyers@efficios.com>
Cc: linux-rt-devel@lists.linux.dev, tj@kernel.org,
	hannes@cmpxchg.org, mkoutny@suse.com,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH v6 23/37] cgroup/cpuset: Reserve the bandwidth in the root domain a task moves to
Date: Mon, 16 Nov 2026 19:56:36 +0000	[thread overview]
Message-ID: <sashiko-outbox-217626@kernel.org> (raw)
In-Reply-To: <20261116190549.230119-24-mathieu.desnoyers@efficios.com>

Thank you for your contribution! Sashiko AI review found 2 potential
issues to consider:
- [High] sched/deadline: unmasked offline CPUs in
  __dl_task_bw_move_cpu() span check
- [Medium] cgroup/cpuset: stale dl_bw_resv_cpu left when cpuset_attach()
  skips task loop
--

commit 1fc4cfdda6a5d5f5e22bff0342024fd03d578a7c
Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

cgroup/cpuset: Reserve the bandwidth in the root domain a task moves to

This commit reserves and moves a deadline task's bandwidth using the
specific destination CPU recorded in task->dl.dl_bw_resv_cpu during
cpuset_can_attach(), keeping per-CPU reservation sums in cpuset_dl_bw so
that attaches to cpusets spanning multiple root domains account bandwidth
in the root domain where the task actually runs.

> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
> index 2f39c3d12550e..0d55f092d9cbb 100644
> --- a/kernel/cgroup/cpuset.c
> +++ b/kernel/cgroup/cpuset.c
[ ... ]
> @@ -3236,15 +3258,20 @@ static int cpuset_can_attach(struct cgroup_taskset *tset)
>  				goto out_unlock;
>  		}
>  
> -		if (dl_task(task)) {
> -			/*
> -			 * Count all migrating DL tasks for cpuset task accounting.
> -			 * Only tasks that need a root-domain bandwidth move
> -			 * contribute to sum_migrate_dl_bw.
> -			 */
> -			cs->nr_migrate_dl_tasks++;
> -			if (dl_task_needs_bw_move(task, cs->effective_cpus))
> -				cs->sum_migrate_dl_bw += task->dl.dl_bw;
> +		if (!dl_task(task))
> +			continue;
> +		/*
> +		 * Sum the bandwidth of the deadline tasks that move to another
> +		 * root domain, per CPU that the move records. That CPU depends
> +		 * on the CPUs the task gets. A cpuset can span more than one
> +		 * root domain, and the affinity of a task can keep it in any
> +		 * of them.
> +		 */
> +		cpuset_attach_cpus(cs, task, cpus_attach);
> +		cpu = dl_task_bw_reserve_cpu(task, cpus_attach);
> +		if (cpu >= 0) {
> +			per_cpu_ptr(&cpuset_dl_bw, cpu)->sum += task->dl.dl_bw;
> +			cpumask_set_cpu(cpu, dl_attach_cpus);
>  		}
>  	}
[ ... ]
> @@ -3407,8 +3446,9 @@ static void cpuset_attach(struct cgroup_taskset *tset)
>  		schedule_flush_migrate_mm();
>  	cs->old_mems_allowed = attach_ctx.nodemask_to;
>  out:
> -	clear_attach_data(&src_cs_head, false);
> -	clear_attach_data(&dst_cs_head, false);
> +	clear_attach_data(&src_cs_head);
> +	clear_attach_data(&dst_cs_head);
> +	cpuset_free_dl_bw();
>  	dec_attach_in_progress_locked();
>  
>  	mutex_unlock(&cpuset_mutex);

[Severity: Medium]
Does cpuset_attach() leave a stale task->dl.dl_bw_resv_cpu behind when it
takes the !attach_ctx.cpus_updated && !attach_ctx.mems_updated fast path?

In cpuset_can_attach(), dl_task_bw_reserve_cpu() runs whenever oldcs != cs,
even if oldcs and cs have identical effective_cpus and effective_mems (for
example, when enabling +cpuset in cgroup.subtree_control or moving a task
between v2 cgroups with the same effective masks).  If the task's bandwidth
is currently counted in another root domain, dl_task_bw_reserve_cpu() sets
task->dl.dl_bw_resv_cpu = cpu.  In cpuset_attach(), the fast path:

kernel/cgroup/cpuset.c:cpuset_attach() {
    ...
	if (cpuset_v2() && !attach_ctx.cpus_updated && !attach_ctx.mems_updated) {
		llist_for_each_entry(cs, dst_cs_head.first, attach_node)
			cs->old_mems_allowed = attach_ctx.nodemask_to;
		goto out;
	}
    ...
}

jumps past cpuset_attach_task() to out:, where cpuset_free_dl_bw() releases
the reservation in cpuset_dl_bw, but dl_task_bw_unreserve() is never called
on the tasks in tset.

If that task is later part of a v2 threaded subtree attach where oldcs == cs
for this thread while another thread in the taskset has oldcs != cs and
cpus_updated == true, cpuset_can_attach() skips this task, and
cpuset_attach_task() consumes the stale dl_bw_resv_cpu.  That moves the
task's bandwidth without a destination reservation and adds moved to
per_cpu(cpuset_dl_bw, cpu).moved, corrupting the root domain accounting and
underflowing b->sum - b->moved in cpuset_free_dl_bw().

> diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
> index 5503820c5b26c..b43106a590256 100644
> --- a/kernel/sched/deadline.c
> +++ b/kernel/sched/deadline.c
[ ... ]
> @@ -3657,10 +3676,64 @@ bool dl_task_needs_bw_move(struct task_struct *p,
>  	 * their bandwidth: dl_add_task_root_domain() leaves them out.
>  	 */
>  	if (!dl_task(p) || dl_entity_is_special(&p->dl) || !p->dl.dl_bw)
> -		return false;
> +		return -1;
>  
>  	cpu = dl_task_root_domain_cpu(p);
> -	return !cpumask_intersects(cpu_rq(cpu)->rd->span, new_mask);
> +	if (cpumask_intersects(cpu_rq(cpu)->rd->span, new_mask))
> +		return -1;
> +	/*
> +	 * A task moved asleep is still on a CPU of the root domain it left.
> +	 * If the new mask holds that CPU, the task wakes up there.
> +	 */
> +	cpu = task_cpu(p);
> +	if (cpu_active(cpu) && cpumask_test_cpu(cpu, new_mask))
> +		return cpu;
> +	cpu = cpumask_first_and_and(new_mask, cpu_active_mask,
> +				    housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT));
> +	if (cpu >= nr_cpu_ids)
> +		cpu = cpumask_any_and(new_mask, cpu_active_mask);
> +	return cpu < nr_cpu_ids ? (int)cpu : -1;
> +}

[Severity: High]
Can offline CPUs in def_root_domain.span cause __dl_task_bw_move_cpu() to
falsely return -1 when attaching a task to top_cpuset?

Every offline possible CPU is attached to def_root_domain (in sched_init()
and detach_destroy_domains()), as are CPUs isolated via isolcpus=domain or
isolated cpuset partitions.  When a task is attached to top_cpuset,
cpuset_attach_cpus() sets new_mask to task_cpu_possible_mask(task) &
~subpartitions_cpus, which includes all offline possible CPUs.

Because cpumask_intersects(cpu_rq(cpu)->rd->span, new_mask) does not mask
against cpu_active_mask, any system with an offline CPU in
cpu_possible_mask will see def_root_domain.span intersect new_mask on that
offline CPU (unless filtered out by p->user_cpus_ptr).  Moving a deadline
task whose bandwidth is in def_root_domain back to top_cpuset will then
return -1 here, skipping both the destination reservation and the bandwidth
move out of def_root_domain.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261116190549.230119-1-mathieu.desnoyers@efficios.com?part=23

  reply	other threads:[~2026-11-16 19:56 UTC|newest]

Thread overview: 53+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-16 19:05 [PATCH v6 00/37] sched/deadline: bandwidth accounting fixes and selftests Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 01/37] sched: Refuse SCHED_FLAG_KEEP_PARAMS that would change the policy or the class Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 02/37] sched/deadline: Do not admit bandwidth for parameters that are kept Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 03/37] sched: Let a task boosted by a deadline waiter change its policy Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 04/37] sched/deadline: Do not divide by zero active CPUs of a root domain Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 05/37] sched/deadline: Refuse dl_server debugfs writes while the rq is offline Mathieu Desnoyers
2026-11-16 19:30   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 06/37] sched: Take cpuset_mutex for a deadline policy change only with pi Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 07/37] sched/deadline: Do not lose a throttled task switched out and back Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 08/37] sched/deadline: Do not replenish a boost from cleared parameters Mathieu Desnoyers
2026-11-16 19:42   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 09/37] sched/deadline: Reset the deadline and the runtime with the parameters Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 10/37] sched/deadline: Reset the donor's runtime at a deboost in the class Mathieu Desnoyers
2026-11-16 19:38   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 11/37] sched/deadline: Release the bandwidth of a task that leaves the class asleep Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 12/37] sched/deadline: Reject SCHED_DEADLINE on exited tasks Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 13/37] sched/deadline: Keep departing bandwidth across a root domain rebuild Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 14/37] sched/deadline: Fix this_bw when a task is changed during a remote wakeup Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 15/37] sched/deadline: Rebuild the root domains' accounting from the task list Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 16/37] sched/deadline: Record the root domain holding a task's bandwidth Mathieu Desnoyers
2026-11-16 19:44   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 17/37] sched/deadline: Leave a dying task out of a rebuild of the accounting Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 18/37] sched/deadline: Check SCHED_DEADLINE affinity under the rq lock Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 19/37] sched/deadline: Move a task's bandwidth only where room was made Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 20/37] cgroup/cpuset: Wait for an attach before rebuilding the sched domains Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 21/37] sched: Make deadline policy and affinity changes wait for a cpuset attach Mathieu Desnoyers
2026-11-16 19:31   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 22/37] cgroup/cpuset: Give back the bandwidth an attach did not move Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 23/37] cgroup/cpuset: Reserve the bandwidth in the root domain a task moves to Mathieu Desnoyers
2026-11-16 19:56   ` sashiko-bot [this message]
2026-11-16 19:05 ` [PATCH v6 24/37] sched/deadline: Make a rebuild count a task in the root domain it runs in Mathieu Desnoyers
2026-11-16 20:01   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 25/37] sched/deadline: Move a task's bandwidth to " Mathieu Desnoyers
2026-11-16 19:47   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 26/37] sched/deadline: Keep a task in the root domain that counts its bandwidth Mathieu Desnoyers
2026-11-16 19:50   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 27/37] sched/deadline: Keep the last CPU a task has in its root domain Mathieu Desnoyers
2026-11-16 19:51   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 28/37] cgroup/cpuset: Refuse to take a deadline task out of " Mathieu Desnoyers
2026-11-16 20:08   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 29/37] sched/deadline: Cancel the replenishment timer with the parameters Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 30/37] sched/deadline: Count the bandwidth a boosted task is given Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 31/37] sched/deadline: Start a period for a task that enters the class blocked Mathieu Desnoyers
2026-11-16 20:12   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 32/37] sched/deadline: Check the runtime of a task that comes back runnable Mathieu Desnoyers
2026-11-16 20:08   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 33/37] sched/deadline: Check the runtime of a task whose parameters change Mathieu Desnoyers
2026-11-16 20:04   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 34/37] sched/debug: Show the deadline bandwidth of each runqueue Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 35/37] selftests/sched: Add a test of SCHED_DEADLINE policy round trips Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 36/37] selftests/sched: Test SCHED_DEADLINE bandwidth across a root domain rebuild Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 37/37] selftests/sched: Add a test of SCHED_DEADLINE bandwidth accounting Mathieu Desnoyers
2026-11-16 19:49   ` sashiko-bot

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=sashiko-outbox-217626@kernel.org \
    --to=sashiko-bot@kernel.org \
    --cc=hannes@cmpxchg.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rt-devel@lists.linux.dev \
    --cc=mathieu.desnoyers@efficios.com \
    --cc=mkoutny@suse.com \
    --cc=sashiko-reviews@lists.linux.dev \
    --cc=tj@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox