All of lore.kernel.org
 help / color / mirror / Atom feed
From: sashiko-bot@kernel.org
To: "Mathieu Desnoyers" <mathieu.desnoyers@efficios.com>
Cc: linux-rt-devel@lists.linux.dev, tj@kernel.org,
	hannes@cmpxchg.org, mkoutny@suse.com,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH v6 23/37] cgroup/cpuset: Reserve the bandwidth in the root domain a task moves to
Date: Mon, 16 Nov 2026 19:56:36 +0000	[thread overview]
Message-ID: <sashiko-outbox-217626@kernel.org> (raw)
In-Reply-To: <20261116190549.230119-24-mathieu.desnoyers@efficios.com>

Thank you for your contribution! Sashiko AI review found 2 potential
issues to consider:
- [High] sched/deadline: unmasked offline CPUs in
  __dl_task_bw_move_cpu() span check
- [Medium] cgroup/cpuset: stale dl_bw_resv_cpu left when cpuset_attach()
  skips task loop
--

commit 1fc4cfdda6a5d5f5e22bff0342024fd03d578a7c
Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

cgroup/cpuset: Reserve the bandwidth in the root domain a task moves to

This commit reserves and moves a deadline task's bandwidth using the
specific destination CPU recorded in task->dl.dl_bw_resv_cpu during
cpuset_can_attach(), keeping per-CPU reservation sums in cpuset_dl_bw so
that attaches to cpusets spanning multiple root domains account bandwidth
in the root domain where the task actually runs.

> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
> index 2f39c3d12550e..0d55f092d9cbb 100644
> --- a/kernel/cgroup/cpuset.c
> +++ b/kernel/cgroup/cpuset.c
[ ... ]
> @@ -3236,15 +3258,20 @@ static int cpuset_can_attach(struct cgroup_taskset *tset)
>  				goto out_unlock;
>  		}
>  
> -		if (dl_task(task)) {
> -			/*
> -			 * Count all migrating DL tasks for cpuset task accounting.
> -			 * Only tasks that need a root-domain bandwidth move
> -			 * contribute to sum_migrate_dl_bw.
> -			 */
> -			cs->nr_migrate_dl_tasks++;
> -			if (dl_task_needs_bw_move(task, cs->effective_cpus))
> -				cs->sum_migrate_dl_bw += task->dl.dl_bw;
> +		if (!dl_task(task))
> +			continue;
> +		/*
> +		 * Sum the bandwidth of the deadline tasks that move to another
> +		 * root domain, per CPU that the move records. That CPU depends
> +		 * on the CPUs the task gets. A cpuset can span more than one
> +		 * root domain, and the affinity of a task can keep it in any
> +		 * of them.
> +		 */
> +		cpuset_attach_cpus(cs, task, cpus_attach);
> +		cpu = dl_task_bw_reserve_cpu(task, cpus_attach);
> +		if (cpu >= 0) {
> +			per_cpu_ptr(&cpuset_dl_bw, cpu)->sum += task->dl.dl_bw;
> +			cpumask_set_cpu(cpu, dl_attach_cpus);
>  		}
>  	}
[ ... ]
> @@ -3407,8 +3446,9 @@ static void cpuset_attach(struct cgroup_taskset *tset)
>  		schedule_flush_migrate_mm();
>  	cs->old_mems_allowed = attach_ctx.nodemask_to;
>  out:
> -	clear_attach_data(&src_cs_head, false);
> -	clear_attach_data(&dst_cs_head, false);
> +	clear_attach_data(&src_cs_head);
> +	clear_attach_data(&dst_cs_head);
> +	cpuset_free_dl_bw();
>  	dec_attach_in_progress_locked();
>  
>  	mutex_unlock(&cpuset_mutex);

[Severity: Medium]
Does cpuset_attach() leave a stale task->dl.dl_bw_resv_cpu behind when it
takes the !attach_ctx.cpus_updated && !attach_ctx.mems_updated fast path?

In cpuset_can_attach(), dl_task_bw_reserve_cpu() runs whenever oldcs != cs,
even if oldcs and cs have identical effective_cpus and effective_mems (for
example, when enabling +cpuset in cgroup.subtree_control or moving a task
between v2 cgroups with the same effective masks).  If the task's bandwidth
is currently counted in another root domain, dl_task_bw_reserve_cpu() sets
task->dl.dl_bw_resv_cpu = cpu.  In cpuset_attach(), the fast path:

kernel/cgroup/cpuset.c:cpuset_attach() {
    ...
	if (cpuset_v2() && !attach_ctx.cpus_updated && !attach_ctx.mems_updated) {
		llist_for_each_entry(cs, dst_cs_head.first, attach_node)
			cs->old_mems_allowed = attach_ctx.nodemask_to;
		goto out;
	}
    ...
}

jumps past cpuset_attach_task() to out:, where cpuset_free_dl_bw() releases
the reservation in cpuset_dl_bw, but dl_task_bw_unreserve() is never called
on the tasks in tset.

If that task is later part of a v2 threaded subtree attach where oldcs == cs
for this thread while another thread in the taskset has oldcs != cs and
cpus_updated == true, cpuset_can_attach() skips this task, and
cpuset_attach_task() consumes the stale dl_bw_resv_cpu.  That moves the
task's bandwidth without a destination reservation and adds moved to
per_cpu(cpuset_dl_bw, cpu).moved, corrupting the root domain accounting and
underflowing b->sum - b->moved in cpuset_free_dl_bw().

> diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
> index 5503820c5b26c..b43106a590256 100644
> --- a/kernel/sched/deadline.c
> +++ b/kernel/sched/deadline.c
[ ... ]
> @@ -3657,10 +3676,64 @@ bool dl_task_needs_bw_move(struct task_struct *p,
>  	 * their bandwidth: dl_add_task_root_domain() leaves them out.
>  	 */
>  	if (!dl_task(p) || dl_entity_is_special(&p->dl) || !p->dl.dl_bw)
> -		return false;
> +		return -1;
>  
>  	cpu = dl_task_root_domain_cpu(p);
> -	return !cpumask_intersects(cpu_rq(cpu)->rd->span, new_mask);
> +	if (cpumask_intersects(cpu_rq(cpu)->rd->span, new_mask))
> +		return -1;
> +	/*
> +	 * A task moved asleep is still on a CPU of the root domain it left.
> +	 * If the new mask holds that CPU, the task wakes up there.
> +	 */
> +	cpu = task_cpu(p);
> +	if (cpu_active(cpu) && cpumask_test_cpu(cpu, new_mask))
> +		return cpu;
> +	cpu = cpumask_first_and_and(new_mask, cpu_active_mask,
> +				    housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT));
> +	if (cpu >= nr_cpu_ids)
> +		cpu = cpumask_any_and(new_mask, cpu_active_mask);
> +	return cpu < nr_cpu_ids ? (int)cpu : -1;
> +}

[Severity: High]
Can offline CPUs in def_root_domain.span cause __dl_task_bw_move_cpu() to
falsely return -1 when attaching a task to top_cpuset?

Every offline possible CPU is attached to def_root_domain (in sched_init()
and detach_destroy_domains()), as are CPUs isolated via isolcpus=domain or
isolated cpuset partitions.  When a task is attached to top_cpuset,
cpuset_attach_cpus() sets new_mask to task_cpu_possible_mask(task) &
~subpartitions_cpus, which includes all offline possible CPUs.

Because cpumask_intersects(cpu_rq(cpu)->rd->span, new_mask) does not mask
against cpu_active_mask, any system with an offline CPU in
cpu_possible_mask will see def_root_domain.span intersect new_mask on that
offline CPU (unless filtered out by p->user_cpus_ptr).  Moving a deadline
task whose bandwidth is in def_root_domain back to top_cpuset will then
return -1 here, skipping both the destination reservation and the bandwidth
move out of def_root_domain.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261116190549.230119-1-mathieu.desnoyers@efficios.com?part=23

  reply	other threads:[~2026-11-16 19:56 UTC|newest]

Thread overview: 59+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-16 19:05 [PATCH v6 00/37] sched/deadline: bandwidth accounting fixes and selftests Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 01/37] sched: Refuse SCHED_FLAG_KEEP_PARAMS that would change the policy or the class Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 02/37] sched/deadline: Do not admit bandwidth for parameters that are kept Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 03/37] sched: Let a task boosted by a deadline waiter change its policy Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 04/37] sched/deadline: Do not divide by zero active CPUs of a root domain Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 05/37] sched/deadline: Refuse dl_server debugfs writes while the rq is offline Mathieu Desnoyers
2026-11-16 19:30   ` sashiko-bot
2026-11-17 14:14     ` Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 06/37] sched: Take cpuset_mutex for a deadline policy change only with pi Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 07/37] sched/deadline: Do not lose a throttled task switched out and back Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 08/37] sched/deadline: Do not replenish a boost from cleared parameters Mathieu Desnoyers
2026-11-16 19:42   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 09/37] sched/deadline: Reset the deadline and the runtime with the parameters Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 10/37] sched/deadline: Reset the donor's runtime at a deboost in the class Mathieu Desnoyers
2026-11-16 19:38   ` sashiko-bot
2026-11-17 14:15     ` Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 11/37] sched/deadline: Release the bandwidth of a task that leaves the class asleep Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 12/37] sched/deadline: Reject SCHED_DEADLINE on exited tasks Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 13/37] sched/deadline: Keep departing bandwidth across a root domain rebuild Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 14/37] sched/deadline: Fix this_bw when a task is changed during a remote wakeup Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 15/37] sched/deadline: Rebuild the root domains' accounting from the task list Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 16/37] sched/deadline: Record the root domain holding a task's bandwidth Mathieu Desnoyers
2026-11-16 19:44   ` sashiko-bot
2026-11-17 14:15     ` Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 17/37] sched/deadline: Leave a dying task out of a rebuild of the accounting Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 18/37] sched/deadline: Check SCHED_DEADLINE affinity under the rq lock Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 19/37] sched/deadline: Move a task's bandwidth only where room was made Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 20/37] cgroup/cpuset: Wait for an attach before rebuilding the sched domains Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 21/37] sched: Make deadline policy and affinity changes wait for a cpuset attach Mathieu Desnoyers
2026-11-16 19:31   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 22/37] cgroup/cpuset: Give back the bandwidth an attach did not move Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 23/37] cgroup/cpuset: Reserve the bandwidth in the root domain a task moves to Mathieu Desnoyers
2026-11-16 19:56   ` sashiko-bot [this message]
2026-11-16 19:05 ` [PATCH v6 24/37] sched/deadline: Make a rebuild count a task in the root domain it runs in Mathieu Desnoyers
2026-11-16 20:01   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 25/37] sched/deadline: Move a task's bandwidth to " Mathieu Desnoyers
2026-11-16 19:47   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 26/37] sched/deadline: Keep a task in the root domain that counts its bandwidth Mathieu Desnoyers
2026-11-16 19:50   ` sashiko-bot
2026-11-17 14:16     ` Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 27/37] sched/deadline: Keep the last CPU a task has in its root domain Mathieu Desnoyers
2026-11-16 19:51   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 28/37] cgroup/cpuset: Refuse to take a deadline task out of " Mathieu Desnoyers
2026-11-16 20:08   ` sashiko-bot
2026-11-17 14:17     ` Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 29/37] sched/deadline: Cancel the replenishment timer with the parameters Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 30/37] sched/deadline: Count the bandwidth a boosted task is given Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 31/37] sched/deadline: Start a period for a task that enters the class blocked Mathieu Desnoyers
2026-11-16 20:12   ` sashiko-bot
2026-11-17 14:18     ` Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 32/37] sched/deadline: Check the runtime of a task that comes back runnable Mathieu Desnoyers
2026-11-16 20:08   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 33/37] sched/deadline: Check the runtime of a task whose parameters change Mathieu Desnoyers
2026-11-16 20:04   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 34/37] sched/debug: Show the deadline bandwidth of each runqueue Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 35/37] selftests/sched: Add a test of SCHED_DEADLINE policy round trips Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 36/37] selftests/sched: Test SCHED_DEADLINE bandwidth across a root domain rebuild Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 37/37] selftests/sched: Add a test of SCHED_DEADLINE bandwidth accounting Mathieu Desnoyers
2026-11-16 19:49   ` sashiko-bot

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=sashiko-outbox-217626@kernel.org \
    --to=sashiko-bot@kernel.org \
    --cc=hannes@cmpxchg.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rt-devel@lists.linux.dev \
    --cc=mathieu.desnoyers@efficios.com \
    --cc=mkoutny@suse.com \
    --cc=sashiko-reviews@lists.linux.dev \
    --cc=tj@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.