All of lore.kernel.org
 help / color / mirror / Atom feed
From: sashiko-bot@kernel.org
To: "Mathieu Desnoyers" <mathieu.desnoyers@efficios.com>
Cc: linux-rt-devel@lists.linux.dev, tj@kernel.org,
	hannes@cmpxchg.org, mkoutny@suse.com,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH v8 16/33] sched/deadline: Keep the last CPU a task has in its root domain
Date: Thu, 19 Nov 2026 20:48:06 +0000	[thread overview]
Message-ID: <sashiko-outbox-224050@kernel.org> (raw)
In-Reply-To: <6e555ba3ad1013206d77ded129a127dadfd3b983.1795116791.git.mathieu.desnoyers@efficios.com>

Thank you for your contribution! Sashiko AI review found 3 potential
issues to consider:
- [Medium] cgroup/cpuset: empty mask in cpuset_cpus_allowed_without() on
  top_cpuset loss
- [Medium] cgroup/cpuset: cpuset_cpus_allowed_without() walks past empty
  valid partition
- [Medium] sched/deadline: missed root domain move when offline
  invalidates PRS_ISOLATED
--

commit 6a0cfaf09c2412ad93623e8c7467bb4c5d10dd61
Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

sched/deadline: Keep the last CPU a task has in its root domain

This commit refuses CPU offline in dl_bw_deactivate() with -EBUSY when
removing the CPU would leave a deadline task with runnable CPUs only
outside the root domain that admitted its bandwidth, or when invalidating
a cpuset partition would move a deadline task to another root domain.

> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
> index 7bdb9e21a7edb..5299652d47b6e 100644
> --- a/kernel/cgroup/cpuset.c
> +++ b/kernel/cgroup/cpuset.c

[ ... ]

> @@ -4312,6 +4334,234 @@ void cpuset_cpus_allowed(struct task_struct *tsk, struct cpumask *pmask)

[ ... ]

> +bool cpuset_cpus_allowed_without(struct task_struct *tsk, int cpu,
> +				 struct cpumask *pmask)
> +{
> +	const struct cpumask *possible = task_cpu_possible_mask(tsk);
> +	struct cpuset *cs = task_cs(tsk), *from;
> +
> +	guard(rcu)();
> +	if (!is_in_v2_mode()) {
> +		/*
> +		 * cgroup v1 moves the tasks of a cpuset left with no CPU to
> +		 * the nearest ancestor that keeps one.
> +		 */
> +		for (from = cs; from; from = parent_cs(from)) {
> +			cpumask_and(pmask, from->effective_cpus, cpu_active_mask);
> +			cpumask_clear_cpu(cpu, pmask);
> +			if (cpumask_intersects(pmask, possible))
> +				break;
> +		}
> +	} else {
> +		/*
> +		 * cgroup v2 works out the CPUs of each cpuset from the top
> +		 * down. A cpuset keeps those of its parent that cpuset.cpus
> +		 * holds, or takes all of them if there are none. Start from
> +		 * the nearest partition root that keeps an active CPU, or from
> +		 * the top cpuset. A partition root left with no CPU of its own
> +		 * can take those of a child partition.
> +		 */
> +		for (from = cs; ; from = parent_cs(from)) {
> +			if (from != &top_cpuset && !is_partition_valid(from))
> +				continue;
> +			cpumask_and(pmask, from->effective_cpus, cpu_active_mask);
> +			cpumask_clear_cpu(cpu, pmask);
> +			if (cpumask_empty(pmask) && from != &top_cpuset &&
> +			    !partition_child_lost(from, cpu, pmask))
> +				cpumask_clear(pmask);

[Severity: Medium]
Does cpuset_cpus_allowed_without() leave pmask empty when from reaches
top_cpuset and @cpu was the last active CPU in top_cpuset.effective_cpus?

When top_cpuset has no active CPUs left outside subpartitions_cpus,
cpuset_handle_hotplug() clears subpartitions_cpus and reclaims all
remaining active CPUs into top_cpuset.effective_cpus:

kernel/cgroup/cpuset.c:cpuset_handle_hotplug() {
    ...
		if (!cpumask_empty(subpartitions_cpus)) {
			if (cpumask_subset(&new_cpus, subpartitions_cpus)) {
				cpumask_clear(subpartitions_cpus);
			} else {
				cpumask_andnot(&new_cpus, &new_cpus,
					       subpartitions_cpus);
			}
		}
		cpumask_copy(top_cpuset.effective_cpus, &new_cpus);
    ...
}

cpuset_offline_moves_dl() skips non-partition descendants whose enclosing
partition is top_cpuset (a == &top_cpuset), leaving their deadline tasks
to dl_task_leaves_span().  Because pmask stays empty at top_cpuset here,
cpuset_cpus_allowed_without() skips effective_cpus_below() and
tsk->user_cpus_ptr filtering and returns false.  dl_task_leaves_span()
then takes the select_fallback_rq() branch and accepts the offline
whenever the task's current root domain has another active CPU (for
example, a boot-isolated CPU in def_root_domain), even if the CPUs
reclaimed from subpartitions_cpus put the task's cpuset in a different
root domain without admission control.

> +			if (!cpumask_empty(pmask) || from == &top_cpuset)
> +				break;
> +		}
> +		effective_cpus_below(cs, from, pmask);

[Severity: Medium]
Can this loop walk past an unpopulated valid parent partition root that
stays valid with an empty effective_cpus mask during hotplug?

Suppose an unpopulated outer partition p_outer delegates all of its
active effective_xcpus to two child partitions: p_inner1 on @cpu (an
isolated partition sharing def_root_domain with a boot-isolated CPU in
top_cpuset) and p_inner2 on another active CPU.  During hotplug,
cpuset_handle_hotplug() visits descendants in pre-order: when it visits
p_outer, p_inner1 is still a valid partition root, so
partition_is_populated(p_outer, NULL) is false, p_inner2 is not
invalidated, and p_outer stays valid with an empty effective_cpus mask.
When cpuset_hotplug_update_tasks() next visits p_inner1, p_inner1 is
invalidated and inherits p_outer's empty effective_cpus mask.

Here, when from walks up from p_inner1 to p_outer, pmask is empty and
partition_child_lost(p_outer, cpu, pmask) returns NULL because
partition_populated_offline(p_outer) is false.  Instead of stopping at
p_outer with an empty mask, the loop continues up to top_cpuset and
returns true with top_cpuset's CPUs intersected with tsk->user_cpus_ptr.
If tsk->user_cpus_ptr restricts the task in p_inner1 to @cpu and the
boot-isolated CPU in top_cpuset, dl_task_leaves_span() sees the
boot-isolated CPU in msk and accepts the offline, whereas actual hotplug
gives p_inner1 an empty effective_cpus mask and dl_task_rebuild_cpu()
falls back to guarantee_active_cpus() (ignoring user_cpus_ptr) and moves
the task's bandwidth to a housekeeping CPU in top_cpuset.

[ ... ]

> +bool cpuset_offline_moves_dl(int cpu)
> +{
> +	struct cgroup_subsys_state *pos_css;
> +	struct cpuset *cp, *a, *p = &top_cpuset, *lost = NULL;
> +	bool found = false;
> +
> +	if (!is_in_v2_mode())
> +		return false;
> +	rcu_read_lock();
> +	cpuset_for_each_descendant_pre(cp, pos_css, &top_cpuset) {
> +		if (cp != &top_cpuset && is_partition_valid(cp) &&
> +		    cpumask_test_cpu(cpu, cp->effective_cpus)) {
> +			p = cp;
> +			break;
> +		}
> +	}
> +	if (p != &top_cpuset) {
> +		lost = partition_child_lost(p, cpu, &dl_offline_cpus);
> +	} else {
> +		cpumask_and(&dl_offline_cpus, top_cpuset.effective_cpus, cpu_active_mask);
> +		cpumask_clear_cpu(cpu, &dl_offline_cpus);
> +		if (cpumask_empty(&dl_offline_cpus) && !cpumask_empty(subpartitions_cpus))
> +			lost = &top_cpuset;
> +	}
> +	if (!lost) {
> +		rcu_read_unlock();
> +		return false;
> +	}
> +	cpuset_for_each_descendant_pre(cp, pos_css, lost) {
> +		if (cp == &top_cpuset)
> +			continue;
> +		/* The partition that gives @cp its CPUs. */
> +		for (a = cp; a != lost && !is_partition_valid(a); a = parent_cs(a))
> +			;
> +		if (a == &top_cpuset ||
> +		    (a->partition_root_state == PRS_ISOLATED &&
> +		     p->partition_root_state == PRS_ISOLATED) ||
> +		    (a == lost && a->partition_root_state != PRS_ISOLATED &&
> +		     p->partition_root_state != PRS_ISOLATED))
> +			continue;
> +		if (!css_tryget_online(&cp->css))
> +			continue;
> +		rcu_read_unlock();
> +		found = dl_task_in_cpuset(cp);
> +		rcu_read_lock();
> +		css_put(&cp->css);
> +		if (found)
> +			break;
> +	}
> +	rcu_read_unlock();
> +	return found;
> +}
> diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
> index c33e73e920ebb..eead45a0c1191 100644
> --- a/kernel/sched/deadline.c
> +++ b/kernel/sched/deadline.c
> @@ -5066,9 +5066,79 @@ static int dl_bw_manage(enum dl_bw_request req, int cpu, u64 dl_bw)
>  	return overflow ? -EBUSY : 0;
>  }
>  
> +/*
> + * Return true if @p has CPUs to run on once @cpu is offline, but none in
> + * the root domain of @cpu, which counts its bandwidth. A dead task runs no
> + * more. It can wait for its 0-lag time, or for its parent to reap it.
> + * Return false for it.
> + */
> +static bool dl_task_leaves_span(struct task_struct *p, int cpu,
> +				const struct cpumask *span)
> +{
> +	struct cpumask *msk;
> +
> +	guard(raw_spinlock_irqsave)(&p->pi_lock);
> +	if (READ_ONCE(p->__state) == TASK_DEAD || dl_task_bw_span(p) != span)
> +		return false;
> +	msk = this_cpu_cpumask_var_ptr(local_cpu_mask_dl);
> +	/* A task in a migrate_disable() section has a cpus_ptr of one CPU. */
> +	cpumask_and(msk, &p->cpus_mask, cpu_active_mask);
> +	cpumask_clear_cpu(cpu, msk);
> +	/* With no CPU left, ask which CPUs the cpuset gives @p. */
> +	if (cpumask_empty(msk) && !cpuset_cpus_allowed_without(p, cpu, msk)) {
> +		/*
> +		 * select_fallback_rq() picks the CPU. It gives @p the CPUs of
> +		 * @span if one of them is active and @p can run on all of them.
> +		 */
> +		if (cpumask_any_and_but(span, cpu_active_mask, cpu) < nr_cpu_ids &&
> +		    cpumask_subset(span, task_cpu_possible_mask(p)))
> +			return false;
> +	}
> +	return !cpumask_empty(msk) && !cpumask_intersects(msk, span);
> +}

[Severity: Medium]
Can an offline that invalidates an isolated partition (PRS_ISOLATED) still
move a deadline task from def_root_domain to a housekeeping root domain
without being refused?

Suppose CPU 1 is isolated at boot via isolcpus=domain,1, while CPU 0 and
CPU 2 are in housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT).  top_cpuset has
effective_cpus = {1, 2}, and a child PRS_ISOLATED partition P has
effective_cpus = {0} with a deadline task recorded on CPU 0 and counted in
def_root_domain (whose pre-offline span is {0, 1}).  When CPU 0 is taken
offline:

1. cpuset_offline_moves_dl(0) finds p = P, calls partition_child_lost()
   which returns NULL because P has no active effective_xcpus left, and
   returns false.

2. dl_task_leaves_span() gets msk = {1, 2} from top_cpuset via
   cpuset_cpus_allowed_without(), sees that msk intersects the pre-offline
   span {0, 1} at CPU 1, and returns false.

3. Once CPU 0 is offline, P is invalidated into top_cpuset and the task
   gets cpus_mask = {1, 2}.  During dl_rebuild_rd_accounting(),
   dl_task_rebuild_cpu() matches the recorded CPU against
   housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT):

kernel/sched/deadline.c:dl_task_rebuild_cpu() {
    ...
	if (recorded >= 0) {
		const struct cpumask *hk = housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT);
		bool rec_hk = cpumask_test_cpu(recorded, hk);
		unsigned int same;
    ...
		for_each_cpu(same, msk) {
			if (cpumask_test_cpu(same, hk) == rec_hk)
				return same;
		}
	}
    ...
}

Because CPU 0 was isolated by PRS_ISOLATED rather than at boot, rec_hk is
true, so dl_task_rebuild_cpu() picks housekeeping CPU 2 instead of
boot-isolated CPU 1 and moves the task's bandwidth into top_cpuset's
housekeeping root domain without admission control.

A similar move happens if top_cpuset holds only boot-isolated CPU 0 (with
a deadline task in top_cpuset) and an empty PRS_ISOLATED partition holds
housekeeping CPU 1: offlining CPU 0 invalidates the PRS_ISOLATED
partition so CPU 1 leaves def_root_domain to form top_cpuset's
housekeeping root domain, while dl_task_leaves_span() still tests against
the pre-offline def_root_domain span {0, 1}.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1795116790.git.mathieu.desnoyers@efficios.com?part=16

  parent reply	other threads:[~2026-11-19 20:48 UTC|newest]

Thread overview: 57+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19 20:15 [PATCH v8 00/33] sched/deadline: bandwidth accounting fixes and selftests Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 01/33] sched/deadline: Keep departing bandwidth across a root domain rebuild Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 02/33] sched/deadline: Rebuild the root domains' accounting from the task list Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 03/33] sched/deadline: Record the root domain holding a task's bandwidth Mathieu Desnoyers
2026-11-19 20:51   ` sashiko-bot
2026-11-20 13:48     ` Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 04/33] sched/deadline: Leave a dying task out of a rebuild of the accounting Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 05/33] selftests/sched: Test SCHED_DEADLINE bandwidth across a root domain rebuild Mathieu Desnoyers
2026-11-19 20:28   ` sashiko-bot
2026-11-20 13:49     ` Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 06/33] selftests/sched: Test the root domain rebuild in deadline_bw_test Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 07/33] sched/deadline: Move a task's bandwidth only where room was made Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 08/33] cgroup/cpuset: Wait for an attach before rebuilding the sched domains Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 09/33] sched: Make deadline policy and affinity changes wait for a cpuset attach Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 10/33] cgroup/cpuset: Give back the bandwidth an attach did not move Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 11/33] cgroup/cpuset: Reserve the bandwidth in the root domain a task moves to Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 12/33] selftests/sched: Test cpuset moves in deadline_bw_test Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 13/33] sched/deadline: Make a rebuild count a task in the root domain it runs in Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 14/33] sched/deadline: Move a task's bandwidth to " Mathieu Desnoyers
2026-11-19 20:53   ` sashiko-bot
2026-11-20 13:49     ` Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 15/33] sched/deadline: Keep a task in the root domain that counts its bandwidth Mathieu Desnoyers
2026-11-19 20:55   ` sashiko-bot
2026-11-20 13:50     ` Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 16/33] sched/deadline: Keep the last CPU a task has in its root domain Mathieu Desnoyers
2026-11-19 20:22   ` Mathieu Desnoyers
2026-11-19 20:48   ` sashiko-bot [this message]
2026-11-20 13:51     ` Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 17/33] cgroup/cpuset: Refuse to take a deadline task out of " Mathieu Desnoyers
2026-11-19 20:43   ` sashiko-bot
2026-11-20 13:51     ` Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 18/33] selftests/sched: Test the root domain of a task in deadline_bw_test Mathieu Desnoyers
2026-11-19 20:27   ` sashiko-bot
2026-11-20 13:51     ` Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 19/33] sched/deadline: Count only active CPUs when spreading extra_bw Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 20/33] rtmutex: Continue the chain walk when a waiter's deadline donor changes Mathieu Desnoyers
2026-11-19 20:16 ` [PATCH v8 21/33] rtmutex: Continue the walk when the owner's top waiter changes lock Mathieu Desnoyers
2026-11-19 20:16 ` [PATCH v8 22/33] selftests/sched: Test the rtmutex chain walk in deadline_bw_test Mathieu Desnoyers
2026-11-19 20:16 ` [PATCH v8 23/33] sched/deadline: Reset the donor's runtime at a deboost in the class Mathieu Desnoyers
2026-11-19 20:53   ` sashiko-bot
2026-11-20 13:52     ` Mathieu Desnoyers
2026-11-19 20:16 ` [PATCH v8 24/33] sched/deadline: Replenish a throttled task boosted while asleep Mathieu Desnoyers
2026-11-19 20:16 ` [PATCH v8 25/33] sched/deadline: Count the bandwidth a boosted task is given Mathieu Desnoyers
2026-11-19 20:16 ` [PATCH v8 26/33] sched/deadline: Start a period for a task that enters the class blocked Mathieu Desnoyers
2026-11-19 20:57   ` sashiko-bot
2026-11-20 13:52     ` Mathieu Desnoyers
2026-11-19 20:16 ` [PATCH v8 27/33] sched/deadline: Start a period at a deboost out of the class Mathieu Desnoyers
2026-11-19 20:47   ` sashiko-bot
2026-11-20 13:53     ` Mathieu Desnoyers
2026-11-19 20:16 ` [PATCH v8 28/33] selftests/sched: Test deadline boosts in deadline_bw_test Mathieu Desnoyers
2026-11-19 20:16 ` [PATCH v8 29/33] sched/deadline: Cancel the replenishment timer with the parameters Mathieu Desnoyers
2026-11-19 20:16 ` [PATCH v8 30/33] sched/deadline: Cap the runtime left of a constrained task at dl_runtime Mathieu Desnoyers
2026-11-19 20:16 ` [PATCH v8 31/33] sched/deadline: Check the runtime of a task that comes back runnable Mathieu Desnoyers
2026-11-19 20:16 ` [PATCH v8 32/33] sched/deadline: Check the runtime of a task whose parameters change Mathieu Desnoyers
2026-11-19 20:59   ` sashiko-bot
2026-11-20 13:53     ` Mathieu Desnoyers
2026-11-19 20:16 ` [PATCH v8 33/33] selftests/sched: Test parameter changes in deadline_bw_test Mathieu Desnoyers

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=sashiko-outbox-224050@kernel.org \
    --to=sashiko-bot@kernel.org \
    --cc=hannes@cmpxchg.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rt-devel@lists.linux.dev \
    --cc=mathieu.desnoyers@efficios.com \
    --cc=mkoutny@suse.com \
    --cc=sashiko-reviews@lists.linux.dev \
    --cc=tj@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.