[PATCH 0/4] sched/deadline: Fix assorted dl_bw accounting bugs

[PATCH 0/4] sched/deadline: Fix assorted dl_bw accounting bugs

[Juri Lelli]

This series fixes several SCHED_DEADLINE bandwidth accounting bugs that
can cause total_bw to leak or underflow, eventually blocking new
SCHED_DEADLINE admissions and CPU offlining with -EBUSY.

Patch 1 fixes stale root domain lookups during cpuset bandwidth
transfers for sleeping tasks, by using the task's cpus_mask instead of
the potentially stale task_rq(p)->rd.

Patch 2 fixes a total_bw leak when switching a non-queued task away
from SCHED_DEADLINE, where neither switched_from_dl() nor the inactive
timer would release the reserved bandwidth.

Patch 3 rejects the contradictory combination of SCHED_FLAG_KEEP_PARAMS
with a policy change that would switch the scheduling class, which
corrupts bandwidth accounting via spurious switched_from_dl() calls.

Patch 4 rejects sched_setattr(SCHED_DEADLINE) on exited (zombie) tasks,
which charge total_bw but never release it since the task will not run
again.

Series also available at

https://github.com/jlelli/linux.git b4/sched-dl-bw-fixes

Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
---
Juri Lelli (4):
      sched/deadline: Fix stale root domain in cpuset bandwidth transfer
      sched/deadline: Fix total_bw leak when switching away from SCHED_DEADLINE while !queued
      sched: Reject SCHED_FLAG_KEEP_PARAMS when it would change scheduling class
      sched/deadline: Reject SCHED_DEADLINE on exited tasks

 kernel/sched/deadline.c | 63 ++++++++++++++++++++++++++++++++++++++-----------
 kernel/sched/syscalls.c | 23 +++++++++++++++++-
 2 files changed, 71 insertions(+), 15 deletions(-)
---
base-commit: b99b7f60c5b473d374c68b53753153b4aa9e0702
change-id: 20261103-b4-sched-dl-bw-fixes-9b921aa91b71

Best regards,
--  
Juri Lelli <juri.lelli@redhat.com>

[PATCH 1/4] sched/deadline: Fix stale root domain in cpuset bandwidth transfer

[Juri Lelli]

dl_task_needs_bw_move() and set_cpus_allowed_dl() use task_rq(p)->rd
to determine a task's current root domain. For a sleeping task, this
is stale: it points to the rq of the CPU the task last ran on, which
may belong to a different root domain after a cpuset move.

This causes incorrect bandwidth accounting when a sleeping DL task is
moved between exclusive cpuset partitions and back:

  1. Move root->partition: task_rq(p)->rd = root, destination in
     partition -> different -> bandwidth transferred correctly.
  2. Move partition->root: task_rq(p)->rd is still root (task never
     ran) -> looks like same domain -> no transfer. Partition keeps
     stale bandwidth, root domain is short.

The root domain then over-admits, and when the task exits total_bw
underflows.

Introduce dl_task_root_domain_cpu() helper that uses the task's
cpus_mask (which still reflects the old/source cpuset at call time)
to find the root domain where the bandwidth is actually accounted,
instead of relying on the potentially stale task_rq(p)->rd.

Use cpus_mask rather than cpus_ptr, as the latter may be temporarily
narrowed to a single CPU by migrate_disable().

Reported-by: Weiheng Qiu <weihengqiuu@gmail.com>
Closes: https://lore.kernel.org/all/14508cb2-805f-41ac-9f6b-bfd7c3d7cb1c@gmail.com/
Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
---
 kernel/sched/deadline.c | 31 ++++++++++++++++++++++++-------
 1 file changed, 24 insertions(+), 7 deletions(-)

diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index c0ebdcde5fe5d..68019dafbaf18 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -3324,14 +3324,28 @@ static void task_woken_dl(struct rq *rq, struct task_struct *p)
 	}
 }
 
+/*
+ * Find the root domain where a DL task's bandwidth is currently
+ * accounted. For a sleeping task that was moved between cpusets
+ * without running, task_rq(p)->rd may be stale. Use the task's
+ * current cpus_mask (not yet updated to the new mask) to find the
+ * actual root domain. Use cpus_mask rather than cpus_ptr, which
+ * may be temporarily narrowed by migrate_disable().
+ */
+static unsigned int dl_task_root_domain_cpu(struct task_struct *p)
+{
+	unsigned int cpu;
+
+	cpu = cpumask_first_and(&p->cpus_mask, cpu_active_mask);
+	BUG_ON(cpu >= nr_cpu_ids);
+	return cpu;
+}
+
 static void set_cpus_allowed_dl(struct task_struct *p,
 				struct affinity_context *ctx)
 {
-	struct rq *rq;
-
 	WARN_ON_ONCE(!dl_task(p));
 
-	rq = task_rq(p);
 	/*
 	 * Migrating a SCHED_DEADLINE task between exclusive
 	 * cpusets (different root_domains) entails a bandwidth
@@ -3339,16 +3353,16 @@ static void set_cpus_allowed_dl(struct task_struct *p,
 	 * domain (see cpuset_can_attach()).
 	 */
 	if (dl_task_needs_bw_move(p, ctx->new_mask)) {
-		struct dl_bw *src_dl_b;
+		unsigned int src_cpu = dl_task_root_domain_cpu(p);
+		struct dl_bw *src_dl_b = &cpu_rq(src_cpu)->rd->dl_bw;
 
-		src_dl_b = dl_bw_of(cpu_of(rq));
 		/*
 		 * We now free resources of the root_domain we are migrating
 		 * off. In the worst case, sched_setattr() may temporary fail
 		 * until we complete the update.
 		 */
 		raw_spin_lock(&src_dl_b->lock);
-		__dl_sub(src_dl_b, p->dl.dl_bw, dl_bw_cpus(task_cpu(p)));
+		__dl_sub(src_dl_b, p->dl.dl_bw, dl_bw_cpus(src_cpu));
 		raw_spin_unlock(&src_dl_b->lock);
 	}
 
@@ -3358,10 +3372,13 @@ static void set_cpus_allowed_dl(struct task_struct *p,
 bool dl_task_needs_bw_move(struct task_struct *p,
 			   const struct cpumask *new_mask)
 {
+	unsigned int cpu;
+
 	if (!dl_task(p))
 		return false;
 
-	return !cpumask_intersects(task_rq(p)->rd->span, new_mask);
+	cpu = dl_task_root_domain_cpu(p);
+	return !cpumask_intersects(cpu_rq(cpu)->rd->span, new_mask);
 }
 
 /* Assumes rq->lock is held */

-- 
2.56.0

[PATCH 2/4] sched/deadline: Fix total_bw leak when switching away from SCHED_DEADLINE while !queued

[Juri Lelli]

When a SCHED_DEADLINE task is switched to another policy while it is not
queued (sleeping or stopped), its bandwidth leaks in the root domain's
total_bw. The leak happens because:

 - switched_from_dl() only calls task_non_contending() for queued tasks.

 - For non-queued tasks, it subtracts running_bw and rq_bw, but never
   releases total_bw from the root domain's dl_bw.

 - The inactive timer is the only path that calls __dl_sub() to release
   total_bw, but it is either not armed (the task never ran as DEADLINE)
   or already fired while the task was still DEADLINE (only dropping
   running_bw at that time).

Repeating this fills total_bw up to the admission cap, after which
sched_setattr(SCHED_DEADLINE) and CPU offline fail with -EBUSY.

Fix this by releasing total_bw directly in switched_from_dl() when the
task is not queued and dl_non_contending is not set (meaning no pending
inactive timer will handle it).

Fixes: 209a0cbda7a0 ("sched/deadline: Improve the tracking of active utilization")
Reported-by: Tejun Heo <tj@kernel.org>
Closes: https://lore.kernel.org/all/2b5fb3f22bca6aa383be8f4781923392@kernel.org/
Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
---
 kernel/sched/deadline.c | 32 +++++++++++++++++++++++++-------
 1 file changed, 25 insertions(+), 7 deletions(-)

diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index 68019dafbaf18..963fda6ddb7f6 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -3547,14 +3547,32 @@ static void switched_from_dl(struct rq *rq, struct task_struct *p)
 	dec_dl_tasks_cs(p);
 
 	if (!task_on_rq_queued(p)) {
-		/*
-		 * Inactive timer is armed. However, p is leaving DEADLINE and
-		 * might migrate away from this rq while continuing to run on
-		 * some other class. We need to remove its contribution from
-		 * this rq running_bw now, or sub_rq_bw (below) will complain.
-		 */
-		if (p->dl.dl_non_contending)
+		if (p->dl.dl_non_contending) {
+			/*
+			 * Inactive timer is armed. However, p is leaving
+			 * DEADLINE and might migrate away from this rq while
+			 * continuing to run on some other class. We need to
+			 * remove its contribution from this rq running_bw now,
+			 * or sub_rq_bw (below) will complain.
+			 *
+			 * inactive_task_timer() will find !dl_task(p) and
+			 * release total_bw via __dl_sub().
+			 */
 			sub_running_bw(&p->dl, &rq->dl);
+		} else {
+			/*
+			 * No pending inactive timer: the task never ran as
+			 * DEADLINE or the timer already fired while still
+			 * DEADLINE (only dropping running_bw at that time).
+			 * Nobody else will release total_bw, do it now.
+			 */
+			unsigned int cpu = dl_task_root_domain_cpu(p);
+			struct dl_bw *dl_b = &cpu_rq(cpu)->rd->dl_bw;
+
+			raw_spin_lock(&dl_b->lock);
+			__dl_sub(dl_b, p->dl.dl_bw, dl_bw_cpus(cpu));
+			raw_spin_unlock(&dl_b->lock);
+		}
 		sub_rq_bw(&p->dl, &rq->dl);
 	}
 

-- 
2.56.0

[PATCH 3/4] sched: Reject SCHED_FLAG_KEEP_PARAMS when it would change scheduling class

[Juri Lelli]

SCHED_FLAG_KEEP_PARAMS was introduced for uclamp (a509a7cd7974) to let
userspace adjust utilization clamping without disturbing existing
scheduling parameters. The flag skips __setscheduler_params() and the
sched_class assignment entirely.

However, when a sched_setattr() call requests a policy change that
would switch the scheduling class (e.g., SCHED_OTHER on a
SCHED_DEADLINE task) while also setting KEEP_PARAMS, the result is
contradictory: DEQUEUE_CLASS is set (because prev_class != next_class),
causing sched_change_begin() to call switched_from_dl(), but the actual
class change never happens. This leaves the task in the original class
with its bandwidth accounting corrupted.

For SCHED_DEADLINE tasks, each such call spuriously releases running_bw
and rq_bw via switched_from_dl(), leading to accounting underflow.

Reject the contradictory combination early with -EINVAL. This does not
affect the intended uclamp use case, which always pairs KEEP_PARAMS
with KEEP_POLICY (keeping the same class).

Fixes: a509a7cd7974 ("sched/uclamp: Extend sched_setattr() to support utilization clamping")
Link: https://lore.kernel.org/all/2b5fb3f22bca6aa383be8f4781923392@kernel.org/
Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
---
 kernel/sched/syscalls.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c
index b215b0ead9a60..014697d454525 100644
--- a/kernel/sched/syscalls.c
+++ b/kernel/sched/syscalls.c
@@ -675,8 +675,19 @@ int __sched_setscheduler(struct task_struct *p,
 	prev_class = p->sched_class;
 	next_class = __setscheduler_class(policy, newprio);
 
-	if (prev_class != next_class)
+	if (prev_class != next_class) {
+		/*
+		 * SCHED_FLAG_KEEP_PARAMS prevents the class change from
+		 * actually happening (it skips __setscheduler_params and
+		 * the sched_class assignment). Reject contradictory
+		 * requests that ask for a class change with KEEP_PARAMS.
+		 */
+		if (attr->sched_flags & SCHED_FLAG_KEEP_PARAMS) {
+			retval = -EINVAL;
+			goto unlock;
+		}
 		queue_flags |= DEQUEUE_CLASS;
+	}
 
 	scoped_guard (sched_change, p, queue_flags) {
 

-- 
2.56.0

[PATCH 4/4] sched/deadline: Reject SCHED_DEADLINE on exited tasks

[Juri Lelli]

sched_setattr(SCHED_DEADLINE) on a zombie task succeeds and charges
total_bw via sched_dl_overflow(), but the task has already done its
final schedule() and will never run again. Nothing releases the reserved
bandwidth. Repeating this fills total_bw to the admission cap, after
which sched_setattr(SCHED_DEADLINE) and CPU offline fail with -EBUSY.

Reject sched_setattr(SCHED_DEADLINE) on exited tasks with -ESRCH.

Reported-by: Weiheng Qiu <weihengqiuu@gmail.com>
Closes: https://lore.kernel.org/all/3f6e6c87-645b-45fd-8fc6-8da5bc5b82d8@gmail.com/
Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
---
 kernel/sched/syscalls.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c
index 014697d454525..af9ba357a2956 100644
--- a/kernel/sched/syscalls.c
+++ b/kernel/sched/syscalls.c
@@ -583,6 +583,16 @@ int __sched_setscheduler(struct task_struct *p,
 	if (retval)
 		goto unlock;
 
+	/*
+	 * Reject SCHED_DEADLINE on exited tasks. A zombie will never be
+	 * scheduled again, so sched_dl_overflow() would charge total_bw
+	 * that nothing will ever release.
+	 */
+	if (dl_policy(policy) && p->exit_state) {
+		retval = -ESRCH;
+		goto unlock;
+	}
+
 	/*
 	 * If not changing anything there's no need to proceed further,
 	 * but store a possible modification of reset_on_fork.

-- 
2.56.0

[tip: sched/urgent] sched/deadline: Reject SCHED_DEADLINE on exited tasks

[tip-bot2 for Juri Lelli]

The following commit has been merged into the sched/urgent branch of tip:

Commit-ID:     a2b67e904401858d9f0ff6a0485c2fab506cc1bf
Gitweb:        https://git.kernel.org/tip/a2b67e904401858d9f0ff6a0485c2fab506cc1bf
Author:        Juri Lelli <juri.lelli@redhat.com>
AuthorDate:    Tue, 03 Nov 2026 12:26:32 +01:00
Committer:     Ingo Molnar <mingo@kernel.org>
CommitterDate: Tue, 03 Nov 2026 14:18:06 +01:00

sched/deadline: Reject SCHED_DEADLINE on exited tasks

sched_setattr(SCHED_DEADLINE) on a zombie task succeeds and charges
total_bw via sched_dl_overflow(), but the task has already done its
final schedule() and will never run again. Nothing releases the reserved
bandwidth. Repeating this fills total_bw to the admission cap, after
which sched_setattr(SCHED_DEADLINE) and CPU offline fail with -EBUSY.

Reject sched_setattr(SCHED_DEADLINE) on exited tasks with -ESRCH.

Reported-by: Weiheng Qiu <weihengqiuu@gmail.com>
Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: https://patch.msgid.link/20261103-b4-sched-dl-bw-fixes-v1-4-dd761fe005b4@redhat.com
Closes: https://lore.kernel.org/all/3f6e6c87-645b-45fd-8fc6-8da5bc5b82d8@gmail.com/
---
 kernel/sched/syscalls.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c
index be351f8..3fa447a 100644
--- a/kernel/sched/syscalls.c
+++ b/kernel/sched/syscalls.c
@@ -584,6 +584,16 @@ recheck:
 		goto unlock;
 
 	/*
+	 * Reject SCHED_DEADLINE on exited tasks. A zombie will never be
+	 * scheduled again, so sched_dl_overflow() would charge total_bw
+	 * that nothing will ever release.
+	 */
+	if (dl_policy(policy) && p->exit_state) {
+		retval = -ESRCH;
+		goto unlock;
+	}
+
+	/*
 	 * If not changing anything there's no need to proceed further,
 	 * but store a possible modification of reset_on_fork.
 	 */

[tip: sched/urgent] sched: Reject SCHED_FLAG_KEEP_PARAMS when it would change scheduling class

[tip-bot2 for Juri Lelli]

The following commit has been merged into the sched/urgent branch of tip:

Commit-ID:     8a70415351ba8b4d80ce0a119a2409c1e3709855
Gitweb:        https://git.kernel.org/tip/8a70415351ba8b4d80ce0a119a2409c1e3709855
Author:        Juri Lelli <juri.lelli@redhat.com>
AuthorDate:    Tue, 03 Nov 2026 12:26:31 +01:00
Committer:     Ingo Molnar <mingo@kernel.org>
CommitterDate: Tue, 03 Nov 2026 14:18:05 +01:00

sched: Reject SCHED_FLAG_KEEP_PARAMS when it would change scheduling class

SCHED_FLAG_KEEP_PARAMS was introduced for uclamp (a509a7cd7974) to let
userspace adjust utilization clamping without disturbing existing
scheduling parameters. The flag skips __setscheduler_params() and the
sched_class assignment entirely.

However, when a sched_setattr() call requests a policy change that
would switch the scheduling class (e.g., SCHED_OTHER on a
SCHED_DEADLINE task) while also setting KEEP_PARAMS, the result is
contradictory: DEQUEUE_CLASS is set (because prev_class != next_class),
causing sched_change_begin() to call switched_from_dl(), but the actual
class change never happens. This leaves the task in the original class
with its bandwidth accounting corrupted.

For SCHED_DEADLINE tasks, each such call spuriously releases running_bw
and rq_bw via switched_from_dl(), leading to accounting underflow.

Reject the contradictory combination early with -EINVAL. This does not
affect the intended uclamp use case, which always pairs KEEP_PARAMS
with KEEP_POLICY (keeping the same class).

Fixes: a509a7cd7974 ("sched/uclamp: Extend sched_setattr() to support utilization clamping")
Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: https://lore.kernel.org/all/2b5fb3f22bca6aa383be8f4781923392@kernel.org/
Link: https://patch.msgid.link/20261103-b4-sched-dl-bw-fixes-v1-3-dd761fe005b4@redhat.com
---
 kernel/sched/syscalls.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c
index 45a127d..be351f8 100644
--- a/kernel/sched/syscalls.c
+++ b/kernel/sched/syscalls.c
@@ -675,8 +675,19 @@ change:
 	prev_class = p->sched_class;
 	next_class = __setscheduler_class(policy, newprio);
 
-	if (prev_class != next_class)
+	if (prev_class != next_class) {
+		/*
+		 * SCHED_FLAG_KEEP_PARAMS prevents the class change from
+		 * actually happening (it skips __setscheduler_params and
+		 * the sched_class assignment). Reject contradictory
+		 * requests that ask for a class change with KEEP_PARAMS.
+		 */
+		if (attr->sched_flags & SCHED_FLAG_KEEP_PARAMS) {
+			retval = -EINVAL;
+			goto unlock;
+		}
 		queue_flags |= DEQUEUE_CLASS;
+	}
 
 	scx_prepare_setscheduler(p, policy);
 

[tip: sched/urgent] sched/deadline: Fix total_bw leak when switching away from SCHED_DEADLINE while !queued

[tip-bot2 for Juri Lelli]

The following commit has been merged into the sched/urgent branch of tip:

Commit-ID:     41a7968022ca3f72b180b4fc2125c66020c8c73a
Gitweb:        https://git.kernel.org/tip/41a7968022ca3f72b180b4fc2125c66020c8c73a
Author:        Juri Lelli <juri.lelli@redhat.com>
AuthorDate:    Tue, 03 Nov 2026 12:26:30 +01:00
Committer:     Ingo Molnar <mingo@kernel.org>
CommitterDate: Tue, 03 Nov 2026 14:18:04 +01:00

sched/deadline: Fix total_bw leak when switching away from SCHED_DEADLINE while !queued

When a SCHED_DEADLINE task is switched to another policy while it is not
queued (sleeping or stopped), its bandwidth leaks in the root domain's
total_bw. The leak happens because:

 - switched_from_dl() only calls task_non_contending() for queued tasks.

 - For non-queued tasks, it subtracts running_bw and rq_bw, but never
   releases total_bw from the root domain's dl_bw.

 - The inactive timer is the only path that calls __dl_sub() to release
   total_bw, but it is either not armed (the task never ran as DEADLINE)
   or already fired while the task was still DEADLINE (only dropping
   running_bw at that time).

Repeating this fills total_bw up to the admission cap, after which
sched_setattr(SCHED_DEADLINE) and CPU offline fail with -EBUSY.

Fix this by releasing total_bw directly in switched_from_dl() when the
task is not queued and dl_non_contending is not set (meaning no pending
inactive timer will handle it).

Fixes: 209a0cbda7a0 ("sched/deadline: Improve the tracking of active utilization")
Reported-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: https://patch.msgid.link/20261103-b4-sched-dl-bw-fixes-v1-2-dd761fe005b4@redhat.com
Closes: https://lore.kernel.org/all/2b5fb3f22bca6aa383be8f4781923392@kernel.org/
---
 kernel/sched/deadline.c | 32 +++++++++++++++++++++++++-------
 1 file changed, 25 insertions(+), 7 deletions(-)

diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index 68019da..963fda6 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -3547,14 +3547,32 @@ static void switched_from_dl(struct rq *rq, struct task_struct *p)
 	dec_dl_tasks_cs(p);
 
 	if (!task_on_rq_queued(p)) {
-		/*
-		 * Inactive timer is armed. However, p is leaving DEADLINE and
-		 * might migrate away from this rq while continuing to run on
-		 * some other class. We need to remove its contribution from
-		 * this rq running_bw now, or sub_rq_bw (below) will complain.
-		 */
-		if (p->dl.dl_non_contending)
+		if (p->dl.dl_non_contending) {
+			/*
+			 * Inactive timer is armed. However, p is leaving
+			 * DEADLINE and might migrate away from this rq while
+			 * continuing to run on some other class. We need to
+			 * remove its contribution from this rq running_bw now,
+			 * or sub_rq_bw (below) will complain.
+			 *
+			 * inactive_task_timer() will find !dl_task(p) and
+			 * release total_bw via __dl_sub().
+			 */
 			sub_running_bw(&p->dl, &rq->dl);
+		} else {
+			/*
+			 * No pending inactive timer: the task never ran as
+			 * DEADLINE or the timer already fired while still
+			 * DEADLINE (only dropping running_bw at that time).
+			 * Nobody else will release total_bw, do it now.
+			 */
+			unsigned int cpu = dl_task_root_domain_cpu(p);
+			struct dl_bw *dl_b = &cpu_rq(cpu)->rd->dl_bw;
+
+			raw_spin_lock(&dl_b->lock);
+			__dl_sub(dl_b, p->dl.dl_bw, dl_bw_cpus(cpu));
+			raw_spin_unlock(&dl_b->lock);
+		}
 		sub_rq_bw(&p->dl, &rq->dl);
 	}
 

[tip: sched/urgent] sched/deadline: Fix stale root domain in cpuset bandwidth transfer

[tip-bot2 for Juri Lelli]

The following commit has been merged into the sched/urgent branch of tip:

Commit-ID:     b4a61fdc19c5c40f295b28ad318a3c4827d7d289
Gitweb:        https://git.kernel.org/tip/b4a61fdc19c5c40f295b28ad318a3c4827d7d289
Author:        Juri Lelli <juri.lelli@redhat.com>
AuthorDate:    Tue, 03 Nov 2026 12:26:29 +01:00
Committer:     Ingo Molnar <mingo@kernel.org>
CommitterDate: Tue, 03 Nov 2026 14:18:04 +01:00

sched/deadline: Fix stale root domain in cpuset bandwidth transfer

dl_task_needs_bw_move() and set_cpus_allowed_dl() use task_rq(p)->rd
to determine a task's current root domain. For a sleeping task, this
is stale: it points to the rq of the CPU the task last ran on, which
may belong to a different root domain after a cpuset move.

This causes incorrect bandwidth accounting when a sleeping DL task is
moved between exclusive cpuset partitions and back:

  1. Move root->partition: task_rq(p)->rd = root, destination in
     partition -> different -> bandwidth transferred correctly.
  2. Move partition->root: task_rq(p)->rd is still root (task never
     ran) -> looks like same domain -> no transfer. Partition keeps
     stale bandwidth, root domain is short.

The root domain then over-admits, and when the task exits total_bw
underflows.

Introduce dl_task_root_domain_cpu() helper that uses the task's
cpus_mask (which still reflects the old/source cpuset at call time)
to find the root domain where the bandwidth is actually accounted,
instead of relying on the potentially stale task_rq(p)->rd.

Use cpus_mask rather than cpus_ptr, as the latter may be temporarily
narrowed to a single CPU by migrate_disable().

Reported-by: Weiheng Qiu <weihengqiuu@gmail.com>
Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: https://patch.msgid.link/20261103-b4-sched-dl-bw-fixes-v1-1-dd761fe005b4@redhat.com
Closes: https://lore.kernel.org/all/14508cb2-805f-41ac-9f6b-bfd7c3d7cb1c@gmail.com/
---
 kernel/sched/deadline.c | 31 ++++++++++++++++++++++++-------
 1 file changed, 24 insertions(+), 7 deletions(-)

diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index c0ebdcd..68019da 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -3324,14 +3324,28 @@ static void task_woken_dl(struct rq *rq, struct task_struct *p)
 	}
 }
 
+/*
+ * Find the root domain where a DL task's bandwidth is currently
+ * accounted. For a sleeping task that was moved between cpusets
+ * without running, task_rq(p)->rd may be stale. Use the task's
+ * current cpus_mask (not yet updated to the new mask) to find the
+ * actual root domain. Use cpus_mask rather than cpus_ptr, which
+ * may be temporarily narrowed by migrate_disable().
+ */
+static unsigned int dl_task_root_domain_cpu(struct task_struct *p)
+{
+	unsigned int cpu;
+
+	cpu = cpumask_first_and(&p->cpus_mask, cpu_active_mask);
+	BUG_ON(cpu >= nr_cpu_ids);
+	return cpu;
+}
+
 static void set_cpus_allowed_dl(struct task_struct *p,
 				struct affinity_context *ctx)
 {
-	struct rq *rq;
-
 	WARN_ON_ONCE(!dl_task(p));
 
-	rq = task_rq(p);
 	/*
 	 * Migrating a SCHED_DEADLINE task between exclusive
 	 * cpusets (different root_domains) entails a bandwidth
@@ -3339,16 +3353,16 @@ static void set_cpus_allowed_dl(struct task_struct *p,
 	 * domain (see cpuset_can_attach()).
 	 */
 	if (dl_task_needs_bw_move(p, ctx->new_mask)) {
-		struct dl_bw *src_dl_b;
+		unsigned int src_cpu = dl_task_root_domain_cpu(p);
+		struct dl_bw *src_dl_b = &cpu_rq(src_cpu)->rd->dl_bw;
 
-		src_dl_b = dl_bw_of(cpu_of(rq));
 		/*
 		 * We now free resources of the root_domain we are migrating
 		 * off. In the worst case, sched_setattr() may temporary fail
 		 * until we complete the update.
 		 */
 		raw_spin_lock(&src_dl_b->lock);
-		__dl_sub(src_dl_b, p->dl.dl_bw, dl_bw_cpus(task_cpu(p)));
+		__dl_sub(src_dl_b, p->dl.dl_bw, dl_bw_cpus(src_cpu));
 		raw_spin_unlock(&src_dl_b->lock);
 	}
 
@@ -3358,10 +3372,13 @@ static void set_cpus_allowed_dl(struct task_struct *p,
 bool dl_task_needs_bw_move(struct task_struct *p,
 			   const struct cpumask *new_mask)
 {
+	unsigned int cpu;
+
 	if (!dl_task(p))
 		return false;
 
-	return !cpumask_intersects(task_rq(p)->rd->span, new_mask);
+	cpu = dl_task_root_domain_cpu(p);
+	return !cpumask_intersects(cpu_rq(cpu)->rd->span, new_mask);
 }
 
 /* Assumes rq->lock is held */

Re: [PATCH 1/4] sched/deadline: Fix stale root domain in cpuset bandwidth transfer

[Marek Szyprowski]

On 03.11.2026 12:26, Juri Lelli wrote:
> dl_task_needs_bw_move() and set_cpus_allowed_dl() use task_rq(p)->rd
> to determine a task's current root domain. For a sleeping task, this
> is stale: it points to the rq of the CPU the task last ran on, which
> may belong to a different root domain after a cpuset move.
>
> This causes incorrect bandwidth accounting when a sleeping DL task is
> moved between exclusive cpuset partitions and back:
>
>   1. Move root->partition: task_rq(p)->rd = root, destination in
>      partition -> different -> bandwidth transferred correctly.
>   2. Move partition->root: task_rq(p)->rd is still root (task never
>      ran) -> looks like same domain -> no transfer. Partition keeps
>      stale bandwidth, root domain is short.
>
> The root domain then over-admits, and when the task exits total_bw
> underflows.
>
> Introduce dl_task_root_domain_cpu() helper that uses the task's
> cpus_mask (which still reflects the old/source cpuset at call time)
> to find the root domain where the bandwidth is actually accounted,
> instead of relying on the potentially stale task_rq(p)->rd.
>
> Use cpus_mask rather than cpus_ptr, as the latter may be temporarily
> narrowed to a single CPU by migrate_disable().
>
> Reported-by: Weiheng Qiu <weihengqiuu@gmail.com>
> Closes: https://lore.kernel.org/all/14508cb2-805f-41ac-9f6b-bfd7c3d7cb1c@gmail.com/
> Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
This patch landed recently in linux-next as commit b4a61fdc19c5
("sched/deadline: Fix stale root domain in cpuset bandwidth transfer").
I found that it triggers kernel BUG during CPU hot-plug tests on some
of my tests boards. Reverting it on top of next-20261104 (together with
dependent 41a7968022ca3) fixes the issue.

Here is the log from Amlogic Meson G12B based Odroid-N2 board
(arch/arm64/boot/dts/amlogic/meson-g12b-odroid-n2.dts):


root@target:~# for i in /sys/devices/system/cpu/cpu[1-9]; do echo 0 >$i/online; done; sleep 4; for i in /sys/devices/system/cpu/cpu[1-9]; do echo 1 >$i/online; done

psci: CPU1 killed (polled 0 ms)
psci: CPU2 killed (polled 0 ms)
psci: CPU3 killed (polled 0 ms)
psci: CPU4 killed (polled 0 ms)
psci: CPU5 killed (polled 0 ms)
CPU1: Booted secondary processor 0x0000000001 [0x410fd034]
arch_timer: CPU2: Trapping CNTVCT access
CPU2: Booted secondary processor 0x0000000100 [0x410fd092]
cpufreq: cpufreq_policy_online: CPU2: Running at unlisted initial frequency: 999999 kHz, changing to: 1000000 kHz
------------[ cut here ]------------
kernel BUG at kernel/sched/deadline.c:3340!
Internal error: Oops - BUG: 00000000f2000800 [#1]  SMP
Modules linked in: ...
CPU: 2 UID: 0 PID: 1403 Comm: sugov:2 Tainted: G         C          7.4.0-rc1+ #16963 PREEMPT(full)
Tainted: [C]=CRAP
Hardware name: Hardkernel ODROID-N2 (DT)
pstate: 600000c5 (nZCv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : dl_task_root_domain_cpu+0x44/0x48
lr : dl_task_root_domain_cpu+0x28/0x48
sp : ffff8000853dbd00
x29: ffff8000853dbd00 x28: 0000000000000000 x27: 0000000000000000
x26: 0000000000000000 x25: 0000000000000000 x24: ffff0000f4768f40
x23: 0000000000000000 x22: ffff8000853dbdc8 x21: ffff0000f4776480
x20: ffff8000853dbe10 x19: ffff800082e2f000 x18: 00000000ffffffff
x17: 616974696e692064 x16: 657473696c6e7520 x15: ffff800083b338c0
x14: 0000000000000001 x13: 0000000000000000 x12: 0000000000000001
x11: 000000000000000a x10: 0000000000000000 x9 : 000000000000008e
x8 : ffff0000f47770c0 x7 : ffff00000f039db0 x6 : 0000000000000040
x5 : 0000000000000003 x4 : 0000000000000001 x3 : 0000000000000000
x2 : 0000000000000006 x1 : 0000000000000006 x0 : 0000000000000006
Call trace:
 dl_task_root_domain_cpu+0x44/0x48 (P)
 dl_task_needs_bw_move.part.0+0x18/0x60
 set_cpus_allowed_dl+0x2c/0xd8
 __set_cpus_allowed_ptr_locked+0x154/0x1f0
 __set_cpus_allowed_ptr+0x68/0xc0
 set_cpus_allowed_ptr+0x34/0x60
 kthread_affine_node+0xe4/0x10c
 kthread+0x124/0x128
 ret_from_fork+0x10/0x20
Code: f9400bf3 a8c27bfd d50323bf d65f03c0 (d4210000)
---[ end trace 0000000000000000 ]---
note: sugov:2[1403] exited with irqs disabled



> ---
>  kernel/sched/deadline.c | 31 ++++++++++++++++++++++++-------
>  1 file changed, 24 insertions(+), 7 deletions(-)
>
> diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
> index c0ebdcde5fe5d..68019dafbaf18 100644
> --- a/kernel/sched/deadline.c
> +++ b/kernel/sched/deadline.c
> @@ -3324,14 +3324,28 @@ static void task_woken_dl(struct rq *rq, struct task_struct *p)
>  	}
>  }
>  
> +/*
> + * Find the root domain where a DL task's bandwidth is currently
> + * accounted. For a sleeping task that was moved between cpusets
> + * without running, task_rq(p)->rd may be stale. Use the task's
> + * current cpus_mask (not yet updated to the new mask) to find the
> + * actual root domain. Use cpus_mask rather than cpus_ptr, which
> + * may be temporarily narrowed by migrate_disable().
> + */
> +static unsigned int dl_task_root_domain_cpu(struct task_struct *p)
> +{
> +	unsigned int cpu;
> +
> +	cpu = cpumask_first_and(&p->cpus_mask, cpu_active_mask);
> +	BUG_ON(cpu >= nr_cpu_ids);
> +	return cpu;
> +}
> +
>  static void set_cpus_allowed_dl(struct task_struct *p,
>  				struct affinity_context *ctx)
>  {
> -	struct rq *rq;
> -
>  	WARN_ON_ONCE(!dl_task(p));
>  
> -	rq = task_rq(p);
>  	/*
>  	 * Migrating a SCHED_DEADLINE task between exclusive
>  	 * cpusets (different root_domains) entails a bandwidth
> @@ -3339,16 +3353,16 @@ static void set_cpus_allowed_dl(struct task_struct *p,
>  	 * domain (see cpuset_can_attach()).
>  	 */
>  	if (dl_task_needs_bw_move(p, ctx->new_mask)) {
> -		struct dl_bw *src_dl_b;
> +		unsigned int src_cpu = dl_task_root_domain_cpu(p);
> +		struct dl_bw *src_dl_b = &cpu_rq(src_cpu)->rd->dl_bw;
>  
> -		src_dl_b = dl_bw_of(cpu_of(rq));
>  		/*
>  		 * We now free resources of the root_domain we are migrating
>  		 * off. In the worst case, sched_setattr() may temporary fail
>  		 * until we complete the update.
>  		 */
>  		raw_spin_lock(&src_dl_b->lock);
> -		__dl_sub(src_dl_b, p->dl.dl_bw, dl_bw_cpus(task_cpu(p)));
> +		__dl_sub(src_dl_b, p->dl.dl_bw, dl_bw_cpus(src_cpu));
>  		raw_spin_unlock(&src_dl_b->lock);
>  	}
>  
> @@ -3358,10 +3372,13 @@ static void set_cpus_allowed_dl(struct task_struct *p,
>  bool dl_task_needs_bw_move(struct task_struct *p,
>  			   const struct cpumask *new_mask)
>  {
> +	unsigned int cpu;
> +
>  	if (!dl_task(p))
>  		return false;
>  
> -	return !cpumask_intersects(task_rq(p)->rd->span, new_mask);
> +	cpu = dl_task_root_domain_cpu(p);
> +	return !cpumask_intersects(cpu_rq(cpu)->rd->span, new_mask);
>  }
>  
>  /* Assumes rq->lock is held */
>
Best regards
-- 
Marek Szyprowski, PhD
Samsung R&D Institute Poland

Re: [PATCH 1/4] sched/deadline: Fix stale root domain in cpuset bandwidth transfer

[Juri Lelli]

Hello,

On 05/11/26 11:20, Marek Szyprowski wrote:
> On 03.11.2026 12:26, Juri Lelli wrote:
> > dl_task_needs_bw_move() and set_cpus_allowed_dl() use task_rq(p)->rd
> > to determine a task's current root domain. For a sleeping task, this
> > is stale: it points to the rq of the CPU the task last ran on, which
> > may belong to a different root domain after a cpuset move.
> >
> > This causes incorrect bandwidth accounting when a sleeping DL task is
> > moved between exclusive cpuset partitions and back:
> >
> >   1. Move root->partition: task_rq(p)->rd = root, destination in
> >      partition -> different -> bandwidth transferred correctly.
> >   2. Move partition->root: task_rq(p)->rd is still root (task never
> >      ran) -> looks like same domain -> no transfer. Partition keeps
> >      stale bandwidth, root domain is short.
> >
> > The root domain then over-admits, and when the task exits total_bw
> > underflows.
> >
> > Introduce dl_task_root_domain_cpu() helper that uses the task's
> > cpus_mask (which still reflects the old/source cpuset at call time)
> > to find the root domain where the bandwidth is actually accounted,
> > instead of relying on the potentially stale task_rq(p)->rd.
> >
> > Use cpus_mask rather than cpus_ptr, as the latter may be temporarily
> > narrowed to a single CPU by migrate_disable().
> >
> > Reported-by: Weiheng Qiu <weihengqiuu@gmail.com>
> > Closes: https://lore.kernel.org/all/14508cb2-805f-41ac-9f6b-bfd7c3d7cb1c@gmail.com/
> > Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
> This patch landed recently in linux-next as commit b4a61fdc19c5
> ("sched/deadline: Fix stale root domain in cpuset bandwidth transfer").
> I found that it triggers kernel BUG during CPU hot-plug tests on some
> of my tests boards. Reverting it on top of next-20261104 (together with
> dependent 41a7968022ca3) fixes the issue.

Others have found issues with the set and we have been working on fixes.

Can you please check if the following addresses it

https://lore.kernel.org/lkml/20261104211207.1392343-1-mathieu.desnoyers@efficios.com/

Thanks,
Juri

Re: [PATCH 1/4] sched/deadline: Fix stale root domain in cpuset bandwidth transfer

[Marek Szyprowski]

On 05.11.2026 11:38, Juri Lelli wrote:
> On 05/11/26 11:20, Marek Szyprowski wrote:
>> On 03.11.2026 12:26, Juri Lelli wrote:
>>> dl_task_needs_bw_move() and set_cpus_allowed_dl() use task_rq(p)->rd
>>> to determine a task's current root domain. For a sleeping task, this
>>> is stale: it points to the rq of the CPU the task last ran on, which
>>> may belong to a different root domain after a cpuset move.
>>>
>>> This causes incorrect bandwidth accounting when a sleeping DL task is
>>> moved between exclusive cpuset partitions and back:
>>>
>>>   1. Move root->partition: task_rq(p)->rd = root, destination in
>>>      partition -> different -> bandwidth transferred correctly.
>>>   2. Move partition->root: task_rq(p)->rd is still root (task never
>>>      ran) -> looks like same domain -> no transfer. Partition keeps
>>>      stale bandwidth, root domain is short.
>>>
>>> The root domain then over-admits, and when the task exits total_bw
>>> underflows.
>>>
>>> Introduce dl_task_root_domain_cpu() helper that uses the task's
>>> cpus_mask (which still reflects the old/source cpuset at call time)
>>> to find the root domain where the bandwidth is actually accounted,
>>> instead of relying on the potentially stale task_rq(p)->rd.
>>>
>>> Use cpus_mask rather than cpus_ptr, as the latter may be temporarily
>>> narrowed to a single CPU by migrate_disable().
>>>
>>> Reported-by: Weiheng Qiu <weihengqiuu@gmail.com>
>>> Closes: https://lore.kernel.org/all/14508cb2-805f-41ac-9f6b-bfd7c3d7cb1c@gmail.com/
>>> Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
>> This patch landed recently in linux-next as commit b4a61fdc19c5
>> ("sched/deadline: Fix stale root domain in cpuset bandwidth transfer").
>> I found that it triggers kernel BUG during CPU hot-plug tests on some
>> of my tests boards. Reverting it on top of next-20261104 (together with
>> dependent 41a7968022ca3) fixes the issue.
> Others have found issues with the set and we have been working on fixes.
>
> Can you please check if the following addresses it
>
> https://lore.kernel.org/lkml/20261104211207.1392343-1-mathieu.desnoyers@efficios.com/

I've applied the whole patchset and ran my tests. Everything works fine now. Thanks!

Best regards
-- 
Marek Szyprowski, PhD
Samsung R&D Institute Poland

Re: [PATCH 1/4] sched/deadline: Fix stale root domain in cpuset bandwidth transfer

[Ingo Molnar]

* Juri Lelli <juri.lelli@redhat.com> wrote:

> Hello,
> 
> On 05/11/26 11:20, Marek Szyprowski wrote:
> > On 03.11.2026 12:26, Juri Lelli wrote:
> > > dl_task_needs_bw_move() and set_cpus_allowed_dl() use task_rq(p)->rd
> > > to determine a task's current root domain. For a sleeping task, this
> > > is stale: it points to the rq of the CPU the task last ran on, which
> > > may belong to a different root domain after a cpuset move.
> > >
> > > This causes incorrect bandwidth accounting when a sleeping DL task is
> > > moved between exclusive cpuset partitions and back:
> > >
> > >   1. Move root->partition: task_rq(p)->rd = root, destination in
> > >      partition -> different -> bandwidth transferred correctly.
> > >   2. Move partition->root: task_rq(p)->rd is still root (task never
> > >      ran) -> looks like same domain -> no transfer. Partition keeps
> > >      stale bandwidth, root domain is short.
> > >
> > > The root domain then over-admits, and when the task exits total_bw
> > > underflows.
> > >
> > > Introduce dl_task_root_domain_cpu() helper that uses the task's
> > > cpus_mask (which still reflects the old/source cpuset at call time)
> > > to find the root domain where the bandwidth is actually accounted,
> > > instead of relying on the potentially stale task_rq(p)->rd.
> > >
> > > Use cpus_mask rather than cpus_ptr, as the latter may be temporarily
> > > narrowed to a single CPU by migrate_disable().
> > >
> > > Reported-by: Weiheng Qiu <weihengqiuu@gmail.com>
> > > Closes: https://lore.kernel.org/all/14508cb2-805f-41ac-9f6b-bfd7c3d7cb1c@gmail.com/
> > > Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
> > This patch landed recently in linux-next as commit b4a61fdc19c5
> > ("sched/deadline: Fix stale root domain in cpuset bandwidth transfer").
> > I found that it triggers kernel BUG during CPU hot-plug tests on some
> > of my tests boards. Reverting it on top of next-20261104 (together with
> > dependent 41a7968022ca3) fixes the issue.
> 
> Others have found issues with the set and we have been working on fixes.
> 
> Can you please check if the following addresses it
> 
> https://lore.kernel.org/lkml/20261104211207.1392343-1-mathieu.desnoyers@efficios.com/

So given that the first patch triggers a crash, I've removed
these fixes from sched/urgent for the time being, waiting
for an updated series.

Mind sending a series that is strictly -stable suitable fixes only?

Thanks,

	Ingo

Re: [PATCH 1/4] sched/deadline: Fix stale root domain in cpuset bandwidth transfer

[Juri Lelli]

Hi Ingo,

On 05/11/26 12:22, Ingo Molnar wrote:
> 
> * Juri Lelli <juri.lelli@redhat.com> wrote:
> 
> > Hello,
> > 
> > On 05/11/26 11:20, Marek Szyprowski wrote:
> > > On 03.11.2026 12:26, Juri Lelli wrote:
> > > > dl_task_needs_bw_move() and set_cpus_allowed_dl() use task_rq(p)->rd
> > > > to determine a task's current root domain. For a sleeping task, this
> > > > is stale: it points to the rq of the CPU the task last ran on, which
> > > > may belong to a different root domain after a cpuset move.
> > > >
> > > > This causes incorrect bandwidth accounting when a sleeping DL task is
> > > > moved between exclusive cpuset partitions and back:
> > > >
> > > >   1. Move root->partition: task_rq(p)->rd = root, destination in
> > > >      partition -> different -> bandwidth transferred correctly.
> > > >   2. Move partition->root: task_rq(p)->rd is still root (task never
> > > >      ran) -> looks like same domain -> no transfer. Partition keeps
> > > >      stale bandwidth, root domain is short.
> > > >
> > > > The root domain then over-admits, and when the task exits total_bw
> > > > underflows.
> > > >
> > > > Introduce dl_task_root_domain_cpu() helper that uses the task's
> > > > cpus_mask (which still reflects the old/source cpuset at call time)
> > > > to find the root domain where the bandwidth is actually accounted,
> > > > instead of relying on the potentially stale task_rq(p)->rd.
> > > >
> > > > Use cpus_mask rather than cpus_ptr, as the latter may be temporarily
> > > > narrowed to a single CPU by migrate_disable().
> > > >
> > > > Reported-by: Weiheng Qiu <weihengqiuu@gmail.com>
> > > > Closes: https://lore.kernel.org/all/14508cb2-805f-41ac-9f6b-bfd7c3d7cb1c@gmail.com/
> > > > Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
> > > This patch landed recently in linux-next as commit b4a61fdc19c5
> > > ("sched/deadline: Fix stale root domain in cpuset bandwidth transfer").
> > > I found that it triggers kernel BUG during CPU hot-plug tests on some
> > > of my tests boards. Reverting it on top of next-20261104 (together with
> > > dependent 41a7968022ca3) fixes the issue.
> > 
> > Others have found issues with the set and we have been working on fixes.
> > 
> > Can you please check if the following addresses it
> > 
> > https://lore.kernel.org/lkml/20261104211207.1392343-1-mathieu.desnoyers@efficios.com/
> 
> So given that the first patch triggers a crash, I've removed
> these fixes from sched/urgent for the time being, waiting
> for an updated series.

Thanks for doing that and apologies for the noise my broken series has
caused.

> Mind sending a series that is strictly -stable suitable fixes only?

Reviewing the above and will work on what the series doesn't cover from
my original set of (attempted) fixes.

Best,
Juri