* [PATCH 00/16] sched/deadline: first bandwidth accounting fixes and selftests
@ 2026-11-18 20:06 Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 01/16] sched: Refuse SCHED_FLAG_KEEP_PARAMS that would change the policy or the class Mathieu Desnoyers
` (15 more replies)
0 siblings, 16 replies; 31+ messages in thread
From: Mathieu Desnoyers @ 2026-11-18 20:06 UTC (permalink / raw)
To: juri.lelli, peterz, mingo, vincent.guittot
Cc: linux-kernel, Mathieu Desnoyers, tglx, will, boqun,
dietmar.eggemann, rostedt, bsegall, mgorman, vschneid,
kprateek.nayak, luca.abeni, mathieu.poirier, longman, ridong.chen,
tj, hannes, mkoutny, shuah, bigeasy, clrkwllms, parri.andrea,
m.szyprowski, kayracizmeci, weihengqiuu, cgroups, linux-kselftest,
linux-rt-devel
Hi Ingo, Juri,
This series fixes 13 bugs of SCHED_DEADLINE. Then it shows the deadline
bandwidth of each runqueue in sched_debug, and adds two selftests. The
fixes come first, so that stable can take patches 1 to 13.
Two of the bugs take the kernel down: a division by zero (patch 4), and
an owner's CPU that locks up (patch 8). One leaves cpuset_mutex locked
(patch 6). One leaves a task that never runs again (patch 7). Most of
the others leave total_bw of a root domain, or this_bw of a runqueue,
wrong. The root domain then admits deadline tasks beyond its capacity,
or refuses every one.
These are the first 14 patches of "sched/deadline: bandwidth accounting
fixes and selftests" v7 [1], as Juri proposed [2]. Their v7 numbers are
1-8, 10, 13, 14, 16, 20 and 37. The rest of v7 will follow as v8 of that
series, on top of this one.
Patch 11 is Juri's, patch 13 is Andrea Parri's. I carry them unchanged.
Sebastian reviewed patch 13. Juri, three of your four fixes that Ingo
dropped from sched/urgent are here:
- 2/4 is patch 10.
- 3/4 is patch 1.
- 4/4 is patch 11.
1/4 stays in the second series.
The fixes
=========
1 Refuse SCHED_FLAG_KEEP_PARAMS requests that would change the policy
or the class.
2 Skip admission for parameters that SCHED_FLAG_KEEP_PARAMS keeps.
3 Let a task boosted by a deadline waiter change its policy.
4 Do not divide by zero active CPUs of a root domain.
5 Refuse a dl_server debugfs write while the runqueue is offline.
6 Take cpuset_mutex for a deadline policy change only with pi, so
that sysrq-n does not leave it locked.
7 Replenish a throttled task switched out and back, so that it runs
again.
8 Do not refill a boost from cleared parameters, so that the owner's
CPU no longer locks up.
9 Reset the deadline and the runtime with the parameters.
10 Release the bandwidth of a task that leaves the class asleep.
11 Reject SCHED_DEADLINE on exited tasks (Juri Lelli).
12 Keep this_bw right when a task changes during a remote wakeup.
13 Check the affinity of a deadline task under the runqueue lock, so
that sched_setaffinity() and sched_setattr() cannot both succeed
(Andrea Parri).
Patch 14 shows the deadline bandwidth of each runqueue in sched_debug.
Patch 15 adds deadline_switch_test. Patch 16 adds deadline_bw_test, with
the 47 of its 112 cases that pass with these fixes. The other cases
come with their fixes, in the second series.
Known limit
===========
Patch 10 clears the parameters of a task that leaves the class asleep.
On v7.4-rc2, such a task kept them. That hid a bug of the rtmutex chain
walk: an owner can keep as its donor a deadline task that left the
class. Another deadline task then gets too little CPU time. With patch
10, the case pi_chain_top_moves shows this in every run. It passes on
v7.4-rc2. The case is new in the second series, with the two rtmutex
fixes that make it pass with patch 10.
Testing
=======
The 52 cases pass wherever they run, with lockdep and no kernel
warning, in 25 QEMU guests:
- 9 guests of 8 vCPUs, one with proxy execution off, and 1 of 4 vCPUs.
- 5 guests with PREEMPT_RT, 4 of 8 vCPUs and 1 of 4.
- 3 guests without CONFIG_CPUSETS, of 4, 4 and 5 vCPUs.
- 4 guests of 5 vCPUs booted with isolcpus=domain,1,2 or
isolcpus=domain,0,1, 2 of them with PREEMPT_RT.
- 1 guest of 8 vCPUs booted with isolcpus=domain,6,7.
- 2 guests with CONFIG_RT_GROUP_SCHED, of 8 and 4 vCPUs.
move_cancelled runs only in the last group. Without CONFIG_CPUSETS, the
cases that need a cpuset skip. With 3 CPUs that are not isolated,
params_waking and rebuild_record_taken skip: they need 4.
On v7.4-rc2, six cases of deadline_bw_test fail in each of 8 guests of 8
vCPUs. Patch 16 lists them. overrun_caught fails in one guest. In
another one, hotplug_server crashed the kernel with the division by zero
that patch 4 fixes. throttled_round_trip and asleep_round_trip of
deadline_switch_test fail too. The sysrq cases did not run there:
sysrq-n leaves cpuset_mutex locked, which patch 6 fixes. In earlier runs
with them, sysrq_normalize failed.
Each of the 14 kernel patches builds alone. The whole series also
builds without CONFIG_CPUSETS, with PREEMPT_RT and with
CONFIG_RT_GROUP_SCHED. The selftests build with no warning.
Not tested: cgroup v1 cpusets, the sched_ext server, and CPUs of
unequal capacity.
How this was developed
======================
An LLM coding assistant wrote these patches, their messages and the
tests, under my direction and review. It did not write patches 11 and
13. I know the concepts behind SCHED_DEADLINE, but not its
implementation well. So I ask those who know this code for a careful
reading of the fixes. A reproducer that stops failing shows that the
symptom goes away. It does not show that the fix is the one you would
have written.
The bugs came from:
- flaky cgroup selftests of a feature I am developing;
- the review of Juri's four fixes, and his review of v5;
- building deadline_bw_test, and reading the code;
- the Sashiko AI reviews of v1 to v7. Each finding was checked against
the code first.
Changes since v7
================
- Reviewed-by from Juri on 4, 5, 8 and 10. He reviewed the v7 code of
5, before the change listed below.
- 5: the refusal covers the whole CPU deactivation, from the Sashiko
review of v7.
- 8: the fix paragraph is Juri's wording.
- 10: its message names asleep_round_trip, which tests it without PI.
The two PI cases it named move to the second series.
- 15: the stack of the filler threads is at least PTHREAD_STACK_MIN.
The semaphores are destroyed.
- The messages credit the Sashiko AI review without the number of the
posting, as Juri suggested.
Based on tip sched/urgent, ec6f6731538e (v7.4-rc2).
[1] https://lore.kernel.org/all/cover.1794922146.git.mathieu.desnoyers@efficios.com/
[2] https://lore.kernel.org/all/av3J3zMfkRplg-N4@jlelli-thinkpadt14gen4.remote.csb/
Andrea Parri (1):
sched/deadline: Check SCHED_DEADLINE affinity under the rq lock
Juri Lelli (1):
sched/deadline: Reject SCHED_DEADLINE on exited tasks
Mathieu Desnoyers (14):
sched: Refuse SCHED_FLAG_KEEP_PARAMS that would change the policy or
the class
sched/deadline: Do not admit bandwidth for parameters that are kept
sched: Let a task boosted by a deadline waiter change its policy
sched/deadline: Do not divide by zero active CPUs of a root domain
sched/deadline: Refuse dl_server debugfs writes while the rq is
offline
sched: Take cpuset_mutex for a deadline policy change only with pi
sched/deadline: Do not lose a throttled task switched out and back
sched/deadline: Do not replenish a boost from cleared parameters
sched/deadline: Reset the deadline and the runtime with the parameters
sched/deadline: Release the bandwidth of a task that leaves the class
asleep
sched/deadline: Fix this_bw when a task is changed during a remote
wakeup
sched/debug: Show the deadline bandwidth of each runqueue
selftests/sched: Add a test of SCHED_DEADLINE policy round trips
selftests/sched: Add a test of SCHED_DEADLINE bandwidth accounting
include/linux/sched.h | 5 +
kernel/sched/core.c | 11 +
kernel/sched/deadline.c | 125 +-
kernel/sched/debug.c | 13 +-
kernel/sched/sched.h | 1 +
kernel/sched/syscalls.c | 103 +-
tools/testing/selftests/Makefile | 1 +
.../selftests/sched/deadline/.gitignore | 2 +
.../testing/selftests/sched/deadline/Makefile | 8 +
tools/testing/selftests/sched/deadline/config | 5 +
.../sched/deadline/deadline_bw_test.c | 5468 +++++++++++++++++
.../sched/deadline/deadline_switch_test.c | 1115 ++++
.../testing/selftests/sched/deadline/settings | 1 +
13 files changed, 6829 insertions(+), 29 deletions(-)
create mode 100644 tools/testing/selftests/sched/deadline/.gitignore
create mode 100644 tools/testing/selftests/sched/deadline/Makefile
create mode 100644 tools/testing/selftests/sched/deadline/config
create mode 100644 tools/testing/selftests/sched/deadline/deadline_bw_test.c
create mode 100644 tools/testing/selftests/sched/deadline/deadline_switch_test.c
create mode 100644 tools/testing/selftests/sched/deadline/settings
base-commit: ec6f6731538e0d45872eb279fdeb477bed152d6f
--
2.47.3
^ permalink raw reply [flat|nested] 31+ messages in thread
* [PATCH 01/16] sched: Refuse SCHED_FLAG_KEEP_PARAMS that would change the policy or the class
2026-11-18 20:06 [PATCH 00/16] sched/deadline: first bandwidth accounting fixes and selftests Mathieu Desnoyers
@ 2026-11-18 20:06 ` Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 02/16] sched/deadline: Do not admit bandwidth for parameters that are kept Mathieu Desnoyers
` (14 subsequent siblings)
15 siblings, 0 replies; 31+ messages in thread
From: Mathieu Desnoyers @ 2026-11-18 20:06 UTC (permalink / raw)
To: juri.lelli, peterz, mingo, vincent.guittot
Cc: linux-kernel, Mathieu Desnoyers, tglx, will, boqun,
dietmar.eggemann, rostedt, bsegall, mgorman, vschneid,
kprateek.nayak, luca.abeni, mathieu.poirier, longman, ridong.chen,
tj, hannes, mkoutny, shuah, bigeasy, clrkwllms, parri.andrea,
m.szyprowski, kayracizmeci, weihengqiuu, cgroups, linux-kselftest,
linux-rt-devel, stable
A sched_setattr() request with SCHED_FLAG_KEEP_PARAMS can ask for
another policy. __sched_setscheduler() then applies only half of the
change, and returns 0:
- SCHED_OTHER to SCHED_BATCH leaves the policy as it was.
- SCHED_DEADLINE to SCHED_OTHER sets DEQUEUE_CLASS. switched_from_dl()
then runs on a task that stays in the class. It would release
bandwidth the task still uses.
- SCHED_DEADLINE on a task of another policy reserves bandwidth in
sched_dl_overflow(). The task keeps its policy and a dl_bw of 0, so
nothing releases that bandwidth. For a task boosted into the deadline
class, it is not even a class change.
SCHED_FLAG_KEEP_PARAMS came with utilization clamping. With
SCHED_FLAG_KEEP_POLICY, it lets sched_setattr() change only the clamps
of a task. So __sched_setscheduler() skips __setscheduler_params(),
which sets p->policy, and the sched_class assignment. Nothing checks
that the request keeps the policy.
In testing on v7.4-rc2, a deadline waiter boosted a SCHED_OTHER mutex
owner with a 40ms slice. One such call on the owner left 0.4 CPU in
total_bw, until the next rebuild of the root domains.
Fix this by refusing a request for another policy or class with
EINVAL. Move sched_dl_overflow() after it, as the last check. Nothing
gives back the bandwidth it reserves. A refused request also leaves
sched_reset_on_fork alone.
With this patch, that call is refused and total_bw does not change. The
params_keep_boosted case of deadline_bw_test, added later in this
series, tests it.
Juri Lelli found the class change, from a report by Tejun Heo. The
Sashiko AI review of this series found the policy change and the boosted
case.
Fixes: a509a7cd7974 ("sched/uclamp: Extend sched_setattr() to support utilization clamping")
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Reviewed-by: Juri Lelli <juri.lelli@redhat.com>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/all/2b5fb3f22bca6aa383be8f4781923392@kernel.org/
Link: https://lore.kernel.org/all/20261103-b4-sched-dl-bw-fixes-v1-3-dd761fe005b4@redhat.com/
---
kernel/sched/syscalls.c | 40 +++++++++++++++++++++++++++++-----------
1 file changed, 29 insertions(+), 11 deletions(-)
diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c
index 0ec401af80d1..89b0d5787f45 100644
--- a/kernel/sched/syscalls.c
+++ b/kernel/sched/syscalls.c
@@ -647,17 +647,6 @@ int __sched_setscheduler(struct task_struct *p,
goto recheck;
}
- /*
- * If setscheduling to SCHED_DEADLINE (or changing the parameters
- * of a SCHED_DEADLINE task) we need to check if enough bandwidth
- * is available.
- */
- if ((dl_policy(policy) || dl_task(p)) && sched_dl_overflow(p, policy, attr)) {
- retval = -EBUSY;
- goto unlock;
- }
-
- p->sched_reset_on_fork = reset_on_fork;
oldprio = p->prio;
newprio = __normal_prio(policy, attr->sched_priority, attr->sched_nice);
@@ -677,9 +666,38 @@ int __sched_setscheduler(struct task_struct *p,
prev_class = p->sched_class;
next_class = __setscheduler_class(policy, newprio);
+ /*
+ * With SCHED_FLAG_KEEP_PARAMS, the change below skips
+ * __setscheduler_params() and the sched_class assignment. The
+ * policy and the class stay as they are. Refuse a request for
+ * another policy or class, rather than carry out half of it.
+ *
+ * The class depends on the priority, which priority inheritance
+ * can raise. It also depends on whether a BPF scheduler is in
+ * charge. So test the class too, not only the policy.
+ */
+ if ((attr->sched_flags & SCHED_FLAG_KEEP_PARAMS) &&
+ (policy != p->policy || prev_class != next_class)) {
+ retval = -EINVAL;
+ goto unlock;
+ }
+
if (prev_class != next_class)
queue_flags |= DEQUEUE_CLASS;
+ /*
+ * If setscheduling to SCHED_DEADLINE (or changing the parameters
+ * of a SCHED_DEADLINE task) we need to check if enough bandwidth
+ * is available. Keep this check last. If a later check failed,
+ * nothing would give back the bandwidth it reserves.
+ */
+ if ((dl_policy(policy) || dl_task(p)) && sched_dl_overflow(p, policy, attr)) {
+ retval = -EBUSY;
+ goto unlock;
+ }
+
+ p->sched_reset_on_fork = reset_on_fork;
+
scx_prepare_setscheduler(p, policy);
scoped_guard (sched_change, p, queue_flags) {
--
2.47.3
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 02/16] sched/deadline: Do not admit bandwidth for parameters that are kept
2026-11-18 20:06 [PATCH 00/16] sched/deadline: first bandwidth accounting fixes and selftests Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 01/16] sched: Refuse SCHED_FLAG_KEEP_PARAMS that would change the policy or the class Mathieu Desnoyers
@ 2026-11-18 20:06 ` Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 03/16] sched: Let a task boosted by a deadline waiter change its policy Mathieu Desnoyers
` (13 subsequent siblings)
15 siblings, 0 replies; 31+ messages in thread
From: Mathieu Desnoyers @ 2026-11-18 20:06 UTC (permalink / raw)
To: juri.lelli, peterz, mingo, vincent.guittot
Cc: linux-kernel, Mathieu Desnoyers, tglx, will, boqun,
dietmar.eggemann, rostedt, bsegall, mgorman, vschneid,
kprateek.nayak, luca.abeni, mathieu.poirier, longman, ridong.chen,
tj, hannes, mkoutny, shuah, bigeasy, clrkwllms, parri.andrea,
m.szyprowski, kayracizmeci, weihengqiuu, cgroups, linux-kselftest,
linux-rt-devel, stable
__sched_setscheduler() runs the deadline admission test on a request
with SCHED_FLAG_KEEP_PARAMS. But the task keeps its parameters: the
call skips __setscheduler_params().
Commit f4dddf90d58d ("sched: Skip priority checks with
SCHED_FLAG_KEEP_PARAMS") made sys_sched_setattr() copy the task's own
parameters into such a request. Then the priority checks pass.
get_params() copies them without a lock.
Another call can change the task's parameters before
__sched_setscheduler() takes the runqueue lock. The request is then
stale, and dl_param_changed() takes it for a change. sched_dl_overflow()
would put the stale bandwidth in place of the task's own in total_bw,
and in this_bw for a sleeping task. But the task keeps its own dl_bw.
Its release at the 0-lag time takes that dl_bw, and the difference
stays behind.
Fix this by skipping sched_dl_overflow() for SCHED_FLAG_KEEP_PARAMS.
There is nothing to admit for parameters that are kept.
Found by reading, after the Sashiko AI review of this series raised the
boosted case of the previous patch. Not reproduced: the parameter change
must land inside that window.
Fixes: a509a7cd7974 ("sched/uclamp: Extend sched_setattr() to support utilization clamping")
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Reviewed-by: Juri Lelli <juri.lelli@redhat.com>
Cc: stable@vger.kernel.org
---
kernel/sched/syscalls.c | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c
index 89b0d5787f45..6f8ce5fd94a1 100644
--- a/kernel/sched/syscalls.c
+++ b/kernel/sched/syscalls.c
@@ -690,8 +690,16 @@ int __sched_setscheduler(struct task_struct *p,
* of a SCHED_DEADLINE task) we need to check if enough bandwidth
* is available. Keep this check last. If a later check failed,
* nothing would give back the bandwidth it reserves.
+ *
+ * With SCHED_FLAG_KEEP_PARAMS, the task keeps its parameters, so
+ * there is nothing to admit. Such a call changes other attributes,
+ * such as the utilization clamps. A deadline task does not use its
+ * clamps, but keeps them for when it leaves the class. The
+ * parameters in attr may also be stale: sys_sched_setattr() read
+ * them from the task without a lock.
*/
- if ((dl_policy(policy) || dl_task(p)) && sched_dl_overflow(p, policy, attr)) {
+ if (!(attr->sched_flags & SCHED_FLAG_KEEP_PARAMS) &&
+ (dl_policy(policy) || dl_task(p)) && sched_dl_overflow(p, policy, attr)) {
retval = -EBUSY;
goto unlock;
}
--
2.47.3
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 03/16] sched: Let a task boosted by a deadline waiter change its policy
2026-11-18 20:06 [PATCH 00/16] sched/deadline: first bandwidth accounting fixes and selftests Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 01/16] sched: Refuse SCHED_FLAG_KEEP_PARAMS that would change the policy or the class Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 02/16] sched/deadline: Do not admit bandwidth for parameters that are kept Mathieu Desnoyers
@ 2026-11-18 20:06 ` Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 04/16] sched/deadline: Do not divide by zero active CPUs of a root domain Mathieu Desnoyers
` (12 subsequent siblings)
15 siblings, 0 replies; 31+ messages in thread
From: Mathieu Desnoyers @ 2026-11-18 20:06 UTC (permalink / raw)
To: juri.lelli, peterz, mingo, vincent.guittot
Cc: linux-kernel, Mathieu Desnoyers, tglx, will, boqun,
dietmar.eggemann, rostedt, bsegall, mgorman, vschneid,
kprateek.nayak, luca.abeni, mathieu.poirier, longman, ridong.chen,
tj, hannes, mkoutny, shuah, bigeasy, clrkwllms, parri.andrea,
m.szyprowski, kayracizmeci, weihengqiuu, cgroups, linux-kselftest,
linux-rt-devel, stable
A SCHED_OTHER or SCHED_FIFO task that holds an rt_mutex can be boosted
into the deadline class by a deadline waiter. Then a sched_setattr() or
sched_setscheduler() call on it fails with EBUSY, unless it asks for
SCHED_DEADLINE. sysrq-n would not reset it either.
The admission test runs "if setscheduling to SCHED_DEADLINE (or
changing the parameters of a SCHED_DEADLINE task)", as its comment
says. sched_dl_overflow() then handles three cases, by the task's
policy: the task enters, leaves or stays in SCHED_DEADLINE. But the
call site tests dl_task(p). That tests the priority, so a boosted task
passes it. For such a task and another policy, sched_dl_overflow()
matches no case and returns -1.
Fix this by testing the policy, with task_has_dl_policy(). A boosted
task has no bandwidth of its own to admit or release.
Now sysrq-n can take a boosted task out of the class.
normalize_rt_tasks() calls __sched_setscheduler() without pi, so the
task gets its normal priority. Nothing then drops the boost. At the
unlock, rt_mutex_setprio() sees no change of priority. The pi_se of the
task would keep pointing to the waiter, which can exit. So also drop the
boost in __setscheduler_dl_pi() when the task leaves the class, as
rt_mutex_setprio() does.
The boosted_owner_policy case of deadline_bw_test, added later in this
series, fails with EBUSY on v7.4-rc2. It passes with this series. The
sysrq_boosted_owner case of deadline_switch_test tests the sysrq-n path.
Fixes: 332ac17ef5bf ("sched/deadline: Add bandwidth management for SCHED_DEADLINE tasks")
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Reviewed-by: Juri Lelli <juri.lelli@redhat.com>
Cc: stable@vger.kernel.org
---
kernel/sched/syscalls.c | 14 +++++++++++++-
1 file changed, 13 insertions(+), 1 deletion(-)
diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c
index 6f8ce5fd94a1..cb115aa9b0b3 100644
--- a/kernel/sched/syscalls.c
+++ b/kernel/sched/syscalls.c
@@ -304,6 +304,13 @@ static inline void __setscheduler_dl_pi(int newprio, int policy,
scope->flags |= ENQUEUE_REPLENISH;
}
}
+
+ /*
+ * Without pi, as from normalize_rt_tasks(), a boosted task can leave
+ * the class. Drop its boost then, as rt_mutex_setprio() does.
+ */
+ if (!dl_prio(newprio))
+ p->dl.pi_se = &p->dl;
}
#else /* !CONFIG_RT_MUTEXES */
static inline void __setscheduler_dl_pi(int newprio, int policy,
@@ -691,6 +698,10 @@ int __sched_setscheduler(struct task_struct *p,
* is available. Keep this check last. If a later check failed,
* nothing would give back the bandwidth it reserves.
*
+ * Test the policy, not the priority. A task boosted into the class
+ * by priority inheritance has no bandwidth of its own to admit or
+ * release.
+ *
* With SCHED_FLAG_KEEP_PARAMS, the task keeps its parameters, so
* there is nothing to admit. Such a call changes other attributes,
* such as the utilization clamps. A deadline task does not use its
@@ -699,7 +710,8 @@ int __sched_setscheduler(struct task_struct *p,
* them from the task without a lock.
*/
if (!(attr->sched_flags & SCHED_FLAG_KEEP_PARAMS) &&
- (dl_policy(policy) || dl_task(p)) && sched_dl_overflow(p, policy, attr)) {
+ (dl_policy(policy) || task_has_dl_policy(p)) &&
+ sched_dl_overflow(p, policy, attr)) {
retval = -EBUSY;
goto unlock;
}
--
2.47.3
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 04/16] sched/deadline: Do not divide by zero active CPUs of a root domain
2026-11-18 20:06 [PATCH 00/16] sched/deadline: first bandwidth accounting fixes and selftests Mathieu Desnoyers
` (2 preceding siblings ...)
2026-11-18 20:06 ` [PATCH 03/16] sched: Let a task boosted by a deadline waiter change its policy Mathieu Desnoyers
@ 2026-11-18 20:06 ` Mathieu Desnoyers
2026-11-18 20:23 ` sashiko-bot
2026-11-18 20:06 ` [PATCH 05/16] sched/deadline: Refuse dl_server debugfs writes while the rq is offline Mathieu Desnoyers
` (11 subsequent siblings)
15 siblings, 1 reply; 31+ messages in thread
From: Mathieu Desnoyers @ 2026-11-18 20:06 UTC (permalink / raw)
To: juri.lelli, peterz, mingo, vincent.guittot
Cc: linux-kernel, Mathieu Desnoyers, tglx, will, boqun,
dietmar.eggemann, rostedt, bsegall, mgorman, vschneid,
kprateek.nayak, luca.abeni, mathieu.poirier, longman, ridong.chen,
tj, hannes, mkoutny, shuah, bigeasy, clrkwllms, parri.andrea,
m.szyprowski, kayracizmeci, weihengqiuu, cgroups, linux-kselftest,
linux-rt-devel, stable
__dl_sub() and __dl_add() divide by their cpus argument. Most callers
pass dl_bw_cpus(), the number of active CPUs of a root domain. That
number can be 0, and the division then traps.
Commit daec57983670 ("sched/deadline: Reclaim bandwidth not used by dl
tasks") added extra_bw, the bandwidth that deadline tasks can reclaim
on a runqueue. A change of a root domain's total_bw is spread over the
extra_bw of its active CPUs.
During hotplug, a CPU sits in the default root domain, which can then
have no active CPU. This lasts from the rebuild in
sched_cpu_deactivate() to the offline, and from the online to
sched_cpu_activate(). A debugfs write to the runtime or period of its
fair or ext server divides there. A write of the current value passes
admission in the window. In 8-CPU VMs of v7.4-rc2, a loop of such writes
oopses at the first offline and online:
Oops: divide error: 0000 [#1] SMP NOPTI
RIP: 0010:dl_server_apply_params+0x335/0x380
Call Trace:
sched_server_write_common.isra.0+0x1c0/0x260
full_proxy_write+0x58/0x90
vfs_write+0xcb/0x550
By reading, two other paths would divide by zero:
- inactive_task_timer(), for a task that left SCHED_DEADLINE or died,
if its CPU went offline.
- sched_setattr(SCHED_DEADLINE) on a task asleep on an offline CPU,
with admission control off.
Fix this by changing total_bw alone when the root domain has no active
CPU.
With this patch, the reproducer survives 20 rounds with no warning.
Fixes: daec57983670 ("sched/deadline: Reclaim bandwidth not used by dl tasks")
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Reviewed-by: Juri Lelli <juri.lelli@redhat.com>
Cc: stable@vger.kernel.org
---
kernel/sched/deadline.c | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index dfae08625342..b614d5fc1a1a 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -189,18 +189,25 @@ void __dl_update(struct dl_bw *dl_b, s64 bw)
}
}
+/*
+ * Spread a change of total_bw over the extra_bw of the active CPUs of the
+ * root domain. @cpus is their number. With no active CPU, as can happen
+ * during CPU hotplug, change total_bw alone.
+ */
static inline
void __dl_sub(struct dl_bw *dl_b, u64 tsk_bw, int cpus)
{
dl_b->total_bw -= tsk_bw;
- __dl_update(dl_b, (s32)tsk_bw / cpus);
+ if (cpus)
+ __dl_update(dl_b, (s32)tsk_bw / cpus);
}
static inline
void __dl_add(struct dl_bw *dl_b, u64 tsk_bw, int cpus)
{
dl_b->total_bw += tsk_bw;
- __dl_update(dl_b, -((s32)tsk_bw / cpus));
+ if (cpus)
+ __dl_update(dl_b, -((s32)tsk_bw / cpus));
}
static inline bool
--
2.47.3
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 05/16] sched/deadline: Refuse dl_server debugfs writes while the rq is offline
2026-11-18 20:06 [PATCH 00/16] sched/deadline: first bandwidth accounting fixes and selftests Mathieu Desnoyers
` (3 preceding siblings ...)
2026-11-18 20:06 ` [PATCH 04/16] sched/deadline: Do not divide by zero active CPUs of a root domain Mathieu Desnoyers
@ 2026-11-18 20:06 ` Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 06/16] sched: Take cpuset_mutex for a deadline policy change only with pi Mathieu Desnoyers
` (10 subsequent siblings)
15 siblings, 0 replies; 31+ messages in thread
From: Mathieu Desnoyers @ 2026-11-18 20:06 UTC (permalink / raw)
To: juri.lelli, peterz, mingo, vincent.guittot
Cc: linux-kernel, Mathieu Desnoyers, tglx, will, boqun,
dietmar.eggemann, rostedt, bsegall, mgorman, vschneid,
kprateek.nayak, luca.abeni, mathieu.poirier, longman, ridong.chen,
tj, hannes, mkoutny, shuah, bigeasy, clrkwllms, parri.andrea,
m.szyprowski, kayracizmeci, weihengqiuu, cgroups, linux-kselftest,
linux-rt-devel, stable
Commit 4043f5498416 ("sched/deadline: Reject debugfs dl_server writes
for offline CPUs") refuses a write to a dl_server's runtime or period
on an offline CPU. Its message says that such a write has no meaningful
semantics.
It tests cpu_online(). But during hotplug the CPU is still online while
it sits in the default root domain. This lasts from the rebuild in
sched_cpu_deactivate() to the offline, and from the online to
sched_cpu_activate(). That root domain does not count the server's
bandwidth. A write in the window still goes through admission. With
admission control off, it changes total_bw there. On v7.4-rc2, a write
of the current value divides by zero in that window. The previous patch
removes the division, not the write.
Fix this by testing rq->online instead. It is set under the runqueue
lock when the CPU joins a root domain that counts it, after
cpu_active() is set. Test cpu_active() too. On the way down, it is
cleared before rq->online.
A write now fails with EBUSY while the CPU goes offline or comes
online, as a write on an offline CPU already does. Writes at any other
time are unchanged.
Applied without the previous patch, this patch alone also stops the
oops. The reproducer survives 20 rounds with no warning, in 3 VMs of 3.
Fixes: d741f297bcea ("sched/fair: Fair server interface")
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Reviewed-by: Juri Lelli <juri.lelli@redhat.com>
Cc: stable@vger.kernel.org
---
kernel/sched/debug.c | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c
index e6a3b516c703..ddb382df454b 100644
--- a/kernel/sched/debug.c
+++ b/kernel/sched/debug.c
@@ -410,7 +410,16 @@ static ssize_t sched_server_write_common(struct file *filp, const char __user *u
return -EINVAL;
}
- if (!cpu_online(cpu_of(rq)))
+ /*
+ * Refuse the write while the runqueue is offline. This
+ * includes the windows where the CPU goes offline or comes
+ * online. The root domain then does not count the server's
+ * bandwidth, and may have no active CPU. rq->online is set
+ * under this lock when the CPU joins a root domain that
+ * counts it. cpu_active() is set before that. On the way
+ * down, cpu_active() is cleared first.
+ */
+ if (!rq->online || !cpu_active(cpu_of(rq)))
return -EBUSY;
update_rq_clock(rq);
--
2.47.3
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 06/16] sched: Take cpuset_mutex for a deadline policy change only with pi
2026-11-18 20:06 [PATCH 00/16] sched/deadline: first bandwidth accounting fixes and selftests Mathieu Desnoyers
` (4 preceding siblings ...)
2026-11-18 20:06 ` [PATCH 05/16] sched/deadline: Refuse dl_server debugfs writes while the rq is offline Mathieu Desnoyers
@ 2026-11-18 20:06 ` Mathieu Desnoyers
2026-11-18 20:17 ` sashiko-bot
2026-11-18 20:06 ` [PATCH 07/16] sched/deadline: Do not lose a throttled task switched out and back Mathieu Desnoyers
` (9 subsequent siblings)
15 siblings, 1 reply; 31+ messages in thread
From: Mathieu Desnoyers @ 2026-11-18 20:06 UTC (permalink / raw)
To: juri.lelli, peterz, mingo, vincent.guittot
Cc: linux-kernel, Mathieu Desnoyers, tglx, will, boqun,
dietmar.eggemann, rostedt, bsegall, mgorman, vschneid,
kprateek.nayak, luca.abeni, mathieu.poirier, longman, ridong.chen,
tj, hannes, mkoutny, shuah, bigeasy, clrkwllms, parri.andrea,
m.szyprowski, kayracizmeci, weihengqiuu, cgroups, linux-kselftest,
linux-rt-devel, stable
__sched_setscheduler() takes cpuset_mutex for a change that involves
SCHED_DEADLINE. On success, it releases it only when pi is set.
normalize_rt_tasks(), for sysrq-n, is the one caller with pi unset. It
runs under read_lock(&tasklist_lock). So sysrq-n takes the mutex in
atomic context for a deadline task, and never releases it. Every later
change that involves SCHED_DEADLINE then blocks forever.
Commit 710da3c8ea7d ("sched/core: Prevent race condition between cpuset
and __sched_setscheduler()") took the cpuset lock only when pi is set.
normalize_rt_tasks() cannot take it under tasklist_lock. Its changelog
accepts this: after sysrq-n, the deadline guarantees are gone anyway.
Commit 111cd11bbc54 ("sched/cpuset: Bring back cpuset_mutex") then took
the mutex only for deadline tasks. It replaced the pi test by a policy
test, and left the release under pi.
Fix this by taking the mutex for a deadline change only when pi is
set.
The policy test reads p->policy before the runqueue lock. It is checked
again under the lock only when the caller gave no policy. Otherwise,
the task can enter SCHED_DEADLINE in between. The call then takes it
out of the class without the mutex. A rebuild of the root domain
accounting could run at the same time. So check again under the
runqueue lock, and retry with the mutex.
sysrq-n on v7.4-rc2, with a deadline task and lockdep on:
sysrq: Nice All RT Tasks
BUG: sleeping function called from invalid context at kernel/locking/mutex.c:623
in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 4356, name: deadline_switch
locks held by deadline_switch/4356: 4, on CPU#3:
...
#3: ffffffff83006098 (tasklist_lock){.+.+}-{3:3}, at: normalize_rt_tasks+0x34/0x150
A sched_setattr(SCHED_DEADLINE) after it never returned, in 8 VMs of 8.
The sysrq_normalize case of deadline_switch_test, added later in this
series, tests this with DEADLINE_TEST_SYSRQ=1. It passes with this
series.
Fixes: 111cd11bbc54 ("sched/cpuset: Bring back cpuset_mutex")
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Reviewed-by: Juri Lelli <juri.lelli@redhat.com>
Cc: stable@vger.kernel.org
---
kernel/sched/syscalls.c | 26 +++++++++++++++++++++++---
1 file changed, 23 insertions(+), 3 deletions(-)
diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c
index cb115aa9b0b3..9b04e77297ae 100644
--- a/kernel/sched/syscalls.c
+++ b/kernel/sched/syscalls.c
@@ -561,9 +561,11 @@ int __sched_setscheduler(struct task_struct *p,
/*
* SCHED_DEADLINE bandwidth accounting relies on stable cpusets
- * information.
+ * information. Do not take cpuset_mutex when pi is unset: the
+ * caller is normalize_rt_tasks(), which holds tasklist_lock and
+ * cannot sleep.
*/
- if (dl_policy(policy) || dl_policy(p->policy)) {
+ if (pi && (dl_policy(policy) || dl_policy(p->policy))) {
cpuset_locked = true;
cpuset_lock();
}
@@ -643,7 +645,13 @@ int __sched_setscheduler(struct task_struct *p,
}
}
- /* Re-check policy now with rq lock held: */
+ /*
+ * The caller gave no policy (SETPARAM_POLICY): the request was
+ * derived from an unlocked read of p->policy above. If the task
+ * changed policy since, re-derive the request from scratch, and
+ * release cpuset_mutex: the re-derived request re-decides whether
+ * it is needed.
+ */
if (unlikely(oldpolicy != -1 && oldpolicy != p->policy)) {
policy = oldpolicy = -1;
task_rq_unlock(rq, p, &rf);
@@ -654,6 +662,18 @@ int __sched_setscheduler(struct task_struct *p,
goto recheck;
}
+ /*
+ * The caller gave an explicit policy, so the request needs no
+ * re-derivation (oldpolicy is -1 above). But the decision to
+ * take cpuset_mutex used the same unlocked read: the task
+ * entered SCHED_DEADLINE since, and the change below would take
+ * it out of the class. Retry to take the mutex.
+ */
+ if (unlikely(pi && !cpuset_locked && task_has_dl_policy(p))) {
+ task_rq_unlock(rq, p, &rf);
+ goto recheck;
+ }
+
oldprio = p->prio;
newprio = __normal_prio(policy, attr->sched_priority, attr->sched_nice);
--
2.47.3
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 07/16] sched/deadline: Do not lose a throttled task switched out and back
2026-11-18 20:06 [PATCH 00/16] sched/deadline: first bandwidth accounting fixes and selftests Mathieu Desnoyers
` (5 preceding siblings ...)
2026-11-18 20:06 ` [PATCH 06/16] sched: Take cpuset_mutex for a deadline policy change only with pi Mathieu Desnoyers
@ 2026-11-18 20:06 ` Mathieu Desnoyers
2026-11-18 20:31 ` sashiko-bot
2026-11-18 20:06 ` [PATCH 08/16] sched/deadline: Do not replenish a boost from cleared parameters Mathieu Desnoyers
` (8 subsequent siblings)
15 siblings, 1 reply; 31+ messages in thread
From: Mathieu Desnoyers @ 2026-11-18 20:06 UTC (permalink / raw)
To: juri.lelli, peterz, mingo, vincent.guittot
Cc: linux-kernel, Mathieu Desnoyers, tglx, will, boqun,
dietmar.eggemann, rostedt, bsegall, mgorman, vschneid,
kprateek.nayak, luca.abeni, mathieu.poirier, longman, ridong.chen,
tj, hannes, mkoutny, shuah, bigeasy, clrkwllms, parri.andrea,
m.szyprowski, kayracizmeci, weihengqiuu, cgroups, linux-kselftest,
linux-rt-devel, stable
A SCHED_DEADLINE task can be throttled, switched to another policy, and
switched back before its 0-lag time. It may then never run again, not
even to die of a fatal signal.
A task that leaves SCHED_DEADLINE keeps its parameters until its 0-lag
time, so that leaving and coming back gains nothing. Commit 209a0cbda7a0
("sched/deadline: Improve the tracking of active utilization") moved
their reset from the replenishment timer to the inactive timer, which
switched_to_dl() cancels.
This misses a task that was throttled when it left. An overrun can put
its 0-lag time after its replenishment timer. That timer fires while
the task is out of the class, and does nothing. On the return of the
task, enqueue_dl_entity() leaves it to that timer, which has fired. The
task is then on no dl_rq, with no timer pending, until it leaves
SCHED_DEADLINE again.
Fix this by replenishing the task when the timer finds it out of the
class, as for a throttled task that is not queued. Its return then
enqueues it.
Clearing dl_throttled alone would not do. It would forgive the overrun.
A yield left set, or the old deadline of a constrained task, would
throttle the task again. The Sashiko AI review of this series found this
in the first version of this patch.
A cgroup selftest under development hung on this bug about once in
1350 runs. The throttled_round_trip case of deadline_switch_test, added
later in this series, fails 8 times out of 8 on v7.3 and v7.4-rc2. It
passes with this patch.
Fixes: 209a0cbda7a0 ("sched/deadline: Improve the tracking of active utilization")
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Acked-by: Juri Lelli <juri.lelli@redhat.com>
Cc: stable@vger.kernel.org
---
kernel/sched/deadline.c | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index b614d5fc1a1a..e4d7cbf4861d 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -1233,9 +1233,18 @@ static enum hrtimer_restart dl_task_timer(struct hrtimer *timer)
/*
* The task might have changed its scheduling policy to something
* different than SCHED_DEADLINE (through switched_from_dl()).
+ * It keeps its parameters until its 0-lag time. If it comes back
+ * before then, nothing else would lift its throttle. So replenish
+ * it now, as the !queued case below does. Its return enqueues it.
*/
- if (!dl_task(p))
+ if (!dl_task(p)) {
+ if (dl_se->dl_throttled && !is_dl_boosted(dl_se)) {
+ sched_clock_tick();
+ update_rq_clock(rq);
+ replenish_dl_entity(dl_se);
+ }
goto unlock;
+ }
/*
* The task might have been boosted by someone else and might be in the
--
2.47.3
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 08/16] sched/deadline: Do not replenish a boost from cleared parameters
2026-11-18 20:06 [PATCH 00/16] sched/deadline: first bandwidth accounting fixes and selftests Mathieu Desnoyers
` (6 preceding siblings ...)
2026-11-18 20:06 ` [PATCH 07/16] sched/deadline: Do not lose a throttled task switched out and back Mathieu Desnoyers
@ 2026-11-18 20:06 ` Mathieu Desnoyers
2026-11-18 20:35 ` sashiko-bot
2026-11-18 20:06 ` [PATCH 09/16] sched/deadline: Reset the deadline and the runtime with the parameters Mathieu Desnoyers
` (7 subsequent siblings)
15 siblings, 1 reply; 31+ messages in thread
From: Mathieu Desnoyers @ 2026-11-18 20:06 UTC (permalink / raw)
To: juri.lelli, peterz, mingo, vincent.guittot
Cc: linux-kernel, Mathieu Desnoyers, tglx, will, boqun,
dietmar.eggemann, rostedt, bsegall, mgorman, vschneid,
kprateek.nayak, luca.abeni, mathieu.poirier, longman, ridong.chen,
tj, hannes, mkoutny, shuah, bigeasy, clrkwllms, parri.andrea,
m.szyprowski, kayracizmeci, weihengqiuu, cgroups, linux-kselftest,
linux-rt-devel, stable
When a SCHED_DEADLINE waiter boosts the owner of an rt_mutex, the owner
runs on the waiter's parameters, through dl.pi_se. Commit 2d3d891d3344
("sched/deadline: Add SCHED_DEADLINE inheritance logic") calls this a
temporary solution. The owner is never throttled. When its runtime runs
out, replenish_dl_entity() refills it at once from the waiter's
dl_runtime.
The waiter can leave SCHED_DEADLINE before the chain walk deboosts the
owner. Its parameters can be cleared in between, at its 0-lag time.
replenish_dl_entity() then loops forever on a dl_runtime of 0. It
holds the rq lock with interrupts off, so the owner's CPU locks up.
O is a SCHED_OTHER task that holds the rt_mutex L. W is a SCHED_DEADLINE
task. T is any other task:
CPU 0 (O) CPU 1 (W) CPU 2 (T)
locks L
blocks on L
rt_mutex_setprio(O, W)
O->dl.pi_se = &W->dl
sleeps
task_non_contending()
arms W's inactive timer
sched_setscheduler(W)
W leaves SCHED_DEADLINE
task_rq_unlock(W)
<W's inactive timer>
inactive_task_timer()
__dl_clear_params()
W->dl.dl_runtime = 0
<tick>
update_curr_dl_se()
enqueue_task_dl()
replenish_dl_entity()
runtime += W->dl.dl_runtime
loops forever
rt_mutex_adjust_pi(W)
rt_mutex_setprio(O, W)
spins on O's rq lock
Fix this by not refilling from cleared parameters. The donor's
parameters are cleared under the donor's rq lock, not the owner's, so
replenish_dl_entity() can see them change under it. Read dl_runtime,
dl_deadline and dl_period once, and bail out if any of them is 0.
On bailing out, the owner gets no new runtime, but stays runnable and
unthrottled. It runs until the chain walk deboosts it, or until it
releases the lock. Keeping it runnable also covers sysrq-n:
normalize_rt_tasks() changes the waiter without a chain walk, so a
throttled owner might never get to release the lock.
The bandwidth the owner consumed while boosted belongs to the donor, so
a negative runtime is reset to 0 rather than charged to the owner's own
reservation. An owner with no parameters of its own gets a deadline of
now. Its deadline must not stay 0: the dl_rq uses an earliest deadline
of 0 to mean that no deadline entity is queued (see inc_dl_deadline()).
Until the deboost (which should happen "soon enough"), such an owner
sorts first on its CPU's dl_rq and runs ahead of the other deadline
tasks there.
The warning is kept for an entity that is not boosted.
The reset bounds the overrun. But a runtime of 0 still counts as
exceeded. So until the deboost, update_curr_dl_se() still requeues the
owner at each update. An owner with SCHED_FLAG_DL_OVERRUN still gets
SIGXCPU.
With a 300ms debug delay before rt_mutex_adjust_pi(), v7.4-rc2 locks
up in 8-CPU VMs. A kernel with this patch does not.
The Sashiko AI review of this series found the lockup.
Fixes: a5e7be3b28a2 ("sched/deadline: Clear dl_entity params when setscheduling to different class")
Suggested-by: Juri Lelli <juri.lelli@redhat.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Reviewed-by: Juri Lelli <juri.lelli@redhat.com>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/all/avL8oOtjKqVNR7-l@jlelli-thinkpadt14gen4.remote.csb/
---
kernel/sched/deadline.c | 40 +++++++++++++++++++++++++++++++++++-----
1 file changed, 35 insertions(+), 5 deletions(-)
diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index e4d7cbf4861d..71b1c155c242 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -807,8 +807,38 @@ static void replenish_dl_entity(struct sched_dl_entity *dl_se)
{
struct dl_rq *dl_rq = dl_rq_of_se(dl_se);
struct rq *rq = rq_of_dl_rq(dl_rq);
+ struct sched_dl_entity *pi_se = pi_of(dl_se);
+ u64 runtime = READ_ONCE(pi_se->dl_runtime);
+ u64 deadline = READ_ONCE(pi_se->dl_deadline);
+ u64 period = READ_ONCE(pi_se->dl_period);
- WARN_ON_ONCE(pi_of(dl_se)->dl_runtime <= 0);
+ /*
+ * A boosted entity runs on the parameters of its donor, the top
+ * waiter of an rt_mutex. The donor can leave the class and have them
+ * cleared before the chain walk deboosts this entity. They are
+ * cleared under the donor's rq lock, which can be another one. So
+ * read each of them once, and check all three. Refilling from a
+ * runtime of 0 would loop forever. Leave the entity runnable
+ * instead: the walk deboosts it, or its task releases the lock.
+ *
+ * Add no runtime. Do not keep a negative one: the overrun comes
+ * from the boost, and the entity's own reservation does not pay for
+ * it. A task with no parameters of its own gets a deadline of now,
+ * as a donor's relative deadline of 0 would give. A deadline of 0
+ * means an empty dl_rq to inc_dl_deadline().
+ */
+ if (unlikely(!runtime || !deadline || !period)) {
+ WARN_ON_ONCE(!is_dl_boosted(dl_se));
+ if (!dl_se->dl_deadline) {
+ dl_se->deadline = rq_clock(rq);
+ dl_se->runtime = 0;
+ }
+ if (dl_se->runtime < 0 || dl_se->dl_yielded)
+ dl_se->runtime = 0;
+ dl_se->dl_yielded = 0;
+ dl_se->dl_throttled = 0;
+ return;
+ }
/*
* This could be the case for a !-dl task that is boosted.
@@ -822,8 +852,8 @@ static void replenish_dl_entity(struct sched_dl_entity *dl_se)
*/
if (dl_se->dl_deadline == 0 ||
(dl_se->dl_defer_armed && dl_entity_overflow(dl_se, rq_clock(rq)))) {
- dl_se->deadline = rq_clock(rq) + pi_of(dl_se)->dl_deadline;
- dl_se->runtime = pi_of(dl_se)->dl_runtime;
+ dl_se->deadline = rq_clock(rq) + deadline;
+ dl_se->runtime = runtime;
}
if (dl_se->dl_yielded && dl_se->runtime > 0)
@@ -836,8 +866,8 @@ static void replenish_dl_entity(struct sched_dl_entity *dl_se)
* arbitrary large.
*/
while (dl_se->runtime <= 0) {
- dl_se->deadline += pi_of(dl_se)->dl_period;
- dl_se->runtime += pi_of(dl_se)->dl_runtime;
+ dl_se->deadline += period;
+ dl_se->runtime += runtime;
}
/*
--
2.47.3
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 09/16] sched/deadline: Reset the deadline and the runtime with the parameters
2026-11-18 20:06 [PATCH 00/16] sched/deadline: first bandwidth accounting fixes and selftests Mathieu Desnoyers
` (7 preceding siblings ...)
2026-11-18 20:06 ` [PATCH 08/16] sched/deadline: Do not replenish a boost from cleared parameters Mathieu Desnoyers
@ 2026-11-18 20:06 ` Mathieu Desnoyers
2026-11-18 20:38 ` sashiko-bot
2026-11-18 20:06 ` [PATCH 10/16] sched/deadline: Release the bandwidth of a task that leaves the class asleep Mathieu Desnoyers
` (6 subsequent siblings)
15 siblings, 1 reply; 31+ messages in thread
From: Mathieu Desnoyers @ 2026-11-18 20:06 UTC (permalink / raw)
To: juri.lelli, peterz, mingo, vincent.guittot
Cc: linux-kernel, Mathieu Desnoyers, tglx, will, boqun,
dietmar.eggemann, rostedt, bsegall, mgorman, vschneid,
kprateek.nayak, luca.abeni, mathieu.poirier, longman, ridong.chen,
tj, hannes, mkoutny, shuah, bigeasy, clrkwllms, parri.andrea,
m.szyprowski, kayracizmeci, weihengqiuu, cgroups, linux-kselftest,
linux-rt-devel, stable
A task that leaves SCHED_DEADLINE keeps its parameters until its 0-lag
time, so that leaving and coming back gains nothing. Commit 209a0cbda7a0
("sched/deadline: Improve the tracking of active utilization") moved
their reset to that time. Before it, the replenishment timer did the
reset at the deadline. So a task back after the reset found its
deadline passed, and started a new period. But __dl_clear_params()
resets neither the deadline nor the runtime.
Now a task back after its 0-lag time can be enqueued on its old
deadline and runtime. With runtime left, its 0-lag time comes before
its deadline. Back before that deadline, it would first spend the runtime
of the reservation it gave back. Back after it, a constrained task
would be throttled until its old period ends. "sched/deadline: Do not
lose a throttled task switched out and back" moves the deadline of a
throttled task on. This widens the window before the deadline to about
a period.
A task with no parameters of its own, boosted by a deadline waiter,
keeps the donor's deadline and runtime after its deboost. If it then
enters SCHED_DEADLINE before that deadline, it runs on the donor's
runtime. On v7.4-rc2, a task of 10ms per 100ms ran 40ms of its first
period.
Fix this by resetting the deadline and the runtime in
__dl_clear_params(), and in switched_from_dl() at the deboost of a task
with no parameters. The next enqueue then starts a new period.
The Sashiko AI review of this series found the first case and the
deboost. The deboost_resets_period case of deadline_bw_test, added later
in this series, tests the deboost.
Fixes: 209a0cbda7a0 ("sched/deadline: Improve the tracking of active utilization")
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Acked-by: Juri Lelli <juri.lelli@redhat.com>
Cc: stable@vger.kernel.org
---
kernel/sched/deadline.c | 16 +++++++++++++++-
1 file changed, 15 insertions(+), 1 deletion(-)
diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index 71b1c155c242..d6d2657dcddd 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -3569,6 +3569,16 @@ static void switched_from_dl(struct rq *rq, struct task_struct *p)
if (task_on_rq_queued(p) && p->dl.dl_runtime)
task_non_contending(&p->dl, false);
+ /*
+ * A task with no parameters of its own was boosted into the class.
+ * Its deadline and runtime are the donor's. Reset them, so that its
+ * next entry into the class starts a new period.
+ */
+ if (!p->dl.dl_runtime) {
+ p->dl.deadline = 0;
+ p->dl.runtime = 0;
+ }
+
/*
* In case a task is setscheduled out from SCHED_DEADLINE we need to
* keep track of that on its cpuset (for correct bandwidth tracking).
@@ -3984,7 +3994,8 @@ bool __checkparam_dl(const struct sched_attr *attr)
}
/*
- * This function clears the sched_dl_entity static params.
+ * Clear the static parameters of @dl_se. Also reset its deadline and its
+ * runtime, so that its next enqueue starts a new period.
*/
static void __dl_clear_params(struct sched_dl_entity *dl_se)
{
@@ -3995,6 +4006,9 @@ static void __dl_clear_params(struct sched_dl_entity *dl_se)
dl_se->dl_bw = 0;
dl_se->dl_density = 0;
+ dl_se->deadline = 0;
+ dl_se->runtime = 0;
+
dl_se->dl_throttled = 0;
dl_se->dl_yielded = 0;
dl_se->dl_non_contending = 0;
--
2.47.3
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 10/16] sched/deadline: Release the bandwidth of a task that leaves the class asleep
2026-11-18 20:06 [PATCH 00/16] sched/deadline: first bandwidth accounting fixes and selftests Mathieu Desnoyers
` (8 preceding siblings ...)
2026-11-18 20:06 ` [PATCH 09/16] sched/deadline: Reset the deadline and the runtime with the parameters Mathieu Desnoyers
@ 2026-11-18 20:06 ` Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 11/16] sched/deadline: Reject SCHED_DEADLINE on exited tasks Mathieu Desnoyers
` (5 subsequent siblings)
15 siblings, 0 replies; 31+ messages in thread
From: Mathieu Desnoyers @ 2026-11-18 20:06 UTC (permalink / raw)
To: juri.lelli, peterz, mingo, vincent.guittot
Cc: linux-kernel, Mathieu Desnoyers, tglx, will, boqun,
dietmar.eggemann, rostedt, bsegall, mgorman, vschneid,
kprateek.nayak, luca.abeni, mathieu.poirier, longman, ridong.chen,
tj, hannes, mkoutny, shuah, bigeasy, clrkwllms, parri.andrea,
m.szyprowski, kayracizmeci, weihengqiuu, cgroups, linux-kselftest,
linux-rt-devel, stable
A task that leaves SCHED_DEADLINE while it is not queued can keep its
bandwidth in total_bw until the next rebuild of the root domains.
Repeated, this fills total_bw up to the admission limit. Then
sched_setattr(SCHED_DEADLINE) and CPU offlining fail with EBUSY.
A task that leaves the class keeps its bandwidth until its 0-lag time.
Releasing it earlier would allow temporary overloads. For a queued task,
switched_from_dl() calls task_non_contending(). It releases the
bandwidth if the 0-lag time is past. Otherwise it arms the inactive
timer, which releases it then. For a task that is not queued,
switched_from_dl() relies on the timer armed when the task blocked.
But the timer is not always pending when the task leaves the class:
- The task blocked after its 0-lag time. The timer was not armed.
- The timer fired while the task was still in the class. It only
removed running_bw.
- The task was woken up on another CPU and is not enqueued there yet.
The wakeup cancelled the timer.
- The task came back to the class asleep. switched_to_dl() cancelled
the timer.
Then nothing releases total_bw.
Fix this in switched_from_dl(). For a task that is not queued and has
no timer pending, call task_non_contending(), as for a queued task. It
arms the timer if the 0-lag time is ahead. Otherwise it releases the
bandwidth now and clears the parameters, as inactive_task_timer() does.
Clearing the parameters keeps later paths from releasing the bandwidth
again:
- A timer handler that already started finds a dl_bw of 0.
- The deboost of a later priority inheritance boost finds no
parameters.
An owner that the task boosts as a waiter can find the parameters
cleared too. "sched/deadline: Do not replenish a boost from cleared
parameters" handles this, so a backport needs it first. Without it, the
owner's CPU can lock up as soon as the waiter leaves the class.
Juri Lelli's first fix, since dropped from tip, released the bandwidth
here without clearing dl_bw. The bandwidth of a boosted task was then
released twice.
Juri Lelli suggested the call to task_non_contending() in review.
The Sashiko AI review of this series found the two cases with the 0-lag
time ahead. deadline_switch_test, added later in this series, tests
the release in asleep_round_trip.
Fixes: 209a0cbda7a0 ("sched/deadline: Improve the tracking of active utilization")
Reported-by: Tejun Heo <tj@kernel.org>
Closes: https://lore.kernel.org/all/2b5fb3f22bca6aa383be8f4781923392@kernel.org/
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Reviewed-by: Juri Lelli <juri.lelli@redhat.com>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/all/20261103-b4-sched-dl-bw-fixes-v1-2-dd761fe005b4@redhat.com/
---
kernel/sched/deadline.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index d6d2657dcddd..f0e03f97fd82 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -3595,6 +3595,14 @@ static void switched_from_dl(struct rq *rq, struct task_struct *p)
if (p->dl.dl_non_contending)
sub_running_bw(&p->dl, &rq->dl);
sub_rq_bw(&p->dl, &rq->dl);
+ /*
+ * Without an armed timer, nothing releases total_bw. Do as for
+ * a queued task: arm the timer if the 0-lag time is ahead, or
+ * release the bandwidth and clear the parameters now. Call
+ * task_non_contending() after sub_rq_bw(), which needs dl_bw.
+ */
+ if (!p->dl.dl_non_contending)
+ task_non_contending(&p->dl, false);
}
/*
--
2.47.3
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 11/16] sched/deadline: Reject SCHED_DEADLINE on exited tasks
2026-11-18 20:06 [PATCH 00/16] sched/deadline: first bandwidth accounting fixes and selftests Mathieu Desnoyers
` (9 preceding siblings ...)
2026-11-18 20:06 ` [PATCH 10/16] sched/deadline: Release the bandwidth of a task that leaves the class asleep Mathieu Desnoyers
@ 2026-11-18 20:06 ` Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 12/16] sched/deadline: Fix this_bw when a task is changed during a remote wakeup Mathieu Desnoyers
` (4 subsequent siblings)
15 siblings, 0 replies; 31+ messages in thread
From: Mathieu Desnoyers @ 2026-11-18 20:06 UTC (permalink / raw)
To: juri.lelli, peterz, mingo, vincent.guittot
Cc: linux-kernel, Mathieu Desnoyers, tglx, will, boqun,
dietmar.eggemann, rostedt, bsegall, mgorman, vschneid,
kprateek.nayak, luca.abeni, mathieu.poirier, longman, ridong.chen,
tj, hannes, mkoutny, shuah, bigeasy, clrkwllms, parri.andrea,
m.szyprowski, kayracizmeci, weihengqiuu, cgroups, linux-kselftest,
linux-rt-devel, stable
From: Juri Lelli <juri.lelli@redhat.com>
sched_setattr(SCHED_DEADLINE) on a zombie task succeeds and charges
total_bw via sched_dl_overflow(), but the task has already done its
final schedule() and will never run again. Nothing releases the reserved
bandwidth. Repeating this fills total_bw to the admission cap, after
which sched_setattr(SCHED_DEADLINE) and CPU offline fail with -EBUSY.
Reject sched_setattr(SCHED_DEADLINE) on exited tasks with -ESRCH.
A task of the class that exited, switched to another policy, runs
switched_from_dl(), which decrements the count of deadline tasks of its
cpuset that cgroup_task_dead() decremented already: at 0 with a deadline
task left in it, the cpuset is skipped by the next rebuild of the root
domains' accounting, and that task's bandwidth with it. Reject that as
well. The deadline_bw_test selftest added later in this series has the
case, death_zombie_switched: on v7.4-rc1 the switch is accepted, and
the one-CPU partition root the zombie ran in admits 9 tasks of 10% after
a rebuild, with a task of 30% still in it, where 6 are due.
Fixes: 332ac17ef5bf ("sched/deadline: Add bandwidth management for SCHED_DEADLINE tasks")
Reported-by: Weiheng Qiu <weihengqiuu@gmail.com>
Closes: https://lore.kernel.org/all/3f6e6c87-645b-45fd-8fc6-8da5bc5b82d8@gmail.com/
Link: https://lore.kernel.org/all/20261103-b4-sched-dl-bw-fixes-v1-4-dd761fe005b4@redhat.com/
Cc: stable@vger.kernel.org
Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
[mathieu: reject a task of the class leaving it too, found by the
Sashiko AI review of the patch]
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
---
kernel/sched/syscalls.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c
index 9b04e77297ae..b87b878be63f 100644
--- a/kernel/sched/syscalls.c
+++ b/kernel/sched/syscalls.c
@@ -592,6 +592,19 @@ int __sched_setscheduler(struct task_struct *p,
if (retval)
goto unlock;
+ /*
+ * Reject SCHED_DEADLINE on exited tasks. A zombie will never be
+ * scheduled again, so sched_dl_overflow() would charge total_bw
+ * that nothing will ever release. And reject a change of policy of
+ * one in the class: cgroup_task_dead() took it off the count of
+ * deadline tasks of its cpuset, which switched_from_dl() would
+ * decrement a second time.
+ */
+ if ((dl_policy(policy) || dl_task(p)) && p->exit_state) {
+ retval = -ESRCH;
+ goto unlock;
+ }
+
/*
* If not changing anything there's no need to proceed further,
* but store a possible modification of reset_on_fork.
--
2.47.3
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 12/16] sched/deadline: Fix this_bw when a task is changed during a remote wakeup
2026-11-18 20:06 [PATCH 00/16] sched/deadline: first bandwidth accounting fixes and selftests Mathieu Desnoyers
` (10 preceding siblings ...)
2026-11-18 20:06 ` [PATCH 11/16] sched/deadline: Reject SCHED_DEADLINE on exited tasks Mathieu Desnoyers
@ 2026-11-18 20:06 ` Mathieu Desnoyers
2026-11-18 20:46 ` sashiko-bot
2026-11-18 20:06 ` [PATCH 13/16] sched/deadline: Check SCHED_DEADLINE affinity under the rq lock Mathieu Desnoyers
` (3 subsequent siblings)
15 siblings, 1 reply; 31+ messages in thread
From: Mathieu Desnoyers @ 2026-11-18 20:06 UTC (permalink / raw)
To: juri.lelli, peterz, mingo, vincent.guittot
Cc: linux-kernel, Mathieu Desnoyers, tglx, will, boqun,
dietmar.eggemann, rostedt, bsegall, mgorman, vschneid,
kprateek.nayak, luca.abeni, mathieu.poirier, longman, ridong.chen,
tj, hannes, mkoutny, shuah, bigeasy, clrkwllms, parri.andrea,
m.szyprowski, kayracizmeci, weihengqiuu, cgroups, linux-kselftest,
linux-rt-devel, stable
The this_bw of a runqueue is the bandwidth of the deadline tasks
assigned to it, running or not. GRUB uses it to compute the inactive
utilization of the runqueue. When a task wakes up on another CPU,
migrate_task_rq_dl() takes its bandwidth off the old runqueue. The
enqueue adds it to the new one.
Between these two steps, no runqueue counts the bandwidth.
try_to_wake_up() can leave the enqueue to the wake list of the new CPU,
and return before it runs. task_cpu() already points at the new CPU. A
change of the task in that window gets this_bw wrong.
In this example, T is a deadline task. S is another task. It changes
the bandwidth of T from "old bw" to "new bw":
CPU 0 CPU 1 CPU 2
T blocks
this_bw of CPU 0 counts T
<timer> try_to_wake_up(T)
set_task_cpu(T, 1)
migrate_task_rq_dl()
sub_rq_bw(): CPU 0 drops T
T on the wake list of CPU 1
return
S: sched_setattr(T)
task_rq_lock(T): CPU 1
T is not queued
dl_change_utilization()
this_bw of CPU 1:
- old bw + new bw
sched_ttwu_pending()
enqueue_task_dl(T)
task_contending()
add_rq_bw():
this_bw of CPU 1:
+ new bw
CPU 1 then counts the new bandwidth twice, less the old one. Repeated,
this makes this_bw drift until it underflows. If T left the class in
that window, switched_from_dl() would take its bandwidth off CPU 1 the
same way. A task that enters the class in that window would be counted
twice, by switched_to_dl() and by the enqueue.
This needs a wakeup that moves the task, so a root domain of more than
one CPU, with busy CPUs. CONFIG_SCHED_PROXY_EXEC=y makes such wakeups
more common. In testing, a task sleeps and wakes in a busy two-CPU
partition root, and another CPU changes its runtime. The kernel warns
within a second in each of 8 VMs:
WARNING: kernel/sched/deadline.c:352 at dl_rq_change_utilization+0x71/0x2e0, CPU#0: wake_race/109
__sub_rq_bw(): this_bw underflow
WARNING: kernel/sched/deadline.c:316 at task_contending+0x244/0x260, CPU#6: swapper/6/0
__add_running_bw(): running_bw overflow
Fix this with a new bit, dl_bw_oncpu, that tells whether a runqueue
counts the bandwidth of the task. add_rq_bw() adds the bandwidth only if
no runqueue counts it. sub_rq_bw() takes it off only if one does.
dl_change_utilization() does nothing while no runqueue counts the
bandwidth. The enqueue adds the new one.
With this patch, none of the 8 VMs warns. deadline_bw_test, added later
in this series, tests this in params_waking. The Sashiko
AI review of this series found the double count, and a bit that the
first version of this patch left set.
Fixes: 8fd27231c330 ("sched/deadline: Track the "total rq utilization" too")
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Reviewed-by: Juri Lelli <juri.lelli@redhat.com>
Cc: stable@vger.kernel.org
---
include/linux/sched.h | 5 +++++
kernel/sched/deadline.c | 39 +++++++++++++++++++++++++++++++++++----
2 files changed, 40 insertions(+), 4 deletions(-)
diff --git a/include/linux/sched.h b/include/linux/sched.h
index e4f6323d7855..88cda9c10cb8 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -706,6 +706,10 @@ struct sched_dl_entity {
* contributes to the root domain's total_bw. Only meaningful for server
* entities (@dl_server == 1). Allows toggling the reservation on/off
* without losing the configured @dl_runtime/@dl_period.
+ *
+ * @dl_bw_oncpu tells if the this_bw of a runqueue counts the task's
+ * @dl_bw: add_rq_bw() and sub_rq_bw() set and clear it, under that
+ * runqueue's lock. Not for servers, which keep @dl_bw_attached.
*/
unsigned int dl_throttled : 1;
unsigned int dl_yielded : 1;
@@ -718,6 +722,7 @@ struct sched_dl_entity {
unsigned int dl_defer_running : 1;
unsigned int dl_defer_idle : 1;
unsigned int dl_bw_attached : 1;
+ unsigned int dl_bw_oncpu : 1;
/*
* Bandwidth enforcement timer. Each -deadline task has its
diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index f0e03f97fd82..c36516210669 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -267,18 +267,35 @@ void __sub_rq_bw(u64 dl_bw, struct dl_rq *dl_rq)
WARN_ON_ONCE(dl_rq->running_bw > dl_rq->this_bw);
}
+/*
+ * @dl_se->dl_bw_oncpu tells whether the this_bw of a runqueue counts the
+ * bandwidth of @dl_se. add_rq_bw() adds the bandwidth only if no runqueue
+ * counts it. sub_rq_bw() subtracts it only if one does.
+ *
+ * A wakeup on another CPU takes the bandwidth off the old runqueue. The
+ * enqueue on the new CPU adds it back later. A change of the task in
+ * between must not count the bandwidth twice, nor take it off a runqueue
+ * that does not count it.
+ *
+ * The servers track their bandwidth with dl_bw_attached. They call
+ * __add_rq_bw() and __sub_rq_bw() directly.
+ */
static inline
void add_rq_bw(struct sched_dl_entity *dl_se, struct dl_rq *dl_rq)
{
- if (!dl_entity_is_special(dl_se))
- __add_rq_bw(dl_se->dl_bw, dl_rq);
+ if (dl_entity_is_special(dl_se) || dl_se->dl_bw_oncpu)
+ return;
+ __add_rq_bw(dl_se->dl_bw, dl_rq);
+ dl_se->dl_bw_oncpu = 1;
}
static inline
void sub_rq_bw(struct sched_dl_entity *dl_se, struct dl_rq *dl_rq)
{
- if (!dl_entity_is_special(dl_se))
- __sub_rq_bw(dl_se->dl_bw, dl_rq);
+ if (dl_entity_is_special(dl_se) || !dl_se->dl_bw_oncpu)
+ return;
+ __sub_rq_bw(dl_se->dl_bw, dl_rq);
+ dl_se->dl_bw_oncpu = 0;
}
static inline
@@ -347,6 +364,14 @@ static void dl_change_utilization(struct task_struct *p, u64 new_bw)
if (task_on_rq_queued(p))
return;
+ /*
+ * The task is waking up on another CPU, and no runqueue counts its
+ * bandwidth. The enqueue adds it, with the new parameters that
+ * __setparam_dl() sets right after this.
+ */
+ if (!p->dl.dl_bw_oncpu)
+ return;
+
dl_rq_change_utilization(task_rq(p), &p->dl, new_bw);
}
@@ -2726,6 +2751,11 @@ static void migrate_task_rq_dl(struct task_struct *p, int new_cpu __maybe_unused
*/
cancel_inactive_timer(&p->dl);
}
+ /*
+ * set_task_cpu() points task_cpu() at the new CPU next. The runqueue
+ * of that CPU counts this bandwidth only once the enqueue there adds
+ * it. Until then, no runqueue counts it, as dl_bw_oncpu now says.
+ */
sub_rq_bw(&p->dl, &rq->dl);
rq_unlock(rq, &rf);
}
@@ -4025,6 +4055,7 @@ static void __dl_clear_params(struct sched_dl_entity *dl_se)
dl_se->dl_defer = 0;
dl_se->dl_defer_running = 0;
dl_se->dl_defer_armed = 0;
+ dl_se->dl_bw_oncpu = 0;
#ifdef CONFIG_RT_MUTEXES
dl_se->pi_se = dl_se;
--
2.47.3
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 13/16] sched/deadline: Check SCHED_DEADLINE affinity under the rq lock
2026-11-18 20:06 [PATCH 00/16] sched/deadline: first bandwidth accounting fixes and selftests Mathieu Desnoyers
` (11 preceding siblings ...)
2026-11-18 20:06 ` [PATCH 12/16] sched/deadline: Fix this_bw when a task is changed during a remote wakeup Mathieu Desnoyers
@ 2026-11-18 20:06 ` Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 14/16] sched/debug: Show the deadline bandwidth of each runqueue Mathieu Desnoyers
` (2 subsequent siblings)
15 siblings, 0 replies; 31+ messages in thread
From: Mathieu Desnoyers @ 2026-11-18 20:06 UTC (permalink / raw)
To: juri.lelli, peterz, mingo, vincent.guittot
Cc: linux-kernel, Mathieu Desnoyers, tglx, will, boqun,
dietmar.eggemann, rostedt, bsegall, mgorman, vschneid,
kprateek.nayak, luca.abeni, mathieu.poirier, longman, ridong.chen,
tj, hannes, mkoutny, shuah, bigeasy, clrkwllms, parri.andrea,
m.szyprowski, kayracizmeci, weihengqiuu, cgroups, linux-kselftest,
linux-rt-devel, stable
From: Andrea Parri <parri.andrea@gmail.com>
A task can end up SCHED_DEADLINE with an affinity narrower than its root
domain when sched_setaffinity() and sched_setattr() race on it. Both
calls succeed, although each fails when made after the other:
sched_setattr() with -EPERM, sched_setaffinity() with -EBUSY.
__sched_setaffinity() calls dl_task_check_affinity() before
__set_cpus_allowed_ptr() takes task_rq_lock(), whereas
__sched_setscheduler() runs its admission test and switches the policy
under a single task_rq_lock(). So both calls can pass their checks:
CPU 1: sched_setaffinity(p, cpu0) CPU 2: sched_setattr(p, DEADLINE)
dl_task_check_affinity()
!task_has_dl_policy(p): return 0
task_rq_lock()
admission test passes
p->policy = SCHED_DEADLINE
task_rq_unlock()
__set_cpus_allowed_ptr()
task_rq_lock()
p->cpus_mask = cpu0
The window includes CPU 1 spinning on task_rq_lock() while CPU 2 holds
it. With one thread narrowing a sleeping task to CPU 0 while another
makes it SCHED_DEADLINE, 923514, 959547 and 983997 of 1000000 rounds
ended with both calls succeeding on three boots of a 4-CPU guest.
Move the check into __set_cpus_allowed_ptr_locked(), next to the
PF_NO_SETAFFINITY re-check, under the same lock as the admission test.
Whichever call takes task_rq_lock() second then sees the other's
result: sched_setaffinity() fails with -EBUSY if p is already
SCHED_DEADLINE, and sched_setattr() fails with -EPERM if p's affinity
is already cpu0.
A new SCA_DL_CHECK flag selects the check, and only
__sched_setaffinity() sets it, so sched_setaffinity() and
relax_compatible_cpus_allowed_ptr() keep the check and other callers
stay unchecked. restrict_cpus_allowed_ptr() already does its deadline
check this way.
__sched_setaffinity() clears SCA_DL_CHECK before the second
__set_cpus_allowed_ptr() call it makes after racing with a cpuset
update, so the cpuset mask still wins there, as it does when
cpuset_update_tasks_cpumask() narrows a deadline task. SCA_CHECK stays
set for the PF_NO_SETAFFINITY re-check.
With this applied, the same test ends with both calls succeeding in 0
of 1000000 rounds on each of three boots, and sched_setaffinity()
fails with -EBUSY whenever it loses.
The race was reported by Sebastian. The fix was written with the help
of an LLM.
Fixes: 332ac17ef5bf ("sched/deadline: Add bandwidth management for SCHED_DEADLINE tasks")
Reported-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Closes: https://lore.kernel.org/20261103081149.DuyWNhVe@linutronix.de
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
Cc: stable@vger.kernel.org
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
---
kernel/sched/core.c | 11 +++++++++++
kernel/sched/sched.h | 1 +
kernel/sched/syscalls.c | 12 +++++++-----
3 files changed, 19 insertions(+), 5 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 5ed660f099a5..32fe889e02fb 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3195,6 +3195,17 @@ static int __set_cpus_allowed_ptr_locked(struct task_struct *p,
goto out;
}
+ /*
+ * Check under the same lock that __sched_setscheduler() holds for
+ * its admission test, so that p cannot become SCHED_DEADLINE
+ * between this check and the mask update below.
+ */
+ if (ctx->flags & SCA_DL_CHECK) {
+ ret = dl_task_check_affinity(p, ctx->new_mask);
+ if (ret)
+ goto out;
+ }
+
if (!(ctx->flags & SCA_MIGRATE_ENABLE)) {
if (cpumask_equal(&p->cpus_mask, ctx->new_mask)) {
if (ctx->flags & SCA_USER)
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 92134ce326a4..20ad13f388cb 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -2972,6 +2972,7 @@ extern struct task_struct *pick_task_idle(struct rq *rq, struct rq_flags *rf);
#define SCA_MIGRATE_DISABLE 0x02
#define SCA_MIGRATE_ENABLE 0x04
#define SCA_USER 0x08
+#define SCA_DL_CHECK 0x10
extern void update_group_capacity(struct sched_domain *sd, int cpu);
diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c
index b87b878be63f..5983232476a9 100644
--- a/kernel/sched/syscalls.c
+++ b/kernel/sched/syscalls.c
@@ -1225,11 +1225,7 @@ int __sched_setaffinity(struct task_struct *p, struct affinity_context *ctx)
cpumask_and(new_mask, ctx->new_mask, cpus_allowed);
ctx->new_mask = new_mask;
- ctx->flags |= SCA_CHECK;
-
- retval = dl_task_check_affinity(p, new_mask);
- if (retval)
- goto out_free_new_mask;
+ ctx->flags |= SCA_CHECK | SCA_DL_CHECK;
retval = __set_cpus_allowed_ptr(p, ctx);
if (retval)
@@ -1258,6 +1254,12 @@ int __sched_setaffinity(struct task_struct *p, struct affinity_context *ctx)
if (empty)
cpumask_copy(new_mask, cpus_allowed);
}
+
+ /*
+ * The cpuset mask must win here, as it does when
+ * cpuset_update_tasks_cpumask() sets it on a deadline task.
+ */
+ ctx->flags &= ~SCA_DL_CHECK;
__set_cpus_allowed_ptr(p, ctx);
retval = -EINVAL;
}
--
2.47.3
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 14/16] sched/debug: Show the deadline bandwidth of each runqueue
2026-11-18 20:06 [PATCH 00/16] sched/deadline: first bandwidth accounting fixes and selftests Mathieu Desnoyers
` (12 preceding siblings ...)
2026-11-18 20:06 ` [PATCH 13/16] sched/deadline: Check SCHED_DEADLINE affinity under the rq lock Mathieu Desnoyers
@ 2026-11-18 20:06 ` Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 15/16] selftests/sched: Add a test of SCHED_DEADLINE policy round trips Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 16/16] selftests/sched: Add a test of SCHED_DEADLINE bandwidth accounting Mathieu Desnoyers
15 siblings, 0 replies; 31+ messages in thread
From: Mathieu Desnoyers @ 2026-11-18 20:06 UTC (permalink / raw)
To: juri.lelli, peterz, mingo, vincent.guittot
Cc: linux-kernel, Mathieu Desnoyers, tglx, will, boqun,
dietmar.eggemann, rostedt, bsegall, mgorman, vschneid,
kprateek.nayak, luca.abeni, mathieu.poirier, longman, ridong.chen,
tj, hannes, mkoutny, shuah, bigeasy, clrkwllms, parri.andrea,
m.szyprowski, kayracizmeci, weihengqiuu, cgroups, linux-kselftest,
linux-rt-devel
For each CPU, sched_debug prints the deadline bandwidth of its root
domain. It does not print the two sums of its runqueue. this_bw is the
bandwidth of the deadline tasks assigned to the CPU. running_bw is the
part of it that belongs to active tasks.
GRUB reclaiming depends on both sums. grub_reclaim() takes this_bw -
running_bw as the inactive bandwidth of the CPU. The more inactive
bandwidth, the slower a reclaiming task uses up its runtime.
The kernel warns when a sum wraps around, and when running_bw exceeds
this_bw. Bandwidth left behind in this_bw triggers no warning.
Print both sums. The deadline_bw_test selftest added later in this
series reads this_bw in its params_waking case. It checks that this_bw
is back to its first value once the tasks of the case are gone.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
---
kernel/sched/debug.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c
index ddb382df454b..2c373adf3196 100644
--- a/kernel/sched/debug.c
+++ b/kernel/sched/debug.c
@@ -1175,6 +1175,8 @@ void print_dl_rq(struct seq_file *m, int cpu, struct dl_rq *dl_rq)
SEQ_printf(m, " .%-30s: %lu\n", #x, (unsigned long)(dl_rq->x))
PU(dl_nr_running);
+ PU(running_bw);
+ PU(this_bw);
dl_bw = &cpu_rq(cpu)->rd->dl_bw;
SEQ_printf(m, " .%-30s: %lld\n", "dl_bw->bw", dl_bw->bw);
SEQ_printf(m, " .%-30s: %lld\n", "dl_bw->total_bw", dl_bw->total_bw);
--
2.47.3
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 15/16] selftests/sched: Add a test of SCHED_DEADLINE policy round trips
2026-11-18 20:06 [PATCH 00/16] sched/deadline: first bandwidth accounting fixes and selftests Mathieu Desnoyers
` (13 preceding siblings ...)
2026-11-18 20:06 ` [PATCH 14/16] sched/debug: Show the deadline bandwidth of each runqueue Mathieu Desnoyers
@ 2026-11-18 20:06 ` Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 16/16] selftests/sched: Add a test of SCHED_DEADLINE bandwidth accounting Mathieu Desnoyers
15 siblings, 0 replies; 31+ messages in thread
From: Mathieu Desnoyers @ 2026-11-18 20:06 UTC (permalink / raw)
To: juri.lelli, peterz, mingo, vincent.guittot
Cc: linux-kernel, Mathieu Desnoyers, tglx, will, boqun,
dietmar.eggemann, rostedt, bsegall, mgorman, vschneid,
kprateek.nayak, luca.abeni, mathieu.poirier, longman, ridong.chen,
tj, hannes, mkoutny, shuah, bigeasy, clrkwllms, parri.andrea,
m.szyprowski, kayracizmeci, weihengqiuu, cgroups, linux-kselftest,
linux-rt-devel
No selftest runs a SCHED_DEADLINE task.
Add deadline_switch_test, with five cases:
- throttled_round_trip: a task of 10ms every 2s spins, so it is
throttled in every period. Right after a throttle, the test switches
it to SCHED_OTHER. The test switches it back 200us after its
deadline, between its replenishment timer and its 0-lag time. It must
then run again.
- asleep_round_trip: a task of 9ms every 10ms runs 0.2ms, then sleeps
50ms. While it sleeps past its 0-lag time, the test switches it to
SCHED_OTHER and back. A leak at each round trip fills the root
domain, and the switch back fails with EBUSY.
- sysrq_normalize: sysrq-n takes a deadline task out of the class.
Another task must then still enter SCHED_DEADLINE.
- sysrq_boosted_owner: sysrq-n takes a SCHED_OTHER task out of the
deadline class while a deadline waiter boosts it. After the unlock,
the task enters SCHED_DEADLINE at 10ms every 100ms. It must not run
more than its runtime in its first period.
- sysrq_running_owner: a SCHED_OTHER task spins alone on a CPU, with a
lock held. A deadline waiter of 100us every 1ms boosts it. The test
starts 2048 sleeping SCHED_FIFO threads, then fires sysrq-n. Every
thread must end, and the owner must no longer be boosted.
The cases need CAP_SYS_NICE, and skip without it. asleep_round_trip
also skips without admission control, when sched_rt_runtime_us is -1.
sysrq-n changes the policy of every real-time user task. So the three
sysrq cases run only with DEADLINE_TEST_SYSRQ set. They also need root
and CONFIG_MAGIC_SYSRQ. The two owner cases run only if
sysrq_normalize passed.
On a kernel with the bugs, asleep_round_trip and sysrq_normalize keep
new tasks out of SCHED_DEADLINE. So they run last, followed only by
the two owner cases.
On v7.4-rc2, in 8-CPU VMs, three of the five cases fail:
- throttled_round_trip. "sched/deadline: Do not lose a throttled task
switched out and back" fixes it.
- asleep_round_trip. "sched/deadline: Release the bandwidth of a task
that leaves the class asleep" fixes it.
- sysrq_normalize. "sched: Take cpuset_mutex for a deadline policy
change only with pi" fixes it.
sysrq_boosted_owner tests "sched: Let a task boosted by a deadline
waiter change its policy". Without the sysrq-n part of that fix, the
owner ran 99-100ms of its first period, in 4 runs of 4. With it, it ran
10-11ms.
sysrq_running_owner tests "sched/deadline: Do not replenish a boost
from cleared parameters". With the early return of that fix disabled,
the guest locked up in 3 runs of 3.
With the whole series, the five cases pass, with and without
PREEMPT_RT.
The test goes in a new selftest target, sched/deadline.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
---
tools/testing/selftests/Makefile | 1 +
.../selftests/sched/deadline/.gitignore | 1 +
.../testing/selftests/sched/deadline/Makefile | 8 +
tools/testing/selftests/sched/deadline/config | 1 +
.../sched/deadline/deadline_switch_test.c | 1115 +++++++++++++++++
5 files changed, 1126 insertions(+)
create mode 100644 tools/testing/selftests/sched/deadline/.gitignore
create mode 100644 tools/testing/selftests/sched/deadline/Makefile
create mode 100644 tools/testing/selftests/sched/deadline/config
create mode 100644 tools/testing/selftests/sched/deadline/deadline_switch_test.c
diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
index 07c3719f8b04..35b0c5e86ba7 100644
--- a/tools/testing/selftests/Makefile
+++ b/tools/testing/selftests/Makefile
@@ -120,6 +120,7 @@ TARGETS += rlimits
TARGETS += rseq
TARGETS += rtc
TARGETS += rust
+TARGETS += sched/deadline
TARGETS += sched_ext
TARGETS += seccomp
TARGETS += sgx
diff --git a/tools/testing/selftests/sched/deadline/.gitignore b/tools/testing/selftests/sched/deadline/.gitignore
new file mode 100644
index 000000000000..9f203c455b91
--- /dev/null
+++ b/tools/testing/selftests/sched/deadline/.gitignore
@@ -0,0 +1 @@
+deadline_switch_test
diff --git a/tools/testing/selftests/sched/deadline/Makefile b/tools/testing/selftests/sched/deadline/Makefile
new file mode 100644
index 000000000000..3f4dc1ec6119
--- /dev/null
+++ b/tools/testing/selftests/sched/deadline/Makefile
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0+
+
+CFLAGS += -O2 -Wall -g -I./ $(KHDR_INCLUDES)
+LDLIBS += -lpthread
+
+TEST_GEN_PROGS := deadline_switch_test
+
+include ../../lib.mk
diff --git a/tools/testing/selftests/sched/deadline/config b/tools/testing/selftests/sched/deadline/config
new file mode 100644
index 000000000000..3a3a7285a708
--- /dev/null
+++ b/tools/testing/selftests/sched/deadline/config
@@ -0,0 +1 @@
+CONFIG_MAGIC_SYSRQ=y
diff --git a/tools/testing/selftests/sched/deadline/deadline_switch_test.c b/tools/testing/selftests/sched/deadline/deadline_switch_test.c
new file mode 100644
index 000000000000..40c7c6bafc2b
--- /dev/null
+++ b/tools/testing/selftests/sched/deadline/deadline_switch_test.c
@@ -0,0 +1,1115 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Test SCHED_DEADLINE tasks that switch to SCHED_OTHER and back:
+ *
+ * - throttled_round_trip: the test switches a throttled task to
+ * SCHED_OTHER. The task's replenishment timer fires while it is out of
+ * SCHED_DEADLINE. The test switches it back after its deadline, before
+ * its 0-lag time. The task must run again.
+ * - asleep_round_trip: a task is switched out and back while it sleeps
+ * past its 0-lag time. Its bandwidth must leave the root domain when it
+ * leaves SCHED_DEADLINE. Otherwise each switch back adds it again, and
+ * sched_setattr() soon fails with EBUSY.
+ * - sysrq_normalize: sysrq-n takes a deadline task out of the class.
+ * Another task must then still enter SCHED_DEADLINE.
+ * - sysrq_boosted_owner: a deadline waiter boosts a sleeping SCHED_OTHER
+ * owner into the deadline class. sysrq-n must end the boost. When the
+ * owner later calls sched_setattr() to enter SCHED_DEADLINE, it must be
+ * throttled at its own runtime.
+ * - sysrq_running_owner: a deadline waiter boosts a running SCHED_OTHER
+ * owner. sysrq-n clears the waiter's parameters before it ends the
+ * boost. The owner must keep running until then.
+ */
+#define _GNU_SOURCE
+#include <errno.h>
+#include <fcntl.h>
+#include <limits.h>
+#include <pthread.h>
+#include <sched.h>
+#include <semaphore.h>
+#include <signal.h>
+#include <stdatomic.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/mman.h>
+#include <sys/prctl.h>
+#include <sys/syscall.h>
+#include <sys/sysinfo.h>
+#include <sys/wait.h>
+#include <time.h>
+#include <unistd.h>
+
+#include "../../kselftest.h"
+
+#ifndef SCHED_DEADLINE
+#define SCHED_DEADLINE 6
+#endif
+
+#define NSEC_PER_USEC 1000ULL
+#define NSEC_PER_MSEC 1000000ULL
+#define NSEC_PER_SEC 1000000000ULL
+
+struct dl_attr {
+ uint32_t size;
+ uint32_t sched_policy;
+ uint64_t sched_flags;
+ int32_t sched_nice;
+ uint32_t sched_priority;
+ uint64_t sched_runtime;
+ uint64_t sched_deadline;
+ uint64_t sched_period;
+};
+
+/* Written by the child, read by the parent. */
+struct shared {
+ uint64_t stamp; /* last time the child ran */
+ uint64_t run_start; /* when it last resumed after a throttle */
+ uint64_t runs; /* resumptions after a throttle */
+ uint64_t sleeps; /* sleeps begun */
+};
+
+static uint64_t now_ns(void)
+{
+ struct timespec ts;
+
+ clock_gettime(CLOCK_MONOTONIC, &ts);
+ return ts.tv_sec * NSEC_PER_SEC + ts.tv_nsec;
+}
+
+static void sleep_ns(uint64_t ns)
+{
+ struct timespec ts = { .tv_sec = ns / NSEC_PER_SEC, .tv_nsec = ns % NSEC_PER_SEC };
+
+ while (nanosleep(&ts, &ts) && errno == EINTR)
+ ;
+}
+
+static void sleep_until(uint64_t t)
+{
+ struct timespec ts = { .tv_sec = t / NSEC_PER_SEC, .tv_nsec = t % NSEC_PER_SEC };
+
+ while (clock_nanosleep(CLOCK_MONOTONIC, TIMER_ABSTIME, &ts, NULL) == EINTR)
+ ;
+}
+
+static int set_deadline(pid_t pid, uint64_t runtime, uint64_t period)
+{
+ struct dl_attr attr = {
+ .size = sizeof(attr),
+ .sched_policy = SCHED_DEADLINE,
+ .sched_runtime = runtime,
+ .sched_deadline = period,
+ .sched_period = period,
+ };
+
+ return syscall(__NR_sched_setattr, pid, &attr, 0) ? -errno : 0;
+}
+
+static int set_other(pid_t pid)
+{
+ struct sched_param sp = { .sched_priority = 0 };
+
+ return sched_setscheduler(pid, SCHED_OTHER, &sp) ? -errno : 0;
+}
+
+static struct shared *sh;
+
+/*
+ * Fork a child that is killed when the test dies, so that it cannot keep
+ * its bandwidth. Return what fork() returns.
+ */
+static pid_t fork_child(void)
+{
+ pid_t parent = getpid();
+ pid_t pid = fork();
+
+ if (!pid && (prctl(PR_SET_PDEATHSIG, SIGKILL) || getppid() != parent))
+ _exit(KSFT_FAIL);
+ return pid;
+}
+
+static void reap(pid_t pid)
+{
+ int status;
+
+ if (pid <= 0)
+ return;
+ if (kill(pid, SIGKILL) && errno != ESRCH)
+ ksft_print_msg("kill %d: %s\n", pid, strerror(errno));
+ /*
+ * A deadline task left throttled for good never runs, even to die.
+ * Switched to SCHED_OTHER, it runs and dies. Ignore the result: the
+ * task may be dead already.
+ */
+ set_other(pid);
+ while (waitpid(pid, &status, 0) != pid) {
+ if (errno == EINTR)
+ continue;
+ ksft_print_msg("waitpid %d: %s\n", pid, strerror(errno));
+ break;
+ }
+}
+
+/*
+ * Report a child that died before the end of its case. Return KSFT_SKIP
+ * if it could not enter SCHED_DEADLINE, and KSFT_FAIL otherwise.
+ */
+static int gone(int status)
+{
+ if (WIFEXITED(status) && WEXITSTATUS(status) == KSFT_SKIP) {
+ ksft_print_msg("SCHED_DEADLINE refused\n");
+ return KSFT_SKIP;
+ }
+ if (WIFSIGNALED(status))
+ ksft_print_msg("the task died of signal %d\n", WTERMSIG(status));
+ else
+ ksft_print_msg("the task exited with %d\n", WEXITSTATUS(status));
+ return KSFT_FAIL;
+}
+
+/*
+ * Spin, so that the task uses up its runtime and is throttled in every
+ * period. Record in sh when it runs, and when it resumes after a
+ * throttle. A throttle stops the task for most of its 2s period. A gap
+ * of 1s or less, such as a preemption, is not a throttle.
+ */
+static void spinner(void)
+{
+ uint64_t last = now_ns();
+
+ for (;;) {
+ uint64_t t = now_ns();
+
+ if (t - last > NSEC_PER_SEC) {
+ __atomic_store_n(&sh->run_start, t, __ATOMIC_RELAXED);
+ __atomic_fetch_add(&sh->runs, 1, __ATOMIC_RELEASE);
+ }
+ __atomic_store_n(&sh->stamp, t, __ATOMIC_RELEASE);
+ last = t;
+ }
+}
+
+/*
+ * Return the runtime @pid has left, from dl.runtime in /proc/<pid>/sched.
+ * It is below zero for a throttled task: the task overran by that much.
+ * Return 1 if it cannot be read.
+ */
+static long long dl_runtime_left(pid_t pid)
+{
+ char path[64], line[256];
+ long long left = 1;
+ FILE *f;
+
+ snprintf(path, sizeof(path), "/proc/%d/sched", pid);
+ f = fopen(path, "r");
+ if (!f)
+ return 1;
+ while (fgets(line, sizeof(line), f)) {
+ char *val = strchr(line, ':');
+
+ if (!strncmp(line, "dl.runtime", 10) && val) {
+ left = atoll(val + 1);
+ break;
+ }
+ }
+ fclose(f);
+ return left;
+}
+
+#define LOST_RUNTIME (10 * NSEC_PER_MSEC)
+#define LOST_PERIOD (2 * NSEC_PER_SEC)
+#define LOST_MARGIN (200 * NSEC_PER_USEC)
+
+/*
+ * The child is a deadline task of 10ms every 2s, and spins. Three times,
+ * the parent does this:
+ * - Wait until the child resumes at a replenishment, at time r, and is
+ * throttled again.
+ * - Switch it to SCHED_OTHER.
+ * - Sleep until r + LOST_PERIOD + LOST_MARGIN. r + LOST_PERIOD is the
+ * child's deadline. Its replenishment timer fires there, on a task out
+ * of SCHED_DEADLINE.
+ * - Switch it back to SCHED_DEADLINE.
+ * - Check that it runs again within 1.5 periods.
+ *
+ * The switch back must come before the 0-lag time. The 0-lag time is after
+ * the deadline by the overrun times period/runtime, which is 200 here.
+ * The overrun is how far the task ran past its runtime before the kernel
+ * throttled it. The parent reads it from /proc. A round whose switch back
+ * comes after the 0-lag time tests nothing, and does not count. The check
+ * keeps half of LOST_MARGIN as a margin. The case is skipped if no round
+ * counts. If the overrun cannot be read, every round counts.
+ */
+static int test_throttled_round_trip(void)
+{
+ int attempt, ret = KSFT_PASS, err, counted = 0;
+ pid_t pid;
+
+ memset(sh, 0, sizeof(*sh));
+ pid = fork_child();
+ if (pid < 0) {
+ ksft_print_msg("fork: %s\n", strerror(errno));
+ return KSFT_FAIL;
+ }
+ if (!pid) {
+ if (set_deadline(0, LOST_RUNTIME, LOST_PERIOD))
+ _exit(KSFT_SKIP);
+ spinner();
+ }
+ for (attempt = 0; attempt < 3 && ret == KSFT_PASS; attempt++) {
+ uint64_t runs, r, s, t;
+ long long left;
+
+ /*
+ * Wait until the task resumes at a replenishment, at time r.
+ * Then wait until it stops running: it is throttled.
+ */
+ runs = __atomic_load_n(&sh->runs, __ATOMIC_ACQUIRE);
+ t = now_ns();
+ while (__atomic_load_n(&sh->runs, __ATOMIC_ACQUIRE) == runs) {
+ if (now_ns() - t > 3 * LOST_PERIOD) {
+ int status;
+
+ if (waitpid(pid, &status, WNOHANG) == pid)
+ return gone(status);
+ ksft_print_msg("the task did not resume within 3 periods\n");
+ reap(pid);
+ return KSFT_FAIL;
+ }
+ sleep_ns(NSEC_PER_MSEC);
+ }
+ r = __atomic_load_n(&sh->run_start, __ATOMIC_RELAXED);
+ do {
+ s = __atomic_load_n(&sh->stamp, __ATOMIC_ACQUIRE);
+ sleep_ns(2 * NSEC_PER_MSEC);
+ } while (__atomic_load_n(&sh->stamp, __ATOMIC_ACQUIRE) != s);
+ /*
+ * The task is throttled until r + LOST_PERIOD. Switch it out
+ * now, and back LOST_MARGIN after that.
+ */
+ left = dl_runtime_left(pid);
+ err = set_other(pid);
+ if (!err) {
+ sleep_until(r + LOST_PERIOD + LOST_MARGIN);
+ err = set_deadline(pid, LOST_RUNTIME, LOST_PERIOD);
+ t = now_ns();
+ }
+ if (err) {
+ ksft_print_msg("policy switch: %s\n", strerror(-err));
+ ret = KSFT_FAIL;
+ break;
+ }
+ /* The deadline is at most r + LOST_PERIOD: r is read after it. */
+ if (left > 0 ||
+ t + LOST_MARGIN / 2 <
+ r + LOST_PERIOD + (uint64_t)-left * (LOST_PERIOD / LOST_RUNTIME))
+ counted++;
+ else
+ ksft_print_msg("an overrun of %lld ns: switched back after the 0-lag time\n",
+ -left);
+ sleep_ns(10 * NSEC_PER_MSEC);
+ s = __atomic_load_n(&sh->stamp, __ATOMIC_ACQUIRE);
+ t = now_ns();
+ while (__atomic_load_n(&sh->stamp, __ATOMIC_ACQUIRE) == s &&
+ now_ns() - t < LOST_PERIOD + LOST_PERIOD / 2)
+ sleep_ns(NSEC_PER_MSEC);
+ if (__atomic_load_n(&sh->stamp, __ATOMIC_ACQUIRE) == s) {
+ ksft_print_msg("throttled, switched out and back: never ran again\n");
+ ret = KSFT_FAIL;
+ }
+ }
+ reap(pid);
+ if (ret == KSFT_PASS && !counted)
+ ret = KSFT_SKIP;
+ return ret;
+}
+
+#define LEAK_RUNTIME (9 * NSEC_PER_MSEC)
+#define LEAK_PERIOD (10 * NSEC_PER_MSEC)
+
+/* Run 0.2ms, then sleep 50ms, forever. Count the sleeps in sh. */
+static void sleeper(void)
+{
+ for (;;) {
+ uint64_t end = now_ns() + 200 * NSEC_PER_USEC;
+
+ while (now_ns() < end)
+ ;
+ __atomic_fetch_add(&sh->sleeps, 1, __ATOMIC_RELEASE);
+ sleep_ns(50 * NSEC_PER_MSEC);
+ }
+}
+
+/*
+ * Switch a sleeping task out of SCHED_DEADLINE and back, over and over.
+ * The task has 90% of a CPU. If each round trip leaks it, the root domain
+ * is full after about as many round trips as it has CPUs.
+ */
+static int test_asleep_round_trip(void)
+{
+ int i, trips = get_nprocs() * 10 / 9 + 3, ret = KSFT_PASS, err;
+ pid_t pid;
+ FILE *f;
+
+ f = fopen("/proc/sys/kernel/sched_rt_runtime_us", "r");
+ if (f) {
+ long rt = 0;
+
+ if (fscanf(f, "%ld", &rt) == 1 && rt < 0) {
+ fclose(f);
+ ksft_print_msg("no admission control: a leak cannot be seen\n");
+ return KSFT_SKIP;
+ }
+ fclose(f);
+ }
+ memset(sh, 0, sizeof(*sh));
+ pid = fork_child();
+ if (pid < 0) {
+ ksft_print_msg("fork: %s\n", strerror(errno));
+ return KSFT_FAIL;
+ }
+ if (!pid) {
+ if (set_deadline(0, LEAK_RUNTIME, LEAK_PERIOD))
+ _exit(KSFT_SKIP);
+ sleeper();
+ }
+ for (i = 0; i < trips; i++) {
+ uint64_t sleeps = __atomic_load_n(&sh->sleeps, __ATOMIC_ACQUIRE);
+ uint64_t t = now_ns();
+
+ while (__atomic_load_n(&sh->sleeps, __ATOMIC_ACQUIRE) == sleeps) {
+ if (now_ns() - t > NSEC_PER_SEC) {
+ int status;
+
+ if (waitpid(pid, &status, WNOHANG) == pid)
+ return gone(status);
+ ksft_print_msg("the task did not run within 1s\n");
+ reap(pid);
+ return KSFT_FAIL;
+ }
+ sleep_ns(NSEC_PER_MSEC);
+ }
+ /*
+ * Wait 5ms. The task is still asleep, and past its 0-lag
+ * time.
+ */
+ sleep_ns(5 * NSEC_PER_MSEC);
+ err = set_other(pid);
+ if (!err)
+ err = set_deadline(pid, LEAK_RUNTIME, LEAK_PERIOD);
+ if (err) {
+ ksft_print_msg("switch back refused after %d of %d round trips: %s\n",
+ i, trips, strerror(-err));
+ ret = KSFT_FAIL;
+ break;
+ }
+ }
+ reap(pid);
+ return ret;
+}
+
+/*
+ * Take a deadline task out of the class with sysrq-n. Then check that
+ * another task can still enter SCHED_DEADLINE.
+ *
+ * sysrq-n calls __sched_setscheduler() on every real-time and deadline
+ * user task, under tasklist_lock, to make it SCHED_OTHER. On a kernel
+ * with the bug, that call takes cpuset_mutex for a deadline task and
+ * never releases it. No task can enter SCHED_DEADLINE after that.
+ *
+ * sysrq-n changes the whole system. So the sysrq cases run last, and only
+ * with DEADLINE_TEST_SYSRQ set.
+ */
+static int test_sysrq_normalize(void)
+{
+ int fd, status, ret = KSFT_PASS;
+ pid_t pid, prober, w;
+ uint64_t t;
+
+ if (!getenv("DEADLINE_TEST_SYSRQ")) {
+ ksft_print_msg("sysrq-n renices every real-time task: set DEADLINE_TEST_SYSRQ=1\n");
+ return KSFT_SKIP;
+ }
+ fd = open("/proc/sysrq-trigger", O_WRONLY);
+ if (fd < 0) {
+ ksft_print_msg("/proc/sysrq-trigger: %s\n", strerror(errno));
+ return KSFT_SKIP;
+ }
+ pid = fork_child();
+ if (pid < 0) {
+ ksft_print_msg("fork: %s\n", strerror(errno));
+ close(fd);
+ return KSFT_FAIL;
+ }
+ if (!pid) {
+ if (set_deadline(0, LOST_RUNTIME, LOST_PERIOD))
+ _exit(KSFT_SKIP);
+ for (;;)
+ pause();
+ }
+ t = now_ns();
+ while (sched_getscheduler(pid) != SCHED_DEADLINE) {
+ if (waitpid(pid, &status, WNOHANG) == pid) {
+ close(fd);
+ return gone(status);
+ }
+ if (now_ns() - t > NSEC_PER_SEC) {
+ ksft_print_msg("the task did not enter SCHED_DEADLINE within 1s\n");
+ close(fd);
+ reap(pid);
+ return KSFT_FAIL;
+ }
+ sleep_ns(NSEC_PER_MSEC);
+ }
+ if (write(fd, "n", 1) != 1) {
+ ksft_print_msg("sysrq-n: %s\n", strerror(errno));
+ close(fd);
+ reap(pid);
+ return KSFT_SKIP;
+ }
+ close(fd);
+ t = now_ns();
+ while (sched_getscheduler(pid) != SCHED_OTHER && now_ns() - t < NSEC_PER_SEC)
+ sleep_ns(NSEC_PER_MSEC);
+ if (sched_getscheduler(pid) != SCHED_OTHER) {
+ ksft_print_msg("still SCHED_DEADLINE after sysrq-n\n");
+ ret = KSFT_FAIL;
+ }
+ /* A new task must still enter SCHED_DEADLINE within 5s. */
+ prober = fork_child();
+ if (prober < 0) {
+ ksft_print_msg("fork: %s\n", strerror(errno));
+ reap(pid);
+ return KSFT_FAIL;
+ }
+ if (!prober)
+ _exit(set_deadline(0, LOST_RUNTIME, LOST_PERIOD) ? 1 : 0);
+ t = now_ns();
+ while ((w = waitpid(prober, &status, WNOHANG)) == 0) {
+ if (now_ns() - t > 5 * NSEC_PER_SEC) {
+ ksft_print_msg("SCHED_DEADLINE admission blocked after sysrq-n\n");
+ reap(pid);
+ /*
+ * The prober is blocked on a mutex. Do not wait for
+ * it.
+ */
+ return KSFT_FAIL;
+ }
+ sleep_ns(NSEC_PER_MSEC);
+ }
+ if (w != prober) {
+ ksft_print_msg("waitpid %d: %s\n", prober, strerror(errno));
+ ret = KSFT_FAIL;
+ } else if (!WIFEXITED(status) || WEXITSTATUS(status)) {
+ ksft_print_msg("SCHED_DEADLINE refused after sysrq-n\n");
+ ret = KSFT_FAIL;
+ }
+ reap(pid);
+ return ret;
+}
+
+/* The priority /proc shows for a deadline task, or a task boosted by one. */
+#define DL_TASK_PRIO (-101)
+
+static pthread_mutex_t pi_lock;
+static sem_t owner_locked, owner_go;
+static pid_t owner_tid;
+static int owner_ms; /* the owner's CPU time in its first period */
+
+static uint64_t thread_time_ns(void)
+{
+ struct timespec ts;
+
+ clock_gettime(CLOCK_THREAD_CPUTIME_ID, &ts);
+ return ts.tv_sec * NSEC_PER_SEC + ts.tv_nsec;
+}
+
+/*
+ * Wait on @s for at most @ms, retrying on signals. Return 0, or -1 on a
+ * timeout or an error.
+ */
+static int sem_wait_for(sem_t *s, int ms)
+{
+ struct timespec ts;
+
+ clock_gettime(CLOCK_REALTIME, &ts);
+ ts.tv_nsec += (long)(ms % 1000) * 1000000L;
+ ts.tv_sec += ms / 1000 + ts.tv_nsec / 1000000000L;
+ ts.tv_nsec %= 1000000000L;
+ while (sem_timedwait(s, &ts)) {
+ if (errno != EINTR)
+ return -1;
+ }
+ return 0;
+}
+
+/* Return the priority @tid runs at, from /proc, or 0. */
+static int prio_of(pid_t tid)
+{
+ char path[64], buf[1024], *p;
+ ssize_t n;
+ int fd, i;
+
+ snprintf(path, sizeof(path), "/proc/%d/stat", tid);
+ fd = open(path, O_RDONLY);
+ if (fd < 0)
+ return 0;
+ n = read(fd, buf, sizeof(buf) - 1);
+ close(fd);
+ if (n <= 0)
+ return 0;
+ buf[n] = 0;
+ /* The priority is field 18, the 16th after the command name. */
+ p = strrchr(buf, ')');
+ for (i = 0; p && i < 16; i++)
+ p = strchr(p + 1, ' ');
+ return p ? atoi(p + 1) : 0;
+}
+
+/* Join @t. Return -1 if the join fails or @t returned an error. */
+static int join(pthread_t t)
+{
+ void *res;
+ int err = pthread_join(t, &res);
+
+ if (err)
+ ksft_print_msg("pthread_join: %s\n", strerror(err));
+ return err || res ? -1 : 0;
+}
+
+/*
+ * Run as the owner thread. Take pi_lock, post owner_locked, and sleep on
+ * owner_go while holding the lock. A deadline waiter boosts the owner
+ * during that sleep. Then drop the lock, become a deadline task of 10ms
+ * every 100ms, and spin for 100ms. Store the CPU time used, in ms, in
+ * owner_ms. Return NULL, or 1 on an error.
+ */
+static void *sleeping_owner(void *arg)
+{
+ uint64_t t0, cpu0;
+ int err;
+
+ owner_tid = syscall(SYS_gettid);
+ err = pthread_mutex_lock(&pi_lock);
+ if (err) {
+ ksft_print_msg("the owner cannot take the lock: %s\n", strerror(err));
+ return (void *)1;
+ }
+ sem_post(&owner_locked);
+ while (sem_wait(&owner_go) && errno == EINTR)
+ ;
+ err = pthread_mutex_unlock(&pi_lock);
+ if (err) {
+ ksft_print_msg("the owner cannot unlock: %s\n", strerror(err));
+ return (void *)1;
+ }
+ err = set_deadline(0, 10 * NSEC_PER_MSEC, 100 * NSEC_PER_MSEC);
+ if (err) {
+ ksft_print_msg("the owner refused as a deadline task: %s\n", strerror(-err));
+ return (void *)1;
+ }
+ t0 = now_ns();
+ cpu0 = thread_time_ns();
+ while (now_ns() - t0 < 100 * NSEC_PER_MSEC)
+ ;
+ owner_ms = (thread_time_ns() - cpu0) / NSEC_PER_MSEC;
+ set_other(0);
+ return NULL;
+}
+
+/*
+ * The waiter thread: become a deadline task of 5ms every 50ms, and wait
+ * up to 300ms for pi_lock. The owner holds the lock all along. Return
+ * NULL if the wait timed out, or 1.
+ */
+static void *boosting_waiter(void *arg)
+{
+ struct timespec ts;
+ int err;
+
+ err = set_deadline(0, 5 * NSEC_PER_MSEC, 50 * NSEC_PER_MSEC);
+ if (err) {
+ ksft_print_msg("the waiter refused as a deadline task: %s\n", strerror(-err));
+ return (void *)1;
+ }
+ clock_gettime(CLOCK_REALTIME, &ts);
+ ts.tv_nsec += 300 * 1000000L;
+ ts.tv_sec += ts.tv_nsec / 1000000000L;
+ ts.tv_nsec %= 1000000000L;
+ err = pthread_mutex_timedlock(&pi_lock, &ts);
+ if (!err)
+ pthread_mutex_unlock(&pi_lock);
+ set_other(0); /* sysrq-n may have done it */
+ if (err != ETIMEDOUT) {
+ ksft_print_msg("the waiter of the lock: %s, not a timeout\n",
+ err ? strerror(err) : "got it");
+ return (void *)1;
+ }
+ return NULL;
+}
+
+/*
+ * Check that sysrq-n ends the boost of a SCHED_OTHER lock owner.
+ *
+ * M is the main thread, O the owner, W the waiter. All three start as
+ * SCHED_OTHER. pi_lock is a priority inheritance mutex.
+ *
+ * M O W
+ * start O
+ * lock pi_lock
+ * post owner_locked
+ * sleep on owner_go
+ * start W
+ * sched_setattr():
+ * deadline,
+ * 5ms every 50ms
+ * timedlock pi_lock,
+ * 300ms: boosts O
+ * see O at deadline priority
+ * write "n" to sysrq-trigger:
+ * O and W leave the class
+ * timedlock times out:
+ * deboosts O
+ * join W
+ * post owner_go
+ * unlock pi_lock
+ * sched_setattr(): deadline,
+ * 10ms every 100ms
+ * spin 100ms, measure
+ * CPU time
+ * join O
+ *
+ * sysrq-n takes O out of the deadline class without a chain walk. The
+ * boost must end there. On a kernel with the bug it does not, and the
+ * deboost at W's timeout sees no change of priority, so it leaves the
+ * boost too. O would then enter SCHED_DEADLINE as a boosted task, and
+ * never be throttled. O must get about 10ms of CPU time in its 100ms. The
+ * case fails above 20ms.
+ *
+ * O sleeps while it is boosted, on purpose. A boosted task runs on its
+ * waiter's parameters. sysrq-n takes W out of the class too, and W's
+ * parameters can then be cleared. On a kernel that refills the runtime
+ * from cleared parameters, a running O would loop in
+ * replenish_dl_entity().
+ *
+ * Run the case only if sysrq_normalize passed. @normalized is its result.
+ * Otherwise sysrq-n may leave cpuset_mutex held, and O would block in
+ * sched_setattr().
+ */
+static int test_sysrq_boosted_owner(int normalized)
+{
+ int fd, err, ret = KSFT_PASS;
+ pthread_mutexattr_t a;
+ pthread_t o, w;
+ uint64_t t;
+
+ if (!getenv("DEADLINE_TEST_SYSRQ")) {
+ ksft_print_msg("sysrq-n renices every real-time task: set DEADLINE_TEST_SYSRQ=1\n");
+ return KSFT_SKIP;
+ }
+ if (normalized != KSFT_PASS) {
+ ksft_print_msg("sysrq_normalize did not pass: sched_setattr() may block\n");
+ return KSFT_SKIP;
+ }
+ fd = open("/proc/sysrq-trigger", O_WRONLY);
+ if (fd < 0) {
+ ksft_print_msg("/proc/sysrq-trigger: %s\n", strerror(errno));
+ return KSFT_SKIP;
+ }
+ err = pthread_mutexattr_init(&a);
+ if (!err) {
+ err = pthread_mutexattr_setprotocol(&a, PTHREAD_PRIO_INHERIT);
+ if (!err)
+ err = pthread_mutex_init(&pi_lock, &a);
+ pthread_mutexattr_destroy(&a);
+ }
+ if (err) {
+ ksft_print_msg("no priority inheritance mutex: %s\n", strerror(err));
+ close(fd);
+ return KSFT_SKIP;
+ }
+ if (sem_init(&owner_locked, 0, 0) || sem_init(&owner_go, 0, 0)) {
+ ksft_print_msg("sem_init: %s\n", strerror(errno));
+ ret = KSFT_FAIL;
+ goto out;
+ }
+ owner_ms = -1;
+ if (pthread_create(&o, NULL, sleeping_owner, NULL)) {
+ ksft_print_msg("the owner thread could not start\n");
+ ret = KSFT_FAIL;
+ goto out_sem;
+ }
+ if (sem_wait_for(&owner_locked, 5000)) {
+ ksft_print_msg("the owner never took the lock\n");
+ sem_post(&owner_go); /* in case it gets there late */
+ join(o);
+ ret = KSFT_FAIL;
+ goto out_sem;
+ }
+ if (pthread_create(&w, NULL, boosting_waiter, NULL)) {
+ ksft_print_msg("the waiter thread could not start\n");
+ ret = KSFT_FAIL;
+ goto unlock;
+ }
+ t = now_ns();
+ while (prio_of(owner_tid) != DL_TASK_PRIO && now_ns() - t < 200 * NSEC_PER_MSEC)
+ sleep_ns(NSEC_PER_MSEC);
+ if (prio_of(owner_tid) != DL_TASK_PRIO) {
+ ksft_print_msg("the waiter never boosted the owner\n");
+ ret = KSFT_FAIL;
+ } else if (write(fd, "n", 1) != 1) {
+ ksft_print_msg("sysrq-n: %s\n", strerror(errno));
+ ret = KSFT_SKIP;
+ }
+ if (join(w))
+ ret = KSFT_FAIL; /* the waiter said why */
+unlock:
+ sem_post(&owner_go);
+ if (join(o)) {
+ ret = KSFT_FAIL; /* the owner said why */
+ } else if (ret == KSFT_PASS && owner_ms > 20) {
+ ksft_print_msg("the owner ran %d ms of its first 100: the boost outlived sysrq-n\n",
+ owner_ms);
+ ret = KSFT_FAIL;
+ }
+out_sem:
+ sem_destroy(&owner_go);
+ sem_destroy(&owner_locked);
+out:
+ close(fd);
+ pthread_mutex_destroy(&pi_lock);
+ return ret;
+}
+
+#define FILLERS 2048
+#define FILLER_STACK (64 * 1024 > PTHREAD_STACK_MIN ? 64 * 1024 : PTHREAD_STACK_MIN)
+
+static atomic_int running_stop;
+static sem_t filler_ready, waiter_go;
+static int filler_pipe[2];
+static int running_cpu;
+
+/*
+ * Run as a filler thread: become SCHED_FIFO, and sleep until the write end
+ * of filler_pipe is closed. Return NULL, or 1 on an error.
+ */
+static void *filler(void *arg)
+{
+ struct sched_param sp = { .sched_priority = 1 };
+ int err = sched_setscheduler(0, SCHED_FIFO, &sp) ? errno : 0;
+ char c;
+
+ sem_post(&filler_ready);
+ if (err)
+ return (void *)1;
+ while (read(filler_pipe[0], &c, 1) < 0 && errno == EINTR)
+ ;
+ return NULL;
+}
+
+/*
+ * Run as the owner thread O of sysrq_running_owner, on running_cpu only.
+ * Take pi_lock and post owner_locked. Spin until running_stop is set, or
+ * for 5s. Then unlock. Return NULL, or 1 on an error.
+ */
+static void *running_owner(void *arg)
+{
+ uint64_t t0;
+ cpu_set_t one;
+ int err;
+
+ owner_tid = syscall(SYS_gettid);
+ CPU_ZERO(&one);
+ CPU_SET(running_cpu, &one);
+ if (sched_setaffinity(0, sizeof(one), &one)) {
+ ksft_print_msg("the owner cannot run on CPU %d: %s\n", running_cpu,
+ strerror(errno));
+ sem_post(&owner_locked);
+ return (void *)1;
+ }
+ err = pthread_mutex_lock(&pi_lock);
+ if (err) {
+ ksft_print_msg("the owner cannot take the lock: %s\n", strerror(err));
+ sem_post(&owner_locked);
+ return (void *)1;
+ }
+ sem_post(&owner_locked);
+ t0 = now_ns();
+ while (!atomic_load(&running_stop) && now_ns() - t0 < 5 * NSEC_PER_SEC)
+ ;
+ err = pthread_mutex_unlock(&pi_lock);
+ if (err) {
+ ksft_print_msg("the owner cannot unlock: %s\n", strerror(err));
+ return (void *)1;
+ }
+ return NULL;
+}
+
+/*
+ * Run as the waiter thread W of sysrq_running_owner. Wait for waiter_go.
+ * Then become a deadline task of 100us every 1ms, and wait for pi_lock.
+ * Return NULL once it got the lock, or 1.
+ */
+static void *running_waiter(void *arg)
+{
+ int err;
+
+ while (sem_wait(&waiter_go) && errno == EINTR)
+ ;
+ err = set_deadline(0, 100 * NSEC_PER_USEC, NSEC_PER_MSEC);
+ if (err) {
+ ksft_print_msg("the waiter refused as a deadline task: %s\n", strerror(-err));
+ return (void *)1;
+ }
+ err = pthread_mutex_lock(&pi_lock);
+ if (!err)
+ pthread_mutex_unlock(&pi_lock);
+ set_other(0); /* sysrq-n did it already */
+ if (err) {
+ ksft_print_msg("the waiter cannot take the lock: %s\n", strerror(err));
+ return (void *)1;
+ }
+ return NULL;
+}
+
+/*
+ * Check that a boosted owner that runs survives the clearing of its
+ * donor's parameters.
+ *
+ * M is the main thread, O the owner, W the waiter. pi_lock is a priority
+ * inheritance mutex. O runs alone on the last CPU. M and the fillers run
+ * on the other CPUs. The fillers are 2048 sleeping SCHED_FIFO threads,
+ * created after W and before O.
+ *
+ * M O W
+ * start W, the fillers, O
+ * lock pi_lock
+ * post owner_locked
+ * spin
+ * post waiter_go
+ * sched_setattr():
+ * deadline,
+ * 100us every 1ms
+ * lock pi_lock:
+ * boosts O
+ * runs on W's runtime,
+ * refilled as it runs out
+ * see O at deadline priority
+ * sleep 20ms
+ * write "n" to sysrq-trigger:
+ * W leaves the class.
+ * Its parameters are
+ * cleared.
+ * Then the fillers leave
+ * their class, one by one.
+ * refills from W's cleared
+ * parameters
+ * Then O leaves the class.
+ * see O not boosted
+ * set running_stop
+ * unlock pi_lock
+ * take pi_lock, unlock
+ * join O and W
+ * end the fillers
+ *
+ * sysrq-n changes the tasks in the order of the task list, with no chain
+ * walk. The fillers keep O boosted from cleared parameters for a few ms.
+ * O refills its runtime in that time. On a kernel with the bug, that
+ * refill loops forever with O's rq lock held. sysrq-n then waits forever
+ * for that lock. The case passes if every thread ends.
+ *
+ * Run the case only if sysrq_normalize passed. @normalized is its result.
+ */
+static int test_sysrq_running_owner(int normalized)
+{
+ int fd, err, i, started = 0, ret = KSFT_PASS;
+ pthread_t o, w, *fill = NULL;
+ cpu_set_t old, others;
+ pthread_mutexattr_t a;
+ pthread_attr_t fa;
+ uint64_t t;
+
+ if (!getenv("DEADLINE_TEST_SYSRQ")) {
+ ksft_print_msg("sysrq-n renices every real-time task: set DEADLINE_TEST_SYSRQ=1\n");
+ return KSFT_SKIP;
+ }
+ if (normalized != KSFT_PASS) {
+ ksft_print_msg("sysrq_normalize did not pass: sched_setattr() may block\n");
+ return KSFT_SKIP;
+ }
+ if (get_nprocs() < 2 || sched_getaffinity(0, sizeof(old), &old)) {
+ ksft_print_msg("needs 2 CPUs\n");
+ return KSFT_SKIP;
+ }
+ for (running_cpu = CPU_SETSIZE - 1; running_cpu >= 0; running_cpu--)
+ if (CPU_ISSET(running_cpu, &old))
+ break;
+ others = old;
+ CPU_CLR(running_cpu, &others);
+ if (!CPU_COUNT(&others)) {
+ ksft_print_msg("needs 2 CPUs\n");
+ return KSFT_SKIP;
+ }
+ fd = open("/proc/sysrq-trigger", O_WRONLY);
+ if (fd < 0) {
+ ksft_print_msg("/proc/sysrq-trigger: %s\n", strerror(errno));
+ return KSFT_SKIP;
+ }
+ err = pthread_mutexattr_init(&a);
+ if (!err) {
+ err = pthread_mutexattr_setprotocol(&a, PTHREAD_PRIO_INHERIT);
+ if (!err)
+ err = pthread_mutex_init(&pi_lock, &a);
+ pthread_mutexattr_destroy(&a);
+ }
+ if (err) {
+ ksft_print_msg("no priority inheritance mutex: %s\n", strerror(err));
+ close(fd);
+ return KSFT_SKIP;
+ }
+ fill = calloc(FILLERS, sizeof(*fill));
+ if (!fill || pipe(filler_pipe)) {
+ ksft_print_msg("setup: %s\n", strerror(errno));
+ ret = KSFT_FAIL;
+ goto out_free;
+ }
+ if (sem_init(&owner_locked, 0, 0) || sem_init(&filler_ready, 0, 0) ||
+ sem_init(&waiter_go, 0, 0)) {
+ ksft_print_msg("sem_init: %s\n", strerror(errno));
+ close(filler_pipe[1]);
+ ret = KSFT_FAIL;
+ goto out_pipe;
+ }
+ err = pthread_attr_init(&fa);
+ if (!err) {
+ err = pthread_attr_setstacksize(&fa, FILLER_STACK);
+ if (err)
+ pthread_attr_destroy(&fa);
+ }
+ if (err) {
+ ksft_print_msg("filler stack of %zu bytes: %s\n", (size_t)FILLER_STACK,
+ strerror(err));
+ close(filler_pipe[1]);
+ ret = KSFT_FAIL;
+ goto out_sem;
+ }
+ running_stop = 0;
+ /* W, then the fillers, then O: the order of the task list */
+ if (pthread_create(&w, NULL, running_waiter, NULL)) {
+ ksft_print_msg("the waiter thread could not start\n");
+ ret = KSFT_FAIL;
+ goto out;
+ }
+ /*
+ * Keep M and the fillers off O's CPU. W keeps every CPU: a deadline
+ * task must be allowed on its whole root domain.
+ */
+ if (sched_setaffinity(0, sizeof(others), &others)) {
+ ksft_print_msg("sched_setaffinity: %s\n", strerror(errno));
+ sem_post(&waiter_go);
+ ret = KSFT_FAIL;
+ goto join_w;
+ }
+ for (started = 0; started < FILLERS; started++) {
+ if (pthread_create(&fill[started], &fa, filler, NULL)) {
+ ksft_print_msg("filler %d could not start\n", started);
+ ret = KSFT_FAIL;
+ break;
+ }
+ }
+ for (i = 0; i < started; i++)
+ if (sem_wait_for(&filler_ready, 5000))
+ ret = KSFT_FAIL;
+ if (ret != KSFT_PASS) {
+ sem_post(&waiter_go);
+ goto join_w;
+ }
+ if (pthread_create(&o, NULL, running_owner, NULL)) {
+ ksft_print_msg("the owner thread could not start\n");
+ sem_post(&waiter_go);
+ ret = KSFT_FAIL;
+ goto join_w;
+ }
+ if (sem_wait_for(&owner_locked, 5000)) {
+ ksft_print_msg("the owner never took the lock\n");
+ ret = KSFT_FAIL;
+ }
+ sem_post(&waiter_go);
+ t = now_ns();
+ while (ret == KSFT_PASS && prio_of(owner_tid) != DL_TASK_PRIO &&
+ now_ns() - t < 200 * NSEC_PER_MSEC)
+ sleep_ns(NSEC_PER_MSEC);
+ if (ret == KSFT_PASS && prio_of(owner_tid) != DL_TASK_PRIO) {
+ ksft_print_msg("the waiter never boosted the owner\n");
+ ret = KSFT_FAIL;
+ }
+ if (ret == KSFT_PASS) {
+ sleep_ns(20 * NSEC_PER_MSEC);
+ if (write(fd, "n", 1) != 1) {
+ ksft_print_msg("sysrq-n: %s\n", strerror(errno));
+ ret = KSFT_SKIP;
+ } else if (prio_of(owner_tid) == DL_TASK_PRIO) {
+ ksft_print_msg("the owner is still boosted after sysrq-n\n");
+ ret = KSFT_FAIL;
+ }
+ }
+ running_stop = 1;
+ if (join(o))
+ ret = KSFT_FAIL; /* the owner said why */
+join_w:
+ if (join(w))
+ ret = KSFT_FAIL; /* the waiter said why */
+out:
+ close(filler_pipe[1]);
+ for (i = 0; i < started; i++)
+ if (join(fill[i]))
+ ret = KSFT_FAIL;
+ sched_setaffinity(0, sizeof(old), &old);
+ pthread_attr_destroy(&fa);
+out_sem:
+ sem_destroy(&waiter_go);
+ sem_destroy(&filler_ready);
+ sem_destroy(&owner_locked);
+out_pipe:
+ close(filler_pipe[0]);
+out_free:
+ free(fill);
+ close(fd);
+ pthread_mutex_destroy(&pi_lock);
+ return ret;
+}
+
+static void report(const char *name, int ret)
+{
+ if (ret == KSFT_PASS)
+ ksft_test_result_pass("%s\n", name);
+ else if (ret == KSFT_SKIP)
+ ksft_test_result_skip("%s\n", name);
+ else
+ ksft_test_result_fail("%s\n", name);
+}
+
+int main(void)
+{
+ int ret;
+
+ sh = mmap(NULL, sizeof(*sh), PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0);
+ if (sh == MAP_FAILED)
+ ksft_exit_fail_msg("mmap: %s\n", strerror(errno));
+ ksft_print_header();
+ ksft_set_plan(5);
+ /*
+ * Keep this order. On a kernel with the bugs, asleep_round_trip fills
+ * the root domain, and sysrq_normalize blocks every later admission.
+ * So sysrq_boosted_owner runs only if sysrq_normalize passed.
+ */
+ report("throttled_round_trip", test_throttled_round_trip());
+ report("asleep_round_trip", test_asleep_round_trip());
+ ret = test_sysrq_normalize();
+ report("sysrq_normalize", ret);
+ report("sysrq_boosted_owner", test_sysrq_boosted_owner(ret));
+ report("sysrq_running_owner", test_sysrq_running_owner(ret));
+ ksft_finished();
+}
--
2.47.3
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 16/16] selftests/sched: Add a test of SCHED_DEADLINE bandwidth accounting
2026-11-18 20:06 [PATCH 00/16] sched/deadline: first bandwidth accounting fixes and selftests Mathieu Desnoyers
` (14 preceding siblings ...)
2026-11-18 20:06 ` [PATCH 15/16] selftests/sched: Add a test of SCHED_DEADLINE policy round trips Mathieu Desnoyers
@ 2026-11-18 20:06 ` Mathieu Desnoyers
2026-11-18 21:04 ` sashiko-bot
15 siblings, 1 reply; 31+ messages in thread
From: Mathieu Desnoyers @ 2026-11-18 20:06 UTC (permalink / raw)
To: juri.lelli, peterz, mingo, vincent.guittot
Cc: linux-kernel, Mathieu Desnoyers, tglx, will, boqun,
dietmar.eggemann, rostedt, bsegall, mgorman, vschneid,
kprateek.nayak, luca.abeni, mathieu.poirier, longman, ridong.chen,
tj, hannes, mkoutny, shuah, bigeasy, clrkwllms, parri.andrea,
m.szyprowski, kayracizmeci, weihengqiuu, cgroups, linux-kselftest,
linux-rt-devel
Upstream has no selftest of the SCHED_DEADLINE bandwidth accounting. A
leak or a double release does not show at once. Later, the root domain
refuses deadline tasks, or admits more than it can run.
Add deadline_bw_test, with 47 cases. Most cases count the tasks of 10ms
every 100ms that a root domain admits, before and after they run. Once
the tasks of the case are gone and past their 0-lag time, the two counts
must be equal. The cases cover:
- priority inheritance, boost and deboost
- parameter changes, SCHED_FLAG_KEEP_PARAMS among them
- cpuset moves
- deaths, sched_yield() and runtime overruns
- a stopped task
- bandwidth reclaiming and fair server changes
- CPU hotplug
- parameter changes while a task wakes on another CPU (params_waking)
The comment above each case gives its scenario. params_waking also
compares the this_bw of its CPUs, which "sched/debug: Show the deadline
bandwidth of each runqueue" prints. Two cases follow reproducers of Juri
Lelli's sched-deadline-tests: stopped_switched_out and
pi_chain_two_owners.
The test needs root, deadline admission control and 3 CPUs. Most cases
need the cgroup v2 cpuset controller, and some need debugfs, CPU hotplug
or 4 CPUs. move_cancelled needs CONFIG_RT_GROUP_SCHED. A case skips
without what it needs.
The message of each fix in this series names the cases that test it.
These cases are not named there. Each fails without its fix, and passes
with it:
- overrun_caught and stopped_switched_out: "sched/deadline: Release the
bandwidth of a task that leaves the class asleep"
- params_keep_class_change: "sched: Refuse SCHED_FLAG_KEEP_PARAMS that
would change the policy or the class"
overrun_caught fails in part of the runs only.
On v7.4-rc2, in 8 VMs of 8 CPUs, six cases fail in every VM:
- boosted_owner_policy
- deboost_resets_period
- death_zombie_switched
- params_keep_boosted
- params_keep_class_change
- stopped_switched_out
overrun_caught fails in one VM. In another one, hotplug_server crashed
the kernel with a divide error in inactive_task_timer().
"sched/deadline: Do not divide by zero active CPUs of a root domain"
fixes it.
Two cases skip there. params_waking needs the this_bw that "sched/debug:
Show the deadline bandwidth of each runqueue" prints. move_cancelled
needs CONFIG_RT_GROUP_SCHED.
With the whole series, every case passes wherever it runs, PREEMPT_RT
included. The test also ran in guests built without cpusets, and in
guests booted with isolcpus=domain,1,2 or isolcpus=domain,0,1.
move_cancelled ran in two guests built with CONFIG_RT_GROUP_SCHED. No
kernel warning is logged.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Link: https://github.com/jlelli/sched-deadline-tests
---
.../selftests/sched/deadline/.gitignore | 1 +
.../testing/selftests/sched/deadline/Makefile | 2 +-
tools/testing/selftests/sched/deadline/config | 4 +
.../sched/deadline/deadline_bw_test.c | 5468 +++++++++++++++++
.../testing/selftests/sched/deadline/settings | 1 +
5 files changed, 5475 insertions(+), 1 deletion(-)
create mode 100644 tools/testing/selftests/sched/deadline/deadline_bw_test.c
create mode 100644 tools/testing/selftests/sched/deadline/settings
diff --git a/tools/testing/selftests/sched/deadline/.gitignore b/tools/testing/selftests/sched/deadline/.gitignore
index 9f203c455b91..5a572318e1ba 100644
--- a/tools/testing/selftests/sched/deadline/.gitignore
+++ b/tools/testing/selftests/sched/deadline/.gitignore
@@ -1 +1,2 @@
deadline_switch_test
+deadline_bw_test
diff --git a/tools/testing/selftests/sched/deadline/Makefile b/tools/testing/selftests/sched/deadline/Makefile
index 3f4dc1ec6119..c3156059bdcf 100644
--- a/tools/testing/selftests/sched/deadline/Makefile
+++ b/tools/testing/selftests/sched/deadline/Makefile
@@ -3,6 +3,6 @@
CFLAGS += -O2 -Wall -g -I./ $(KHDR_INCLUDES)
LDLIBS += -lpthread
-TEST_GEN_PROGS := deadline_switch_test
+TEST_GEN_PROGS := deadline_switch_test deadline_bw_test
include ../../lib.mk
diff --git a/tools/testing/selftests/sched/deadline/config b/tools/testing/selftests/sched/deadline/config
index 3a3a7285a708..b19db0c962e0 100644
--- a/tools/testing/selftests/sched/deadline/config
+++ b/tools/testing/selftests/sched/deadline/config
@@ -1 +1,5 @@
+CONFIG_CGROUPS=y
+CONFIG_CPUSETS=y
+CONFIG_DEBUG_FS=y
+CONFIG_HOTPLUG_CPU=y
CONFIG_MAGIC_SYSRQ=y
diff --git a/tools/testing/selftests/sched/deadline/deadline_bw_test.c b/tools/testing/selftests/sched/deadline/deadline_bw_test.c
new file mode 100644
index 000000000000..1e64402f90df
--- /dev/null
+++ b/tools/testing/selftests/sched/deadline/deadline_bw_test.c
@@ -0,0 +1,5468 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Test the bandwidth accounting of SCHED_DEADLINE. The deadline tasks
+ * of the cases go through:
+ * - priority inheritance
+ * - parameter changes
+ * - cpuset moves
+ * - deaths, yields and runtime overruns
+ * - bandwidth reclaiming and fair server changes
+ * - CPU hotplug
+ *
+ * Most cases measure the free bandwidth of a root domain before and after
+ * they run. probe() counts the tasks of 10ms every 100ms that the root
+ * domain admits. Once the tasks of the case are gone and their 0-lag time
+ * is past, the two counts must be equal. Fewer means leaked bandwidth.
+ * More means bandwidth released twice, which lets the root domain admit
+ * deadline tasks beyond its capacity. A deadline task that a case keeps
+ * running must also make progress.
+ *
+ * The reclaiming cases measure the share of a CPU that a deadline task
+ * with SCHED_FLAG_RECLAIM gets. That share follows extra_bw: the free
+ * bandwidth that the task's runqueue can reclaim. probe() follows the
+ * root domain's total_bw instead.
+ *
+ * The CPUs that isolcpus="domain" isolates are in the default root
+ * domain: the root domain of the isolated CPUs. The other CPUs are the
+ * housekeeping CPUs, in the housekeeping root domain. The test leaves the
+ * isolated CPUs out, and keeps its tasks off them.
+ *
+ * The cases of p_cases[] run with P, a partition root of the last
+ * housekeeping CPU. P is made anew for each case. Making it rebuilds the
+ * accounting, which clears what an earlier case left.
+ *
+ * The cases of hp_cases[] run after them, without P. They take CPUs
+ * offline. The test skips them if no CPU can go offline.
+ *
+ * The server cases change the fair server of a CPU through debugfs, and
+ * the test then puts it back.
+ *
+ * The test needs root, deadline admission control and 3 housekeeping
+ * CPUs. Some cases need 4. Most also need the cgroup v2 cpuset
+ * controller. Some need debugfs or CPU hotplug.
+ */
+#define _GNU_SOURCE
+#include <errno.h>
+#include <fcntl.h>
+#include <limits.h>
+#include <pthread.h>
+#include <sched.h>
+#include <semaphore.h>
+#include <signal.h>
+#include <stdatomic.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/mman.h>
+#include <sys/prctl.h>
+#include <sys/stat.h>
+#include <sys/syscall.h>
+#include <sys/wait.h>
+#include <time.h>
+#include <unistd.h>
+#include <linux/futex.h>
+
+#include "../../kselftest.h"
+
+#ifndef SCHED_DEADLINE
+#define SCHED_DEADLINE 6
+#endif
+#ifndef SCHED_FLAG_RECLAIM
+#define SCHED_FLAG_RECLAIM 0x02
+#endif
+#ifndef SCHED_FLAG_DL_OVERRUN
+#define SCHED_FLAG_DL_OVERRUN 0x04
+#endif
+#ifndef SCHED_FLAG_KEEP_PARAMS
+#define SCHED_FLAG_KEEP_PARAMS 0x10
+#endif
+
+#define NSEC_PER_USEC 1000ULL
+#define NSEC_PER_MSEC 1000000ULL
+#define NSEC_PER_SEC 1000000000ULL
+
+/* The threads of probe() and hold() do nothing: a small stack will do. */
+#define THREAD_STACK (64 * 1024 > PTHREAD_STACK_MIN ? 64 * 1024 : PTHREAD_STACK_MIN)
+
+#define PROBE_RUNTIME (10 * NSEC_PER_MSEC)
+#define PROBE_PERIOD (100 * NSEC_PER_MSEC)
+#define PERIOD (100 * NSEC_PER_MSEC)
+
+struct dl_attr {
+ uint32_t size;
+ uint32_t sched_policy;
+ uint64_t sched_flags;
+ int32_t sched_nice;
+ uint32_t sched_priority;
+ uint64_t sched_runtime;
+ uint64_t sched_deadline;
+ uint64_t sched_period;
+};
+
+static char root[PATH_MAX], cg_p[PATH_MAX], cg_q[PATH_MAX];
+/* The online CPUs at the start, ascending, except the isolated ones */
+static int ncpus, *online;
+static int nisol, *isol; /* the online CPUs that isolcpus="domain" isolates */
+static char isolated[4096]; /* the CPU list of /sys/devices/system/cpu/isolated */
+
+/* Return the @n-th highest online CPU, counting from 1. */
+static int top_cpu(int n)
+{
+ return online[ncpus - n];
+}
+
+/*
+ * clock_gettime() cannot fail in now_ns() and thread_time_ns(). Both
+ * clocks are always supported, and the pointer is valid.
+ */
+static uint64_t now_ns(void)
+{
+ struct timespec ts;
+
+ clock_gettime(CLOCK_MONOTONIC, &ts);
+ return ts.tv_sec * NSEC_PER_SEC + ts.tv_nsec;
+}
+
+static uint64_t thread_time_ns(void)
+{
+ struct timespec ts;
+
+ clock_gettime(CLOCK_THREAD_CPUTIME_ID, &ts);
+ return ts.tv_sec * NSEC_PER_SEC + ts.tv_nsec;
+}
+
+static void sleep_ns(uint64_t ns)
+{
+ struct timespec ts = { .tv_sec = ns / NSEC_PER_SEC, .tv_nsec = ns % NSEC_PER_SEC };
+
+ while (nanosleep(&ts, &ts) && errno == EINTR)
+ ;
+}
+
+static void spin_ns(uint64_t ns)
+{
+ uint64_t end = thread_time_ns() + ns;
+
+ while (thread_time_ns() < end)
+ ;
+}
+
+/*
+ * Poll @cond every @step ns, for at most @ms. Return 0 once it holds, or
+ * -1 on a timeout. The bound makes a child that never started, or was
+ * refused SCHED_DEADLINE, fail its case instead of hanging it.
+ */
+#define wait_for_step(ms, step, cond) \
+({ \
+ uint64_t wait_end = now_ns() + (uint64_t)(ms) * NSEC_PER_MSEC; \
+ int wait_ret = -1; \
+ \
+ while (1) { \
+ if (cond) { \
+ wait_ret = 0; \
+ break; \
+ } \
+ if (now_ns() >= wait_end) \
+ break; \
+ sleep_ns(step); \
+ } \
+ wait_ret; \
+})
+
+#define wait_for(ms, cond) wait_for_step(ms, NSEC_PER_MSEC, cond)
+
+/* Wait on @s. Retry when a signal interrupts the wait. */
+static void sem_wait_sig(sem_t *s)
+{
+ while (sem_wait(s)) {
+ if (errno == EINTR)
+ continue;
+ ksft_print_msg("sem_wait: %s\n", strerror(errno));
+ return;
+ }
+}
+
+/*
+ * Wait on @s for at most @ms, retrying on signals. Return 0, or -1 on a
+ * timeout or an error.
+ */
+static int sem_wait_for(sem_t *s, int ms)
+{
+ struct timespec ts;
+
+ clock_gettime(CLOCK_REALTIME, &ts);
+ ts.tv_nsec += (long)(ms % 1000) * 1000000L;
+ ts.tv_sec += ms / 1000 + ts.tv_nsec / 1000000000L;
+ ts.tv_nsec %= 1000000000L;
+ while (sem_timedwait(s, &ts)) {
+ if (errno == EINTR)
+ continue;
+ if (errno != ETIMEDOUT)
+ ksft_print_msg("sem_timedwait: %s\n", strerror(errno));
+ return -1;
+ }
+ return 0;
+}
+
+/*
+ * Fork a child that is killed when the thread that forked it dies, so
+ * that it cannot keep its bandwidth. Every caller forks from the main
+ * thread of its process. Print a message with @what if fork() fails.
+ * Return what fork() returns. The caller must handle -1.
+ */
+static pid_t fork_child(const char *what)
+{
+ pid_t parent = getpid();
+ pid_t pid = fork();
+
+ if (pid < 0)
+ ksft_print_msg("fork for %s: %s\n", what, strerror(errno));
+ else if (!pid && (prctl(PR_SET_PDEATHSIG, SIGKILL) || getppid() != parent))
+ _exit(KSFT_FAIL);
+ return pid;
+}
+
+static void reap(pid_t pid)
+{
+ int status;
+
+ if (pid <= 0)
+ return;
+ if (kill(pid, SIGKILL) && errno != ESRCH)
+ ksft_print_msg("kill %d: %s\n", pid, strerror(errno));
+ while (waitpid(pid, &status, 0) != pid) {
+ if (errno == EINTR)
+ continue;
+ ksft_print_msg("waitpid %d: %s\n", pid, strerror(errno));
+ break;
+ }
+}
+
+static void join(pthread_t t)
+{
+ int err = pthread_join(t, NULL);
+
+ if (err)
+ ksft_print_msg("pthread_join: %s\n", strerror(err));
+}
+
+static pid_t gettid_(void)
+{
+ return syscall(SYS_gettid);
+}
+
+static int set_deadline_flags(pid_t tid, uint64_t runtime, uint64_t period,
+ uint64_t flags)
+{
+ struct dl_attr attr = {
+ .size = sizeof(attr),
+ .sched_policy = SCHED_DEADLINE,
+ .sched_flags = flags,
+ .sched_runtime = runtime,
+ .sched_deadline = period,
+ .sched_period = period,
+ };
+
+ return syscall(SYS_sched_setattr, tid, &attr, 0) ? -errno : 0;
+}
+
+static int set_deadline(pid_t tid, uint64_t runtime, uint64_t period)
+{
+ return set_deadline_flags(tid, runtime, period, 0);
+}
+
+/* Make @tid a deadline task with a relative deadline shorter than @period. */
+static int set_constrained(pid_t tid, uint64_t runtime, uint64_t deadline, uint64_t period)
+{
+ struct dl_attr attr = {
+ .size = sizeof(attr),
+ .sched_policy = SCHED_DEADLINE,
+ .sched_runtime = runtime,
+ .sched_deadline = deadline,
+ .sched_period = period,
+ };
+
+ return syscall(SYS_sched_setattr, tid, &attr, 0) ? -errno : 0;
+}
+
+static int set_other(pid_t tid)
+{
+ struct dl_attr attr = { .size = sizeof(attr), .sched_policy = SCHED_OTHER };
+
+ return syscall(SYS_sched_setattr, tid, &attr, 0) ? -errno : 0;
+}
+
+static int write_file(const char *dir, const char *name, const char *val)
+{
+ char path[PATH_MAX];
+ int fd, err = 0;
+ ssize_t n;
+
+ if (snprintf(path, sizeof(path), "%s/%s", dir, name) >= (int)sizeof(path))
+ return -ENAMETOOLONG;
+ fd = open(path, O_WRONLY);
+ if (fd < 0)
+ return -errno;
+ n = write(fd, val, strlen(val));
+ if (n < 0)
+ err = -errno;
+ else if (n != (ssize_t)strlen(val))
+ err = -EIO;
+ if (close(fd) && !err)
+ err = -errno;
+ return err;
+}
+
+static int read_file(const char *path, char *buf, size_t len)
+{
+ int fd = open(path, O_RDONLY);
+ ssize_t n;
+
+ if (fd < 0)
+ return -1;
+ do {
+ n = read(fd, buf, len - 1);
+ } while (n < 0 && errno == EINTR);
+ close(fd);
+ if (n < 0)
+ return -1;
+ buf[n] = 0;
+ return 0;
+}
+
+static int enter(const char *cg, pid_t pid)
+{
+ char buf[16];
+
+ snprintf(buf, sizeof(buf), "%d", pid);
+ return write_file(cg, "cgroup.procs", buf);
+}
+
+/* Make @cg a partition root of the CPUs in @cpus. Return 0, or -1. */
+static int make_partition_cpus(const char *cg, const char *cpus)
+{
+ char path[PATH_MAX + 32], buf[32];
+
+ if (mkdir(cg, 0755) && errno != EEXIST)
+ return -1;
+ if (write_file(cg, "cpuset.cpus", cpus) ||
+ write_file(cg, "cpuset.cpus.partition", "root"))
+ return -1;
+ snprintf(path, sizeof(path), "%s/cpuset.cpus.partition", cg);
+ if (read_file(path, buf, sizeof(buf)) || strcmp(buf, "root\n"))
+ return -1;
+ return 0;
+}
+
+/* Make @cg a partition root of CPU @cpu. Return 0, or -1. */
+static int make_partition(const char *cg, int cpu)
+{
+ char buf[32];
+
+ snprintf(buf, sizeof(buf), "%d", cpu);
+ return make_partition_cpus(cg, buf);
+}
+
+static void remove_partition(const char *cg)
+{
+ int err = write_file(cg, "cpuset.cpus.partition", "member");
+
+ if (err && err != -ENOENT)
+ ksft_print_msg("%s back to a member: %s\n", cg, strerror(-err));
+ /* A cgroup whose last task just died takes a moment to empty. */
+ if (wait_for(1000, !rmdir(cg) || errno == ENOENT))
+ ksft_print_msg("rmdir %s: %s\n", cg, strerror(errno));
+}
+
+/* Make P a partition root of the last online CPU. Return 0, or -1. */
+static int make_p(void)
+{
+ return make_partition(cg_p, top_cpu(1));
+}
+
+static void remove_p(void)
+{
+ remove_partition(cg_p);
+}
+
+/*
+ * Return field @n of /proc/@pid/stat, or @fail. Fields are numbered as
+ * in proc(5), and @n must be 3 or more: a field after the command name.
+ */
+static int stat_field(pid_t pid, int n, int fail)
+{
+ char path[64], buf[1024], *p;
+ int i;
+
+ snprintf(path, sizeof(path), "/proc/%d/stat", pid);
+ if (read_file(path, buf, sizeof(buf)))
+ return fail;
+ p = strrchr(buf, ')');
+ for (i = 0; p && i < n - 2; i++)
+ p = strchr(p + 1, ' ');
+ return p ? atoi(p + 1) : fail;
+}
+
+/* Return the CPU @pid last ran on, from /proc, or -1. */
+static int task_cpu_of(pid_t pid)
+{
+ return stat_field(pid, 39, -1);
+}
+
+/* The priority /proc shows for a deadline task, or a task boosted by one. */
+#define DL_TASK_PRIO (-101)
+
+/* Return the priority @pid runs at, from /proc, or 0. */
+static int prio_of(pid_t pid)
+{
+ return stat_field(pid, 18, 0);
+}
+
+/* Return the CPU time @t has used, or 0 if it cannot be read. */
+static uint64_t cpu_time_of(pthread_t t)
+{
+ static int told;
+ struct timespec ts;
+ clockid_t c;
+ int err;
+
+ err = pthread_getcpuclockid(t, &c);
+ if (!err && clock_gettime(c, &ts))
+ err = errno;
+ if (err) {
+ if (!told++)
+ ksft_print_msg("the CPU time of a thread cannot be read: %s\n",
+ strerror(err));
+ return 0;
+ }
+ return ts.tv_sec * NSEC_PER_SEC + ts.tv_nsec;
+}
+
+/* Return the state letter of @pid in /proc, or 0. */
+static char proc_state_of(pid_t pid)
+{
+ char path[64], buf[1024], *p;
+
+ snprintf(path, sizeof(path), "/proc/%d/stat", pid);
+ if (read_file(path, buf, sizeof(buf)))
+ return 0;
+ p = strrchr(buf, ')');
+ if (!p || p[1] != ' ')
+ return 0;
+ return p[2];
+}
+
+/*
+ * Wait a period. Then return the CPU share, in percent, that @t uses
+ * over the next @ns.
+ */
+static int share_of(pthread_t t, uint64_t ns)
+{
+ uint64_t cpu, wall;
+
+ sleep_ns(PERIOD); /* a period of its own to settle in */
+ cpu = cpu_time_of(t);
+ wall = now_ns();
+ sleep_ns(ns);
+ cpu = cpu_time_of(t) - cpu;
+ wall = now_ns() - wall;
+ return wall ? (int)(100 * cpu / wall) : -1;
+}
+
+struct probe_slot {
+ pthread_t t;
+ int err;
+};
+
+static sem_t probe_done;
+static int probe_hold[2];
+
+static void *probe_thread(void *arg)
+{
+ struct probe_slot *s = arg;
+ char c;
+
+ s->err = set_deadline(0, PROBE_RUNTIME, PROBE_PERIOD);
+ if (sem_post(&probe_done))
+ ksft_print_msg("sem_post: %s\n", strerror(errno));
+ /* Sleep until the probe closes the write end. */
+ while (!s->err) {
+ ssize_t r = read(probe_hold[0], &c, 1);
+
+ if (r >= 0)
+ break;
+ if (errno != EINTR) {
+ s->err = -errno;
+ break;
+ }
+ }
+ return NULL;
+}
+
+/*
+ * Count the deadline tasks of PROBE_RUNTIME every PROBE_PERIOD that the
+ * root domain of @cg admits. A child process joins @cg, or stays in the
+ * caller's cgroup if @cg is NULL. Its threads become deadline tasks one
+ * at a time, until one is refused. Then they all exit. Return the number
+ * admitted, or -1 on an error.
+ *
+ * The threads barely run, so their bandwidth is released when they exit.
+ * probe() still waits three periods for their 0-lag times.
+ */
+static int probe(const char *cg)
+{
+ int fd[2], n = -1, max = 16 * ncpus + 16;
+ pid_t pid;
+
+ if (pipe(fd)) {
+ ksft_print_msg("pipe: %s\n", strerror(errno));
+ return -1;
+ }
+ pid = fork_child("the admission check");
+ if (!pid) {
+ struct probe_slot *s = calloc(max, sizeof(*s));
+ pthread_attr_t a;
+ int i, k = 0;
+
+ close(fd[0]);
+ if (!s || sem_init(&probe_done, 0, 0) || pipe(probe_hold) ||
+ (cg && enter(cg, getpid())))
+ _exit(1);
+ if (pthread_attr_init(&a) || pthread_attr_setstacksize(&a, THREAD_STACK))
+ _exit(1);
+ for (i = 0; i < max; i++) {
+ if (pthread_create(&s[i].t, &a, probe_thread, &s[i])) {
+ k = -1; /* an error, not a refusal */
+ break;
+ }
+ if (sem_wait_for(&probe_done, 10000)) {
+ k = -1;
+ i++; /* join that thread too */
+ break;
+ }
+ if (s[i].err) {
+ if (s[i].err != -EBUSY)
+ k = -1;
+ join(s[i].t);
+ break;
+ }
+ if (k >= 0)
+ k++;
+ }
+ if (i == max)
+ k = -1;
+ close(probe_hold[1]);
+ while (--i >= 0)
+ join(s[i].t);
+ if (write(fd[1], &k, sizeof(k)) != sizeof(k))
+ _exit(1);
+ _exit(0);
+ }
+ close(fd[1]);
+ if (pid < 0 || read(fd[0], &n, sizeof(n)) != sizeof(n))
+ n = -1;
+ close(fd[0]);
+ reap(pid);
+ sleep_ns(3 * PROBE_PERIOD);
+ return n;
+}
+
+/*
+ * Run @fn in a child process in @cg. Every thread of the case is then in
+ * @cg, and dies with the child. Return KSFT_PASS, KSFT_FAIL or KSFT_SKIP.
+ */
+static int run_in(const char *cg, int (*fn)(void))
+{
+ pid_t pid = fork_child("a case");
+ int status;
+
+ if (!pid) {
+ int err = cg ? enter(cg, getpid()) : 0;
+
+ if (err) {
+ ksft_print_msg("the case cannot enter %s: %s\n", cg, strerror(-err));
+ _exit(KSFT_SKIP);
+ }
+ _exit(fn());
+ }
+ if (pid < 0)
+ return KSFT_FAIL;
+ while (waitpid(pid, &status, 0) != pid) {
+ if (errno == EINTR)
+ continue;
+ ksft_print_msg("waitpid %d: %s\n", pid, strerror(errno));
+ return KSFT_FAIL;
+ }
+ if (!WIFEXITED(status)) {
+ ksft_print_msg("the case died of signal %d\n",
+ WIFSIGNALED(status) ? WTERMSIG(status) : 0);
+ return KSFT_FAIL;
+ }
+ return WEXITSTATUS(status);
+}
+
+/*
+ * Run @fn in @cg with run_in(), and probe the root domain of @cg before
+ * and after. Return KSFT_SKIP if the first probe admits no task. Return
+ * KSFT_FAIL if a probe fails, or if @fn passed and the two probes differ.
+ * Otherwise return the result of @fn.
+ */
+static int measured(const char *name, const char *cg, int (*fn)(void))
+{
+ int before, after, ret;
+
+ before = probe(cg);
+ if (before <= 0) {
+ ksft_print_msg("%s: %s\n", name, before ? "the probe failed" : "no room to probe");
+ return before ? KSFT_FAIL : KSFT_SKIP;
+ }
+ ret = run_in(cg, fn);
+ sleep_ns(3 * PERIOD);
+ after = probe(cg);
+ if (after < 0) {
+ ksft_print_msg("%s: the probe failed after the case\n", name);
+ ret = KSFT_FAIL;
+ } else if (ret == KSFT_PASS && after != before) {
+ ksft_print_msg("%s: %d probe tasks admitted before, %d after: %s\n",
+ name, before, after,
+ after < before ? "bandwidth leaked" : "bandwidth released twice");
+ ret = KSFT_FAIL;
+ }
+ return ret;
+}
+
+/* Priority inheritance */
+
+static pthread_mutex_t pi_lock;
+static sem_t owner_ready, owner_go;
+static atomic_int owner_holds, owner_tid;
+
+/*
+ * Set up pi_lock, a priority inheritance mutex, and the owner's
+ * semaphores. Return 0, or -1 if they cannot be set up.
+ */
+static int pi_init(void)
+{
+ pthread_mutexattr_t a;
+ int err;
+
+ err = pthread_mutexattr_init(&a);
+ if (!err) {
+ err = pthread_mutexattr_setprotocol(&a, PTHREAD_PRIO_INHERIT);
+ if (!err)
+ err = pthread_mutex_init(&pi_lock, &a);
+ pthread_mutexattr_destroy(&a);
+ }
+ if (err) {
+ ksft_print_msg("no priority inheritance mutex: %s\n", strerror(err));
+ return -1;
+ }
+ if (sem_init(&owner_ready, 0, 0) || sem_init(&owner_go, 0, 0)) {
+ ksft_print_msg("sem_init: %s\n", strerror(errno));
+ return -1;
+ }
+ owner_holds = 0;
+ return 0;
+}
+
+/* Undo pi_init(). The owner and the waiters must be gone. */
+static void pi_fini(void)
+{
+ pthread_mutex_destroy(&pi_lock);
+ sem_destroy(&owner_ready);
+ sem_destroy(&owner_go);
+}
+
+/*
+ * Make the calling thread a deadline task, and wait up to @ms for
+ * pi_lock. Then switch back to SCHED_OTHER. Return 0 if the wait ends as
+ * @timeout expects: a timeout if it is set, the lock if not. Return -1
+ * otherwise.
+ */
+static int wait_lock_for(int ms, int timeout)
+{
+ struct timespec ts;
+ int err;
+
+ err = set_deadline(0, 5 * NSEC_PER_MSEC, 50 * NSEC_PER_MSEC);
+ if (err) {
+ ksft_print_msg("a waiter refused as a deadline task: %s\n", strerror(-err));
+ return -1;
+ }
+ clock_gettime(CLOCK_REALTIME, &ts);
+ ts.tv_nsec += (long)ms * 1000000L;
+ ts.tv_sec += ts.tv_nsec / 1000000000L;
+ ts.tv_nsec %= 1000000000L;
+ err = pthread_mutex_timedlock(&pi_lock, &ts);
+ if (!err) {
+ int e = pthread_mutex_unlock(&pi_lock);
+
+ if (e)
+ ksft_print_msg("a waiter cannot unlock: %s\n", strerror(e));
+ }
+ if (set_other(0)) {
+ ksft_print_msg("a waiter cannot leave SCHED_DEADLINE\n");
+ return -1;
+ }
+ if (err != (timeout ? ETIMEDOUT : 0)) {
+ ksft_print_msg("a waiter of the lock: %s, not %s\n", err ? strerror(err) : "got it",
+ timeout ? "a timeout" : "the lock");
+ return -1;
+ }
+ return 0;
+}
+
+/*
+ * Take pi_lock as the owner, and return 0. On a failure, report it, set
+ * owner_holds to -1 and return -1.
+ */
+static int take_pi_lock(void)
+{
+ int err = pthread_mutex_lock(&pi_lock);
+
+ if (err) {
+ ksft_print_msg("an owner cannot take the lock: %s\n", strerror(err));
+ owner_holds = -1;
+ }
+ return err ? -1 : 0;
+}
+
+static void drop_pi_lock(void)
+{
+ int err = pthread_mutex_unlock(&pi_lock);
+
+ if (err)
+ ksft_print_msg("an owner cannot unlock: %s\n", strerror(err));
+}
+
+/*
+ * A chain of two priority inheritance mutexes, M1 and M2. The comment of
+ * pi_chain_two_owners() describes the threads D1, N1 and N2. chain_tid[]
+ * holds their tids, in that order. chain_step says how far the case is:
+ * - 1: N2 holds M2. N2 sets it.
+ * - 2: the main thread has checked the boost of N1. N1 may now block on
+ * M2.
+ * - 3: the main thread is done. N2 may now unlock M2.
+ */
+static pthread_mutex_t chain_m1, chain_m2;
+static atomic_int chain_tid[3], chain_step, chain_err;
+
+/*
+ * Wait up to 2s for chain_step to reach @step. Return 0 once it does, or
+ * once a thread of the chain has set chain_err. Return -1 on a timeout.
+ */
+static int chain_wait(int step)
+{
+ return wait_for(2000, chain_step >= step || chain_err);
+}
+
+/* N2: take M2, and hold it until step 3. */
+static void *chain_n2(void *arg)
+{
+ chain_tid[2] = gettid_();
+ if (pthread_mutex_lock(&chain_m2)) {
+ chain_err = 1;
+ return arg;
+ }
+ chain_step = 1;
+ if (chain_wait(3))
+ chain_err = 1;
+ pthread_mutex_unlock(&chain_m2);
+ return arg;
+}
+
+/* N1: take M1. At step 2, block on M2, which N2 holds. */
+static void *chain_n1(void *arg)
+{
+ if (pthread_mutex_lock(&chain_m1)) {
+ chain_err = 1;
+ return arg;
+ }
+ chain_tid[1] = gettid_(); /* N1 holds M1 */
+ if (chain_wait(2) || pthread_mutex_lock(&chain_m2)) {
+ chain_err = 1;
+ } else {
+ pthread_mutex_unlock(&chain_m2);
+ }
+ pthread_mutex_unlock(&chain_m1);
+ return arg;
+}
+
+/* D1: become a deadline task and block on M1, which N1 holds. */
+static void *chain_d1(void *arg)
+{
+ struct timespec ts;
+
+ chain_tid[0] = gettid_();
+ if (set_deadline(0, 5 * NSEC_PER_MSEC, 50 * NSEC_PER_MSEC)) {
+ chain_err = 1;
+ return arg;
+ }
+ clock_gettime(CLOCK_REALTIME, &ts);
+ ts.tv_sec += 5;
+ if (pthread_mutex_timedlock(&chain_m1, &ts))
+ chain_err = 1;
+ else
+ pthread_mutex_unlock(&chain_m1);
+ if (set_other(0))
+ chain_err = 1;
+ return arg;
+}
+
+/*
+ * Boost two SCHED_OTHER tasks through a chain of two mutexes:
+ *
+ * D1 (deadline) waits for M1, held by N1
+ * N1 (boosted) waits for M2, held by N2
+ *
+ * N1 is boosted when D1 blocks. N2 is boosted when N1 then blocks: the
+ * boost goes through N1, which has no deadline parameters of its own.
+ * Both must be in the deadline class then. Once N2 unlocks, the three
+ * must finish.
+ */
+static int pi_chain_two_owners(void)
+{
+ pthread_mutexattr_t a;
+ int ret = KSFT_PASS, i;
+ pthread_t t[3];
+
+ if (pthread_mutexattr_init(&a) ||
+ pthread_mutexattr_setprotocol(&a, PTHREAD_PRIO_INHERIT) ||
+ pthread_mutex_init(&chain_m1, &a) || pthread_mutex_init(&chain_m2, &a)) {
+ ksft_print_msg("no priority inheritance mutex\n");
+ return KSFT_SKIP;
+ }
+ chain_step = 0;
+ chain_err = 0;
+ for (i = 0; i < 3; i++)
+ chain_tid[i] = 0;
+ if (pthread_create(&t[2], NULL, chain_n2, NULL) || chain_wait(1) ||
+ pthread_create(&t[1], NULL, chain_n1, NULL) ||
+ wait_for(2000, chain_tid[1]) ||
+ pthread_create(&t[0], NULL, chain_d1, NULL)) {
+ ksft_print_msg("the threads of the chain could not start\n");
+ return KSFT_FAIL;
+ }
+ if (wait_for(2000, prio_of(chain_tid[1]) == DL_TASK_PRIO || chain_err)) {
+ ksft_print_msg("N1 is not boosted by D1\n");
+ ret = KSFT_FAIL;
+ }
+ chain_step = 2;
+ if (ret == KSFT_PASS &&
+ wait_for(2000, prio_of(chain_tid[2]) == DL_TASK_PRIO || chain_err)) {
+ ksft_print_msg("N2 is not boosted through N1\n");
+ ret = KSFT_FAIL;
+ }
+ chain_step = 3;
+ for (i = 0; i < 3; i++)
+ join(t[i]);
+ if (chain_err) {
+ ksft_print_msg("a thread of the chain failed\n");
+ ret = KSFT_FAIL;
+ }
+ pthread_mutex_destroy(&chain_m1);
+ pthread_mutex_destroy(&chain_m2);
+ return ret;
+}
+
+/*
+ * Run as the owner thread. With @arg (was_dl) set, first become a
+ * deadline task of 30ms every 100ms, run 1ms, and sleep on owner_go. The
+ * main thread switches the owner to SCHED_OTHER during that sleep. Then
+ * take pi_lock, and sleep 400ms while holding it. Return NULL, or 1 on an
+ * error.
+ */
+static void *sleeping_owner(void *arg)
+{
+ int was_dl = (long)arg;
+
+ owner_tid = gettid_();
+ if (was_dl) {
+ if (set_deadline(0, 30 * NSEC_PER_MSEC, PERIOD)) {
+ owner_holds = -1;
+ sem_post(&owner_ready);
+ return (void *)1;
+ }
+ spin_ns(NSEC_PER_MSEC);
+ /*
+ * Sleep. The main thread switches this thread to SCHED_OTHER
+ * 200ms later, past its 0-lag time.
+ */
+ sem_post(&owner_ready);
+ sem_wait_sig(&owner_go);
+ }
+ if (take_pi_lock())
+ return (void *)1;
+ owner_holds = 2;
+ sleep_ns(400 * NSEC_PER_MSEC);
+ drop_pi_lock();
+ return NULL;
+}
+
+/*
+ * A SCHED_OTHER owner takes pi_lock and sleeps 400ms while holding it.
+ * The main thread then waits 20ms for the lock, @rounds times. For each
+ * wait it becomes a deadline task of 5ms every 50ms. Each wait boosts the
+ * sleeping owner, and its timeout deboosts it. Every wait must time out.
+ *
+ * With @was_dl, the owner was a deadline task of 30ms every 100ms before.
+ * It ran 1ms and slept. The main thread switched it to SCHED_OTHER 200ms
+ * later, past its 0-lag time.
+ */
+static int pi_asleep(int was_dl, int rounds)
+{
+ pthread_t o;
+ int r, ret = KSFT_PASS;
+
+ if (pi_init())
+ return KSFT_SKIP;
+ if (pthread_create(&o, NULL, sleeping_owner, (void *)(long)was_dl)) {
+ ksft_print_msg("the owner thread could not start\n");
+ return KSFT_FAIL;
+ }
+ if (was_dl) {
+ if (sem_wait_for(&owner_ready, 5000)) {
+ ksft_print_msg("the owner did not become a deadline task\n");
+ ret = KSFT_FAIL;
+ } else {
+ sleep_ns(200 * NSEC_PER_MSEC);
+ if (set_other(owner_tid)) {
+ ksft_print_msg("the owner cannot leave SCHED_DEADLINE\n");
+ ret = KSFT_FAIL;
+ }
+ }
+ sem_post(&owner_go);
+ }
+ if (ret == KSFT_PASS && (wait_for(5000, owner_holds) || owner_holds != 2)) {
+ ksft_print_msg("the owner never took the lock\n");
+ ret = KSFT_FAIL;
+ }
+ if (ret == KSFT_PASS)
+ sleep_ns(20 * NSEC_PER_MSEC);
+ for (r = 0; r < rounds && ret == KSFT_PASS; r++) {
+ if (wait_lock_for(20, 1)) {
+ ksft_print_msg("round %d of %d failed\n", r + 1, rounds);
+ ret = KSFT_FAIL;
+ }
+ }
+ join(o);
+ return ret;
+}
+
+static int pi_other_owner_asleep(void)
+{
+ return pi_asleep(0, 5);
+}
+
+/*
+ * Run as the owner thread. Become a deadline task of 30ms every 100ms,
+ * and take pi_lock. Hold it for 300ms, spinning 2ms and sleeping 5ms in
+ * turn. A waiter boosts the owner when its deadline is the earlier one.
+ */
+static void *dl_owner(void *arg)
+{
+ uint64_t end;
+
+ if (set_deadline(0, 30 * NSEC_PER_MSEC, PERIOD)) {
+ owner_holds = -1;
+ return (void *)1;
+ }
+ if (take_pi_lock())
+ return (void *)1;
+ owner_holds = 2;
+ end = now_ns() + 300 * NSEC_PER_MSEC;
+ while (now_ns() < end) {
+ spin_ns(2 * NSEC_PER_MSEC);
+ sleep_ns(5 * NSEC_PER_MSEC);
+ }
+ drop_pi_lock();
+ return NULL;
+}
+
+/*
+ * A deadline owner of 30ms every 100ms holds pi_lock for 300ms. The main
+ * thread waits 20ms for the lock, five times. For each wait it becomes a
+ * deadline task of 5ms every 50ms. Every wait must time out, and the
+ * owner must end without an error.
+ */
+static int pi_deadline_owner(void)
+{
+ int r, ret = KSFT_PASS, err;
+ pthread_t o;
+ void *res;
+
+ if (pi_init())
+ return KSFT_SKIP;
+ if (pthread_create(&o, NULL, dl_owner, NULL)) {
+ ksft_print_msg("the owner thread could not start\n");
+ return KSFT_FAIL;
+ }
+ if (wait_for(5000, owner_holds) || owner_holds != 2) {
+ ksft_print_msg("the deadline owner never took the lock\n");
+ ret = KSFT_FAIL;
+ }
+ for (r = 0; r < 5 && ret == KSFT_PASS; r++)
+ if (wait_lock_for(20, 1))
+ ret = KSFT_FAIL;
+ err = pthread_join(o, &res);
+ if (err)
+ ksft_print_msg("pthread_join: %s\n", strerror(err));
+ return res || err ? KSFT_FAIL : ret;
+}
+
+/*
+ * The owner thread: take pi_lock and spin 50ms with it. The unlock
+ * deboosts it while it runs.
+ */
+static void *running_owner(void *arg)
+{
+ if (take_pi_lock())
+ return (void *)1;
+ owner_holds = 2;
+ spin_ns(50 * NSEC_PER_MSEC);
+ drop_pi_lock();
+ return NULL;
+}
+
+/*
+ * A SCHED_OTHER owner takes pi_lock and spins 50ms. The main thread waits
+ * for the lock as a deadline task of 5ms every 50ms, which boosts the
+ * running owner. The owner's unlock deboosts it while it runs. The main
+ * thread must get the lock within 1s. Do this five times.
+ */
+static int pi_unlock_running(void)
+{
+ pthread_t o;
+ int r, ret = KSFT_PASS;
+
+ for (r = 0; r < 5 && ret == KSFT_PASS; r++) {
+ if (pi_init())
+ return KSFT_SKIP;
+ if (pthread_create(&o, NULL, running_owner, NULL)) {
+ ksft_print_msg("the owner thread could not start\n");
+ return KSFT_FAIL;
+ }
+ if (wait_for_step(5000, 100 * NSEC_PER_USEC, owner_holds) ||
+ owner_holds != 2) {
+ ksft_print_msg("the owner never took the lock\n");
+ ret = KSFT_FAIL;
+ } else if (wait_lock_for(1000, 0)) {
+ ret = KSFT_FAIL;
+ }
+ join(o);
+ pi_fini();
+ }
+ return ret;
+}
+
+/* Parameter changes */
+
+static atomic_int t_state; /* 1: about to sleep, or spinning */
+static _Atomic pid_t t_tid;
+static sem_t t_exit;
+
+/*
+ * The sleeper thread: become a deadline task of 20ms every 100ms. Run
+ * @arg ns of a fresh period, then sleep until t_exit is posted.
+ */
+static void *runs_then_sleeps(void *arg)
+{
+ uint64_t run = (uint64_t)(long)arg;
+
+ t_tid = gettid_();
+ if (set_deadline(0, 20 * NSEC_PER_MSEC, PERIOD)) {
+ t_state = -1;
+ return (void *)1;
+ }
+ sleep_ns(PERIOD); /* a fresh period */
+ spin_ns(run);
+ t_state = 1;
+ sem_wait_sig(&t_exit);
+ return NULL;
+}
+
+/*
+ * Start runs_then_sleeps() in @t, to run @run ns. Return 0 once it is
+ * about to sleep, or -1.
+ */
+static int start_sleeper(pthread_t *t, uint64_t run)
+{
+ t_state = 0;
+ if (sem_init(&t_exit, 0, 0)) {
+ ksft_print_msg("sem_init: %s\n", strerror(errno));
+ return -1;
+ }
+ if (pthread_create(t, NULL, runs_then_sleeps, (void *)(long)run)) {
+ ksft_print_msg("the sleeper thread could not start\n");
+ return -1;
+ }
+ if (wait_for_step(5000, 100 * NSEC_PER_USEC, t_state) || t_state != 1) {
+ ksft_print_msg("the sleeper was refused SCHED_DEADLINE\n");
+ sem_post(&t_exit); /* in case it gets there late */
+ join(*t);
+ return -1;
+ }
+ return 0;
+}
+
+static void stop_sleeper(pthread_t t)
+{
+ if (sem_post(&t_exit))
+ ksft_print_msg("sem_post: %s\n", strerror(errno));
+ join(t);
+}
+
+/* Return 0 if the root domain of @cg admits @want probe tasks now, or -1. */
+static int admits_in(const char *cg, int want, const char *what)
+{
+ int n = probe(cg);
+
+ if (n != want) {
+ ksft_print_msg("%s: %d probe tasks admitted, %d expected\n", what, n, want);
+ return -1;
+ }
+ return 0;
+}
+
+/* Do the same check in the root domain of the caller's cgroup. */
+static int admits(int want, const char *what)
+{
+ return admits_in(NULL, want, what);
+}
+
+/*
+ * A deadline task of 20ms every 100ms sleeps in P, past its 0-lag time.
+ * The main thread changes its runtime to 40ms, 10ms, 30ms and 20ms. After
+ * each change, P must admit one probe task fewer for each 10ms of the new
+ * runtime.
+ */
+static int params_asleep(void)
+{
+ static const uint64_t rt[] = { 40, 10, 30, 20 };
+ int base, i, ret = KSFT_PASS;
+ pthread_t t;
+
+ if (start_sleeper(&t, NSEC_PER_MSEC))
+ return KSFT_FAIL;
+ sleep_ns(2 * PERIOD);
+ /*
+ * probe(NULL) probes P: this process is in P. The sleeper holds 20%,
+ * which is two probe tasks. base is what P admits without it.
+ */
+ base = probe(NULL) + 2;
+ for (i = 0; i < 4 && ret == KSFT_PASS; i++) {
+ char what[64];
+
+ if (set_deadline(t_tid, rt[i] * NSEC_PER_MSEC, PERIOD)) {
+ ret = KSFT_FAIL;
+ break;
+ }
+ snprintf(what, sizeof(what), "asleep, changed to %llums every 100ms",
+ (unsigned long long)rt[i]);
+ if (admits(base - (int)(rt[i] / 10), what))
+ ret = KSFT_FAIL;
+ }
+ stop_sleeper(t);
+ return ret;
+}
+
+/*
+ * A deadline task of 20ms every 100ms runs 5ms in a new period, then
+ * sleeps. Its 0-lag time is then about 20ms ahead. 1ms later, the main
+ * thread changes its runtime to 40ms, then to 10ms. measured() checks
+ * that P admits as many probe tasks after the case as before.
+ */
+static int params_non_contending(void)
+{
+ pthread_t t;
+ int ret = KSFT_PASS;
+
+ if (start_sleeper(&t, 5 * NSEC_PER_MSEC))
+ return KSFT_FAIL;
+ sleep_ns(NSEC_PER_MSEC);
+ if (set_deadline(t_tid, 40 * NSEC_PER_MSEC, PERIOD) ||
+ set_deadline(t_tid, 10 * NSEC_PER_MSEC, PERIOD))
+ ret = KSFT_FAIL;
+ stop_sleeper(t);
+ return ret;
+}
+
+static atomic_int spin_stop;
+/* Set before dl_spinner() starts. */
+static uint64_t spin_runtime, spin_flags;
+
+static void *dl_spinner(void *arg)
+{
+ t_tid = gettid_();
+ if (set_deadline_flags(0, spin_runtime, PERIOD, spin_flags)) {
+ t_state = -1;
+ return (void *)1;
+ }
+ t_state = 1;
+ while (!spin_stop)
+ ;
+ return NULL;
+}
+
+/*
+ * Start dl_spinner() in @t, a deadline task of @runtime every 100ms with
+ * @flags. Return 0, or -1.
+ */
+static int start_spinner(pthread_t *t, uint64_t runtime, uint64_t flags)
+{
+ t_state = 0;
+ spin_stop = 0;
+ spin_runtime = runtime;
+ spin_flags = flags;
+ if (pthread_create(t, NULL, dl_spinner, NULL)) {
+ ksft_print_msg("the spinner thread could not start\n");
+ return -1;
+ }
+ if (!wait_for(5000, t_state) && t_state == 1)
+ return 0;
+ spin_stop = 1; /* in case it gets there late */
+ join(*t);
+ return -1;
+}
+
+static void stop_spinner(pthread_t t)
+{
+ spin_stop = 1;
+ /* The thread may have left the class already. That is fine. */
+ set_other(t_tid);
+ join(t);
+}
+
+/*
+ * Wait until @t stops using CPU time: it is throttled. Return 0, or -1
+ * if that takes over 1s or its CPU time cannot be read.
+ */
+static int wait_throttled(pthread_t t)
+{
+ uint64_t end = now_ns() + NSEC_PER_SEC, last = cpu_time_of(t);
+
+ while (last && now_ns() < end) {
+ uint64_t cur;
+
+ sleep_ns(3 * NSEC_PER_MSEC);
+ cur = cpu_time_of(t);
+ if (cur == last)
+ return 0;
+ last = cur;
+ }
+ return -1;
+}
+
+/* Return 0 if @t runs 1ms more within @ns, or -1. */
+static int progresses(pthread_t t, uint64_t ns)
+{
+ uint64_t start = cpu_time_of(t), end = now_ns() + ns;
+
+ while (now_ns() < end) {
+ if (cpu_time_of(t) > start + NSEC_PER_MSEC)
+ return 0;
+ sleep_ns(2 * NSEC_PER_MSEC);
+ }
+ return -1;
+}
+
+/*
+ * A deadline task of 10ms every 100ms spins. Five times, wait until it is
+ * throttled, then change its runtime from the main thread: to 20ms, then
+ * back to 10ms, in turn. After each change it must run 1ms more within
+ * three periods.
+ */
+static int params_throttled(void)
+{
+ int r, ret = KSFT_PASS;
+ pthread_t t;
+
+ if (start_spinner(&t, 10 * NSEC_PER_MSEC, 0))
+ return KSFT_FAIL;
+ for (r = 0; r < 5 && ret == KSFT_PASS; r++) {
+ if (wait_throttled(t) ||
+ set_deadline(t_tid, (r & 1 ? 10 : 20) * NSEC_PER_MSEC, PERIOD)) {
+ ret = KSFT_FAIL;
+ break;
+ }
+ if (progresses(t, 3 * PERIOD)) {
+ ksft_print_msg("throttled, parameters changed: never ran again\n");
+ ret = KSFT_FAIL;
+ }
+ }
+ stop_spinner(t);
+ return ret;
+}
+
+/*
+ * A deadline task of 20ms every 100ms sleeps in P. The main thread tries
+ * to raise its runtime by more than P has free. sched_setattr() must fail
+ * with EBUSY, and P must admit as many probe tasks after as before.
+ */
+static int params_refused(void)
+{
+ pthread_t t;
+ int ret = KSFT_PASS, base, err;
+
+ if (start_sleeper(&t, NSEC_PER_MSEC))
+ return KSFT_FAIL;
+ sleep_ns(2 * PERIOD);
+ base = probe(NULL);
+ /*
+ * P has room for base probe tasks of 10%, and not for one more. So
+ * growing the task's runtime by (base + 1) * 10ms must be refused,
+ * whatever the admission limit.
+ */
+ if (20 + 10 * (base + 1) > 100) {
+ stop_sleeper(t);
+ ksft_print_msg("P has more room than one of its tasks can take\n");
+ return KSFT_SKIP;
+ }
+ err = set_deadline(t_tid, (20 + 10 * (uint64_t)(base + 1)) * NSEC_PER_MSEC, PERIOD);
+ if (err != -EBUSY) {
+ ksft_print_msg("an increase beyond P's room: %s, not EBUSY\n",
+ err ? strerror(-err) : "accepted");
+ ret = KSFT_FAIL;
+ } else if (admits(base, "after a refused increase")) {
+ ret = KSFT_FAIL;
+ }
+ stop_sleeper(t);
+ return ret;
+}
+
+/*
+ * A SCHED_OTHER task with a slice of 40ms asks for SCHED_DEADLINE with
+ * SCHED_FLAG_KEEP_PARAMS. That changes the class, so it must be refused
+ * with EINVAL. It must not reserve bandwidth for the parameters it would
+ * keep, which take the slice as their runtime.
+ */
+static int params_keep_class_change(void)
+{
+ struct dl_attr attr = {
+ .size = sizeof(attr),
+ .sched_policy = SCHED_OTHER,
+ .sched_runtime = 40 * NSEC_PER_MSEC,
+ };
+ int err;
+
+ if (syscall(SYS_sched_setattr, 0, &attr, 0)) {
+ ksft_print_msg("a slice of its own is refused: %s\n", strerror(errno));
+ return KSFT_SKIP;
+ }
+ attr.sched_policy = SCHED_DEADLINE;
+ attr.sched_flags = SCHED_FLAG_KEEP_PARAMS;
+ attr.sched_deadline = PERIOD;
+ attr.sched_period = PERIOD;
+ err = syscall(SYS_sched_setattr, 0, &attr, 0) ? -errno : 0;
+ if (err != -EINVAL) {
+ ksft_print_msg("to SCHED_DEADLINE with SCHED_FLAG_KEEP_PARAMS: %s, not EINVAL\n",
+ err ? strerror(-err) : "accepted");
+ return KSFT_FAIL;
+ }
+ return KSFT_PASS;
+}
+
+/*
+ * The waiter thread: wait up to 300ms for pi_lock as a deadline task,
+ * which boosts the owner. Return NULL if the wait timed out.
+ */
+static void *boosting_waiter(void *arg)
+{
+ return (void *)(long)wait_lock_for(300, 1);
+}
+
+/*
+ * Start a boosting_waiter() thread in @w. Wait up to 200ms for it to boost
+ * the owner. Return 0 once the owner is boosted. Otherwise report the
+ * failure, join the waiter and return -1.
+ */
+static int start_boost(pthread_t *w)
+{
+ if (pthread_create(w, NULL, boosting_waiter, NULL)) {
+ ksft_print_msg("the waiter thread could not start\n");
+ return -1;
+ }
+ if (!wait_for(200, prio_of(owner_tid) == DL_TASK_PRIO))
+ return 0;
+ ksft_print_msg("the waiter never boosted the owner\n");
+ join(*w);
+ return -1;
+}
+
+/*
+ * Join the boosting_waiter() thread @w. Return -1 if its wait did not
+ * time out.
+ */
+static int join_waiter(pthread_t w)
+{
+ void *res;
+ int err = pthread_join(w, &res);
+
+ if (err)
+ ksft_print_msg("pthread_join: %s\n", strerror(err));
+ return err || res ? -1 : 0;
+}
+
+/*
+ * A SCHED_OTHER owner holds pi_lock and sleeps. The main thread gives it
+ * a slice of 40ms with sched_setattr(). A deadline waiter then boosts the
+ * owner into the deadline class. The main thread then calls
+ * sched_setattr() on the owner, with SCHED_DEADLINE and
+ * SCHED_FLAG_KEEP_PARAMS.
+ *
+ * The call must fail with EINVAL. The boost put the owner in the deadline
+ * class, but its policy is still SCHED_OTHER. So the call asks for another
+ * policy, which SCHED_FLAG_KEEP_PARAMS cannot give. If the call were
+ * admitted, it would reserve bandwidth for the 40ms slice, and nothing
+ * would release it.
+ */
+static int params_keep_boosted(void)
+{
+ struct dl_attr slice = {
+ .size = sizeof(slice),
+ .sched_policy = SCHED_OTHER,
+ .sched_runtime = 40 * NSEC_PER_MSEC, /* the slice to keep */
+ };
+ struct dl_attr keep = {
+ .size = sizeof(keep),
+ .sched_policy = SCHED_DEADLINE,
+ .sched_flags = SCHED_FLAG_KEEP_PARAMS,
+ .sched_deadline = PERIOD,
+ .sched_period = PERIOD,
+ };
+ int ret = KSFT_PASS, err;
+ pthread_t o, w;
+
+ if (pi_init())
+ return KSFT_SKIP;
+ if (pthread_create(&o, NULL, sleeping_owner, (void *)0L)) {
+ ksft_print_msg("the owner thread could not start\n");
+ return KSFT_FAIL;
+ }
+ if (wait_for(5000, owner_holds) || owner_holds != 2) {
+ ksft_print_msg("the owner never took the lock\n");
+ join(o);
+ return KSFT_FAIL;
+ }
+ if (syscall(SYS_sched_setattr, owner_tid, &slice, 0)) {
+ ksft_print_msg("the owner cannot take a slice of its own: %s\n",
+ strerror(errno));
+ ret = KSFT_SKIP;
+ goto out;
+ }
+ if (start_boost(&w)) {
+ ret = KSFT_FAIL;
+ goto out;
+ }
+ err = syscall(SYS_sched_setattr, owner_tid, &keep, 0) ? -errno : 0;
+ if (err != -EINVAL) {
+ ksft_print_msg("KEEP_PARAMS to SCHED_DEADLINE on a boosted owner: %s, not EINVAL\n",
+ err ? strerror(-err) : "accepted");
+ ret = KSFT_FAIL;
+ }
+ if (join_waiter(w))
+ ret = KSFT_FAIL; /* the waiter said why */
+out:
+ join(o);
+ return ret;
+}
+
+/*
+ * A deadline waiter boosts a sleeping SCHED_OTHER owner. The main thread
+ * then calls sched_setattr() on the owner, with SCHED_OTHER. The call
+ * must succeed while the owner is boosted.
+ */
+static int boosted_owner_policy(void)
+{
+ int ret = KSFT_PASS, err;
+ pthread_t o, w;
+
+ if (pi_init())
+ return KSFT_SKIP;
+ if (pthread_create(&o, NULL, sleeping_owner, (void *)0L)) {
+ ksft_print_msg("the owner thread could not start\n");
+ return KSFT_FAIL;
+ }
+ if (wait_for(5000, owner_holds) || owner_holds != 2) {
+ ksft_print_msg("the owner never took the lock\n");
+ join(o);
+ return KSFT_FAIL;
+ }
+ if (start_boost(&w)) {
+ join(o);
+ return KSFT_FAIL;
+ }
+ err = set_other(owner_tid);
+ if (err) {
+ ksft_print_msg("SCHED_OTHER on a boosted owner: %s\n", strerror(-err));
+ ret = KSFT_FAIL;
+ }
+ if (join_waiter(w))
+ ret = KSFT_FAIL; /* the waiter said why */
+ join(o);
+ return ret;
+}
+
+static atomic_int owner_first_period_ms;
+static _Atomic pid_t donor_tid;
+
+/*
+ * Run as the owner O of deboost_case(). @arg is its @mode. Store the CPU
+ * time of step 4, in ms, in owner_first_period_ms. Return NULL, 1 on an
+ * error, or 2 if no waiter boosted it within 1s.
+ */
+static void *deboosted_owner(void *arg)
+{
+ uint64_t t0, cpu0;
+ int err;
+
+ owner_tid = gettid_();
+ if (arg == (void *)1L) {
+ err = set_deadline(0, 60 * NSEC_PER_MSEC, PERIOD);
+ if (err) {
+ ksft_print_msg("the owner refused as a deadline task: %s\n",
+ strerror(-err));
+ return (void *)1;
+ }
+ spin_ns(NSEC_PER_MSEC);
+ sem_post(&owner_ready);
+ sem_wait_sig(&owner_go);
+ }
+ if (take_pi_lock())
+ return (void *)1;
+ owner_holds = 2;
+ t0 = now_ns();
+ while (prio_of(owner_tid) != DL_TASK_PRIO) {
+ if (now_ns() - t0 > NSEC_PER_SEC) {
+ drop_pi_lock();
+ return (void *)2; /* never boosted */
+ }
+ }
+ if (arg == (void *)2L) {
+ err = set_deadline(0, 10 * NSEC_PER_MSEC, PERIOD);
+ if (err) {
+ ksft_print_msg("the boosted owner refused as a deadline task: %s\n",
+ strerror(-err));
+ drop_pi_lock();
+ return (void *)1;
+ }
+ }
+ spin_ns(5 * NSEC_PER_MSEC);
+ drop_pi_lock();
+ err = arg == (void *)2L ? 0 : set_deadline(0, 10 * NSEC_PER_MSEC, PERIOD);
+ if (err) {
+ ksft_print_msg("the owner refused as a deadline task: %s\n", strerror(-err));
+ return (void *)1;
+ }
+ t0 = now_ns();
+ cpu0 = thread_time_ns();
+ while (now_ns() - t0 < PERIOD)
+ ;
+ owner_first_period_ms = (thread_time_ns() - cpu0) / NSEC_PER_MSEC;
+ return NULL;
+}
+
+/*
+ * The waiter thread: take pi_lock as a deadline task of 40ms every
+ * 100ms. It boosts the owner, which then runs on the waiter's runtime.
+ * Return NULL once it got the lock.
+ */
+static void *donor_waiter(void *arg)
+{
+ int err;
+
+ donor_tid = gettid_();
+ err = set_deadline(0, 40 * NSEC_PER_MSEC, PERIOD);
+ if (err) {
+ ksft_print_msg("a waiter refused as a deadline task: %s\n", strerror(-err));
+ return (void *)1;
+ }
+ err = take_pi_lock();
+ if (!err)
+ drop_pi_lock();
+ if (set_other(0))
+ ksft_print_msg("a waiter cannot leave SCHED_DEADLINE\n");
+ return (void *)(long)err;
+}
+
+/*
+ * Run a deboost case. The owner O starts as SCHED_OTHER. The donor D is
+ * the waiter that boosts it: a deadline task of 40ms every 100ms.
+ * 1. O takes pi_lock and spins.
+ * 2. D blocks on pi_lock. This boosts O, which then runs on D's runtime.
+ * 3. O spins 5ms more and unlocks. The unlock deboosts O.
+ * 4. O becomes a deadline task of 10ms every 100ms. It spins for 100ms
+ * and measures its CPU time.
+ *
+ * O must get about 10ms in step 4. The case fails above 20ms: O then
+ * still runs on the runtime that the boost left.
+ *
+ * @mode changes one step:
+ * - 0: nothing.
+ * - 1: before step 1, O runs 1ms as a deadline task of 60ms every 100ms,
+ * and sleeps. The main thread switches it to SCHED_OTHER 300ms later,
+ * past its 0-lag time.
+ * - 2: O becomes the deadline task of step 4 in step 3 instead, before
+ * the 5ms. It stays in the deadline class at the deboost.
+ */
+static int deboost_case(long mode)
+{
+ int err, ret = KSFT_PASS;
+ void *o_ret, *w_ret;
+ pthread_t o, w;
+
+ if (pi_init())
+ return KSFT_SKIP;
+ owner_first_period_ms = -1;
+ if (pthread_create(&o, NULL, deboosted_owner, (void *)mode)) {
+ ksft_print_msg("the owner thread could not start\n");
+ return KSFT_FAIL;
+ }
+ if (mode == 1) {
+ if (sem_wait_for(&owner_ready, 5000)) {
+ /* The owner may get there late. Let it end. */
+ sem_post(&owner_go);
+ join(o);
+ return KSFT_FAIL; /* the owner said why */
+ }
+ sleep_ns(3 * PERIOD);
+ err = proc_state_of(owner_tid) == 'S' ? set_other(owner_tid) : -EAGAIN;
+ sem_post(&owner_go);
+ if (err) {
+ ksft_print_msg("the owner could not leave the class asleep: %s\n",
+ strerror(-err));
+ join(o);
+ return KSFT_FAIL;
+ }
+ }
+ if (wait_for(5000, owner_holds) || owner_holds != 2) {
+ ksft_print_msg("the owner never took the lock\n");
+ join(o);
+ return KSFT_FAIL;
+ }
+ if (pthread_create(&w, NULL, donor_waiter, NULL)) {
+ ksft_print_msg("the waiter thread could not start\n");
+ join(o);
+ return KSFT_FAIL;
+ }
+ err = pthread_join(w, &w_ret);
+ if (err) {
+ ksft_print_msg("pthread_join: %s\n", strerror(err));
+ ret = KSFT_FAIL;
+ } else if (w_ret) {
+ ret = KSFT_FAIL; /* the waiter said why */
+ }
+ err = pthread_join(o, &o_ret);
+ if (err) {
+ ksft_print_msg("pthread_join: %s\n", strerror(err));
+ ret = KSFT_FAIL;
+ } else if (o_ret == (void *)2) {
+ ksft_print_msg("the waiter never boosted the owner\n");
+ ret = KSFT_FAIL;
+ } else if (owner_first_period_ms < 0) {
+ ret = KSFT_FAIL;
+ } else if (owner_first_period_ms > 20) {
+ ksft_print_msg("the owner ran %d ms of its first 100: the donor's runtime\n",
+ (int)owner_first_period_ms);
+ ret = KSFT_FAIL;
+ }
+ return ret;
+}
+
+/* Run deboost_case() with a plain SCHED_OTHER owner. */
+static int deboost_resets_period(void)
+{
+ return deboost_case(0);
+}
+
+/*
+ * Return the runtime that the deadline task @tid has left, in ns, from
+ * dl.runtime in /proc/<tid>/sched. Return -1 if it cannot be read.
+ */
+static long long dl_runtime_left(pid_t tid)
+{
+ char path[64], line[256];
+ long long left = -1;
+ FILE *f;
+
+ snprintf(path, sizeof(path), "/proc/%d/sched", tid);
+ f = fopen(path, "r");
+ if (!f)
+ return -1;
+ while (fgets(line, sizeof(line), f)) {
+ char *val = strchr(line, ':');
+
+ if (!strncmp(line, "dl.runtime", 10) && val) {
+ left = atoll(val + 1);
+ break;
+ }
+ }
+ fclose(f);
+ return left;
+}
+
+#define REFILL_OWN_RUNTIME (5 * NSEC_PER_MSEC)
+#define REFILL_DONOR_RUNTIME (2 * NSEC_PER_MSEC)
+#define REFILL_DONOR_PERIOD (10 * NSEC_PER_MSEC)
+
+static sem_t refill_donor_ready;
+static atomic_int refill_donor_locking;
+static _Atomic long long refill_owner_left;
+
+/*
+ * Run as the donor D of refill_case(). Become a deadline task of 2ms
+ * every 10ms and sleep until the owner holds pi_lock. Then block on
+ * pi_lock. Return NULL once it got the lock, or 1 on an error.
+ */
+static void *refill_donor(void *arg)
+{
+ int err;
+
+ donor_tid = gettid_();
+ err = set_deadline(0, REFILL_DONOR_RUNTIME, REFILL_DONOR_PERIOD);
+ if (err) {
+ ksft_print_msg("the donor refused as a deadline task: %s\n", strerror(-err));
+ donor_tid = -1;
+ sem_post(&refill_donor_ready);
+ return (void *)1;
+ }
+ sem_post(&refill_donor_ready);
+ sem_wait_sig(&owner_go);
+ refill_donor_locking = 1;
+ err = take_pi_lock();
+ if (!err)
+ drop_pi_lock();
+ if (set_other(0))
+ ksft_print_msg("the donor cannot leave SCHED_DEADLINE\n");
+ return (void *)(long)err;
+}
+
+/*
+ * Run as the owner O of refill_case(). @arg is the CPU time, in ms, that
+ * O spins with the lock once it is boosted. Store the runtime O has left
+ * after the unlock in refill_owner_left. Return NULL, 1 on an error, or 2
+ * if the donor did not block on the lock within 1s.
+ */
+static void *refill_owner(void *arg)
+{
+ uint64_t t0;
+ int err;
+
+ owner_tid = gettid_();
+ err = set_deadline(0, REFILL_OWN_RUNTIME, PERIOD);
+ if (err) {
+ ksft_print_msg("the owner refused as a deadline task: %s\n", strerror(-err));
+ return (void *)1;
+ }
+ spin_ns(2 * NSEC_PER_MSEC);
+ if (take_pi_lock())
+ return (void *)1;
+ owner_holds = 2;
+ sem_post(&owner_go);
+ t0 = now_ns();
+ while (!refill_donor_locking || proc_state_of(donor_tid) != 'S') {
+ if (now_ns() - t0 > NSEC_PER_SEC) {
+ drop_pi_lock();
+ set_other(0);
+ return (void *)2;
+ }
+ }
+ spin_ns((long)arg * NSEC_PER_MSEC);
+ drop_pi_lock();
+ refill_owner_left = dl_runtime_left(owner_tid);
+ if (set_other(0))
+ ksft_print_msg("the owner cannot leave SCHED_DEADLINE\n");
+ return NULL;
+}
+
+/*
+ * Run a case on the runtime that a deadline owner has after its boost.
+ * O and D share the CPU of P.
+ *
+ * owner O: 5ms every 100ms donor D: 2ms every 10ms
+ *
+ * sleeps
+ * starts its period, runs 2ms
+ * takes pi_lock
+ * wakes D up
+ * preempts O: its deadline is
+ * earlier
+ * blocks on pi_lock, boosts O
+ * runs @spin_ms more
+ * unlocks: the boost ends
+ * takes pi_lock, unlocks
+ * reads its dl.runtime
+ *
+ * A boosted task that runs out of runtime is not throttled. Its runtime
+ * is refilled from the parameters of its donor: 2ms here.
+ *
+ * - @spin_ms is 8: O runs out of its own 5ms, and is refilled from D at
+ * least once. At the unlock it holds what the last refill left, 2ms at
+ * most. That is not the runtime of its own reservation. O must start a
+ * new period at the unlock, with 5ms. The case fails if O has 3.5ms or
+ * less.
+ * - @spin_ms is 1: O has used about 3ms of its own 5ms at the unlock, and
+ * was never refilled. It must keep what it has left. A new period would
+ * hand it a full runtime at every contended unlock. The case fails if O
+ * has 4ms or more.
+ */
+static int refill_case(long spin_ms)
+{
+ void *o_ret = NULL, *d_ret = NULL;
+ int err, ret = KSFT_PASS;
+ pthread_t o, d;
+ long long left;
+
+ if (pi_init())
+ return KSFT_SKIP;
+ if (sem_init(&refill_donor_ready, 0, 0)) {
+ ksft_print_msg("sem_init: %s\n", strerror(errno));
+ pi_fini();
+ return KSFT_SKIP;
+ }
+ refill_donor_locking = 0;
+ refill_owner_left = -1;
+ donor_tid = 0;
+ if (pthread_create(&d, NULL, refill_donor, NULL)) {
+ ksft_print_msg("the donor thread could not start\n");
+ return KSFT_FAIL;
+ }
+ if (sem_wait_for(&refill_donor_ready, 5000) || donor_tid <= 0) {
+ ksft_print_msg("the donor is not ready\n");
+ sem_post(&owner_go);
+ join(d);
+ return KSFT_FAIL;
+ }
+ if (pthread_create(&o, NULL, refill_owner, (void *)spin_ms)) {
+ ksft_print_msg("the owner thread could not start\n");
+ sem_post(&owner_go);
+ join(d);
+ return KSFT_FAIL;
+ }
+ err = pthread_join(o, &o_ret);
+ if (err) {
+ ksft_print_msg("pthread_join: %s\n", strerror(err));
+ ret = KSFT_FAIL;
+ }
+ if (o_ret == (void *)1)
+ sem_post(&owner_go); /* the owner never woke the donor up */
+ err = pthread_join(d, &d_ret);
+ if (err) {
+ ksft_print_msg("pthread_join: %s\n", strerror(err));
+ ret = KSFT_FAIL;
+ }
+ left = refill_owner_left;
+ if (o_ret == (void *)2) {
+ ksft_print_msg("the donor never blocked on the lock\n");
+ ret = KSFT_FAIL;
+ } else if (o_ret || d_ret) {
+ ret = KSFT_FAIL; /* the thread said why */
+ } else if (left < 0) {
+ ksft_print_msg("the runtime of the owner cannot be read\n");
+ ret = KSFT_SKIP;
+ } else if (spin_ms == 8 && left <= 3500 * NSEC_PER_USEC) {
+ ksft_print_msg("the owner has %lld us after the unlock: what the donor's refill left\n",
+ left / NSEC_PER_USEC);
+ ret = KSFT_FAIL;
+ } else if (spin_ms == 1 && left >= 4 * NSEC_PER_MSEC) {
+ ksft_print_msg("the owner has %lld us after the unlock: a new runtime, with no refill from the donor\n",
+ left / NSEC_PER_USEC);
+ ret = KSFT_FAIL;
+ }
+ return ret;
+}
+
+/* Run refill_case() with an owner that stays on its own runtime. */
+static int deboost_keeps_own_runtime(void)
+{
+ return refill_case(1);
+}
+
+static sem_t ret_ready, ret_go;
+static _Atomic pid_t ret_tid;
+static atomic_int ret_first_period_ms;
+
+/*
+ * Run as the thread of the return cases. Become a deadline task of 50ms
+ * every 100ms and run 20ms. 30ms of runtime are left, and the 0-lag time
+ * is about 20ms ahead. Then change to new parameters, as @arg says:
+ * - 0: the thread switches itself to SCHED_OTHER and back to
+ * SCHED_DEADLINE, while it runs.
+ * - 1: the thread sleeps. The main thread switches it to SCHED_OTHER and
+ * back.
+ * - 2: the thread changes its own parameters while it runs. It stays in
+ * SCHED_DEADLINE.
+ * - 3: as 2, while a donor boosts the thread. The thread takes pi_lock and
+ * wakes the donor up. The donor blocks on pi_lock. The thread then
+ * changes its parameters, and unlocks.
+ * - 4: as 2, to a constrained deadline: 8ms in 20ms, every 100ms.
+ * - 5: the thread sleeps. The main thread changes it to 8ms in 20ms,
+ * every 100ms.
+ * The new parameters are 10ms every 100ms, except in modes 4 and 5.
+ * Then spin for 100ms, and store the CPU time used, in ms, in
+ * ret_first_period_ms. Return NULL, or 1 on an error.
+ */
+static void *returner(void *arg)
+{
+ long mode = (long)arg;
+ uint64_t t0, cpu0;
+ int err = 0;
+
+ ret_tid = gettid_();
+ err = set_deadline(0, 50 * NSEC_PER_MSEC, PERIOD);
+ if (err) {
+ ksft_print_msg("the task refused as a deadline task: %s\n", strerror(-err));
+ return (void *)1;
+ }
+ spin_ns(20 * NSEC_PER_MSEC);
+ if (mode == 1 || mode == 5) {
+ sem_post(&ret_ready);
+ sem_wait_sig(&ret_go);
+ } else if (mode == 4) {
+ err = set_constrained(0, 8 * NSEC_PER_MSEC, 20 * NSEC_PER_MSEC, PERIOD);
+ } else if (mode == 3) {
+ if (take_pi_lock())
+ return (void *)1;
+ sem_post(&owner_go);
+ t0 = now_ns();
+ while (!refill_donor_locking || proc_state_of(donor_tid) != 'S') {
+ if (now_ns() - t0 > NSEC_PER_SEC) {
+ ksft_print_msg("the donor did not block on the lock\n");
+ drop_pi_lock();
+ return (void *)1;
+ }
+ }
+ err = set_deadline(0, 10 * NSEC_PER_MSEC, PERIOD);
+ drop_pi_lock();
+ } else {
+ err = mode ? 0 : set_other(0);
+ if (!err)
+ err = set_deadline(0, 10 * NSEC_PER_MSEC, PERIOD);
+ }
+ if (err) {
+ ksft_print_msg("the task could not change: %s\n", strerror(-err));
+ return (void *)1;
+ }
+ t0 = now_ns();
+ cpu0 = thread_time_ns();
+ while (now_ns() - t0 < PERIOD)
+ ;
+ ret_first_period_ms = (thread_time_ns() - cpu0) / NSEC_PER_MSEC;
+ return NULL;
+}
+
+/*
+ * Run a return case: returner() with @mode as its argument. The thread
+ * must get about 10ms of CPU time in the 100ms after the change, 8ms in
+ * modes 4 and 5. That is its new runtime, not the 30ms left from the old
+ * one. The case fails above 20ms.
+ */
+static int return_case(long mode)
+{
+ void *t_ret, *d_ret = NULL;
+ int err, ret = KSFT_PASS;
+ pthread_t t, d;
+
+ if (sem_init(&ret_ready, 0, 0) || sem_init(&ret_go, 0, 0)) {
+ ksft_print_msg("sem_init: %s\n", strerror(errno));
+ return KSFT_FAIL;
+ }
+ ret_first_period_ms = -1;
+ if (mode == 3) {
+ /* The donor: a deadline task of 2ms every 10ms */
+ if (pi_init() || sem_init(&refill_donor_ready, 0, 0))
+ return KSFT_SKIP;
+ refill_donor_locking = 0;
+ donor_tid = 0;
+ if (pthread_create(&d, NULL, refill_donor, NULL)) {
+ ksft_print_msg("the donor thread could not start\n");
+ return KSFT_FAIL;
+ }
+ if (sem_wait_for(&refill_donor_ready, 5000) || donor_tid <= 0) {
+ ksft_print_msg("the donor is not ready\n");
+ sem_post(&owner_go);
+ join(d);
+ return KSFT_FAIL;
+ }
+ }
+ if (pthread_create(&t, NULL, returner, (void *)mode)) {
+ ksft_print_msg("the task thread could not start\n");
+ if (mode == 3) {
+ sem_post(&owner_go);
+ join(d);
+ }
+ return KSFT_FAIL;
+ }
+ if (mode == 1 || mode == 5) {
+ if (sem_wait_for(&ret_ready, 5000)) {
+ /* The thread may get there late. Let it end. */
+ sem_post(&ret_go);
+ join(t);
+ return KSFT_FAIL; /* the task said why */
+ }
+ /* Before its 0-lag time, about 20ms ahead */
+ err = wait_for(10, proc_state_of(ret_tid) == 'S') ? -EAGAIN : 0;
+ if (!err && mode == 1)
+ err = set_other(ret_tid);
+ if (!err && mode == 1)
+ err = set_deadline(ret_tid, 10 * NSEC_PER_MSEC, PERIOD);
+ if (!err && mode == 5)
+ err = set_constrained(ret_tid, 8 * NSEC_PER_MSEC, 20 * NSEC_PER_MSEC,
+ PERIOD);
+ sem_post(&ret_go);
+ if (err) {
+ ksft_print_msg("the task could not change asleep: %s\n",
+ strerror(-err));
+ join(t);
+ return KSFT_FAIL;
+ }
+ }
+ err = pthread_join(t, &t_ret);
+ if (mode == 3) {
+ if (t_ret)
+ sem_post(&owner_go); /* the task never woke the donor up */
+ if (pthread_join(d, &d_ret) || d_ret)
+ ret = KSFT_FAIL;
+ pi_fini();
+ sem_destroy(&refill_donor_ready);
+ }
+ if (err) {
+ ksft_print_msg("pthread_join: %s\n", strerror(err));
+ ret = KSFT_FAIL;
+ } else if (t_ret || ret_first_period_ms < 0) {
+ ret = KSFT_FAIL; /* the task said why */
+ } else if (ret_first_period_ms > 20) {
+ ksft_print_msg("the task ran %d ms of its first 100: the old runtime\n",
+ (int)ret_first_period_ms);
+ ret = KSFT_FAIL;
+ }
+ sem_destroy(&ret_ready);
+ sem_destroy(&ret_go);
+ return ret;
+}
+
+/* The same, for a task that leaves and comes back while it sleeps. */
+static int return_asleep_new_params(void)
+{
+ return return_case(1);
+}
+
+static sem_t rb_donor_go;
+static atomic_int rb_abort, rb_donor_locking, rb_donor_done, rb_waiter_locking;
+static _Atomic pid_t rb_waiter_tid;
+static _Atomic long long rb_spin_ns;
+
+/*
+ * Run as the owner O of deboost_new_params_reboost(). Its comment has the
+ * steps. Store in rb_spin_ns the time that O's last spin of 20ms takes.
+ * Return NULL, or 1 on an error.
+ */
+static void *reboost_owner(void *arg)
+{
+ uint64_t t0;
+ int err;
+
+ owner_tid = gettid_();
+ if (take_pi_lock())
+ return (void *)1;
+ owner_holds = 2;
+ sem_wait_sig(&owner_go);
+ if (rb_abort) {
+ drop_pi_lock();
+ return NULL;
+ }
+ err = set_deadline(0, 50 * NSEC_PER_MSEC, PERIOD);
+ if (err) {
+ ksft_print_msg("the owner refused as a deadline task: %s\n",
+ strerror(-err));
+ goto out;
+ }
+ sem_post(&rb_donor_go);
+ t0 = now_ns();
+ while (!rb_donor_locking || proc_state_of(donor_tid) != 'S') {
+ if (now_ns() - t0 > NSEC_PER_SEC) {
+ ksft_print_msg("the donor did not block on the lock\n");
+ err = -1;
+ goto out;
+ }
+ }
+ err = set_deadline(rb_waiter_tid, 2 * NSEC_PER_MSEC, PERIOD);
+ if (err) {
+ ksft_print_msg("the waiter refused as a deadline task: %s\n",
+ strerror(-err));
+ goto out;
+ }
+ err = set_deadline(0, 5 * NSEC_PER_MSEC, PERIOD);
+ if (err) {
+ ksft_print_msg("the owner could not change: %s\n",
+ strerror(-err));
+ goto out;
+ }
+ t0 = now_ns();
+ while (!rb_donor_done) {
+ if (now_ns() - t0 > NSEC_PER_SEC) {
+ ksft_print_msg("the donor did not time out\n");
+ err = -1;
+ goto out;
+ }
+ }
+ t0 = now_ns();
+ spin_ns(20 * NSEC_PER_MSEC);
+ rb_spin_ns = now_ns() - t0;
+out:
+ drop_pi_lock();
+ if (set_other(0))
+ ksft_print_msg("the owner cannot leave SCHED_DEADLINE\n");
+ return err ? (void *)1 : NULL;
+}
+
+/*
+ * Run as the donor D of deboost_new_params_reboost(). Become a deadline
+ * task of 5ms every 50ms, and sleep until rb_donor_go. Then wait 20ms for
+ * pi_lock. Return NULL if the wait timed out or the case gave up, or 1
+ * otherwise.
+ */
+static void *reboost_donor(void *arg)
+{
+ struct timespec ts;
+ int err;
+
+ err = set_deadline(0, 5 * NSEC_PER_MSEC, PERIOD / 2);
+ if (err) {
+ ksft_print_msg("the donor refused as a deadline task: %s\n",
+ strerror(-err));
+ donor_tid = -1;
+ return (void *)1;
+ }
+ donor_tid = gettid_();
+ sem_wait_sig(&rb_donor_go);
+ if (rb_abort) {
+ set_other(0);
+ return NULL;
+ }
+ clock_gettime(CLOCK_REALTIME, &ts);
+ ts.tv_nsec += 20 * 1000000L;
+ ts.tv_sec += ts.tv_nsec / 1000000000L;
+ ts.tv_nsec %= 1000000000L;
+ rb_donor_locking = 1;
+ err = pthread_mutex_timedlock(&pi_lock, &ts);
+ rb_donor_done = 1;
+ if (!err)
+ pthread_mutex_unlock(&pi_lock);
+ if (set_other(0))
+ ksft_print_msg("the donor cannot leave SCHED_DEADLINE\n");
+ if (err != ETIMEDOUT) {
+ ksft_print_msg("the donor: %s, not a timeout\n",
+ err ? strerror(err) : "got pi_lock");
+ return (void *)1;
+ }
+ return NULL;
+}
+
+/*
+ * Run as the waiter W of deboost_new_params_reboost(). Wait for pi_lock
+ * as a fair task. The owner makes W a deadline task during the wait.
+ * Return NULL, or 1 on an error.
+ */
+static void *reboost_waiter(void *arg)
+{
+ rb_waiter_tid = gettid_();
+ rb_waiter_locking = 1;
+ if (take_pi_lock())
+ return (void *)1;
+ drop_pi_lock();
+ if (set_other(0))
+ ksft_print_msg("the waiter cannot leave SCHED_DEADLINE\n");
+ return NULL;
+}
+
+/*
+ * Check that a deboost boosts the owner again when the check of its new
+ * parameters moves its deadline after the one of its top waiter. The
+ * threads share the CPU of P:
+ *
+ * owner O donor D waiter W
+ * fair 5ms every 50ms fair
+ *
+ * takes pi_lock sleeps
+ * blocks on pi_lock
+ * becomes 50ms every 100ms
+ * wakes D up
+ * preempts O
+ * waits 20ms for
+ * pi_lock, boosts O
+ * makes W 2ms every 100ms
+ * changes itself to 5ms
+ * every 100ms
+ * spins
+ * times out: W is now
+ * the top waiter
+ * spins 20ms
+ * unlocks
+ * takes pi_lock
+ *
+ * W gets its deadline after O does, so W's deadline is the later one.
+ * W blocks as a fair task: a deadline W could not preempt O to block.
+ *
+ * O has run about 20ms of its 50ms when D times out. The deboost checks
+ * the runtime left against the new parameters. It is too much, so O
+ * starts a new period: 5ms and a deadline 100ms ahead. W's deadline is
+ * now the earlier one. W must boost O again. A boosted O is refilled
+ * from W, so its last spin takes about 20ms. Without the boost, O is
+ * throttled once its 5ms run out, until its new deadline. The case fails
+ * if the spin takes more than 50ms.
+ */
+static int deboost_new_params_reboost(void)
+{
+ void *o_ret = NULL, *d_ret = NULL, *w_ret = NULL;
+ int ret = KSFT_FAIL, have_d = 0, have_w = 0;
+ pthread_t o, d, w;
+
+ if (pi_init())
+ return KSFT_SKIP;
+ if (sem_init(&rb_donor_go, 0, 0)) {
+ ksft_print_msg("sem_init: %s\n", strerror(errno));
+ pi_fini();
+ return KSFT_FAIL;
+ }
+ donor_tid = 0;
+ rb_abort = 0;
+ rb_donor_locking = 0;
+ rb_donor_done = 0;
+ rb_waiter_locking = 0;
+ rb_waiter_tid = 0;
+ rb_spin_ns = -1;
+ if (pthread_create(&o, NULL, reboost_owner, NULL)) {
+ ksft_print_msg("the owner thread could not start\n");
+ goto out_fini;
+ }
+ if (wait_for(5000, owner_holds) || owner_holds != 2) {
+ ksft_print_msg("the owner never took pi_lock\n");
+ goto out;
+ }
+ have_d = !pthread_create(&d, NULL, reboost_donor, NULL);
+ if (!have_d || wait_for(1000, donor_tid &&
+ proc_state_of(donor_tid) == 'S') ||
+ donor_tid < 0) {
+ ksft_print_msg("the donor never slept\n");
+ goto out;
+ }
+ have_w = !pthread_create(&w, NULL, reboost_waiter, NULL);
+ if (!have_w || wait_for(1000, rb_waiter_locking &&
+ proc_state_of(rb_waiter_tid) == 'S')) {
+ ksft_print_msg("the waiter never waited on pi_lock\n");
+ goto out;
+ }
+ ret = KSFT_PASS;
+out:
+ /* On a failure, the owner and the donor end without the steps. */
+ rb_abort = ret != KSFT_PASS;
+ sem_post(&owner_go);
+ if (pthread_join(o, &o_ret))
+ ret = KSFT_FAIL;
+ sem_post(&rb_donor_go);
+ if (have_d && pthread_join(d, &d_ret))
+ ret = KSFT_FAIL;
+ if (have_w && pthread_join(w, &w_ret))
+ ret = KSFT_FAIL;
+ if (o_ret || d_ret || w_ret) {
+ ret = KSFT_FAIL; /* the thread said why */
+ } else if (ret == KSFT_PASS && rb_spin_ns > PERIOD / 2) {
+ ksft_print_msg("the spin took %lld ms: not boosted again\n",
+ rb_spin_ns / (long long)NSEC_PER_MSEC);
+ ret = KSFT_FAIL;
+ }
+out_fini:
+ sem_destroy(&rb_donor_go);
+ pi_fini();
+ return ret;
+}
+
+static pthread_mutex_t pi_lock2;
+static sem_t holder_go;
+static atomic_int holder_holds, chain_waits, donor_done, x_waits, l2_takes;
+static atomic_int chain_rank, x_rank;
+static _Atomic pid_t x_tid, fair_tid;
+
+/* The holder thread: take pi_lock2, and hold it until holder_go. */
+static void *l2_holder(void *arg)
+{
+ int err = pthread_mutex_lock(&pi_lock2);
+
+ if (err) {
+ ksft_print_msg("the holder cannot take pi_lock2: %s\n", strerror(err));
+ holder_holds = -1;
+ return (void *)1;
+ }
+ holder_holds = 1;
+ sem_wait_sig(&holder_go);
+ pthread_mutex_unlock(&pi_lock2);
+ return NULL;
+}
+
+/*
+ * Take pi_lock2. Store in @rank how many threads took it before the
+ * caller. Then unlock it. Return 0, or -1 if the lock cannot be taken.
+ */
+static int take_l2_ranked(atomic_int *rank)
+{
+ int err = pthread_mutex_lock(&pi_lock2);
+
+ if (err) {
+ ksft_print_msg("a waiter cannot take pi_lock2: %s\n", strerror(err));
+ return -1;
+ }
+ *rank = l2_takes++;
+ pthread_mutex_unlock(&pi_lock2);
+ return 0;
+}
+
+/*
+ * Run as the owner thread, a deadline task of 10ms every 200ms. Take
+ * pi_lock, and sleep until owner_go is posted. The donor boosts the owner
+ * during that sleep. Then spin 20ms. The owner's 10ms run out, and the
+ * boost refills its runtime from the donor's 40ms. Then wait on pi_lock2
+ * while holding pi_lock. The donor's timeout deboosts the owner during
+ * that wait. The owner then has more runtime left than its own 10ms.
+ * Return NULL, 1 on an error, or 2 if the spin took over 50ms: the owner
+ * was throttled, so it was not boosted.
+ */
+static void *chain_owner(void *arg)
+{
+ uint64_t t0;
+ int err;
+
+ owner_tid = gettid_();
+ err = set_deadline(0, 10 * NSEC_PER_MSEC, 2 * PERIOD);
+ if (err) {
+ ksft_print_msg("the owner refused as a deadline task: %s\n", strerror(-err));
+ owner_holds = -1;
+ return (void *)1;
+ }
+ if (take_pi_lock())
+ return (void *)1;
+ owner_holds = 2;
+ sem_wait_sig(&owner_go);
+ t0 = now_ns();
+ spin_ns(20 * NSEC_PER_MSEC);
+ if (now_ns() - t0 > PERIOD / 2) {
+ drop_pi_lock();
+ return (void *)2; /* throttled: never boosted */
+ }
+ chain_waits = 1;
+ err = take_l2_ranked(&chain_rank);
+ drop_pi_lock();
+ return err ? (void *)1 : NULL;
+}
+
+/*
+ * The donor thread: a deadline task of 40ms every 100ms. Wait 150ms for
+ * pi_lock. Its timeout deboosts the owner, which waits on pi_lock2 then.
+ */
+static void *timed_donor(void *arg)
+{
+ struct timespec ts;
+ int err;
+
+ donor_tid = gettid_();
+ err = set_deadline(0, 40 * NSEC_PER_MSEC, PERIOD);
+ if (err) {
+ ksft_print_msg("the donor refused as a deadline task: %s\n", strerror(-err));
+ return (void *)1;
+ }
+ clock_gettime(CLOCK_REALTIME, &ts);
+ ts.tv_nsec += 150 * 1000000L;
+ ts.tv_sec += ts.tv_nsec / 1000000000L;
+ ts.tv_nsec %= 1000000000L;
+ err = pthread_mutex_timedlock(&pi_lock, &ts);
+ donor_done = 1;
+ if (!err)
+ pthread_mutex_unlock(&pi_lock);
+ if (set_other(0))
+ ksft_print_msg("the donor cannot leave SCHED_DEADLINE\n");
+ if (err != ETIMEDOUT) {
+ ksft_print_msg("the donor: %s, not a timeout\n",
+ err ? strerror(err) : "got pi_lock");
+ return (void *)1;
+ }
+ return NULL;
+}
+
+/*
+ * The other waiter: a deadline task of 5ms every 50ms. Its deadline is
+ * earlier than the one the owner gets at the deboost. Wait on pi_lock2.
+ */
+static void *early_waiter(void *arg)
+{
+ int err;
+
+ x_tid = gettid_();
+ err = set_deadline(0, 5 * NSEC_PER_MSEC, PERIOD / 2);
+ if (err) {
+ ksft_print_msg("a waiter refused as a deadline task: %s\n", strerror(-err));
+ return (void *)1;
+ }
+ x_waits = 1;
+ err = take_l2_ranked(&x_rank);
+ if (set_other(0))
+ ksft_print_msg("a waiter cannot leave SCHED_DEADLINE\n");
+ return err ? (void *)1 : NULL;
+}
+
+/* Set up pi_lock2 and holder_go. Return 0, or -1. */
+static int pi_init_lock2(void)
+{
+ pthread_mutexattr_t a;
+ int err;
+
+ err = pthread_mutexattr_init(&a);
+ if (!err) {
+ err = pthread_mutexattr_setprotocol(&a, PTHREAD_PRIO_INHERIT);
+ if (!err)
+ err = pthread_mutex_init(&pi_lock2, &a);
+ pthread_mutexattr_destroy(&a);
+ }
+ if (err) {
+ ksft_print_msg("no second priority inheritance mutex: %s\n", strerror(err));
+ return -1;
+ }
+ if (sem_init(&holder_go, 0, 0)) {
+ ksft_print_msg("sem_init: %s\n", strerror(errno));
+ pthread_mutex_destroy(&pi_lock2);
+ return -1;
+ }
+ holder_holds = 0;
+ chain_waits = 0;
+ donor_done = 0;
+ x_waits = 0;
+ l2_takes = 0;
+ chain_rank = -1;
+ x_rank = -1;
+ donor_tid = 0;
+ x_tid = 0;
+ fair_tid = 0;
+ return 0;
+}
+
+/*
+ * Check the order of the waiters of pi_lock2 after a deboost. The threads
+ * are:
+ * - the holder: SCHED_OTHER. It has pi_lock2.
+ * - the owner: deadline, 10ms every 200ms. It has pi_lock.
+ * - the donor: deadline, 40ms every 100ms.
+ * - the other waiter: deadline, 5ms every 50ms.
+ *
+ * 1. The donor waits on pi_lock, and boosts the owner.
+ * 2. The owner spins on the donor's runtime, then waits on pi_lock2.
+ * 3. The other waiter waits on pi_lock2.
+ * 4. The donor times out after 150ms. The deboost starts a new period
+ * for the owner, which has more runtime left than its own 10ms.
+ * 5. The holder drops pi_lock2.
+ *
+ * The other waiter must take pi_lock2 first: its deadline is earlier
+ * than the owner's new one. At step 4 the owner is not queued, and the
+ * chain walk copies its deadline into its waiter node on pi_lock2. A
+ * deadline of 0 in the node would put the owner first.
+ */
+static int deboost_blocked_keeps_order(void)
+{
+ void *o_ret = NULL, *d_ret = NULL, *x_ret = NULL;
+ int ret = KSFT_FAIL, have_o = 0, have_d = 0, have_x = 0;
+ pthread_t h, o, d, x;
+
+ if (pi_init())
+ return KSFT_SKIP;
+ if (pi_init_lock2()) {
+ pi_fini();
+ return KSFT_SKIP;
+ }
+ if (pthread_create(&h, NULL, l2_holder, NULL)) {
+ ksft_print_msg("the holder thread could not start\n");
+ goto out_fini;
+ }
+ if (wait_for(5000, holder_holds) || holder_holds != 1) {
+ ksft_print_msg("the holder never took pi_lock2\n");
+ goto out;
+ }
+ have_o = !pthread_create(&o, NULL, chain_owner, NULL);
+ if (!have_o || wait_for(5000, owner_holds) || owner_holds != 2) {
+ ksft_print_msg("the owner never took pi_lock\n");
+ goto out;
+ }
+ have_d = !pthread_create(&d, NULL, timed_donor, NULL);
+ if (!have_d || wait_for(1000, donor_tid && proc_state_of(donor_tid) == 'S')) {
+ ksft_print_msg("the donor never waited on pi_lock\n");
+ goto out;
+ }
+ sem_post(&owner_go);
+ if (wait_for(1000, chain_waits && proc_state_of(owner_tid) == 'S')) {
+ ksft_print_msg("the owner never waited on pi_lock2\n");
+ goto out;
+ }
+ if (donor_done) {
+ ksft_print_msg("the donor timed out before the owner waited on pi_lock2\n");
+ goto out;
+ }
+ have_x = !pthread_create(&x, NULL, early_waiter, NULL);
+ if (!have_x || wait_for(1000, x_waits && proc_state_of(x_tid) == 'S')) {
+ ksft_print_msg("the other waiter never waited on pi_lock2\n");
+ goto out;
+ }
+ /* The donor's timeout deboosts the owner, which waits on pi_lock2. */
+ have_d = 0;
+ if (!pthread_join(d, &d_ret) && !d_ret)
+ ret = KSFT_PASS;
+out:
+ /* The owner may get there late. Let it end. */
+ sem_post(&owner_go);
+ sem_post(&holder_go);
+ if (have_d)
+ join(d);
+ if (have_x && pthread_join(x, &x_ret))
+ ret = KSFT_FAIL;
+ if (have_o && pthread_join(o, &o_ret))
+ ret = KSFT_FAIL;
+ join(h);
+ if (o_ret == (void *)2) {
+ ksft_print_msg("the donor never boosted the owner\n");
+ ret = KSFT_FAIL;
+ } else if (o_ret || x_ret) {
+ ret = KSFT_FAIL; /* the thread said why */
+ } else if (ret == KSFT_PASS && chain_rank < x_rank) {
+ ksft_print_msg("the deboosted owner took pi_lock2 before the earlier waiter\n");
+ ret = KSFT_FAIL;
+ }
+out_fini:
+ pthread_mutex_destroy(&pi_lock2);
+ sem_destroy(&holder_go);
+ pi_fini();
+ return ret;
+}
+
+/*
+ * The threads of pi_chain_donor_leaves(): O, M, W, Z and Y. leave_tid[]
+ * holds their tids. leave_y_ns is the CPU time Y gets in O's window, or
+ * -1.
+ */
+enum { LV_O, LV_M, LV_W, LV_Z, LV_Y, LV_N };
+static _Atomic pid_t leave_tid[LV_N];
+static atomic_int leave_err, leave_m_locking, leave_w_locking, leave_y_stop;
+static _Atomic long long leave_y_ns;
+static sem_t leave_o_go, leave_m_go;
+
+/*
+ * O: take pi_lock2 (L2), and sleep until leave_o_go. Then spin 100ms.
+ * Measure the CPU time of Y (@arg) over the next 200ms, and unlock L2.
+ */
+static void *leave_o(void *arg)
+{
+ pthread_t y = *(pthread_t *)arg;
+ uint64_t t0, y0;
+
+ if (pthread_mutex_lock(&pi_lock2)) {
+ leave_err = 1;
+ return arg;
+ }
+ leave_tid[LV_O] = gettid_(); /* O holds L2 */
+ sem_wait_sig(&leave_o_go);
+ t0 = now_ns();
+ while (now_ns() - t0 < PERIOD)
+ ;
+ y0 = cpu_time_of(y);
+ t0 = now_ns();
+ while (now_ns() - t0 < 2 * PERIOD)
+ ;
+ leave_y_ns = cpu_time_of(y) - y0;
+ pthread_mutex_unlock(&pi_lock2);
+ return NULL;
+}
+
+/*
+ * M: become a deadline task of 10ms every 1s, and take pi_lock (L1). At
+ * leave_m_go, block on L2, which O holds.
+ */
+static void *leave_m(void *arg)
+{
+ if (set_deadline(0, 10 * NSEC_PER_MSEC, NSEC_PER_SEC) ||
+ pthread_mutex_lock(&pi_lock)) {
+ leave_err = 1;
+ return arg;
+ }
+ leave_tid[LV_M] = gettid_(); /* M holds L1 */
+ sem_wait_sig(&leave_m_go);
+ leave_m_locking = 1;
+ if (pthread_mutex_lock(&pi_lock2))
+ leave_err = 1;
+ else
+ pthread_mutex_unlock(&pi_lock2);
+ pthread_mutex_unlock(&pi_lock);
+ if (set_other(0))
+ leave_err = 1;
+ return arg;
+}
+
+/*
+ * W and Z: become a deadline task of @arg ms every 40 times that, and
+ * block on L1. W is the one of 5ms. The main thread switches it to
+ * SCHED_OTHER while it waits.
+ */
+static void *leave_waiter(void *arg)
+{
+ uint64_t runtime = (long)arg * NSEC_PER_MSEC;
+ int w = (long)arg == 5;
+
+ if (set_deadline(0, runtime, 40 * runtime)) {
+ leave_err = 1;
+ return arg;
+ }
+ leave_tid[w ? LV_W : LV_Z] = gettid_();
+ if (w)
+ leave_w_locking = 1;
+ if (pthread_mutex_lock(&pi_lock))
+ leave_err = 1;
+ else
+ pthread_mutex_unlock(&pi_lock);
+ if (set_other(0))
+ leave_err = 1;
+ return arg;
+}
+
+/* Y: become a deadline task of 30ms every 100ms, and spin. */
+static void *leave_y(void *arg)
+{
+ if (set_deadline(0, 30 * NSEC_PER_MSEC, PERIOD)) {
+ leave_err = 1;
+ return arg;
+ }
+ leave_tid[LV_Y] = gettid_();
+ while (!leave_y_stop)
+ ;
+ return arg;
+}
+
+/*
+ * A chain walk must pass a new donor on to the end of the chain. This
+ * holds even when the sort keys of the task in the middle do not change.
+ *
+ * L1 is pi_lock and L2 is pi_lock2. All threads run on the CPU of P.
+ * Each deadline thread calls sched_setattr() on itself when it starts.
+ * The threads are:
+ * - O: SCHED_OTHER.
+ * - M: a deadline task of 10ms every 1s.
+ * - W: a deadline task of 5ms every 200ms.
+ * - Z: a deadline task of 2ms every 80ms.
+ * - Y: a deadline task of 30ms every 100ms. It spins all along.
+ *
+ * 1. O takes L2, and sleeps.
+ * 2. M takes L1, and sleeps.
+ * 3. W blocks on L1. W's deadline is earlier than M's, so W boosts M. M
+ * keeps its own deadline.
+ * 4. M blocks on L2. M passes W on to O, as O's donor. O enters the
+ * deadline class. It sleeps, so it gets no runtime yet.
+ * 5. Z blocks on L1. Z's deadline is earlier than W's, so Z becomes M's
+ * donor. M sleeps, so its deadline does not move. The chain walk must
+ * go on to O and make Z O's donor too.
+ * 6. The main thread switches W to SCHED_OTHER. W sleeps and has run
+ * almost nothing, so its parameters are cleared at once.
+ * 7. 250ms later, the main thread wakes O up. O has no deadline ahead, so
+ * its wakeup starts a new period from its donor. O spins 100ms.
+ * Then it measures the CPU time Y gets over 200ms, and unlocks L2.
+ *
+ * With Z as its donor, O gets 2ms every 80ms. Y's deadline is soon the
+ * earlier one, and Y gets about its 60ms. With W as its donor, O runs on
+ * cleared parameters. Each refill gives it no runtime and a deadline of
+ * about now. O then runs before Y all the time, and Y gets close to
+ * nothing. The case fails if Y gets less than 30ms.
+ *
+ * W does not exit before O unlocks L2. It waits for L1, which M releases
+ * after it gets L2.
+ */
+static int pi_chain_donor_leaves(void)
+{
+ int ret = KSFT_FAIL, i, have[LV_N] = { 0 };
+ pthread_t t[LV_N];
+ long long y_ns;
+ int err;
+
+ if (pi_init())
+ return KSFT_SKIP;
+ if (pi_init_lock2()) {
+ pi_fini();
+ return KSFT_SKIP;
+ }
+ if (sem_init(&leave_o_go, 0, 0) || sem_init(&leave_m_go, 0, 0)) {
+ ksft_print_msg("sem_init: %s\n", strerror(errno));
+ goto out_fini;
+ }
+ for (i = 0; i < LV_N; i++)
+ leave_tid[i] = 0;
+ leave_err = 0;
+ leave_m_locking = 0;
+ leave_w_locking = 0;
+ leave_y_stop = 0;
+ leave_y_ns = -1;
+ have[LV_Y] = !pthread_create(&t[LV_Y], NULL, leave_y, NULL);
+ if (!have[LV_Y] || wait_for(1000, leave_tid[LV_Y] || leave_err)) {
+ ksft_print_msg("Y never became a deadline task\n");
+ goto out;
+ }
+ have[LV_O] = !pthread_create(&t[LV_O], NULL, leave_o, &t[LV_Y]);
+ if (!have[LV_O] || wait_for(1000, leave_tid[LV_O] || leave_err)) {
+ ksft_print_msg("O never took L2\n");
+ goto out;
+ }
+ have[LV_M] = !pthread_create(&t[LV_M], NULL, leave_m, NULL);
+ if (!have[LV_M] || wait_for(1000, leave_tid[LV_M] || leave_err)) {
+ ksft_print_msg("M never took L1\n");
+ goto out;
+ }
+ have[LV_W] = !pthread_create(&t[LV_W], NULL, leave_waiter, (void *)5L);
+ if (!have[LV_W] ||
+ wait_for(1000, (leave_w_locking && proc_state_of(leave_tid[LV_W]) == 'S') ||
+ leave_err)) {
+ ksft_print_msg("W never blocked on L1\n");
+ goto out;
+ }
+ sem_post(&leave_m_go);
+ if (wait_for(1000, (leave_m_locking && prio_of(leave_tid[LV_O]) == DL_TASK_PRIO) ||
+ leave_err)) {
+ ksft_print_msg("O was not boosted through M\n");
+ goto out;
+ }
+ have[LV_Z] = !pthread_create(&t[LV_Z], NULL, leave_waiter, (void *)2L);
+ if (!have[LV_Z] ||
+ wait_for(1000, (leave_tid[LV_Z] && proc_state_of(leave_tid[LV_Z]) == 'S') ||
+ leave_err)) {
+ ksft_print_msg("Z never blocked on L1\n");
+ goto out;
+ }
+ sleep_ns(10 * NSEC_PER_MSEC); /* the chain walk of Z is done */
+ err = leave_err ? -EINVAL : set_other(leave_tid[LV_W]);
+ if (err) {
+ ksft_print_msg("W cannot leave SCHED_DEADLINE: %s\n", strerror(-err));
+ goto out;
+ }
+ sleep_ns(250 * NSEC_PER_MSEC);
+ ret = KSFT_PASS;
+out:
+ /* Let every thread end: O unlocks L2, then M unlocks L1. */
+ sem_post(&leave_m_go);
+ sem_post(&leave_o_go);
+ for (i = LV_O; i < LV_Y; i++)
+ if (have[i])
+ join(t[i]);
+ leave_y_stop = 1;
+ if (have[LV_Y])
+ join(t[LV_Y]);
+ y_ns = leave_y_ns;
+ if (leave_err) {
+ ksft_print_msg("a thread of the chain failed\n");
+ ret = KSFT_FAIL;
+ } else if (ret == KSFT_PASS && y_ns < 30 * (long long)NSEC_PER_MSEC) {
+ ksft_print_msg("Y ran %lld ms of 200: O ran on the cleared parameters of W\n",
+ y_ns / (long long)NSEC_PER_MSEC);
+ ret = KSFT_FAIL;
+ }
+ sem_destroy(&leave_o_go);
+ sem_destroy(&leave_m_go);
+out_fini:
+ pthread_mutex_destroy(&pi_lock2);
+ sem_destroy(&holder_go);
+ pi_fini();
+ return ret;
+}
+
+/* cpuset moves, from the main process, outside P */
+
+/* Shared with the processes of a case: atomic in shared memory. */
+static atomic_int *mv_state;
+
+/*
+ * Wait up to 5s for a child to report in mv_state[0]. Return 0 if it is
+ * up (1). Return -1 if it was refused SCHED_DEADLINE (-1), could not
+ * start its threads (-2) or did not report. @fast polls every 100us.
+ */
+static int child_up(const char *what, int fast)
+{
+ if (wait_for_step(5000, fast ? 100 * NSEC_PER_USEC : NSEC_PER_MSEC, mv_state[0])) {
+ ksft_print_msg("%s: the task never started\n", what);
+ return -1;
+ }
+ if (mv_state[0] == -2) {
+ ksft_print_msg("%s: the process cannot start its threads\n", what);
+ return -1;
+ }
+ if (mv_state[0] != 1) {
+ ksft_print_msg("%s: the task was refused SCHED_DEADLINE\n", what);
+ return -1;
+ }
+ return 0;
+}
+
+enum mv_mode { MV_ASLEEP, MV_THROTTLED, MV_YIELDED, MV_NON_CONTENDING };
+
+/*
+ * Fork a process in the caller's cgroup. The process becomes a deadline
+ * task of @runtime every 100ms, and does what @mode says:
+ * - MV_ASLEEP: run 1ms, set mv_state[0] to 1, then sleep.
+ * - MV_THROTTLED: set mv_state[0] to 1, then spin and count in
+ * mv_state[1].
+ * - MV_YIELDED: set mv_state[0] to 1. Then spin 0.5ms, count in
+ * mv_state[1] and yield, in a loop.
+ * - MV_NON_CONTENDING: sleep a period, run a quarter of its runtime, set
+ * mv_state[0] to 1, then sleep. Its 0-lag time is still ahead.
+ * If it is refused SCHED_DEADLINE, the process sets mv_state[0] to -1 and
+ * exits. Return its pid, or -1.
+ */
+static pid_t mover(enum mv_mode mode, uint64_t runtime)
+{
+ pid_t pid = fork_child("a deadline task");
+
+ if (!pid) {
+ if (set_deadline(0, runtime, PERIOD)) {
+ mv_state[0] = -1;
+ _exit(1);
+ }
+ if (mode == MV_ASLEEP) {
+ spin_ns(NSEC_PER_MSEC);
+ *mv_state = 1;
+ for (;;)
+ pause();
+ }
+ if (mode == MV_NON_CONTENDING) {
+ sleep_ns(PERIOD);
+ spin_ns(runtime / 4);
+ *mv_state = 1;
+ for (;;)
+ pause();
+ }
+ if (mode == MV_YIELDED) {
+ *mv_state = 1;
+ for (;;) {
+ spin_ns(NSEC_PER_MSEC / 2);
+ mv_state[1]++;
+ sched_yield();
+ }
+ }
+ *mv_state = 1;
+ for (;;)
+ mv_state[1]++;
+ }
+ return pid;
+}
+
+static int wait_spinner(int moving);
+
+/*
+ * A deadline task of 60ms every 100ms sleeps in P. Move a second sleeping
+ * task of 60ms every 100ms into P from outside. The move must fail with
+ * EBUSY. P and the root cgroup must each admit as many probe tasks after
+ * as before.
+ */
+static int move_refused(void)
+{
+ int base_p, base_r, ret = KSFT_PASS, err;
+ pid_t in_p, pid;
+
+ base_p = probe(cg_p);
+ base_r = probe(root);
+ mv_state[0] = 0;
+ in_p = fork_child("a deadline task of P");
+ if (!in_p) {
+ if (enter(cg_p, getpid()) || set_deadline(0, 60 * NSEC_PER_MSEC, PERIOD))
+ _exit(1);
+ mv_state[0] = 1;
+ for (;;)
+ pause();
+ }
+ if (in_p < 0 || child_up("a task of 60% in P", 0)) {
+ reap(in_p);
+ return KSFT_FAIL;
+ }
+ mv_state[0] = 0;
+ pid = mover(MV_ASLEEP, 60 * NSEC_PER_MSEC);
+ if (pid < 0 || child_up("a task of 60% to move into P", 0)) {
+ reap(pid);
+ reap(in_p);
+ return KSFT_FAIL;
+ }
+ err = enter(cg_p, pid);
+ if (err != -EBUSY) {
+ ksft_print_msg("a move beyond P's room: %s, not EBUSY\n",
+ err ? strerror(-err) : "accepted");
+ ret = KSFT_FAIL;
+ }
+ reap(pid);
+ reap(in_p);
+ sleep_ns(3 * PERIOD);
+ if (ret == KSFT_PASS && (probe(cg_p) != base_p || probe(root) != base_r)) {
+ ksft_print_msg("a refused move changed the free bandwidth\n");
+ ret = KSFT_FAIL;
+ }
+ return ret;
+}
+
+/*
+ * A runtime of 1024ns every 2s gives a bandwidth of 0. A cpuset move
+ * skips such a task: it has no bandwidth to move.
+ *
+ * A deadline task of 60ms every 100ms sleeps in P. A second task sleeps
+ * outside P. The main process makes it a deadline task of 1024ns every
+ * 2s, moves it into P, and wakes it. It then runs on P's CPU. The main
+ * process then raises it to 60ms every 100ms. P has no room for that, so
+ * sched_setattr() must fail with EBUSY. The kernel must check the request
+ * against P's root domain, not against the one the task left.
+ */
+static int move_zero_bw(void)
+{
+ int base_p, base_r, ret = KSFT_PASS, err, zfd[2];
+ pid_t in_p, pid;
+
+ if (pipe(zfd))
+ return KSFT_FAIL;
+ base_p = probe(cg_p);
+ base_r = probe(root);
+ mv_state[0] = 0;
+ in_p = fork_child("a deadline task of P");
+ if (!in_p) {
+ if (enter(cg_p, getpid()) || set_deadline(0, 60 * NSEC_PER_MSEC, PERIOD))
+ _exit(1);
+ mv_state[0] = 1;
+ for (;;)
+ pause();
+ }
+ if (in_p < 0 || child_up("a task of 60% in P", 0)) {
+ reap(in_p);
+ ret = KSFT_FAIL;
+ goto out;
+ }
+ mv_state[0] = 0;
+ pid = fork_child("a task with no bandwidth");
+ if (!pid) {
+ char c;
+
+ close(zfd[1]);
+ mv_state[0] = 1;
+ while (read(zfd[0], &c, 1) == 1)
+ ;
+ _exit(0);
+ }
+ if (pid < 0 || child_up("a task to move into P", 0) ||
+ wait_for(1000, proc_state_of(pid) == 'S')) {
+ reap(pid);
+ reap(in_p);
+ ret = KSFT_FAIL;
+ goto out;
+ }
+ /*
+ * Set the parameters from here, while the task sleeps. With a runtime
+ * of 1024ns, the task could be throttled before it sleeps again, for
+ * up to its period of 2s.
+ */
+ err = set_deadline(pid, 1024, 2 * NSEC_PER_SEC);
+ if (err) {
+ ksft_print_msg("a runtime of 1024ns every 2s: %s\n", strerror(-err));
+ ret = KSFT_FAIL;
+ } else if (enter(cg_p, pid)) {
+ ksft_print_msg("a task with no bandwidth: its move into P refused\n");
+ ret = KSFT_FAIL;
+ } else if (write(zfd[1], "w", 1) != 1 || wait_for(1000, task_cpu_of(pid) == top_cpu(1))) {
+ ksft_print_msg("a task moved into P did not wake on P's CPU\n");
+ ret = KSFT_FAIL;
+ } else {
+ err = set_deadline(pid, 60 * NSEC_PER_MSEC, PERIOD);
+ if (err != -EBUSY) {
+ ksft_print_msg("0%% to 60%% beside a task of 60%% in P: %s, not EBUSY\n",
+ err ? strerror(-err) : "accepted");
+ ret = KSFT_FAIL;
+ }
+ }
+ /* Throttled, it would wait up to 2s to run and die. */
+ set_other(pid);
+ reap(pid);
+ reap(in_p);
+out:
+ close(zfd[0]);
+ close(zfd[1]);
+ sleep_ns(3 * PERIOD);
+ if (ret == KSFT_PASS && (probe(cg_p) != base_p || probe(root) != base_r)) {
+ ksft_print_msg("a refused change changed the free bandwidth\n");
+ ret = KSFT_FAIL;
+ }
+ return ret;
+}
+
+static void *pauser(void *arg)
+{
+ for (;;)
+ pause();
+ return arg;
+}
+
+/* What late_setter() does, and the result. */
+static struct {
+ pid_t pid;
+ uint64_t runtime;
+ uint64_t delay;
+ atomic_int go;
+ int ret;
+ uint64_t done;
+} late;
+
+/*
+ * Make late.pid a deadline task of late.runtime every 100ms, late.delay
+ * ns after late.go. Store the result in late.ret and the time it returned
+ * in late.done.
+ */
+static void *late_setter(void *arg)
+{
+ uint64_t start;
+
+ while (!late.go)
+ ;
+ start = now_ns();
+ while (now_ns() - start < late.delay)
+ ;
+ late.ret = set_deadline(late.pid, late.runtime, PERIOD);
+ late.done = now_ns();
+ return arg;
+}
+
+static void no_op(int sig)
+{
+}
+
+/* The CPU that dl_sleeper_tid() first runs on, or -1 for any CPU */
+static int sleeper_cpu = -1;
+
+/*
+ * Run as the deadline thread of pinned_mover(): a deadline task of 30%.
+ * Store the tid in mv_state[3], set mv_state[0] to 1 and sleep. A signal
+ * wakes the thread up. It then sleeps again. If mv_state[4] is set, this
+ * thread exits instead, and the process stays.
+ *
+ * With sleeper_cpu set, first run 1ms pinned on that CPU, then restore
+ * the previous affinity. Do this before sched_setattr(): the affinity of
+ * a deadline task must include every CPU of its root domain.
+ */
+static void *dl_sleeper_tid(void *arg)
+{
+ if (sleeper_cpu >= 0) {
+ cpu_set_t all, one;
+
+ CPU_ZERO(&one);
+ CPU_SET(sleeper_cpu, &one);
+ if (sched_getaffinity(0, sizeof(all), &all) ||
+ sched_setaffinity(0, sizeof(one), &one)) {
+ mv_state[0] = -2;
+ return arg;
+ }
+ spin_ns(NSEC_PER_MSEC);
+ if (sched_setaffinity(0, sizeof(all), &all)) {
+ mv_state[0] = -2;
+ return arg;
+ }
+ }
+ if (set_deadline(0, 30 * NSEC_PER_MSEC, PERIOD)) {
+ mv_state[0] = -1;
+ return arg;
+ }
+ mv_state[3] = gettid_();
+ mv_state[0] = 1;
+ while (!mv_state[4])
+ pause();
+ syscall(SYS_exit, 0);
+ return arg;
+}
+
+/*
+ * Run as the last thread of pinned_mover() with @fifo. Become a
+ * SCHED_FIFO thread, set mv_state[5] to 1, or to -1 if SCHED_FIFO is
+ * refused, and sleep.
+ */
+static void *fifo_sleeper(void *arg)
+{
+ struct sched_param sp = { .sched_priority = 1 };
+
+ mv_state[5] = sched_setscheduler(0, SCHED_FIFO, &sp) ? -1 : 1;
+ for (;;)
+ pause();
+ return arg;
+}
+
+/*
+ * Fork a process of @threads sleeping threads, then one deadline thread,
+ * dl_sleeper_tid(). A cgroup move of the process handles the deadline
+ * thread after the sleeping ones, which make the move long. With @fifo,
+ * add a SCHED_FIFO thread, fifo_sleeper(), after it. Return the pid, or
+ * -1.
+ */
+static pid_t pinned_mover(int threads, int fifo)
+{
+ pid_t pid = fork_child("a wide deadline process");
+
+ if (!pid) {
+ pthread_attr_t a;
+ pthread_t t;
+ int i;
+
+ if (pthread_attr_init(&a) || pthread_attr_setstacksize(&a, THREAD_STACK))
+ goto no_threads;
+ for (i = 0; i < threads; i++) {
+ if (pthread_create(&t, &a, pauser, NULL))
+ goto no_threads;
+ }
+ signal(SIGUSR1, no_op);
+ if (pthread_create(&t, NULL, dl_sleeper_tid, NULL))
+ goto no_threads;
+ if (fifo && (wait_for(5000, mv_state[0]) ||
+ pthread_create(&t, NULL, fifo_sleeper, NULL)))
+ goto no_threads;
+ for (;;)
+ pause();
+no_threads:
+ mv_state[0] = -2;
+ _exit(1);
+ }
+ return pid;
+}
+
+static int hold(int n);
+static void release_held(void);
+
+/*
+ * Have the cpu controller refuse a move after cpuset_can_attach()
+ * reserved room. cpuset_cancel_attach() must then give the room back.
+ *
+ * The process is pinned_mover() with a SCHED_FIFO thread: 2048 sleeping
+ * threads, a sleeping deadline thread of 30%, and the SCHED_FIFO thread.
+ * It moves from the root cgroup into P. cpuset_can_attach() reserves
+ * room in P for the deadline thread. With CONFIG_RT_GROUP_SCHED, the cpu
+ * controller then refuses the SCHED_FIFO thread: P has no real-time
+ * runtime.
+ *
+ * Then a thread of the test, late_setter(), changes the runtime of the
+ * deadline thread during 20 such moves. sched_setattr() waits for the
+ * end of a move in progress. It must return, and succeed, once the move
+ * is cancelled.
+ *
+ * Skip if the cpu controller accepts the move.
+ */
+static int move_cancelled(void)
+{
+ int with_p, with_r, ret = KSFT_PASS, i, err;
+ const char *name = "move_cancelled";
+ uint64_t start, len;
+ pid_t pid;
+
+ mv_state[0] = 0;
+ mv_state[3] = 0;
+ mv_state[4] = 0;
+ mv_state[5] = 0;
+ pid = pinned_mover(2048, 1);
+ if (pid < 0 || child_up(name, 0) || wait_for(5000, mv_state[5])) {
+ reap(pid);
+ return mv_state[0] == -2 ? KSFT_SKIP : KSFT_FAIL;
+ }
+ if (mv_state[5] < 0 || write_file(root, "cgroup.subtree_control", "+cpu")) {
+ ksft_print_msg("%s: no SCHED_FIFO thread or no cpu controller\n", name);
+ reap(pid);
+ return KSFT_SKIP;
+ }
+ /* The reference: the deadline thread asleep in the root cgroup. */
+ sleep_ns(2 * PERIOD);
+ with_p = probe(cg_p);
+ with_r = probe(root);
+ start = now_ns();
+ err = enter(cg_p, pid);
+ len = now_ns() - start;
+ if (!err) {
+ ksft_print_msg("%s: the cpu controller accepts a SCHED_FIFO thread: needs CONFIG_RT_GROUP_SCHED\n",
+ name);
+ ret = KSFT_SKIP;
+ goto out;
+ }
+ for (i = 0; i < 5; i++) {
+ if (!enter(cg_p, pid)) {
+ ksft_print_msg("%s: a move accepted\n", name);
+ ret = KSFT_FAIL;
+ goto out;
+ }
+ }
+ sleep_ns(2 * PERIOD);
+ if (admits_in(cg_p, with_p, "P, after cancelled moves") ||
+ admits_in(root, with_r, "the root domain of the process")) {
+ ret = KSFT_FAIL;
+ goto out;
+ }
+
+ late.pid = mv_state[3];
+ for (i = 0; i < 20 && ret == KSFT_PASS; i++) {
+ pthread_t t;
+
+ late.go = 0;
+ late.done = 0;
+ late.runtime = (i % 2 ? 30 : 20) * NSEC_PER_MSEC;
+ late.delay = len * i / 20;
+ if (pthread_create(&t, NULL, late_setter, NULL)) {
+ ret = KSFT_FAIL;
+ break;
+ }
+ sleep_ns(NSEC_PER_MSEC);
+ late.go = 1;
+ if (!enter(cg_p, pid)) {
+ ksft_print_msg("%s: a move accepted\n", name);
+ ret = KSFT_FAIL;
+ }
+ if (wait_for(5000, late.done)) {
+ ksft_print_msg("%s: sched_setattr() still waits after a cancelled move\n",
+ name);
+ ret = KSFT_FAIL;
+ break; /* the thread cannot be joined */
+ }
+ join(t);
+ if (late.ret) {
+ ksft_print_msg("%s: sched_setattr: %s\n", name, strerror(-late.ret));
+ ret = KSFT_FAIL;
+ }
+ }
+ sleep_ns(3 * PERIOD);
+ if (ret == KSFT_PASS &&
+ (admits_in(cg_p, with_p, "P, after the cancelled moves") ||
+ admits_in(root, with_r, "the root domain of the process")))
+ ret = KSFT_FAIL;
+out:
+ reap(pid);
+ if (write_file(root, "cgroup.subtree_control", "-cpu"))
+ ksft_print_msg("%s: the cpu controller stays enabled\n", name);
+ return ret;
+}
+
+/* Deaths, in P */
+
+enum die_mode { DIE_RUNNING, DIE_SWITCHED_OUT, DIE_SHRUNK };
+
+/*
+ * Fork a deadline task of 30% that runs 5ms of a fresh period and exits
+ * before its 0-lag time. @mode says what it does before it exits:
+ * - DIE_RUNNING: nothing.
+ * - DIE_SWITCHED_OUT: switch to SCHED_OTHER.
+ * - DIE_SHRUNK: shrink to 10%. Its 0-lag time is then in the past.
+ *
+ * Its bandwidth is released at the 0-lag time, or at the exit for
+ * DIE_SHRUNK.
+ */
+static int die_case(enum die_mode mode, const char *what)
+{
+ pid_t pid = fork_child("a task that dies");
+ int status;
+
+ if (!pid) {
+ if (set_deadline(0, 30 * NSEC_PER_MSEC, PERIOD))
+ _exit(1);
+ sleep_ns(PERIOD);
+ spin_ns(5 * NSEC_PER_MSEC);
+ if (mode == DIE_SWITCHED_OUT && set_other(0))
+ _exit(2);
+ if (mode == DIE_SHRUNK && set_deadline(0, 10 * NSEC_PER_MSEC, PERIOD))
+ _exit(3);
+ _exit(0);
+ }
+ if (pid < 0 || waitpid(pid, &status, 0) != pid || !WIFEXITED(status) ||
+ WEXITSTATUS(status)) {
+ ksft_print_msg("a task that dies %s failed\n", what);
+ return KSFT_FAIL;
+ }
+ return KSFT_PASS;
+}
+
+static int death_running(void)
+{
+ return die_case(DIE_RUNNING, "running");
+}
+
+static int death_switched_out(void)
+{
+ return die_case(DIE_SWITCHED_OUT, "switched out");
+}
+
+static int death_shrunk(void)
+{
+ return die_case(DIE_SHRUNK, "shrunk");
+}
+
+/*
+ * Stop a running deadline task of 30% with SIGSTOP, switch it to
+ * SCHED_OTHER while it is stopped, and kill it. The task is on no
+ * runqueue when it leaves SCHED_DEADLINE. Do it five times.
+ */
+static int stopped_switched_out(void)
+{
+ const char *what = "a deadline task to stop";
+ int i, err;
+
+ for (i = 0; i < 5; i++) {
+ pid_t pid;
+
+ mv_state[0] = 0;
+ pid = fork_child(what);
+ if (!pid) {
+ if (set_deadline(0, 30 * NSEC_PER_MSEC, PERIOD)) {
+ mv_state[0] = -1;
+ _exit(1);
+ }
+ mv_state[0] = 1;
+ for (;;)
+ spin_ns(NSEC_PER_MSEC);
+ }
+ if (pid < 0 || child_up(what, 0)) {
+ reap(pid);
+ return KSFT_FAIL;
+ }
+ err = 0;
+ if (kill(pid, SIGSTOP) || wait_for(1000, proc_state_of(pid) == 'T')) {
+ ksft_print_msg("the deadline task did not stop\n");
+ err = 1;
+ } else if (set_other(pid)) {
+ ksft_print_msg("the stopped task cannot leave SCHED_DEADLINE\n");
+ err = 1;
+ }
+ reap(pid);
+ if (err)
+ return KSFT_FAIL;
+ }
+ return KSFT_PASS;
+}
+
+/*
+ * With @moving, wait until the spinner's count in mv_state[1] moves: it
+ * runs. Without, wait until the count stops for 5ms: it is throttled.
+ * Return 0, or -1 after 1s.
+ */
+static int wait_spinner(int moving)
+{
+ uint64_t end = now_ns() + NSEC_PER_SEC;
+ int last = mv_state[1];
+
+ while (now_ns() < end) {
+ sleep_ns(moving ? NSEC_PER_MSEC : 5 * NSEC_PER_MSEC);
+ if (moving ? mv_state[1] != last : mv_state[1] == last)
+ return 0;
+ last = mv_state[1];
+ }
+ return -1;
+}
+
+enum kill_when { KILL_AFTER, KILL_THROTTLED, KILL_RUNNING };
+
+/*
+ * Start a deadline task of 30% with mover() in @mode, then kill it as
+ * @when says:
+ * - KILL_AFTER: @after ns after it is up.
+ * - KILL_THROTTLED: once it is throttled.
+ * - KILL_RUNNING: while it runs.
+ *
+ * The killer first leaves P. It then watches and kills from another CPU,
+ * and does not take the task's CPU.
+ */
+static int kill_case(enum mv_mode mode, enum kill_when when, uint64_t after, const char *what)
+{
+ int ret = KSFT_PASS;
+ pid_t pid;
+
+ mv_state[0] = 0;
+ mv_state[1] = 0;
+ pid = mover(mode, 30 * NSEC_PER_MSEC);
+ if (pid < 0) {
+ ret = KSFT_FAIL;
+ goto out;
+ }
+ if (enter(root, getpid())) {
+ ksft_print_msg("killed %s: the killer cannot leave P\n", what);
+ ret = KSFT_FAIL;
+ goto out;
+ }
+ if (child_up(what, 1)) {
+ ret = KSFT_FAIL;
+ } else if (when != KILL_AFTER && wait_spinner(when == KILL_RUNNING)) {
+ ksft_print_msg("killed %s: never %s\n", what,
+ when == KILL_RUNNING ? "ran" : "throttled");
+ ret = KSFT_FAIL;
+ } else {
+ sleep_ns(after);
+ }
+out:
+ reap(pid);
+ return ret;
+}
+
+static int death_killed_asleep(void)
+{
+ return kill_case(MV_ASLEEP, KILL_AFTER, 2 * PERIOD, "asleep");
+}
+
+static int death_killed_non_contending(void)
+{
+ return kill_case(MV_NON_CONTENDING, KILL_AFTER, 0, "before its 0-lag time");
+}
+
+static int death_killed_throttled(void)
+{
+ return kill_case(MV_THROTTLED, KILL_THROTTLED, 0, "throttled");
+}
+
+static int death_killed_running(void)
+{
+ return kill_case(MV_THROTTLED, KILL_RUNNING, 0, "running");
+}
+
+/*
+ * A robust priority inheritance lock shared between processes.
+ * mv_state[0] holds the state of its owner, mv_state[2] that of its
+ * waiter.
+ */
+static pthread_mutex_t *shm_lock;
+
+/* Set up shm_lock. Return 0, or -1 if it cannot be set up. */
+static int shm_lock_init(void)
+{
+ pthread_mutexattr_t a;
+ int err;
+
+ err = pthread_mutexattr_init(&a);
+ if (!err)
+ err = pthread_mutexattr_setpshared(&a, PTHREAD_PROCESS_SHARED);
+ if (!err)
+ err = pthread_mutexattr_setprotocol(&a, PTHREAD_PRIO_INHERIT);
+ if (!err)
+ err = pthread_mutexattr_setrobust(&a, PTHREAD_MUTEX_ROBUST);
+ if (!err)
+ err = pthread_mutex_init(shm_lock, &a);
+ if (err) {
+ ksft_print_msg("no shared priority inheritance mutex: %s\n", strerror(err));
+ return -1;
+ }
+ mv_state[0] = 0;
+ mv_state[2] = 0;
+ return 0;
+}
+
+/*
+ * Fork a process that takes shm_lock and sleeps with it, as a deadline
+ * task of 30% if @dl. Return its pid, or -1.
+ */
+static pid_t lock_owner(int dl)
+{
+ pid_t pid = fork_child("the owner of the lock");
+
+ if (!pid) {
+ if ((dl && set_deadline(0, 30 * NSEC_PER_MSEC, PERIOD)) ||
+ pthread_mutex_lock(shm_lock)) {
+ mv_state[0] = -1;
+ _exit(1);
+ }
+ mv_state[0] = 1;
+ for (;;)
+ pause();
+ }
+ return pid;
+}
+
+/*
+ * Fork a deadline task of 5ms every 50ms that waits for shm_lock. It sets
+ * mv_state[2] to 1 before the wait, and to 2 once it got the lock. Return
+ * its pid, or -1.
+ */
+static pid_t lock_waiter(void)
+{
+ pid_t pid = fork_child("a waiter of the lock");
+
+ if (!pid) {
+ int err;
+
+ if (set_deadline(0, 5 * NSEC_PER_MSEC, 50 * NSEC_PER_MSEC)) {
+ mv_state[2] = -1;
+ _exit(1);
+ }
+ mv_state[2] = 1;
+ err = pthread_mutex_lock(shm_lock);
+ if (err == EOWNERDEAD)
+ err = pthread_mutex_consistent(shm_lock);
+ if (!err)
+ pthread_mutex_unlock(shm_lock);
+ mv_state[2] = err ? -2 : 2;
+ for (;;)
+ pause();
+ }
+ return pid;
+}
+
+/* Wait up to 1s for @var to reach @want. Return 0, or -1. */
+static int wait_state(atomic_int *var, int want)
+{
+ uint64_t end = now_ns() + NSEC_PER_SEC;
+
+ while (now_ns() < end) {
+ if (*var == want)
+ return 0;
+ sleep_ns(NSEC_PER_MSEC);
+ }
+ return -1;
+}
+
+/*
+ * Kill one side of a priority inheritance boost. lock_owner() sleeps
+ * with shm_lock, as a deadline task of 30% if @dl_owner. lock_waiter(),
+ * a deadline task of 5ms every 50ms, blocks on the lock and boosts the
+ * owner. Kill the owner if @kill_owner: the waiter must then get the
+ * lock. If not, kill the waiter.
+ *
+ * A killed owner does not die boosted. Its exit first releases its
+ * priority inheritance futexes in futex_exit_exec_release(), which
+ * deboosts it.
+ */
+static int pi_death(int dl_owner, int kill_owner, const char *what)
+{
+ int ret = KSFT_PASS;
+ pid_t owner, waiter = -1;
+
+ if (shm_lock_init())
+ return KSFT_SKIP;
+ owner = lock_owner(dl_owner);
+ if (owner < 0)
+ return KSFT_FAIL;
+ if (wait_state(&mv_state[0], 1)) {
+ ksft_print_msg("%s: no owner\n", what);
+ ret = KSFT_FAIL;
+ goto out;
+ }
+ waiter = lock_waiter();
+ if (waiter < 0) {
+ ret = KSFT_FAIL;
+ goto out;
+ }
+ if (wait_state(&mv_state[2], 1)) {
+ ksft_print_msg("%s: no waiter\n", what);
+ ret = KSFT_FAIL;
+ goto out;
+ }
+ sleep_ns(50 * NSEC_PER_MSEC); /* blocked, boosting the owner */
+ if (kill_owner) {
+ reap(owner);
+ owner = -1;
+ if (wait_state(&mv_state[2], 2)) {
+ ksft_print_msg("%s: the waiter did not get the lock\n", what);
+ ret = KSFT_FAIL;
+ }
+ } else {
+ reap(waiter);
+ waiter = -1;
+ sleep_ns(50 * NSEC_PER_MSEC); /* the owner deboosted asleep */
+ }
+out:
+ reap(waiter);
+ reap(owner);
+ return ret;
+}
+
+static int death_pi_waiter(void)
+{
+ return pi_death(0, 0, "a waiter killed");
+}
+
+static int death_pi_owner(void)
+{
+ return pi_death(0, 1, "a boosted owner killed");
+}
+
+static int death_pi_deadline_owner(void)
+{
+ return pi_death(1, 1, "a boosted deadline owner killed");
+}
+
+/*
+ * Run as thread @arg of death_group(), a deadline task with a period of
+ * 100ms. Each thread counts itself in mv_state[0] once it has run 1ms.
+ * - Thread 0, of 10ms: run 1ms, then sleep.
+ * - Thread 1, of 10ms: spin and count in mv_state[1], until throttled.
+ * - Thread 2, of 60ms: wait for the killer to set mv_state[2]. Then
+ * spin and count in mv_state[3].
+ */
+static void *group_thread(void *arg)
+{
+ long kind = (long)arg;
+
+ if (set_deadline(0, (kind == 2 ? 60 : 10) * NSEC_PER_MSEC, PERIOD)) {
+ mv_state[0] = -1000;
+ return NULL;
+ }
+ if (kind == 2)
+ while (!mv_state[2])
+ sleep_ns(NSEC_PER_MSEC);
+ spin_ns(NSEC_PER_MSEC);
+ mv_state[0]++;
+ if (!kind)
+ for (;;)
+ pause();
+ for (;;)
+ mv_state[kind == 2 ? 3 : 1]++;
+ return NULL;
+}
+
+/*
+ * Kill a process of three deadline threads with one SIGKILL, while one
+ * sleeps, one is throttled and one runs. See group_thread().
+ */
+static int death_group(void)
+{
+ int last, ret = KSFT_PASS;
+ pid_t pid;
+
+ mv_state[0] = mv_state[1] = mv_state[2] = mv_state[3] = 0;
+ pid = fork_child("a group of deadline threads");
+ if (pid < 0)
+ return KSFT_FAIL;
+ if (!pid) {
+ pthread_t t;
+ long i;
+
+ for (i = 0; i < 3; i++)
+ if (pthread_create(&t, NULL, group_thread, (void *)i))
+ mv_state[0] = -1000;
+ for (;;)
+ pause();
+ }
+ /*
+ * Kill from outside P. P has one CPU: in P, the killer could not
+ * run while thread 2 runs.
+ */
+ if (enter(root, getpid())) {
+ ksft_print_msg("the killer cannot leave P\n");
+ ret = KSFT_FAIL;
+ goto out;
+ }
+ if (wait_for(5000, mv_state[0] < 0 || mv_state[0] == 2) || mv_state[0] != 2) {
+ ksft_print_msg("only %d threads of the group started\n", (int)mv_state[0]);
+ ret = KSFT_FAIL;
+ goto out;
+ }
+ if (wait_spinner(0)) {
+ ksft_print_msg("the first spinner never throttled\n");
+ ret = KSFT_FAIL;
+ goto out;
+ }
+ last = mv_state[3];
+ mv_state[2] = 1;
+ if (wait_for(5000, mv_state[0] < 0 || mv_state[3] != last) || mv_state[0] != 3) {
+ ksft_print_msg("the second spinner never ran\n");
+ ret = KSFT_FAIL;
+ }
+out:
+ /* One asleep, one throttled, one running. */
+ reap(pid);
+ return ret;
+}
+
+/*
+ * Two deadline tasks are in P. One, of 30%, sleeps. The other, of 10%,
+ * exits and stays a zombie. Switching the zombie to SCHED_OTHER must be
+ * refused with ESRCH.
+ *
+ * On kernels where each cpuset counts its deadline tasks,
+ * cgroup_task_dead() took the zombie off P's count. The switch would
+ * run switched_from_dl() and take it off a second time. P's count would
+ * then be 0 with the task of 30% still in P. A rebuild of the accounting
+ * would skip P, and lose the bandwidth of that task. So rebuild the
+ * accounting, and check that P still counts the 30%.
+ */
+static int death_zombie_switched(void)
+{
+ int base, ret = KSFT_PASS, err;
+ pid_t in_p, zombie;
+
+ base = probe(cg_p);
+ if (base < 4) {
+ ksft_print_msg("no room in P for a task of 30%% and a probe\n");
+ return KSFT_SKIP;
+ }
+ mv_state[0] = 0;
+ in_p = fork_child("a deadline task of P");
+ if (!in_p) {
+ if (enter(cg_p, getpid()) || set_deadline(0, 30 * NSEC_PER_MSEC, PERIOD))
+ _exit(1);
+ mv_state[0] = 1;
+ for (;;)
+ pause();
+ }
+ if (in_p < 0 || child_up("a task of 30% in P", 0)) {
+ reap(in_p);
+ return KSFT_FAIL;
+ }
+ mv_state[0] = 0;
+ zombie = fork_child("a deadline task that exits");
+ if (!zombie) {
+ if (enter(cg_p, getpid()) || set_deadline(0, 10 * NSEC_PER_MSEC, PERIOD))
+ _exit(1);
+ mv_state[0] = 1;
+ _exit(0);
+ }
+ if (zombie < 0 || child_up("a task of 10% in P that exits", 0) ||
+ wait_for(1000, proc_state_of(zombie) == 'Z')) {
+ ksft_print_msg("no zombie deadline task\n");
+ reap(zombie);
+ reap(in_p);
+ return KSFT_FAIL;
+ }
+ /* Its bandwidth is released at its 0-lag time, within a period. */
+ sleep_ns(3 * PERIOD);
+ err = set_other(zombie);
+ if (err != -ESRCH) {
+ ksft_print_msg("SCHED_OTHER on a zombie deadline task: %s, not ESRCH\n",
+ err ? strerror(-err) : "accepted");
+ ret = KSFT_FAIL;
+ }
+ /*
+ * Rebuild the accounting: make Q beside P, then remove it. Q is a
+ * second partition root, of the CPU below P's.
+ */
+ if (make_partition(cg_q, top_cpu(2))) {
+ ksft_print_msg("no partition root Q beside P\n");
+ ret = KSFT_FAIL;
+ }
+ remove_partition(cg_q);
+ if (admits_in(cg_p, base - 3, "P rebuilt, a task of 30% and a zombie in it"))
+ ret = KSFT_FAIL;
+ reap(zombie);
+ reap(in_p);
+ sleep_ns(3 * PERIOD);
+ if (ret == KSFT_PASS && admits_in(cg_p, base, "P, its tasks gone"))
+ ret = KSFT_FAIL;
+ return ret;
+}
+
+/* sched_yield() */
+
+static atomic_int y_rounds; /* the yields of the yielder */
+
+/*
+ * Run as the thread of start_yielder(): a deadline task of 30% that
+ * yields after 0.5ms of every period. sched_yield() throttles it until
+ * its next period, with runtime left. So it is throttled most of the
+ * time. t_state is 2 while it is in the yield.
+ */
+static void *yielder(void *arg)
+{
+ t_tid = gettid_();
+ if (set_deadline(0, 30 * NSEC_PER_MSEC, PERIOD)) {
+ t_state = -1;
+ return (void *)1;
+ }
+ t_state = 1;
+ while (!spin_stop) {
+ spin_ns(NSEC_PER_MSEC / 2);
+ y_rounds++;
+ t_state = 2;
+ sched_yield();
+ t_state = 1;
+ }
+ return NULL;
+}
+
+static int start_yielder(pthread_t *t)
+{
+ t_state = 0;
+ spin_stop = 0;
+ y_rounds = 0;
+ if (pthread_create(t, NULL, yielder, NULL)) {
+ ksft_print_msg("the yielder thread could not start\n");
+ return -1;
+ }
+ if (!wait_for(5000, t_state) && t_state > 0)
+ return 0;
+ spin_stop = 1; /* in case it gets there late */
+ join(*t);
+ return -1;
+}
+
+/*
+ * Switch a deadline task to SCHED_OTHER while sched_yield() throttles it.
+ * Its bandwidth must come back, and it must run again.
+ */
+static int yield_switched_out(void)
+{
+ int base, ret = KSFT_PASS;
+ pthread_t t;
+
+ base = probe(NULL);
+ if (base < 3) {
+ ksft_print_msg("no room for a task of 30%%\n");
+ return KSFT_SKIP;
+ }
+ if (start_yielder(&t))
+ return KSFT_FAIL;
+ if (wait_throttled(t) || t_state != 2) {
+ ksft_print_msg("the task never yielded\n");
+ ret = KSFT_FAIL;
+ } else if (set_other(t_tid)) {
+ ret = KSFT_FAIL;
+ } else {
+ int last = y_rounds;
+
+ sleep_ns(2 * PERIOD);
+ if (y_rounds == last) {
+ ksft_print_msg("yielded, then switched out: never ran again\n");
+ ret = KSFT_FAIL;
+ }
+ }
+ stop_spinner(t);
+ sleep_ns(3 * PERIOD); /* its 0-lag time */
+ if (ret == KSFT_PASS && admits(base, "yielded, then switched out"))
+ ret = KSFT_FAIL;
+ return ret;
+}
+
+/* Kill a deadline task while sched_yield() throttles it. */
+static int yield_killed(void)
+{
+ pid_t pid;
+
+ mv_state[0] = 0;
+ pid = fork_child("a task that yields");
+ if (!pid) {
+ if (set_deadline(0, 30 * NSEC_PER_MSEC, PERIOD)) {
+ mv_state[0] = -1;
+ _exit(1);
+ }
+ mv_state[0] = 1;
+ for (;;) {
+ spin_ns(NSEC_PER_MSEC / 2);
+ sched_yield();
+ }
+ }
+ if (pid < 0)
+ return KSFT_FAIL;
+ if (child_up("a task that yields", 0)) {
+ reap(pid);
+ return KSFT_FAIL;
+ }
+ sleep_ns(2 * PERIOD);
+ reap(pid);
+ return KSFT_PASS;
+}
+
+/* SCHED_FLAG_DL_OVERRUN */
+
+static atomic_int got_xcpu;
+
+static void catch_xcpu(int sig)
+{
+ got_xcpu = 1;
+}
+
+/*
+ * Run as the thread of overrun_caught(): a deadline task of 10% with
+ * SCHED_FLAG_DL_OVERRUN. Run 15ms, past the runtime, then sleep until
+ * spin_stop is set.
+ */
+static void *overruns(void *arg)
+{
+ t_tid = gettid_();
+ if (set_deadline_flags(0, 10 * NSEC_PER_MSEC, PERIOD, SCHED_FLAG_DL_OVERRUN)) {
+ t_state = -1;
+ return (void *)1;
+ }
+ t_state = 1;
+ spin_ns(15 * NSEC_PER_MSEC);
+ while (!spin_stop)
+ sleep_ns(10 * NSEC_PER_MSEC);
+ return NULL;
+}
+
+/*
+ * When a task with SCHED_FLAG_DL_OVERRUN overruns its runtime, its thread
+ * group gets SIGXCPU. Catch it here. The task's bandwidth must stay
+ * reserved after the overrun, and come back when the task leaves
+ * SCHED_DEADLINE.
+ */
+static int overrun_caught(void)
+{
+ int base, ret = KSFT_PASS;
+ uint64_t end;
+ pthread_t t;
+
+ if (signal(SIGXCPU, catch_xcpu) == SIG_ERR)
+ return KSFT_FAIL;
+ base = probe(NULL);
+ if (base < 1) {
+ ksft_print_msg("no room for a task of 10%%\n");
+ return KSFT_SKIP;
+ }
+ got_xcpu = 0;
+ t_state = 0;
+ spin_stop = 0;
+ if (pthread_create(&t, NULL, overruns, NULL))
+ return KSFT_FAIL;
+ while (!t_state)
+ sleep_ns(NSEC_PER_MSEC);
+ end = now_ns() + 2 * NSEC_PER_SEC;
+ while (!got_xcpu && now_ns() < end)
+ sleep_ns(5 * NSEC_PER_MSEC);
+ if (t_state != 1) {
+ ret = KSFT_FAIL;
+ } else if (!got_xcpu) {
+ ksft_print_msg("it overran its runtime, no SIGXCPU in 2s\n");
+ ret = KSFT_FAIL;
+ } else if (admits(base - 1, "an overrun told")) {
+ ret = KSFT_FAIL; /* the overrun gave its 10% back */
+ }
+ stop_spinner(t);
+ sleep_ns(3 * PERIOD); /* its 0-lag time */
+ if (ret == KSFT_PASS && admits(base, "after an overrun was told"))
+ ret = KSFT_FAIL;
+ return ret;
+}
+
+/*
+ * Run as thread @arg of overrun_group_death(), a deadline task. Thread 0
+ * overruns its runtime. Threads 1 and 2 count themselves in mv_state[0],
+ * and sleep.
+ */
+static void *og_thread(void *arg)
+{
+ switch ((int)(long)arg) {
+ case 0: /* overruns its runtime: SIGXCPU to the group */
+ if (set_deadline_flags(0, 10 * NSEC_PER_MSEC, PERIOD,
+ SCHED_FLAG_DL_OVERRUN))
+ _exit(1);
+ /*
+ * Wait for the other two first. On P's one CPU, this
+ * thread has the earliest deadline of the three. It would
+ * overrun, and kill them, before they reach the state the
+ * signal must find them in.
+ */
+ if (wait_for(5000, mv_state[0] >= 2))
+ _exit(2);
+ spin_ns(30 * NSEC_PER_MSEC);
+ break;
+ case 1: /* asleep, its 0-lag time past */
+ if (set_deadline(0, 20 * NSEC_PER_MSEC, PERIOD))
+ _exit(1);
+ spin_ns(NSEC_PER_MSEC);
+ mv_state[0]++;
+ break;
+ case 2: /* asleep, mostly before its 0-lag time */
+ if (set_deadline(0, 15 * NSEC_PER_MSEC, PERIOD))
+ _exit(1);
+ mv_state[0]++;
+ for (;;) {
+ spin_ns(3 * NSEC_PER_MSEC);
+ sleep_ns(22 * NSEC_PER_MSEC);
+ }
+ }
+ for (;;)
+ pause();
+}
+
+/*
+ * Let the SIGXCPU of an overrun kill the whole thread group. The
+ * bandwidth of its three deadline threads must come back.
+ */
+static int overrun_group_death(void)
+{
+ pid_t pid;
+ int i, status;
+
+ mv_state[0] = 0;
+ pid = fork_child("a group of deadline threads");
+ if (!pid) {
+ pthread_t t;
+
+ for (i = 0; i < 3; i++)
+ if (pthread_create(&t, NULL, og_thread, (void *)(long)i))
+ _exit(3);
+ for (;;)
+ pause();
+ }
+ if (pid < 0)
+ return KSFT_FAIL;
+ for (i = 0; i < 300; i++) {
+ if (waitpid(pid, &status, WNOHANG) == pid)
+ break;
+ sleep_ns(10 * NSEC_PER_MSEC);
+ }
+ if (i == 300) {
+ reap(pid);
+ ksft_print_msg("an overrun of 10ms every 100ms: no SIGXCPU in 3s\n");
+ return KSFT_FAIL;
+ }
+ if (WIFEXITED(status)) {
+ switch (WEXITSTATUS(status)) {
+ case 1:
+ ksft_print_msg("a thread of the group was refused SCHED_DEADLINE\n");
+ return KSFT_SKIP;
+ case 2:
+ ksft_print_msg("the other two threads of the group were not ready in 5s\n");
+ break;
+ default:
+ ksft_print_msg("a thread of the group could not start\n");
+ }
+ return KSFT_FAIL;
+ }
+ if (!WIFSIGNALED(status) || WTERMSIG(status) != SIGXCPU) {
+ ksft_print_msg("the group of an overrun died of signal %d, not SIGXCPU\n",
+ WIFSIGNALED(status) ? WTERMSIG(status) : 0);
+ return KSFT_FAIL;
+ }
+ return KSFT_PASS;
+}
+
+/* SCHED_FLAG_RECLAIM */
+
+#define RECLAIM_MIN 40 /* least % for a reclaiming task of 10% */
+#define PLAIN_MAX 25 /* most % for one without the flag */
+#define SHARE_SLACK 15 /* % two measures of a share may differ */
+#define SHARE_NS (400 * NSEC_PER_MSEC)
+
+/*
+ * Measure the CPU share of a spinning deadline task of 10%, on P's CPU,
+ * without SCHED_FLAG_RECLAIM and then with it. With the flag, the task
+ * runs past its runtime: it takes the bandwidth of its root domain that
+ * no deadline task uses. It must get at most PLAIN_MAX percent without
+ * the flag, and at least RECLAIM_MIN percent with it.
+ */
+static int reclaim_runs(void)
+{
+ int plain, grub, ret = KSFT_PASS;
+ pthread_t t;
+
+ if (start_spinner(&t, 10 * NSEC_PER_MSEC, 0)) {
+ ksft_print_msg("no room for a task of 10%%\n");
+ return KSFT_SKIP;
+ }
+ plain = share_of(t, SHARE_NS);
+ stop_spinner(t);
+ if (start_spinner(&t, 10 * NSEC_PER_MSEC, SCHED_FLAG_RECLAIM))
+ return KSFT_FAIL;
+ grub = share_of(t, SHARE_NS);
+ stop_spinner(t);
+ if (plain > PLAIN_MAX || grub < RECLAIM_MIN) {
+ ksft_print_msg("of a CPU, a task of 10%% got %d%%, with SCHED_FLAG_RECLAIM %d%%\n",
+ plain, grub);
+ ret = KSFT_FAIL;
+ }
+ return ret;
+}
+
+/*
+ * Switch a reclaiming task out of SCHED_DEADLINE before its 0-lag time,
+ * and back. Its bandwidth stays until the inactive timer releases it.
+ * Afterwards, the task must reclaim as much as before.
+ */
+static int reclaim_switched(void)
+{
+ int before, after = 0, r, ret = KSFT_PASS;
+ pthread_t t;
+
+ if (start_spinner(&t, 10 * NSEC_PER_MSEC, SCHED_FLAG_RECLAIM)) {
+ ksft_print_msg("no room for a task of 10%%\n");
+ return KSFT_SKIP;
+ }
+ before = share_of(t, SHARE_NS);
+ for (r = 0; r < 4; r++) {
+ if (set_other(t_tid)) {
+ ret = KSFT_FAIL;
+ break;
+ }
+ if (r & 1)
+ sleep_ns(5 * NSEC_PER_MSEC);
+ if (set_deadline_flags(t_tid, 10 * NSEC_PER_MSEC, PERIOD,
+ SCHED_FLAG_RECLAIM)) {
+ ksft_print_msg("round %d of 4: back to SCHED_DEADLINE refused\n",
+ r + 1);
+ ret = KSFT_FAIL;
+ break;
+ }
+ sleep_ns(PERIOD);
+ }
+ if (ret == KSFT_PASS)
+ after = share_of(t, SHARE_NS);
+ stop_spinner(t);
+ if (ret == KSFT_PASS &&
+ (after < before - SHARE_SLACK || after > before + SHARE_SLACK)) {
+ ksft_print_msg("reclaiming: %d%% of a CPU, %d%% after 4 round trips\n",
+ before, after);
+ ret = KSFT_FAIL;
+ }
+ return ret;
+}
+
+/* Deadline servers */
+
+#define SERVER_RUNTIME (100 * NSEC_PER_MSEC) /* the cases start at 10% */
+#define SERVER_PERIOD (1000 * NSEC_PER_MSEC)
+
+static const char server_dir[] = "/sys/kernel/debug/sched/fair_server";
+static uint64_t *server_found; /* each fair server's runtime and period */
+
+static int server_get(int cpu, const char *name, uint64_t *val)
+{
+ char path[96], buf[32];
+
+ snprintf(path, sizeof(path), "%s/cpu%d/%s", server_dir, cpu, name);
+ if (read_file(path, buf, sizeof(buf)))
+ return -1;
+ *val = strtoull(buf, NULL, 10);
+ return 0;
+}
+
+static int server_put(int cpu, const char *name, uint64_t val)
+{
+ char dir[96], buf[32];
+
+ snprintf(dir, sizeof(dir), "%s/cpu%d", server_dir, cpu);
+ snprintf(buf, sizeof(buf), "%llu", (unsigned long long)val);
+ return write_file(dir, name, buf);
+}
+
+/*
+ * Set the fair server of @cpu to @runtime every @period.
+ * Return 0, or -errno.
+ */
+static int server_set(int cpu, uint64_t runtime, uint64_t period)
+{
+ uint64_t cur;
+ int err;
+
+ if (!server_found || server_get(cpu, "period", &cur))
+ return -ENOENT;
+ /* Each write keeps the runtime within the period. */
+ if (runtime <= cur) {
+ err = server_put(cpu, "runtime", runtime);
+ return err ? err : server_put(cpu, "period", period);
+ }
+ err = server_put(cpu, "period", period);
+ return err ? err : server_put(cpu, "runtime", runtime);
+}
+
+/*
+ * Record the runtime and period of each fair server. Without debugfs,
+ * record none: the server cases are then skipped.
+ */
+static void server_init(void)
+{
+ int i;
+
+ server_found = calloc(2 * ncpus, sizeof(*server_found));
+ for (i = 0; server_found && i < ncpus; i++) {
+ if (server_get(online[i], "runtime", &server_found[2 * i]) ||
+ server_get(online[i], "period", &server_found[2 * i + 1])) {
+ free(server_found);
+ server_found = NULL;
+ ksft_print_msg("no fair server in debugfs: server cases skipped\n");
+ }
+ }
+}
+
+/* Put the fair servers back as the test found them. */
+static void server_restore(void)
+{
+ uint64_t runtime, period;
+ int i;
+
+ for (i = 0; server_found && i < ncpus; i++) {
+ int cpu = online[i], err;
+
+ if (server_get(cpu, "runtime", &runtime) || server_get(cpu, "period", &period))
+ continue;
+ if (runtime == server_found[2 * i] && period == server_found[2 * i + 1])
+ continue;
+ /*
+ * Retry for up to 1s. While departing tasks still hold
+ * their bandwidth, a root domain can refuse any change of a
+ * server, a decrease too. Their 0-lag times are a period or
+ * two away.
+ */
+ err = wait_for(1000, !server_set(cpu, server_found[2 * i],
+ server_found[2 * i + 1]));
+ if (err)
+ ksft_print_msg("the fair server of CPU %d stays changed\n", cpu);
+ }
+}
+
+/* Deadline threads of 10% every 100ms that sleep until released. */
+static pthread_t *held;
+static int nheld, held_err;
+static sem_t held_up, held_go;
+
+static void *holder(void *arg)
+{
+ int err = set_deadline(0, PROBE_RUNTIME, PROBE_PERIOD);
+
+ held_err = err;
+ if (sem_post(&held_up))
+ ksft_print_msg("sem_post: %s\n", strerror(errno));
+ if (!err)
+ sem_wait_sig(&held_go);
+ return NULL;
+}
+
+/*
+ * Start @n more holders. Return 0 once they are all admitted, or -1. The
+ * caller must call release_held(), on a failure too.
+ */
+static int hold(int n)
+{
+ int max = 16 * ncpus + 16, err;
+ pthread_attr_t a;
+
+ if (!held) {
+ held = calloc(max, sizeof(*held));
+ if (!held || sem_init(&held_up, 0, 0) || sem_init(&held_go, 0, 0)) {
+ ksft_print_msg("the holders cannot be set up\n");
+ return -1;
+ }
+ }
+ err = pthread_attr_init(&a);
+ if (!err)
+ err = pthread_attr_setstacksize(&a, THREAD_STACK);
+ if (err) {
+ ksft_print_msg("pthread_attr: %s\n", strerror(err));
+ return -1;
+ }
+ for (; n > 0; n--) {
+ if (nheld == max || pthread_create(&held[nheld], &a, holder, NULL)) {
+ ksft_print_msg("a holder thread could not start\n");
+ return -1;
+ }
+ if (sem_wait_for(&held_up, 10000)) {
+ ksft_print_msg("a holder did not answer\n");
+ nheld++; /* release_held() joins it */
+ return -1;
+ }
+ if (held_err) {
+ join(held[nheld]);
+ return -1;
+ }
+ nheld++;
+ }
+ return 0;
+}
+
+static void release_held(void)
+{
+ int i;
+
+ for (i = 0; i < nheld; i++)
+ if (sem_post(&held_go))
+ ksft_print_msg("sem_post: %s\n", strerror(errno));
+ for (i = 0; i < nheld; i++)
+ join(held[i]);
+ /* Drain the semaphore counts a late or refused holder left. */
+ while (held && (!sem_trywait(&held_up) || !sem_trywait(&held_go)))
+ ;
+ nheld = 0;
+ sleep_ns(3 * PERIOD); /* their 0-lag times */
+}
+
+/*
+ * Change the runtime and the period of the fair server of P's CPU. Its
+ * bandwidth comes out of what P's root domain has for deadline tasks.
+ * From 100ms every second (10%), P must admit two probe tasks fewer at
+ * 30%, and one fewer at 100ms every 500ms. With the server off, it must
+ * admit one more. A change that P has no room for must be refused with
+ * EBUSY, and leave the server as it was.
+ */
+static int server_params(void)
+{
+ static const struct {
+ uint64_t runtime, period;
+ int more;
+ const char *what;
+ } step[] = {
+ { 3 * SERVER_RUNTIME, SERVER_PERIOD, -2, "a fair server of 30%" },
+ { SERVER_RUNTIME, SERVER_PERIOD / 2, -1, "a fair server of 100ms every 500ms" },
+ { 0, SERVER_PERIOD, 1, "the fair server off" },
+ { SERVER_RUNTIME, SERVER_PERIOD, 0, "the fair server back to 10%" },
+ };
+ int cpu = top_cpu(1), base, err, i, ret = KSFT_PASS;
+ uint64_t runtime;
+
+ if (server_set(cpu, SERVER_RUNTIME, SERVER_PERIOD)) {
+ ksft_print_msg("the fair server of CPU %d cannot be set\n", cpu);
+ return KSFT_SKIP;
+ }
+ base = probe(NULL);
+ if (base < 2) {
+ ksft_print_msg("no room in P for two probe tasks\n");
+ return KSFT_SKIP;
+ }
+ for (i = 0; i < (int)ARRAY_SIZE(step); i++) {
+ err = server_set(cpu, step[i].runtime, step[i].period);
+ if (err) {
+ ksft_print_msg("%s: %s\n", step[i].what, strerror(-err));
+ return KSFT_FAIL;
+ }
+ if (admits(base + step[i].more, step[i].what))
+ return KSFT_FAIL;
+ }
+ /*
+ * With base - 1 probe tasks held, P has room for 10% more, and not
+ * for 20%. So the server may grow to 20%, and not to 30%.
+ */
+ if (hold(base - 1)) {
+ release_held();
+ return KSFT_FAIL;
+ }
+ err = server_put(cpu, "runtime", 2 * SERVER_RUNTIME);
+ if (err) {
+ ksft_print_msg("a fair server of 20%% with room for it: %s\n", strerror(-err));
+ ret = KSFT_FAIL;
+ } else {
+ err = server_put(cpu, "runtime", 3 * SERVER_RUNTIME);
+ if (err != -EBUSY) {
+ ksft_print_msg("a fair server of 30%% with no room for it: %s, not EBUSY\n",
+ err ? strerror(-err) : "accepted");
+ ret = KSFT_FAIL;
+ } else if (server_get(cpu, "runtime", &runtime) ||
+ runtime != 2 * SERVER_RUNTIME) {
+ ksft_print_msg("a refused change changed the fair server\n");
+ ret = KSFT_FAIL;
+ }
+ }
+ if (server_set(cpu, SERVER_RUNTIME, SERVER_PERIOD))
+ ret = KSFT_FAIL;
+ release_held();
+ if (ret == KSFT_PASS && admits(base, "after a refused change of the fair server"))
+ ret = KSFT_FAIL;
+ return ret;
+}
+
+/*
+ * A rebuild of the root domains' accounting must count each fair server
+ * with its current runtime, not its runtime at boot. Check it for the
+ * server of P's CPU, set to 30%:
+ * - In P, as Q is made and unmade beside P.
+ * - In the root domain of the other CPUs, once P is unmade.
+ * - In P again, once P is made anew.
+ */
+static int server_rebuilds(void)
+{
+ int cpu = top_cpu(1), base, n10, n30;
+
+ if (server_set(cpu, SERVER_RUNTIME, SERVER_PERIOD)) {
+ ksft_print_msg("the fair server of CPU %d cannot be set\n", cpu);
+ return KSFT_SKIP;
+ }
+ base = probe(cg_p);
+ if (base < 2) {
+ ksft_print_msg("no room in P for two probe tasks\n");
+ return KSFT_SKIP;
+ }
+ if (server_set(cpu, 3 * SERVER_RUNTIME, SERVER_PERIOD) ||
+ admits_in(cg_p, base - 2, "P, a fair server of 30%"))
+ return KSFT_FAIL;
+ if (make_partition(cg_q, top_cpu(2))) {
+ remove_partition(cg_q);
+ ksft_print_msg("no partition root Q beside P\n");
+ return KSFT_FAIL;
+ }
+ if (admits_in(cg_p, base - 2, "P, Q made beside it")) {
+ remove_partition(cg_q);
+ return KSFT_FAIL;
+ }
+ remove_partition(cg_q);
+ if (admits_in(cg_p, base - 2, "P, Q unmade beside it"))
+ return KSFT_FAIL;
+ /* Unmake P: its CPU and its server join the other CPUs. */
+ remove_p();
+ n30 = probe(root);
+ if (server_set(cpu, SERVER_RUNTIME, SERVER_PERIOD))
+ return KSFT_FAIL;
+ n10 = probe(root);
+ if (n10 != n30 + 2) {
+ ksft_print_msg("P unmade: %d probe tasks, its CPU's server at 30%%, %d at 10%%\n",
+ n30, n10);
+ return KSFT_FAIL;
+ }
+ if (server_set(cpu, 3 * SERVER_RUNTIME, SERVER_PERIOD) || make_p()) {
+ ksft_print_msg("P made anew, a fair server of 30%%: failed\n");
+ return KSFT_FAIL;
+ }
+ if (admits_in(cg_p, base - 2, "P made anew, a fair server of 30%") ||
+ server_set(cpu, SERVER_RUNTIME, SERVER_PERIOD) ||
+ admits_in(cg_p, base, "P, the fair server back to 10%"))
+ return KSFT_FAIL;
+ return KSFT_PASS;
+}
+
+/* Wait for the futex @f, in shared memory, to be set. */
+static void wait_set(atomic_int *f)
+{
+ while (!*f)
+ syscall(SYS_futex, f, FUTEX_WAIT, 0, NULL, NULL, 0);
+}
+
+static void set_flag(atomic_int *f, int val)
+{
+ *f = val;
+ syscall(SYS_futex, f, FUTEX_WAKE, INT_MAX, NULL, NULL, 0);
+}
+
+/*
+ * Fork a process of P that spins while mv_state[4 + @rt] is set, and
+ * sleeps otherwise. It is a fair task, or a real-time one if @rt. The
+ * real-time one sleeps 1ms every 50ms, to let the real-time kthreads of
+ * P's CPU run: PREEMPT_RT runs RCU and the timers there. Return the pid,
+ * or -1.
+ */
+static pid_t server_client(int rt)
+{
+ pid_t pid = fork_child(rt ? "a real-time task of P" : "a fair task of P");
+
+ if (!pid) {
+ struct sched_param sp = { .sched_priority = 1 };
+
+ if (enter(cg_p, getpid()) ||
+ (rt && sched_setscheduler(0, SCHED_FIFO, &sp)))
+ _exit(1);
+ mv_state[0]++;
+ for (;;) {
+ wait_set(&mv_state[4 + rt]);
+ spin_ns(rt ? 50 * NSEC_PER_MSEC : NSEC_PER_MSEC);
+ if (rt)
+ sleep_ns(NSEC_PER_MSEC);
+ }
+ }
+ return pid;
+}
+
+/* Return the CPU share of process @pid over @ns, in percent, or -1. */
+static int proc_share(pid_t pid, uint64_t ns)
+{
+ struct timespec t0, t1;
+ uint64_t wall = now_ns();
+ clockid_t c;
+
+ if (clock_getcpuclockid(pid, &c) || clock_gettime(c, &t0))
+ return -1;
+ sleep_ns(ns);
+ if (clock_gettime(c, &t1))
+ return -1;
+ wall = now_ns() - wall;
+ return (int)(100 * ((t1.tv_sec - t0.tv_sec) * NSEC_PER_SEC +
+ t1.tv_nsec - t0.tv_nsec) / wall);
+}
+
+/* % that the share of a starved task may differ from its server's bandwidth */
+#define STARVED_SLACK 8
+
+/*
+ * Change the fair server of P's CPU, from outside P, in each state a
+ * change can find it in:
+ * - stopped
+ * - deferred while a fair task of P runs
+ * - deferred while a real-time task starves that fair task
+ * - serving it
+ * - deferred again after serving it
+ * - deferred while the CPU idles
+ *
+ * A fair task and a SCHED_FIFO task of P, see server_client(), put the
+ * server in these states.
+ *
+ * Each change starts a period of the server. The server defers serving a
+ * starved task to its zero-laxity time. That is 0.7s into the period at
+ * 30%, and 0.9s at 10%. Afterwards, the starved task must get the
+ * server's bandwidth, at 30% then at 10%. P's free bandwidth must then be
+ * what it was.
+ *
+ * A change cannot find the server in two more states. Throttled after
+ * serving, the server is deferred anew at once: the last of the served
+ * task's runtime is charged to it as background running. And on a CPU
+ * with no tick in idle, its idle wait begins and ends in one timer
+ * callback.
+ */
+static int server_states(void)
+{
+ static const struct {
+ int fair, rt, ms;
+ } step[] = {
+ { 0, 0, 1500 }, /* stopped: nothing of P ran for a period */
+ { 1, 0, 300 }, /* deferred while the fair task runs */
+ { 1, 1, 300 }, /* deferred, fair task starved: before 0.9s */
+ { 1, 1, 800 }, /* serving it: after 0.7s */
+ { 1, 1, 1500 }, /* deferred again: served from 0.9s to 1s */
+ { 0, 0, 600 }, /* deferred while the CPU idles */
+ };
+ int cpu = top_cpu(1), base, i, s30 = -1, s10 = -1, ret = KSFT_PASS;
+ pid_t fair, rt;
+
+ if (server_set(cpu, SERVER_RUNTIME, SERVER_PERIOD)) {
+ ksft_print_msg("the fair server of CPU %d cannot be set\n", cpu);
+ return KSFT_SKIP;
+ }
+ base = probe(cg_p);
+ if (base <= 0) {
+ ksft_print_msg("no room in P to probe\n");
+ return KSFT_SKIP;
+ }
+ mv_state[0] = 0;
+ set_flag(&mv_state[4], 0);
+ set_flag(&mv_state[5], 0);
+ fair = server_client(0);
+ rt = server_client(1);
+ if (fair < 0 || rt < 0 || wait_for(5000, mv_state[0] >= 2)) {
+ if (fair >= 0 && rt >= 0)
+ ksft_print_msg("a fair or real-time task of P could not start\n");
+ ret = KSFT_FAIL;
+ }
+ /*
+ * Even steps set the server's runtime to 30%, odd ones back to 10%.
+ * The period stays.
+ */
+ for (i = 0; i < (int)ARRAY_SIZE(step) && ret == KSFT_PASS; i++) {
+ set_flag(&mv_state[4], step[i].fair);
+ set_flag(&mv_state[5], step[i].rt);
+ sleep_ns(step[i].ms * NSEC_PER_MSEC);
+ if (server_put(cpu, "runtime", (i & 1 ? 1 : 3) * SERVER_RUNTIME)) {
+ ksft_print_msg("change %d of the fair server refused\n", i + 1);
+ ret = KSFT_FAIL;
+ }
+ }
+ if (ret == KSFT_PASS) {
+ set_flag(&mv_state[4], 1);
+ set_flag(&mv_state[5], 1);
+ if (server_put(cpu, "runtime", 3 * SERVER_RUNTIME))
+ ret = KSFT_FAIL;
+ s30 = proc_share(fair, 2 * SERVER_PERIOD);
+ if (server_put(cpu, "runtime", SERVER_RUNTIME))
+ ret = KSFT_FAIL;
+ s10 = proc_share(fair, 2 * SERVER_PERIOD);
+ }
+ set_flag(&mv_state[5], 0);
+ set_flag(&mv_state[4], 0);
+ reap(rt);
+ reap(fair);
+ if (ret == KSFT_PASS &&
+ (s30 < 30 - STARVED_SLACK || s30 > 30 + STARVED_SLACK ||
+ s10 < 10 - STARVED_SLACK || s10 > 10 + STARVED_SLACK)) {
+ ksft_print_msg("a starved fair task got %d%% of a server of 30%%, %d%% of 10%%\n",
+ s30, s10);
+ ret = KSFT_FAIL;
+ }
+ sleep_ns(3 * PERIOD);
+ if (ret == KSFT_PASS && admits_in(cg_p, base, "after the fair server's changes"))
+ ret = KSFT_FAIL;
+ return ret;
+}
+
+/*
+ * A reclaiming deadline task of P gets the share of P's CPU that its root
+ * domain leaves free. P's fair server takes from that share. The task
+ * must get:
+ * - less once the server grows from 10% to 30%.
+ * - as much after a rebuild of the accounting (Q made and unmade).
+ * - what it had at first once the server is back at 10%.
+ *
+ * Then put the server back as the test found it, so that P's free
+ * bandwidth is what it was.
+ */
+static int server_reclaim(void)
+{
+ int cpu = top_cpu(1), s10, s30, rebuilt, back, ret = KSFT_PASS;
+ pthread_t t;
+
+ if (server_set(cpu, SERVER_RUNTIME, SERVER_PERIOD)) {
+ ksft_print_msg("the fair server of CPU %d cannot be set\n", cpu);
+ return KSFT_SKIP;
+ }
+ if (start_spinner(&t, 10 * NSEC_PER_MSEC, SCHED_FLAG_RECLAIM)) {
+ ksft_print_msg("no room for a task of 10%%\n");
+ return KSFT_SKIP;
+ }
+ s10 = share_of(t, SHARE_NS);
+ if (server_set(cpu, 3 * SERVER_RUNTIME, SERVER_PERIOD))
+ ret = KSFT_FAIL;
+ s30 = share_of(t, SHARE_NS);
+ if (make_partition(cg_q, top_cpu(2)))
+ ret = KSFT_FAIL;
+ remove_partition(cg_q);
+ rebuilt = share_of(t, SHARE_NS);
+ if (server_set(cpu, SERVER_RUNTIME, SERVER_PERIOD))
+ ret = KSFT_FAIL;
+ back = share_of(t, SHARE_NS);
+ stop_spinner(t);
+ server_restore();
+ if (ret == KSFT_PASS &&
+ (s30 > s10 - SHARE_SLACK ||
+ rebuilt < s30 - SHARE_SLACK || rebuilt > s30 + SHARE_SLACK ||
+ back < s10 - SHARE_SLACK || back > s10 + SHARE_SLACK)) {
+ ksft_print_msg("reclaiming: %d%%, %d%% at 30%%, %d%% rebuilt, %d%% back at 10%%\n",
+ s10, s30, rebuilt, back);
+ ret = KSFT_FAIL;
+ }
+ return ret;
+}
+
+/* Parameter changes while a task wakes on another CPU */
+
+#define WAKE_NS (5 * NSEC_PER_SEC) /* how long the changes go on */
+#define WAKE_BURST 2000 /* pairs of changes per clock reading */
+
+/* Read this_bw and running_bw of @cpu from sched_debug. Return 0, or -1. */
+static int rq_bw_of(int cpu, long *this_bw, long *running_bw)
+{
+ char line[256], want[32];
+ FILE *f = fopen("/sys/kernel/debug/sched/debug", "r");
+ int in = 0, got = 0;
+
+ if (!f)
+ return -1;
+ snprintf(want, sizeof(want), "dl_rq[%d]:", cpu);
+ while (got < 2 && fgets(line, sizeof(line), f)) {
+ char *val = strchr(line, ':');
+
+ if (!strncmp(line, want, strlen(want)))
+ in = 1;
+ else if (in && !strncmp(line, "dl_rq[", 6))
+ break; /* the next CPU's */
+ else if (!in || !val)
+ continue;
+ else if (strstr(line, ".running_bw") && sscanf(val + 1, "%ld", running_bw) == 1)
+ got++;
+ else if (strstr(line, ".this_bw") && sscanf(val + 1, "%ld", this_bw) == 1)
+ got++;
+ }
+ fclose(f);
+ return got == 2 ? 0 : -1;
+}
+
+static atomic_int *wk_state; /* 0: tasks up, 1: wakeups, 2: stop */
+
+/*
+ * Fork a deadline task of @runtime every @period in Q. It counts itself
+ * in wk_state[0], then spins. With @sleeps, it instead runs 20us and
+ * sleeps 80us in a loop, and counts its wakeups in wk_state[1]. It exits
+ * once wk_state[2] is set. Return the pid, or -1.
+ */
+static pid_t waking_task(uint64_t runtime, uint64_t period, int sleeps)
+{
+ pid_t pid = fork_child("a waking deadline task");
+
+ if (!pid) {
+ if (enter(cg_q, getpid()) || set_deadline(0, runtime, period)) {
+ wk_state[0] = -1;
+ _exit(1);
+ }
+ wk_state[0]++;
+ for (;;) {
+ if (wk_state[2])
+ _exit(0);
+ spin_ns(sleeps ? 20 * NSEC_PER_USEC : NSEC_PER_MSEC);
+ if (sleeps) {
+ sleep_ns(80 * NSEC_PER_USEC);
+ wk_state[1]++;
+ }
+ }
+ }
+ return pid;
+}
+
+/*
+ * Fork a process that changes the parameters of @pid over and over, until
+ * wk_state[2] is set. Return its pid, or -1.
+ */
+static pid_t changer_proc(pid_t pid)
+{
+ pid_t self = fork_child("a changer of the parameters");
+
+ if (!self) {
+ while (!wk_state[2]) {
+ if (set_deadline(pid, 20 * NSEC_PER_MSEC, PERIOD) ||
+ set_deadline(pid, 10 * NSEC_PER_MSEC, PERIOD))
+ break;
+ }
+ _exit(0);
+ }
+ return self;
+}
+
+/*
+ * Change the parameters of a deadline task while its wakeups move it
+ * between two CPUs.
+ *
+ * Q is a partition root of two CPUs. Three deadline tasks run in Q, see
+ * waking_task(). Two of 20ms every 50ms spin, and hold both CPUs with
+ * earlier deadlines. The third is a task of 10ms every 100ms. It runs
+ * 20us and sleeps 80us in a loop. It wakes on whichever of the two CPUs
+ * has the later deadline. So a wakeup can move it to the other CPU.
+ *
+ * Two processes outside Q change the runtime of the third task for 5s:
+ * the test and changer_proc(). Each switches it between 20ms and 10ms in
+ * a loop.
+ *
+ * A wakeup that moves a task takes its bandwidth off the runqueue it
+ * slept on. The enqueue on the new runqueue adds it there, and may be
+ * left to the CPU the task wakes on. Between the two, no runqueue counts
+ * the bandwidth. A change in that window must not change this_bw of
+ * either runqueue. Once the tasks of the case are gone, this_bw of both
+ * CPUs must be what it was before.
+ *
+ * Check this_bw only then. sched_debug prints running_bw and this_bw of
+ * a CPU one after the other, with no lock held in between. Every
+ * parameter change takes the bandwidth of a task off its runqueue and
+ * puts it back. So a reading taken during the changes can straddle one
+ * of them.
+ */
+static int params_waking(void)
+{
+ int cpu_a = top_cpu(3), cpu_b = top_cpu(2), i, ret = KSFT_PASS, changes = 0;
+ long ta, ra, tb, rb, ta2, ra2, tb2, rb2;
+ pid_t pid[3] = { -1, -1, -1 }, changer = -1;
+ uint64_t end;
+ char cpus[32];
+
+ if (ncpus < 4) {
+ ksft_print_msg("needs 4 non-isolated CPUs\n");
+ return KSFT_SKIP;
+ }
+ snprintf(cpus, sizeof(cpus), "%d,%d", cpu_a, cpu_b);
+ if (make_partition_cpus(cg_q, cpus)) {
+ remove_partition(cg_q);
+ ksft_print_msg("no partition root of two CPUs\n");
+ return KSFT_SKIP;
+ }
+ if (rq_bw_of(cpu_a, &ta, &ra) || rq_bw_of(cpu_b, &tb, &rb)) {
+ remove_partition(cg_q);
+ ksft_print_msg("sched_debug tells of neither CPU's this_bw\n");
+ return KSFT_SKIP;
+ }
+ wk_state = mv_state + 8;
+ wk_state[0] = wk_state[1] = wk_state[2] = 0;
+ /* Two of 40% hold both CPUs with an earlier deadline than the third. */
+ pid[0] = waking_task(20 * NSEC_PER_MSEC, 50 * NSEC_PER_MSEC, 0);
+ pid[1] = waking_task(20 * NSEC_PER_MSEC, 50 * NSEC_PER_MSEC, 0);
+ pid[2] = waking_task(10 * NSEC_PER_MSEC, PERIOD, 1);
+ if (pid[0] < 0 || pid[1] < 0 || pid[2] < 0) {
+ ret = KSFT_FAIL;
+ goto out;
+ }
+ wait_for(5000, wk_state[0] >= 3 || wk_state[0] < 0);
+ if (wk_state[0] != 3) {
+ ksft_print_msg("no room for the deadline tasks of a partition of two CPUs\n");
+ ret = KSFT_SKIP;
+ goto out;
+ }
+ changer = changer_proc(pid[2]);
+ if (changer < 0) {
+ ret = KSFT_FAIL;
+ goto out;
+ }
+ end = now_ns() + WAKE_NS;
+ while (now_ns() < end) {
+ for (i = 0; i < WAKE_BURST; i++, changes += 2) {
+ if (set_deadline(pid[2], 20 * NSEC_PER_MSEC, PERIOD) ||
+ set_deadline(pid[2], 10 * NSEC_PER_MSEC, PERIOD)) {
+ ksft_print_msg("a change of the waking task refused\n");
+ ret = KSFT_FAIL;
+ goto out;
+ }
+ }
+ }
+ if (!wk_state[1]) {
+ ksft_print_msg("the waking task never woke\n");
+ ret = KSFT_FAIL;
+ }
+out:
+ wk_state[2] = 1;
+ reap(changer);
+ for (i = 0; i < 3; i++)
+ reap(pid[i]);
+ sleep_ns(3 * PERIOD);
+ if (ret == KSFT_PASS) {
+ if (rq_bw_of(cpu_a, &ta2, &ra2) || rq_bw_of(cpu_b, &tb2, &rb2)) {
+ ret = KSFT_FAIL;
+ } else if (ta2 != ta || tb2 != tb) {
+ ksft_print_msg("%d changes: this_bw %ld and %ld, was %ld and %ld\n",
+ changes, ta2, tb2, ta, tb);
+ ret = KSFT_FAIL;
+ }
+ }
+ remove_partition(cg_q);
+ return ret;
+}
+
+/* CPU hotplug, from the main process, with no P */
+
+static int hp_cpu = -1;
+
+/* Return whether the online file of @cpu can be written. */
+static int cpu_hotpluggable(int cpu)
+{
+ char path[64];
+
+ snprintf(path, sizeof(path), "/sys/devices/system/cpu/cpu%d/online", cpu);
+ return !access(path, W_OK);
+}
+
+static int cpu_online(int cpu, int on)
+{
+ char dir[64];
+
+ snprintf(dir, sizeof(dir), "/sys/devices/system/cpu/cpu%d", cpu);
+ return write_file(dir, "online", on ? "1" : "0");
+}
+
+/* Bring @cpu back online: every case that follows needs it. */
+static void cpu_up(int cpu)
+{
+ int err = cpu_online(cpu, 1);
+
+ if (err)
+ ksft_print_msg("CPU %d cannot come back online: %s\n", cpu, strerror(-err));
+}
+
+/*
+ * Deadline sleepers take most of the free bandwidth. Taking hp_cpu
+ * offline must then fail with EBUSY. Once the sleepers are gone, the
+ * housekeeping root domain must admit what it did before.
+ */
+static int hotplug_refused(void)
+{
+ int base = probe(NULL), n, i, ret = KSFT_PASS, err;
+ pid_t *pid;
+
+ if (base < 12) {
+ ksft_print_msg("needs room for 12 probe tasks\n");
+ return KSFT_SKIP;
+ }
+ /*
+ * The sleepers of 30% leave 30% to 60% of a CPU free. A CPU that
+ * goes offline takes more than that, so the offline must fail.
+ */
+ n = base / 3 - 1;
+ pid = calloc(n, sizeof(*pid));
+ if (!pid) {
+ ksft_print_msg("calloc: %s\n", strerror(errno));
+ return KSFT_FAIL;
+ }
+ for (i = 0; i < n; i++) {
+ mv_state[0] = 0;
+ pid[i] = mover(MV_ASLEEP, 30 * NSEC_PER_MSEC);
+ if (pid[i] < 0 || child_up("a deadline sleeper", 0)) {
+ while (i >= 0)
+ reap(pid[i--]);
+ free(pid);
+ return KSFT_FAIL;
+ }
+ }
+ err = cpu_online(hp_cpu, 0);
+ if (!err) {
+ ksft_print_msg("CPU %d went offline with its deadline bandwidth in use\n", hp_cpu);
+ cpu_up(hp_cpu);
+ ret = KSFT_FAIL;
+ } else if (err != -EBUSY) {
+ ksft_print_msg("offline refused with %s, not EBUSY\n", strerror(-err));
+ ret = KSFT_FAIL;
+ }
+ for (i = 0; i < n; i++)
+ reap(pid[i]);
+ free(pid);
+ sleep_ns(3 * PERIOD);
+ if (ret == KSFT_PASS && probe(NULL) != base) {
+ ksft_print_msg("a refused offline changed the free bandwidth\n");
+ ret = KSFT_FAIL;
+ }
+ return ret;
+}
+
+/*
+ * Read dl_bw->total_bw of @cpu from sched_debug: the bandwidth that the
+ * root domain of @cpu counts. Return it, or -1.
+ */
+static long long total_bw_of(int cpu)
+{
+ char line[256], want[32];
+ FILE *f = fopen("/sys/kernel/debug/sched/debug", "r");
+ long long bw = -1;
+ int in = 0;
+
+ if (!f)
+ return -1;
+ snprintf(want, sizeof(want), "dl_rq[%d]:", cpu);
+ while (fgets(line, sizeof(line), f)) {
+ char *val = strchr(line, ':');
+
+ if (!strncmp(line, want, strlen(want)))
+ in = 1;
+ else if (in && !strncmp(line, "dl_rq[", 6))
+ break; /* the next CPU's */
+ else if (in && val && strstr(line, ".dl_bw->total_bw") &&
+ sscanf(val + 1, "%lld", &bw) == 1)
+ break;
+ }
+ fclose(f);
+ return bw;
+}
+
+/*
+ * The bandwidth of a fair server must leave the root domain with its CPU
+ * and come back with it, whatever its size. Probe the housekeeping root
+ * domain with:
+ * - the server of hp_cpu at 10%, the CPU online: the base;
+ * - the server at 10%, the CPU offline;
+ * - the server at 30%, the CPU online: two probe tasks fewer than the
+ * base;
+ * - the server at 30%, the CPU offline: as many as at 10%, offline.
+ * With the CPU back and its server at 30%, the root domain must admit two
+ * fewer than the base again. With the server back at 10%, it must admit
+ * the base.
+ */
+static int hotplug_server(void)
+{
+ int base, off10, off30;
+
+ if (server_set(hp_cpu, SERVER_RUNTIME, SERVER_PERIOD)) {
+ ksft_print_msg("the fair server of CPU %d cannot be set\n", hp_cpu);
+ return KSFT_SKIP;
+ }
+ base = probe(NULL);
+ if (cpu_online(hp_cpu, 0)) {
+ ksft_print_msg("offline of CPU %d refused\n", hp_cpu);
+ return KSFT_FAIL;
+ }
+ off10 = probe(NULL);
+ cpu_up(hp_cpu);
+ if (server_set(hp_cpu, 3 * SERVER_RUNTIME, SERVER_PERIOD) ||
+ admits(base - 2, "a fair server of 30%"))
+ return KSFT_FAIL;
+ if (cpu_online(hp_cpu, 0)) {
+ ksft_print_msg("offline of CPU %d refused, its fair server at 30%%\n", hp_cpu);
+ return KSFT_FAIL;
+ }
+ off30 = probe(NULL);
+ cpu_up(hp_cpu);
+ if (off30 != off10) {
+ ksft_print_msg("CPU %d offline: %d probe tasks, its server at 10%%, %d at 30%%\n",
+ hp_cpu, off10, off30);
+ return KSFT_FAIL;
+ }
+ if (admits(base - 2, "a fair server of 30%, its CPU back online") ||
+ server_set(hp_cpu, SERVER_RUNTIME, SERVER_PERIOD) ||
+ admits(base, "the fair server back to 10%"))
+ return KSFT_FAIL;
+ return KSFT_PASS;
+}
+
+/* Return whether @cpu is in @list, a CPU list as sysfs prints it. */
+static int in_cpu_list(const char *list, int cpu)
+{
+ const char *p = list;
+
+ while (*p >= '0' && *p <= '9') {
+ char *end;
+ long lo = strtol(p, &end, 10), hi = lo;
+
+ if (*end == '-')
+ hi = strtol(end + 1, &end, 10);
+ if (cpu >= lo && cpu <= hi)
+ return 1;
+ p = *end == ',' ? end + 1 : end;
+ }
+ return 0;
+}
+
+/*
+ * Wake @pid, a task that sleeps in read() on a pipe: write a byte to @fd.
+ * The task counts its wakeups in mv_state[1]. Wait until it has run and
+ * sleeps again. Return 0, or -1.
+ */
+static int fallback_wake(int fd, pid_t pid)
+{
+ int n = mv_state[1];
+
+ if (write(fd, "w", 1) != 1) {
+ ksft_print_msg("write to the deadline task: %s\n", strerror(errno));
+ return -1;
+ }
+ if (wait_for(1000, mv_state[1] != n) ||
+ wait_for(1000, proc_state_of(pid) == 'S')) {
+ ksft_print_msg("the deadline task did not wake up and sleep again\n");
+ return -1;
+ }
+ return 0;
+}
+
+/*
+ * The cpuset of the cases below: Q in the root cgroup. cs_setup() names it
+ * and enables the cpuset controller of the root cgroup if it is off.
+ * cs_cleanup() removes Q, and turns the controller off again if
+ * cs_setup() turned it on.
+ */
+static char cs_q[PATH_MAX];
+static int cs_added;
+
+static int cs_setup(void)
+{
+ char path[PATH_MAX + 32], buf[256];
+
+ cs_added = 0;
+ if (!root[0] ||
+ snprintf(cs_q, sizeof(cs_q), "%s/deadline_bw_q", root) >= (int)sizeof(cs_q))
+ return -1;
+ snprintf(path, sizeof(path), "%s/cgroup.subtree_control", root);
+ if (read_file(path, buf, sizeof(buf)))
+ return -1;
+ if (!strstr(buf, "cpuset")) {
+ if (write_file(root, "cgroup.subtree_control", "+cpuset"))
+ return -1;
+ cs_added = 1;
+ }
+ return 0;
+}
+
+static void cs_cleanup(void)
+{
+ remove_partition(cs_q);
+ if (cs_added && write_file(root, "cgroup.subtree_control", "-cpuset"))
+ ksft_print_msg("the cpuset controller stays enabled\n");
+}
+
+/*
+ * Return whether the partition file of @cg holds @what. If not, and
+ * @tell, print what it holds.
+ */
+static int partition_has(const char *cg, const char *what, int tell)
+{
+ char path[PATH_MAX + 32], buf[256];
+
+ snprintf(path, sizeof(path), "%s/cpuset.cpus.partition", cg);
+ if (read_file(path, buf, sizeof(buf)))
+ return 0;
+ if (strstr(buf, what))
+ return 1;
+ if (tell)
+ ksft_print_msg("%s/cpuset.cpus.partition: %s", cg, buf);
+ return 0;
+}
+
+/*
+ * Read total_bw of the root domain of @d into @def, and of the root
+ * domain of @h into @hk. Return 0, or -1.
+ */
+static int totals_of(int d, int h, long long *def, long long *hk)
+{
+ *def = total_bw_of(d);
+ *hk = total_bw_of(h);
+ if (*def == -1 || *hk == -1) {
+ ksft_print_msg("no total_bw in /sys/kernel/debug/sched/debug\n");
+ return -1;
+ }
+ return 0;
+}
+
+/*
+ * Return KSFT_PASS if total_bw of the root domains of @d and @h is
+ * @def and @hk, else KSFT_FAIL. @what names the step in the message.
+ */
+static int totals_check(const char *what, int d, int h, long long def, long long hk)
+{
+ long long now_def, now_hk;
+
+ if (totals_of(d, h, &now_def, &now_hk))
+ return KSFT_FAIL;
+ if (now_def == def && now_hk == hk)
+ return KSFT_PASS;
+ ksft_print_msg("%s: total_bw %lld, %lld, not %lld, %lld\n",
+ what, now_def, now_hk, def, hk);
+ return KSFT_FAIL;
+}
+
+/*
+ * A rebuild must count a task where its cpuset lets it run. The kernel
+ * records, with the bandwidth of a task, a CPU of the root domain that
+ * counts it. A partition root can take that CPU later. A, B and D are the
+ * three highest housekeeping CPUs. C is the first one. Q is a partition
+ * root of the other housekeeping CPUs, C among them. The steps:
+ * 1. Take D offline. The housekeeping root domain then has A and B only.
+ * 2. A task in the root cgroup sets its affinity to A and B. It becomes a
+ * deadline task of 10% and sleeps. Bring D back online. The task keeps
+ * its affinity.
+ * 3. Take A and B offline. No CPU of the task's mask is active. Its
+ * cpuset gives it D, so the rebuild counts it on D. D is recorded.
+ * 4. Make Q a member, then a partition root of D instead of C. The
+ * cpuset of the task now gives it C. Its mask is still A and B.
+ * Q must not count the task. The root domain of C must count it. Wake
+ * the task: it must run on C, and total_bw must not change.
+ *
+ * The references of total_bw are taken once the task is gone, with the
+ * CPUs as in step 4.
+ */
+static int rebuild_record_taken(void)
+{
+ int a = top_cpu(3), b = top_cpu(2), c = online[0], d = top_cpu(1);
+ int fd[2] = { -1, -1 }, i, n, err, cpu, ret = KSFT_SKIP;
+ long long q, h, q0, h0, bw;
+ char buf[4096];
+ pid_t pid = -1;
+
+ if (ncpus < 4 || !cpu_hotpluggable(a) || !cpu_hotpluggable(b) ||
+ !cpu_hotpluggable(d)) {
+ ksft_print_msg("needs 4 CPUs, 3 of them hotpluggable\n");
+ return KSFT_SKIP;
+ }
+ if (cs_setup())
+ return KSFT_SKIP;
+ /* Q has every housekeeping CPU but A, B and D. */
+ for (i = n = 0; i < ncpus - 3; i++)
+ n += snprintf(buf + n, sizeof(buf) - n, "%s%d", n ? "," : "",
+ online[i]);
+ if (make_partition_cpus(cs_q, buf)) {
+ ksft_print_msg("Q cannot be a partition root of CPUs %s\n", buf);
+ goto out;
+ }
+ ret = KSFT_FAIL;
+ err = cpu_online(d, 0);
+ if (err) {
+ ksft_print_msg("offline of CPU %d: %s\n", d, strerror(-err));
+ goto out;
+ }
+ bw = total_bw_of(a);
+ if (bw == -1 || pipe(fd)) {
+ ksft_print_msg("no total_bw of CPU %d, or no pipe\n", a);
+ goto out;
+ }
+ mv_state[0] = 0;
+ mv_state[1] = 0;
+ pid = fork_child("a deadline task");
+ if (!pid) {
+ cpu_set_t set;
+ char ch;
+
+ CPU_ZERO(&set);
+ CPU_SET(a, &set);
+ CPU_SET(b, &set);
+ if (sched_setaffinity(0, sizeof(set), &set) ||
+ set_deadline(0, 10 * NSEC_PER_MSEC, PERIOD)) {
+ mv_state[0] = -1;
+ _exit(1);
+ }
+ mv_state[0] = 1;
+ while (read(fd[0], &ch, 1) == 1)
+ mv_state[1]++;
+ _exit(0);
+ }
+ if (pid < 0 || child_up("rebuild_record_taken", 0) ||
+ wait_for(1000, proc_state_of(pid) == 'S'))
+ goto out;
+ bw = total_bw_of(a) - bw;
+ if (bw <= 0) {
+ ksft_print_msg("the deadline task adds %lld to total_bw\n", bw);
+ goto out;
+ }
+ err = cpu_online(d, 1);
+ if (!err)
+ err = cpu_online(a, 0);
+ if (!err)
+ err = cpu_online(b, 0);
+ if (err) {
+ ksft_print_msg("hotplug of CPUs %d, %d, %d: %s\n", d, a, b,
+ strerror(-err));
+ goto out;
+ }
+ /*
+ * Q has D and every CPU it had but C. One write of cpuset.cpus
+ * would make Q an invalid partition root: the check sees the root
+ * cgroup lose D, and get C back too late.
+ */
+ for (i = 1, n = 0; i < ncpus - 3; i++)
+ n += snprintf(buf + n, sizeof(buf) - n, "%d,", online[i]);
+ snprintf(buf + n, sizeof(buf) - n, "%d", d);
+ err = write_file(cs_q, "cpuset.cpus.partition", "member");
+ if (err || make_partition_cpus(cs_q, buf)) {
+ ksft_print_msg("Q cannot be a partition root of CPUs %s\n", buf);
+ partition_has(cs_q, "root\n", 1);
+ goto out;
+ }
+ q = total_bw_of(d);
+ h = total_bw_of(c);
+ if (fallback_wake(fd[1], pid))
+ goto out;
+ /* Go on after a failure: the totals tell more. */
+ cpu = task_cpu_of(pid);
+ if (cpu != c)
+ ksft_print_msg("woken on CPU %d, not %d\n", cpu, c);
+ else if (!totals_check("woken", d, c, q, h))
+ ret = KSFT_PASS;
+ reap(pid);
+ pid = -1;
+ sleep_ns(3 * PERIOD);
+ if (totals_of(d, c, &q0, &h0)) {
+ ret = KSFT_FAIL;
+ goto out;
+ }
+ if (q != q0 || h != h0 + bw) {
+ ksft_print_msg("Q counts %lld of %lld, the root cgroup %lld\n",
+ q - q0, bw, h - h0);
+ ret = KSFT_FAIL;
+ }
+out:
+ reap(pid);
+ if (fd[0] >= 0) {
+ close(fd[0]);
+ close(fd[1]);
+ }
+ cpu_up(a);
+ cpu_up(b);
+ cpu_up(d);
+ cs_cleanup();
+ return ret;
+}
+
+/* Setup */
+
+static int find_cgroup2(void)
+{
+ char line[PATH_MAX + 64], dev[64], dir[PATH_MAX], type[64];
+ FILE *f = fopen("/proc/self/mounts", "r");
+ int found = 0;
+
+ while (f && !found && fgets(line, sizeof(line), f)) {
+ if (sscanf(line, "%63s %4095s %63s", dev, dir, type) == 3 &&
+ !strcmp(type, "cgroup2")) {
+ snprintf(root, sizeof(root), "%s", dir);
+ found = 1;
+ }
+ }
+ if (f)
+ fclose(f);
+ return found ? 0 : -1;
+}
+
+static void report(const char *name, int ret)
+{
+ if (ret == KSFT_PASS)
+ ksft_test_result_pass("%s\n", name);
+ else if (ret == KSFT_SKIP)
+ ksft_test_result_skip("%s\n", name);
+ else
+ ksft_test_result_fail("%s\n", name);
+}
+
+/* The tables of the cases, and main() */
+
+enum where {
+ MEASURED_IN_P, /* in a child in P, probed before and after */
+ IN_P, /* in a child in P, which measures itself */
+ FROM_OUTSIDE, /* in the main process, outside P */
+};
+
+static const struct p_case {
+ const char *name;
+ int (*fn)(void);
+ enum where where;
+} p_cases[] = {
+ { "pi_deadline_owner", pi_deadline_owner, MEASURED_IN_P },
+ { "pi_other_owner_asleep", pi_other_owner_asleep, MEASURED_IN_P },
+ { "pi_unlock_running", pi_unlock_running, MEASURED_IN_P },
+ { "pi_chain_two_owners", pi_chain_two_owners, MEASURED_IN_P },
+ { "params_asleep", params_asleep, IN_P },
+ { "params_non_contending", params_non_contending, MEASURED_IN_P },
+ { "params_throttled", params_throttled, MEASURED_IN_P },
+ { "params_refused", params_refused, IN_P },
+ { "params_keep_class_change", params_keep_class_change, MEASURED_IN_P },
+ { "params_keep_boosted", params_keep_boosted, MEASURED_IN_P },
+ { "boosted_owner_policy", boosted_owner_policy, MEASURED_IN_P },
+ { "deboost_resets_period", deboost_resets_period, MEASURED_IN_P },
+ { "deboost_keeps_own_runtime", deboost_keeps_own_runtime, MEASURED_IN_P },
+ { "return_asleep_new_params", return_asleep_new_params, MEASURED_IN_P },
+ { "deboost_new_params_reboost", deboost_new_params_reboost, MEASURED_IN_P },
+ { "deboost_blocked_keeps_order", deboost_blocked_keeps_order, MEASURED_IN_P },
+ { "pi_chain_donor_leaves", pi_chain_donor_leaves, MEASURED_IN_P },
+ { "move_refused", move_refused, FROM_OUTSIDE },
+ { "move_zero_bw", move_zero_bw, FROM_OUTSIDE },
+ { "move_cancelled", move_cancelled, FROM_OUTSIDE },
+ { "death_running", death_running, MEASURED_IN_P },
+ { "death_switched_out", death_switched_out, MEASURED_IN_P },
+ { "death_shrunk", death_shrunk, MEASURED_IN_P },
+ { "stopped_switched_out", stopped_switched_out, MEASURED_IN_P },
+ { "death_killed_asleep", death_killed_asleep, MEASURED_IN_P },
+ { "death_killed_non_contending", death_killed_non_contending, MEASURED_IN_P },
+ { "death_killed_throttled", death_killed_throttled, MEASURED_IN_P },
+ { "death_killed_running", death_killed_running, MEASURED_IN_P },
+ { "death_pi_waiter", death_pi_waiter, MEASURED_IN_P },
+ { "death_pi_owner", death_pi_owner, MEASURED_IN_P },
+ { "death_pi_deadline_owner", death_pi_deadline_owner, MEASURED_IN_P },
+ { "death_group", death_group, MEASURED_IN_P },
+ { "death_zombie_switched", death_zombie_switched, FROM_OUTSIDE },
+ { "yield_switched_out", yield_switched_out, IN_P },
+ { "yield_killed", yield_killed, MEASURED_IN_P },
+ { "overrun_caught", overrun_caught, IN_P },
+ { "overrun_group_death", overrun_group_death, MEASURED_IN_P },
+ { "reclaim_runs", reclaim_runs, MEASURED_IN_P },
+ { "reclaim_switched", reclaim_switched, MEASURED_IN_P },
+ { "server_params", server_params, IN_P },
+ { "server_rebuilds", server_rebuilds, FROM_OUTSIDE },
+ { "server_states", server_states, FROM_OUTSIDE },
+ { "server_reclaim", server_reclaim, MEASURED_IN_P },
+ { "params_waking", params_waking, FROM_OUTSIDE },
+};
+
+#define N_P_CASES ((int)ARRAY_SIZE(p_cases))
+
+static const struct hp_case {
+ const char *name;
+ int (*fn)(void);
+ int no_hp; /* takes no CPU offline */
+} hp_cases[] = {
+ { "hotplug_refused", hotplug_refused },
+ { "hotplug_server", hotplug_server },
+ { "rebuild_record_taken", rebuild_record_taken },
+};
+
+#define N_HP_CASES ((int)ARRAY_SIZE(hp_cases))
+
+/* Run the cases of p_cases[], each with a P made anew. */
+static void run_p_cases(void)
+{
+ char path[PATH_MAX + 32], buf[256];
+ int i, ret, added = 0, told = 0, ok;
+
+ ok = !find_cgroup2() &&
+ snprintf(cg_p, sizeof(cg_p), "%s/deadline_bw_p", root) < (int)sizeof(cg_p) &&
+ snprintf(cg_q, sizeof(cg_q), "%s/deadline_bw_q", root) < (int)sizeof(cg_q);
+ if (ok) {
+ /*
+ * Enable the cpuset controller for the children of the
+ * root while the cases run.
+ */
+ snprintf(path, sizeof(path), "%s/cgroup.subtree_control", root);
+ ok = !read_file(path, buf, sizeof(buf));
+ added = ok && !strstr(buf, "cpuset");
+ if (added && write_file(root, "cgroup.subtree_control", "+cpuset")) {
+ ok = 0;
+ added = 0;
+ }
+ }
+ for (i = 0; i < N_P_CASES; i++) {
+ const struct p_case *c = &p_cases[i];
+
+ if (ok && make_p()) {
+ remove_p();
+ ok = 0;
+ }
+ if (!ok) {
+ if (!told++)
+ ksft_print_msg("no one-CPU partition root: cases in P skipped\n");
+ ksft_test_result_skip("%s\n", c->name);
+ continue;
+ }
+ if (c->where == MEASURED_IN_P)
+ ret = measured(c->name, cg_p, c->fn);
+ else if (c->where == IN_P)
+ ret = run_in(cg_p, c->fn);
+ else
+ ret = c->fn();
+ remove_p();
+ server_restore();
+ report(c->name, ret);
+ }
+ if (added && write_file(root, "cgroup.subtree_control", "-cpuset"))
+ ksft_print_msg("the cpuset controller stays enabled\n");
+}
+
+/*
+ * Read the online CPUs into *@cpus, in ascending order. Read those that
+ * isolcpus="domain" isolates if @iso, the others if not. Return their
+ * count, or -1.
+ */
+static int read_online(int **cpus, int iso)
+{
+ static char buf[65536]; /* a sysfs list fits in a page */
+ char *p = buf;
+ int n = 0, max = 0;
+
+ if (read_file("/sys/devices/system/cpu/online", buf, sizeof(buf)) ||
+ strlen(buf) == sizeof(buf) - 1)
+ return -1;
+ while (*p >= '0' && *p <= '9') {
+ long lo = strtol(p, &p, 10), hi = lo;
+
+ if (*p == '-')
+ hi = strtol(p + 1, &p, 10);
+ for (; lo <= hi; lo++) {
+ if (in_cpu_list(isolated, lo) != iso)
+ continue;
+ if (n == max) {
+ int *more = realloc(*cpus, (max = 2 * max + 64) * sizeof(**cpus));
+
+ if (!more)
+ return -1;
+ *cpus = more;
+ }
+ (*cpus)[n++] = lo;
+ }
+ if (*p == ',')
+ p++;
+ }
+ return n;
+}
+
+int main(void)
+{
+ char buf[64];
+ cpu_set_t set;
+ int i;
+
+ ksft_print_header();
+ if (geteuid())
+ ksft_exit_skip("needs root\n");
+ /* No such file: nothing isolated. */
+ if (read_file("/sys/devices/system/cpu/isolated", isolated, sizeof(isolated)))
+ isolated[0] = 0;
+ ncpus = read_online(&online, 0);
+ nisol = read_online(&isol, 1);
+ if (ncpus < 0 || nisol < 0)
+ ksft_exit_fail_msg("the online CPUs cannot be read\n");
+ if (ncpus < 3)
+ ksft_exit_skip("needs 3 CPUs not isolated by isolcpus=domain\n");
+ /*
+ * Keep the tasks of the test off the isolated CPUs: a deadline task
+ * there is admitted in the root domain of the isolated CPUs. Use
+ * sched_setaffinity() for that. A change of the partitions gives the
+ * tasks of the root cpuset all its CPUs again, the isolated CPUs
+ * too. But the kernel keeps them within an affinity set by
+ * sched_setaffinity(). The children inherit this affinity.
+ */
+ CPU_ZERO(&set);
+ for (i = 0; i < ncpus; i++)
+ CPU_SET(online[i], &set);
+ if (sched_setaffinity(0, sizeof(set), &set))
+ ksft_exit_fail_msg("sched_setaffinity: %s\n", strerror(errno));
+ if (read_file("/proc/sys/kernel/sched_rt_runtime_us", buf, sizeof(buf)) || atol(buf) < 0)
+ ksft_exit_skip("no deadline admission control\n");
+ mv_state = mmap(NULL, 4096, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0);
+ if (mv_state == MAP_FAILED)
+ ksft_exit_fail_msg("mmap: %s\n", strerror(errno));
+ shm_lock = (pthread_mutex_t *)((char *)mv_state + 256);
+
+ ksft_set_plan(N_P_CASES + N_HP_CASES);
+ server_init();
+ run_p_cases();
+
+ for (i = 1; i < ncpus && hp_cpu < 0; i++) {
+ if (cpu_hotpluggable(top_cpu(i)))
+ hp_cpu = top_cpu(i);
+ }
+ if (hp_cpu < 0)
+ ksft_print_msg("no CPU hotplug: hotplug cases skipped\n");
+ for (i = 0; i < N_HP_CASES; i++) {
+ if (hp_cpu < 0 && !hp_cases[i].no_hp) {
+ ksft_test_result_skip("%s\n", hp_cases[i].name);
+ continue;
+ }
+ report(hp_cases[i].name, hp_cases[i].fn());
+ server_restore();
+ }
+ ksft_finished();
+}
diff --git a/tools/testing/selftests/sched/deadline/settings b/tools/testing/selftests/sched/deadline/settings
new file mode 100644
index 000000000000..a62d2fa1275c
--- /dev/null
+++ b/tools/testing/selftests/sched/deadline/settings
@@ -0,0 +1 @@
+timeout=600
--
2.47.3
^ permalink raw reply related [flat|nested] 31+ messages in thread
* Re: [PATCH 06/16] sched: Take cpuset_mutex for a deadline policy change only with pi
2026-11-18 20:06 ` [PATCH 06/16] sched: Take cpuset_mutex for a deadline policy change only with pi Mathieu Desnoyers
@ 2026-11-18 20:17 ` sashiko-bot
2026-11-18 21:09 ` Mathieu Desnoyers
0 siblings, 1 reply; 31+ messages in thread
From: sashiko-bot @ 2026-11-18 20:17 UTC (permalink / raw)
To: Mathieu Desnoyers; +Cc: linux-rt-devel, tj, hannes, mkoutny, linux-kernel
Thank you for your contribution! Sashiko AI review found no new issues
in this patch, but noticed 1 related pre-existing issue it does not
introduce:
- [High] sched/deadline: bandwidth leak in __sched_setscheduler() on
TASK_DEAD task [1]
--
commit eb6bc35bf21202354f98c206678837ac0c913d54
Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
sched: Take cpuset_mutex for a deadline policy change only with pi
This commit avoids taking cpuset_mutex in __sched_setscheduler() when pi
is unset, and retries under the runqueue lock if the target task entered
SCHED_DEADLINE after the unlocked policy check.
> diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c
> index cb115aa9b0b33..9b04e77297aeb 100644
> --- a/kernel/sched/syscalls.c
> +++ b/kernel/sched/syscalls.c
> @@ -561,9 +561,11 @@ int __sched_setscheduler(struct task_struct *p,
>
> /*
> * SCHED_DEADLINE bandwidth accounting relies on stable cpusets
> - * information.
> + * information. Do not take cpuset_mutex when pi is unset: the
> + * caller is normalize_rt_tasks(), which holds tasklist_lock and
> + * cannot sleep.
> */
> - if (dl_policy(policy) || dl_policy(p->policy)) {
> + if (pi && (dl_policy(policy) || dl_policy(p->policy))) {
> cpuset_locked = true;
> cpuset_lock();
> }
> @@ -643,7 +645,13 @@ int __sched_setscheduler(struct task_struct *p,
> }
> }
>
> - /* Re-check policy now with rq lock held: */
> + /*
> + * The caller gave no policy (SETPARAM_POLICY): the request was
> + * derived from an unlocked read of p->policy above. If the task
> + * changed policy since, re-derive the request from scratch, and
> + * release cpuset_mutex: the re-derived request re-decides whether
> + * it is needed.
> + */
> if (unlikely(oldpolicy != -1 && oldpolicy != p->policy)) {
> policy = oldpolicy = -1;
> task_rq_unlock(rq, p, &rf);
> @@ -654,6 +662,18 @@ int __sched_setscheduler(struct task_struct *p,
> goto recheck;
> }
>
> + /*
> + * The caller gave an explicit policy, so the request needs no
> + * re-derivation (oldpolicy is -1 above). But the decision to
> + * take cpuset_mutex used the same unlocked read: the task
> + * entered SCHED_DEADLINE since, and the change below would take
> + * it out of the class. Retry to take the mutex.
> + */
> + if (unlikely(pi && !cpuset_locked && task_has_dl_policy(p))) {
> + task_rq_unlock(rq, p, &rf);
> + goto recheck;
> + }
> +
> oldprio = p->prio;
>
> newprio = __normal_prio(policy, attr->sched_priority, attr->sched_nice);
[Severity: High]
This problem wasn't introduced by this patch, but can
__sched_setscheduler() leak deadline bandwidth and cpuset deadline task
counters if called on a zombie task [1]?
A task that has reached TASK_DEAD and been deactivated from its runqueue
remains reachable by PID until reaped. If sched_setattr() sets
SCHED_DEADLINE on it, sched_dl_overflow() adds the requested bandwidth to
dl_b->total_bw and rq->dl.extra_bw, and switched_to_dl() increments
cs->nr_deadline_tasks via inc_dl_tasks_cs() and adds the bandwidth to
rq->dl.this_bw:
kernel/sched/deadline.c:switched_to_dl() {
...
inc_dl_tasks_cs(p);
/* If p is not queued we will update its parameters at next wakeup. */
if (!task_on_rq_queued(p)) {
add_rq_bw(&p->dl, &rq->dl);
return;
}
...
}
Because a TASK_DEAD task never wakes or runs again and release_task() does
not release deadline bandwidth or decrement cs->nr_deadline_tasks, the
bandwidth and cpuset task count are leaked until reboot, blocking cpuset
changes and eventually causing __dl_overflow() to reject all new
SCHED_DEADLINE admissions with -EBUSY.
[1] https://sashiko.dev/#/bug/linux-6b520332-dcc7-4d6b-b09a-5ec47df1d3cc
--
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1795031605.git.mathieu.desnoyers@efficios.com?part=6
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 04/16] sched/deadline: Do not divide by zero active CPUs of a root domain
2026-11-18 20:06 ` [PATCH 04/16] sched/deadline: Do not divide by zero active CPUs of a root domain Mathieu Desnoyers
@ 2026-11-18 20:23 ` sashiko-bot
2026-11-18 21:08 ` Mathieu Desnoyers
0 siblings, 1 reply; 31+ messages in thread
From: sashiko-bot @ 2026-11-18 20:23 UTC (permalink / raw)
To: Mathieu Desnoyers; +Cc: linux-rt-devel, tj, hannes, mkoutny, linux-kernel
Thank you for your contribution! Sashiko AI review found no new issues
in this patch, but noticed 1 related pre-existing issue it does not
introduce:
- [High] sched/deadline: integer overflow in __dl_sub() and __dl_add()
[1]
--
commit 976d925f6742563bd97b6e12f465ca24230d49b8
Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
sched/deadline: Do not divide by zero active CPUs of a root domain
This commit guards the extra_bw updates in __dl_sub() and __dl_add()
against a zero active CPU count in a root domain during CPU hotplug.
> diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
> index dfae08625342e..b614d5fc1a1ac 100644
> --- a/kernel/sched/deadline.c
> +++ b/kernel/sched/deadline.c
> @@ -189,18 +189,25 @@ void __dl_update(struct dl_bw *dl_b, s64 bw)
> }
> }
>
> +/*
> + * Spread a change of total_bw over the extra_bw of the active CPUs of the
> + * root domain. @cpus is their number. With no active CPU, as can happen
> + * during CPU hotplug, change total_bw alone.
> + */
> static inline
> void __dl_sub(struct dl_bw *dl_b, u64 tsk_bw, int cpus)
> {
> dl_b->total_bw -= tsk_bw;
> - __dl_update(dl_b, (s32)tsk_bw / cpus);
> + if (cpus)
> + __dl_update(dl_b, (s32)tsk_bw / cpus);
> }
>
> static inline
> void __dl_add(struct dl_bw *dl_b, u64 tsk_bw, int cpus)
> {
> dl_b->total_bw += tsk_bw;
> - __dl_update(dl_b, -((s32)tsk_bw / cpus));
> + if (cpus)
> + __dl_update(dl_b, -((s32)tsk_bw / cpus));
> }
[Severity: High]
This problem wasn't introduced by this patch, but can casting tsk_bw to
s32 in __dl_sub() and __dl_add() overflow and flip the sign [1]?
During a large cpuset bandwidth migration (cpuset_reserve_dl_bw() ->
dl_bw_alloc()), tsk_bw can exceed S32_MAX on systems with many CPUs.
Truncating tsk_bw to a negative s32 before dividing by cpus reverses the
sign of the delta passed to __dl_update(), corrupting per-runqueue
extra_bw accounting and breaking SCHED_DEADLINE admission control.
[1] https://sashiko.dev/#/bug/linux-ebf16093-b39b-4cd9-a580-85c7bb17302b
--
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1795031605.git.mathieu.desnoyers@efficios.com?part=4
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 07/16] sched/deadline: Do not lose a throttled task switched out and back
2026-11-18 20:06 ` [PATCH 07/16] sched/deadline: Do not lose a throttled task switched out and back Mathieu Desnoyers
@ 2026-11-18 20:31 ` sashiko-bot
2026-11-18 21:09 ` Mathieu Desnoyers
0 siblings, 1 reply; 31+ messages in thread
From: sashiko-bot @ 2026-11-18 20:31 UTC (permalink / raw)
To: Mathieu Desnoyers; +Cc: linux-rt-devel, tj, hannes, mkoutny, linux-kernel
Thank you for your contribution! Sashiko AI review found no new issues
in this patch, but noticed 1 related pre-existing issue it does not
introduce:
- [High] sched/deadline: stale dl_timer and deadline after
__dl_clear_params() [1]
--
commit e6978eb9fe65441d4ddb2d45bfc7400e644913a5
Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
sched/deadline: Do not lose a throttled task switched out and back
This commit replenishes a throttled SCHED_DEADLINE task in dl_task_timer()
if the task has switched to another scheduling policy, so that it can be
enqueued if it switches back to SCHED_DEADLINE before its 0-lag time.
> diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
> index b614d5fc1a1ac..e4d7cbf4861de 100644
> --- a/kernel/sched/deadline.c
> +++ b/kernel/sched/deadline.c
> @@ -1233,9 +1233,18 @@ static enum hrtimer_restart dl_task_timer(struct hrtimer *timer)
> /*
> * The task might have changed its scheduling policy to something
> * different than SCHED_DEADLINE (through switched_from_dl()).
> + * It keeps its parameters until its 0-lag time. If it comes back
> + * before then, nothing else would lift its throttle. So replenish
> + * it now, as the !queued case below does. Its return enqueues it.
> */
> - if (!dl_task(p))
> + if (!dl_task(p)) {
> + if (dl_se->dl_throttled && !is_dl_boosted(dl_se)) {
> + sched_clock_tick();
> + update_rq_clock(rq);
> + replenish_dl_entity(dl_se);
> + }
> goto unlock;
> + }
[Severity: High]
This problem wasn't introduced by this patch, but in the opposite order
where the 0-lag time elapses before dl_se->dl_timer fires [1], does
__dl_clear_params() leave a stale dl_se->dl_timer armed along with stale
dl_se->deadline and dl_se->runtime values?
When a task calls sched_yield() with remaining runtime (or a
constrained-deadline task overruns slightly), its 0-lag time is earlier
than dl_next_period(dl_se). If the task then switches out of
SCHED_DEADLINE, task_non_contending() or inactive_task_timer() calls
__dl_clear_params(), which clears dl_se->dl_throttled without cancelling
dl_se->dl_timer or resetting dl_se->deadline and dl_se->runtime:
kernel/sched/deadline.c:__dl_clear_params() {
dl_se->dl_runtime = 0;
dl_se->dl_deadline = 0;
dl_se->dl_period = 0;
dl_se->flags = 0;
dl_se->dl_bw = 0;
dl_se->dl_density = 0;
dl_se->dl_throttled = 0;
dl_se->dl_yielded = 0;
...
}
If the task switches back to SCHED_DEADLINE before the old period ends,
enqueue_dl_entity() with ENQUEUE_MOVE sees dl_se->deadline still in the
future and skips setup_new_dl_entity(), running the task with stale
runtime and deadline values. If the task then throttles under its new
reservation while the old dl_se->dl_timer is still queued,
start_dl_timer() sees hrtimer_is_queued(&dl_se->dl_timer) and skips
reprogramming it, delaying replenishment until the old timer expires.
[1] https://sashiko.dev/#/bug/linux-5f16aeeb-544a-4da3-a8a9-3a7be85e8892
--
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1795031605.git.mathieu.desnoyers@efficios.com?part=7
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 08/16] sched/deadline: Do not replenish a boost from cleared parameters
2026-11-18 20:06 ` [PATCH 08/16] sched/deadline: Do not replenish a boost from cleared parameters Mathieu Desnoyers
@ 2026-11-18 20:35 ` sashiko-bot
2026-11-18 21:10 ` Mathieu Desnoyers
0 siblings, 1 reply; 31+ messages in thread
From: sashiko-bot @ 2026-11-18 20:35 UTC (permalink / raw)
To: Mathieu Desnoyers; +Cc: linux-rt-devel, tj, hannes, mkoutny, linux-kernel
Thank you for your contribution! Sashiko AI review found no new issues
in this patch, but noticed 1 related pre-existing issue it does not
introduce:
- [High] sched/deadline: use-after-free of pi_se via
rt_mutex_adjust_prio_chain() [1]
--
commit 961bc9d5ebb4610c191709612415310c77835eeb
Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
sched/deadline: Do not replenish a boost from cleared parameters
This commit prevents an infinite loop in replenish_dl_entity() when an
rt_mutex donor leaves SCHED_DEADLINE and has its deadline parameters
cleared before the priority-inheritance chain walk deboosts the lock
owner.
Link: https://lore.kernel.org/all/avL8oOtjKqVNR7-l@jlelli-thinkpadt14gen4.remote.csb/
> diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
> index e4d7cbf4861de..71b1c155c2424 100644
> --- a/kernel/sched/deadline.c
> +++ b/kernel/sched/deadline.c
> @@ -807,8 +807,38 @@ static void replenish_dl_entity(struct sched_dl_entity *dl_se)
> {
> struct dl_rq *dl_rq = dl_rq_of_se(dl_se);
> struct rq *rq = rq_of_dl_rq(dl_rq);
> + struct sched_dl_entity *pi_se = pi_of(dl_se);
> + u64 runtime = READ_ONCE(pi_se->dl_runtime);
> + u64 deadline = READ_ONCE(pi_se->dl_deadline);
> + u64 period = READ_ONCE(pi_se->dl_period);
[Severity: High]
This problem wasn't introduced by this patch, but can pi_se point to a
freed task_struct after a nested rt_mutex wait is aborted [1]?
Suppose deadline task M with positive remaining runtime holds lock L1, and
deadline task W with an earlier deadline blocks on L1. rt_mutex_setprio()
sets M->dl.pi_se = &W->dl while replenish_dl_entity() leaves
M->dl.deadline unchanged, and when M blocks on L2 held by O, O->dl.pi_se
becomes &W->dl.
When W aborts its wait on L1, remove_waiter() resets M->dl.pi_se to &M->dl
without changing M->dl.deadline and calls rt_mutex_adjust_prio_chain() on
M:
kernel/locking/rtmutex.c:rt_mutex_adjust_prio_chain() {
...
if (rt_waiter_node_equal(&waiter->tree, task_to_waiter_node(task))) {
if (!detect_deadlock)
goto out_unlock_pi;
...
}
Because rt_waiter_node_equal() compares only prio and deadline, it sees no
change for M's waiter on L2 and stops the walk before reaching O. Once W
exits and is freed, replenishing O here dereferences a dangling O->dl.pi_se
(use-after-free).
[1] https://sashiko.dev/#/bug/linux-e1386e80-9bd4-4219-9441-8f56e84f7735
--
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1795031605.git.mathieu.desnoyers@efficios.com?part=8
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 09/16] sched/deadline: Reset the deadline and the runtime with the parameters
2026-11-18 20:06 ` [PATCH 09/16] sched/deadline: Reset the deadline and the runtime with the parameters Mathieu Desnoyers
@ 2026-11-18 20:38 ` sashiko-bot
2026-11-18 21:10 ` Mathieu Desnoyers
0 siblings, 1 reply; 31+ messages in thread
From: sashiko-bot @ 2026-11-18 20:38 UTC (permalink / raw)
To: Mathieu Desnoyers; +Cc: linux-rt-devel, tj, hannes, mkoutny, linux-kernel
Thank you for your contribution! Sashiko AI review found 2 potential
issues to consider:
- [Medium] sched/deadline: donor runtime and deadline retained after PI
boost in switched_from_dl()
- [Medium] sched/deadline: uncanceled dl_timer in __dl_clear_params()
unthrottles new period early
--
commit fd3c8e10959b630f111829811b626124918b2948
Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
sched/deadline: Reset the deadline and the runtime with the parameters
This commit resets dl_se->deadline and dl_se->runtime in
__dl_clear_params() and in switched_from_dl() when a boosted task with no
parameters of its own leaves SCHED_DEADLINE, so that the next entry into
the class starts a new period.
> diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
> index 71b1c155c2424..d6d2657dcddd6 100644
> --- a/kernel/sched/deadline.c
> +++ b/kernel/sched/deadline.c
> + /*
> + * A task with no parameters of its own was boosted into the class.
> + * Its deadline and runtime are the donor's. Reset them, so that its
> + * next entry into the class starts a new period.
> + */
> + if (!p->dl.dl_runtime) {
> + p->dl.deadline = 0;
> + p->dl.runtime = 0;
> + }
> +
> /*
> * In case a task is setscheduled out from SCHED_DEADLINE we need to
> * keep track of that on its cpuset (for correct bandwidth tracking).
[Severity: Medium]
Can a task still retain the donor's runtime and deadline after a PI
deboost if p->dl.dl_runtime is non-zero?
If a non-deadline task is PI-boosted by a SCHED_DEADLINE waiter and
sched_setattr() switches it to SCHED_DEADLINE while still boosted,
__sched_setscheduler() sees prev_class == next_class == &dl_sched_class
and skips switched_from_dl() and switched_to_dl(). __setparam_dl() sets
p->dl.dl_runtime to the task's new reservation without clearing the donor's
p->dl.deadline and p->dl.runtime (and if the task is sleeping, skipping
switched_to_dl() also misses add_rq_bw(), causing a later __sub_rq_bw()
underflow warning). When rt_mutex_setprio() later deboosts the task, it
remains in dl_sched_class with the donor's future deadline, so
enqueue_dl_entity() skips setup_new_dl_entity() and the task runs on the
donor's remaining runtime.
Similarly, if a task that recently left SCHED_DEADLINE is PI-boosted
before its 0-lag inactive_timer expires, switched_to_dl() cancels
inactive_timer while p->dl.dl_runtime is still non-zero, and
replenish_dl_entity() overwrites p->dl.deadline and p->dl.runtime from the
donor. On deboost, !p->dl.dl_runtime is false, so switched_from_dl()
leaves the donor's deadline and runtime in place and passes them to
task_non_contending().
> @@ -3984,7 +3994,8 @@ bool __checkparam_dl(const struct sched_attr *attr)
> }
>
> /*
> - * This function clears the sched_dl_entity static params.
> + * Clear the static parameters of @dl_se. Also reset its deadline and its
> + * runtime, so that its next enqueue starts a new period.
> */
> static void __dl_clear_params(struct sched_dl_entity *dl_se)
> {
> @@ -3995,6 +4006,9 @@ static void __dl_clear_params(struct sched_dl_entity *dl_se)
> dl_se->dl_bw = 0;
> dl_se->dl_density = 0;
>
> + dl_se->deadline = 0;
> + dl_se->runtime = 0;
> +
> dl_se->dl_throttled = 0;
> dl_se->dl_yielded = 0;
> dl_se->dl_non_contending = 0;
[Severity: Medium]
If dl_se->dl_timer was armed before the task left SCHED_DEADLINE, can
leaving it queued in __dl_clear_params() unthrottle the task's new period
early?
If a task throttles (for example via sched_yield() while its 0-lag time is
still before the end of its period) and switches out of SCHED_DEADLINE,
dl_se->dl_timer remains armed for the old period's replenishment time.
When inactive_task_timer() fires at the 0-lag time, __dl_clear_params()
now resets dl_se->deadline, dl_se->runtime, and dl_se->dl_throttled so
that re-entering SCHED_DEADLINE starts a fresh period with a later
deadline, but dl_se->dl_timer is not canceled. If the task re-enters
SCHED_DEADLINE and throttles in its new period before the old timer
expires, start_dl_timer() sees hrtimer_is_queued(&dl_se->dl_timer) and
does not reprogram it:
kernel/sched/deadline.c:start_dl_timer() {
...
if (!hrtimer_is_queued(timer)) {
if (!dl_server(dl_se))
get_task_struct(dl_task_of(dl_se));
hrtimer_start(timer, act, HRTIMER_MODE_ABS_HARD);
}
return 1;
}
When the stale timer fires at the old period boundary, dl_task_timer()
replenishes and unthrottles the task before its new period ends.
--
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1795031605.git.mathieu.desnoyers@efficios.com?part=9
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 12/16] sched/deadline: Fix this_bw when a task is changed during a remote wakeup
2026-11-18 20:06 ` [PATCH 12/16] sched/deadline: Fix this_bw when a task is changed during a remote wakeup Mathieu Desnoyers
@ 2026-11-18 20:46 ` sashiko-bot
2026-11-18 21:11 ` Mathieu Desnoyers
0 siblings, 1 reply; 31+ messages in thread
From: sashiko-bot @ 2026-11-18 20:46 UTC (permalink / raw)
To: Mathieu Desnoyers; +Cc: linux-rt-devel, tj, hannes, mkoutny, linux-kernel
Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] sched/deadline: this_bw desync in add_rq_bw() for PI-boosted
tasks
--
commit 31f5271550e5299348b26e477800f7ac60f7e6c1
Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
sched/deadline: Fix this_bw when a task is changed during a remote wakeup
This commit introduces a dl_bw_oncpu bit in struct sched_dl_entity to track
whether a runqueue's this_bw currently accounts for the task's bandwidth,
preventing bandwidth drift when parameters change during a remote wakeup.
> diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
> index f0e03f97fd82f..c36516210669d 100644
> --- a/kernel/sched/deadline.c
> +++ b/kernel/sched/deadline.c
> @@ -267,18 +267,35 @@ void __sub_rq_bw(u64 dl_bw, struct dl_rq *dl_rq)
> WARN_ON_ONCE(dl_rq->running_bw > dl_rq->this_bw);
> }
>
> +/*
> + * @dl_se->dl_bw_oncpu tells whether the this_bw of a runqueue counts the
> + * bandwidth of @dl_se. add_rq_bw() adds the bandwidth only if no runqueue
> + * counts it. sub_rq_bw() subtracts it only if one does.
> + *
> + * A wakeup on another CPU takes the bandwidth off the old runqueue. The
> + * enqueue on the new CPU adds it back later. A change of the task in
> + * between must not count the bandwidth twice, nor take it off a runqueue
> + * that does not count it.
> + *
> + * The servers track their bandwidth with dl_bw_attached. They call
> + * __add_rq_bw() and __sub_rq_bw() directly.
> + */
> static inline
> void add_rq_bw(struct sched_dl_entity *dl_se, struct dl_rq *dl_rq)
> {
> - if (!dl_entity_is_special(dl_se))
> - __add_rq_bw(dl_se->dl_bw, dl_rq);
> + if (dl_entity_is_special(dl_se) || dl_se->dl_bw_oncpu)
> + return;
> + __add_rq_bw(dl_se->dl_bw, dl_rq);
> + dl_se->dl_bw_oncpu = 1;
> }
[Severity: Medium]
Can add_rq_bw() set dl_bw_oncpu = 1 while dl_se->dl_bw is 0 for a PI-boosted
non-deadline task, causing dl_rq->this_bw to miss a later SCHED_DEADLINE
bandwidth assignment?
When a non-deadline task (dl_runtime == 0, dl_bw == 0) is PI-boosted into
dl_sched_class via rt_mutex_setprio(), enqueue_dl_entity(ENQUEUE_RESTORE) or
switched_to_dl() calls add_rq_bw(), which adds 0 to this_bw and sets
dl_bw_oncpu = 1.
If sched_setattr() assigns SCHED_DEADLINE with new_bw > 0 while the boosted
task is unqueued (either sleeping or on a remote wake_list), prev_class and
next_class are both &dl_sched_class so ENQUEUE_CLASS is not set, and
!task_has_dl_policy(p) is true so sched_dl_overflow() skips
dl_change_utilization(). __setparam_dl() then sets p->dl.dl_bw = new_bw
while dl_bw_oncpu is already 1 and this_bw still has 0.
When the task finishes waking up, task_contending() calls add_rq_bw(), which
returns early because dl_bw_oncpu is 1, and then add_running_bw() adds new_bw
to running_bw, triggering WARN_ON_ONCE(dl_rq->running_bw > dl_rq->this_bw).
When the task later migrates, leaves SCHED_DEADLINE, or exits, sub_rq_bw()
subtracts new_bw and underflows this_bw. Should add_rq_bw() also return
early when !dl_se->dl_bw (or !dl_se->dl_runtime)?
--
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1795031605.git.mathieu.desnoyers@efficios.com?part=12
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 16/16] selftests/sched: Add a test of SCHED_DEADLINE bandwidth accounting
2026-11-18 20:06 ` [PATCH 16/16] selftests/sched: Add a test of SCHED_DEADLINE bandwidth accounting Mathieu Desnoyers
@ 2026-11-18 21:04 ` sashiko-bot
2026-11-18 21:12 ` Mathieu Desnoyers
0 siblings, 1 reply; 31+ messages in thread
From: sashiko-bot @ 2026-11-18 21:04 UTC (permalink / raw)
To: Mathieu Desnoyers; +Cc: linux-rt-devel, tj, hannes, mkoutny, linux-kernel
Thank you for your contribution! Sashiko AI review found 9 potential
issues to consider:
- [Medium] selftests/sched: sub-cgroup start and unconditional cpu
toggle in move_cancelled()
- [Medium] selftests/sched: broken PI and runtime logic in
deboost_new_params_reboost()
- [Medium] selftests/sched: pi_chain_donor_leaves() fails due to
unpropagated dl.pi_se
- [Medium] selftests/sched: rebuild_record_taken() fails under
isolcpus=domain,0,1
- [Low] selftests/sched: rebuild_record_taken() fails instead of
skipping without debugfs
- [Low] selftests/sched: data race on late.done and unjoined thread in
move_cancelled()
- [Low] selftests/sched: unchecked -1 error return from probe() in
bandwidth comparisons
- [Low] selftests/sched: unused variables, write-only fair_tid, and dead
mode branches
- [Low] selftests/sched: re-init of active robust mutex shm_lock in
shm_lock_init()
--
commit ec25de481fdb21f02eb990530342399d3485450d
Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
selftests/sched: Add a test of SCHED_DEADLINE bandwidth accounting
This commit adds deadline_bw_test, a kselftest covering SCHED_DEADLINE
bandwidth accounting across priority inheritance, parameter changes,
cpuset moves, task deaths, yields, runtime overruns, bandwidth
reclaiming, fair server changes, and CPU hotplug.
Link: https://github.com/jlelli/sched-deadline-tests
> diff --git a/tools/testing/selftests/sched/deadline/deadline_bw_test.c b/tools/testing/selftests/sched/deadline/deadline_bw_test.c
> new file mode 100644
> index 0000000000000..1e64402f90df0
> --- /dev/null
> +++ b/tools/testing/selftests/sched/deadline/deadline_bw_test.c
[ ... ]
> + sem_post(&rb_donor_go);
> + t0 = now_ns();
> + while (!rb_donor_locking || proc_state_of(donor_tid) != 'S') {
> + if (now_ns() - t0 > NSEC_PER_SEC) {
> + ksft_print_msg("the donor did not block on the lock\n");
> + err = -1;
> + goto out;
> + }
> + }
> + err = set_deadline(rb_waiter_tid, 2 * NSEC_PER_MSEC, PERIOD);
> + if (err) {
> + ksft_print_msg("the waiter refused as a deadline task: %s\n",
> + strerror(-err));
> + goto out;
> + }
> + err = set_deadline(0, 5 * NSEC_PER_MSEC, PERIOD);
[Severity: Medium]
Does deboost_new_params_reboost() exercise reboosting after a deboost as
its comment describes?
When reboost_owner() calls set_deadline() on rb_waiter_tid while W is
blocked on pi_lock, W is not queued on a runqueue, so __setparam_dl()
updates W's static parameters and switched_to_dl() returns without setting
W->dl.deadline. rt_mutex_adjust_pi() then copies W->dl.deadline (0) into
W's waiter node on pi_lock, which sorts ahead of D's deadline in
rt_waiter_node_less() and makes W the top waiter boosting O immediately,
before D times out.
Also, when O calls set_deadline() on itself while running, the kernel
enqueues O with ENQUEUE_RESTORE | ENQUEUE_MOVE without replenishing, and
rt_mutex_setprio() on deboost does not compare O's remaining runtime
against its new static runtime or start a new period, so the test passes
without testing a reboost on deboost.
[ ... ]
> +static pthread_mutex_t pi_lock2;
> +static sem_t holder_go;
> +static atomic_int holder_holds, chain_waits, donor_done, x_waits, l2_takes;
> +static atomic_int chain_rank, x_rank;
> +static _Atomic pid_t x_tid, fair_tid;
[ ... ]
> + donor_tid = 0;
> + x_tid = 0;
> + fair_tid = 0;
> + return 0;
> +}
[Severity: Low]
fair_tid is set in pi_init_lock2() but never read, which triggers a
-Wunused-but-set-variable warning under -Wall. Similarly, nisol and isol
in main() are populated by read_online(&isol, 1) and never used or freed,
sleeper_cpu is never modified from -1 (leaving the sleeper_cpu >= 0 block
in dl_sleeper_tid() unreachable), struct hp_case::no_hp is never set, and
several helper modes (was_dl != 0 in pi_asleep(), modes 1 and 2 in
deboost_case(), spin_ms == 8 in refill_case(), modes 0 and 2..5 in
return_case(), and MV_YIELDED in mover()) have no callers.
[ ... ]
> + have[LV_Z] = !pthread_create(&t[LV_Z], NULL, leave_waiter, (void *)2L);
> + if (!have[LV_Z] ||
> + wait_for(1000, (leave_tid[LV_Z] && proc_state_of(leave_tid[LV_Z]) == 'S') ||
> + leave_err)) {
> + ksft_print_msg("Z never blocked on L1\n");
> + goto out;
> + }
> + sleep_ns(10 * NSEC_PER_MSEC); /* the chain walk of Z is done */
> + err = leave_err ? -EINVAL : set_other(leave_tid[LV_W]);
[Severity: Medium]
Does pi_chain_donor_leaves() fail because Z is never propagated past M to
O as O's donor?
When Z blocks on L1, task_blocks_on_rt_mutex() updates M->dl.pi_se to
&Z->dl, but M is sleeping on L2, so M->prio (-1) and M->dl.deadline do not
change. In rt_mutex_adjust_prio_chain(), rt_waiter_node_equal() sees that
M's waiter node on L2 still matches M's prio and deadline and stops without
requeueing M on L2 or calling rt_mutex_adjust_prio() on O:
kernel/locking/rtmutex.c:rt_mutex_adjust_prio_chain() {
...
if (rt_waiter_node_equal(&waiter->tree, task_to_waiter_node(task))) {
if (!detect_deadlock)
goto out_unlock_pi;
else
requeue = false;
}
...
}
O->dl.pi_se therefore stays &W->dl. After set_other(leave_tid[LV_W]) and
the 250ms sleep clear W's deadline parameters, waking O replenishes O from
W's zeroed parameters with runtime = 0 and deadline = rq_clock(rq), so O
preempts and starves Y on P's CPU and fails the y_ns < 30ms check.
[ ... ]
> +/* What late_setter() does, and the result. */
> +static struct {
> + pid_t pid;
> + uint64_t runtime;
> + uint64_t delay;
> + atomic_int go;
> + int ret;
> + uint64_t done;
> +} late;
> +
> +/*
> + * Make late.pid a deadline task of late.runtime every 100ms, late.delay
> + * ns after late.go. Store the result in late.ret and the time it returned
> + * in late.done.
> + */
> +static void *late_setter(void *arg)
> +{
> + uint64_t start;
> +
> + while (!late.go)
> + ;
> + start = now_ns();
> + while (now_ns() - start < late.delay)
> + ;
> + late.ret = set_deadline(late.pid, late.runtime, PERIOD);
> + late.done = now_ns();
> + return arg;
> +}
[ ... ]
> +static int move_cancelled(void)
> +{
> + int with_p, with_r, ret = KSFT_PASS, i, err;
> + const char *name = "move_cancelled";
> + uint64_t start, len;
> + pid_t pid;
> +
> + mv_state[0] = 0;
> + mv_state[3] = 0;
> + mv_state[4] = 0;
> + mv_state[5] = 0;
> + pid = pinned_mover(2048, 1);
> + if (pid < 0 || child_up(name, 0) || wait_for(5000, mv_state[5])) {
> + reap(pid);
> + return mv_state[0] == -2 ? KSFT_SKIP : KSFT_FAIL;
> + }
> + if (mv_state[5] < 0 || write_file(root, "cgroup.subtree_control", "+cpu")) {
> + ksft_print_msg("%s: no SCHED_FIFO thread or no cpu controller\n", name);
> + reap(pid);
> + return KSFT_SKIP;
> + }
[ ... ]
> + if (wait_for(5000, late.done)) {
> + ksft_print_msg("%s: sched_setattr() still waits after a cancelled move\n",
> + name);
> + ret = KSFT_FAIL;
> + break; /* the thread cannot be joined */
> + }
> + join(t);
> + if (late.ret) {
> + ksft_print_msg("%s: sched_setattr: %s\n", name, strerror(-late.ret));
> + ret = KSFT_FAIL;
> + }
> + }
> + sleep_ns(3 * PERIOD);
> + if (ret == KSFT_PASS &&
> + (admits_in(cg_p, with_p, "P, after the cancelled moves") ||
> + admits_in(root, with_r, "the root domain of the process")))
> + ret = KSFT_FAIL;
> +out:
> + reap(pid);
> + if (write_file(root, "cgroup.subtree_control", "-cpu"))
> + ksft_print_msg("%s: the cpu controller stays enabled\n", name);
> + return ret;
> +}
[Severity: Medium]
What happens in move_cancelled() when deadline_bw_test is started from a
non-root cgroup (such as a systemd session or service cgroup)?
Unlike kill_case() and death_group(), main() and move_cancelled() never
move the test process into root before forking pinned_mover(2048, 1).
With CONFIG_RT_GROUP_SCHED=y, a non-root cgroup has no RT runtime by
default, so either fifo_sleeper() fails sched_setscheduler(SCHED_FIFO)
with EPERM when +cpu is already enabled, or enabling +cpu on root fails
with EINVAL while the child cgroup contains a SCHED_FIFO task, causing
move_cancelled() to always skip. Also, unlike run_p_cases() and
cs_setup() with +cpuset, move_cancelled() unconditionally writes -cpu to
root/cgroup.subtree_control at exit even if the cpu controller was already
enabled before the test ran.
[Severity: Low]
In the same loop, late.done is a plain uint64_t written by late_setter()
while move_cancelled() polls it in wait_for(5000, late.done), which is a
data race. If that wait times out, breaking out before join(t) also
leaves t unjoined in the main process even though reap(pid) at out: kills
pid and unblocks late_setter().
[ ... ]
> +/* Set up shm_lock. Return 0, or -1 if it cannot be set up. */
> +static int shm_lock_init(void)
> +{
> + pthread_mutexattr_t a;
> + int err;
> +
> + err = pthread_mutexattr_init(&a);
> + if (!err)
> + err = pthread_mutexattr_setpshared(&a, PTHREAD_PROCESS_SHARED);
> + if (!err)
> + err = pthread_mutexattr_setprotocol(&a, PTHREAD_PRIO_INHERIT);
> + if (!err)
> + err = pthread_mutexattr_setrobust(&a, PTHREAD_MUTEX_ROBUST);
> + if (!err)
> + err = pthread_mutex_init(shm_lock, &a);
[ ... ]
> + } else {
> + reap(waiter);
> + waiter = -1;
> + sleep_ns(50 * NSEC_PER_MSEC); /* the owner deboosted asleep */
> + }
> +out:
> + reap(waiter);
> + reap(owner);
> + return ret;
> +}
[Severity: Low]
Does shm_lock_init() re-initialize shm_lock without destroying it across
death_pi_waiter, death_pi_owner, and death_pi_deadline_owner?
shm_lock lives in MAP_SHARED memory, and in death_pi_waiter, reap(owner)
kills the owner with SIGKILL while it still holds shm_lock, leaving the
robust mutex in the FUTEX_OWNER_DIED state when the next test case calls
pthread_mutex_init(shm_lock, &a). shm_lock_init() (and
pi_chain_two_owners()) also omits pthread_mutexattr_destroy(&a).
[ ... ]
> +static int hotplug_server(void)
> +{
> + int base, off10, off30;
> +
> + if (server_set(hp_cpu, SERVER_RUNTIME, SERVER_PERIOD)) {
> + ksft_print_msg("the fair server of CPU %d cannot be set\n", hp_cpu);
> + return KSFT_SKIP;
> + }
> + base = probe(NULL);
> + if (cpu_online(hp_cpu, 0)) {
> + ksft_print_msg("offline of CPU %d refused\n", hp_cpu);
> + return KSFT_FAIL;
> + }
> + off10 = probe(NULL);
> + cpu_up(hp_cpu);
> + if (server_set(hp_cpu, 3 * SERVER_RUNTIME, SERVER_PERIOD) ||
> + admits(base - 2, "a fair server of 30%"))
> + return KSFT_FAIL;
> + if (cpu_online(hp_cpu, 0)) {
> + ksft_print_msg("offline of CPU %d refused, its fair server at 30%%\n", hp_cpu);
> + return KSFT_FAIL;
> + }
> + off30 = probe(NULL);
> + cpu_up(hp_cpu);
> + if (off30 != off10) {
[Severity: Low]
Can hotplug_server() pass when probe(NULL) fails and returns -1?
If both offline probes return -1, off30 != off10 evaluates to false; base
is not checked for <= 0 either. move_refused() and move_zero_bw()
similarly compare probe(cg_p) != base_p || probe(root) != base_r without
checking whether base_p or base_r failed with -1.
[ ... ]
> +static int rebuild_record_taken(void)
> +{
> + int a = top_cpu(3), b = top_cpu(2), c = online[0], d = top_cpu(1);
> + int fd[2] = { -1, -1 }, i, n, err, cpu, ret = KSFT_SKIP;
> + long long q, h, q0, h0, bw;
> + char buf[4096];
> + pid_t pid = -1;
> +
> + if (ncpus < 4 || !cpu_hotpluggable(a) || !cpu_hotpluggable(b) ||
> + !cpu_hotpluggable(d)) {
> + ksft_print_msg("needs 4 CPUs, 3 of them hotpluggable\n");
> + return KSFT_SKIP;
> + }
> + if (cs_setup())
> + return KSFT_SKIP;
> + /* Q has every housekeeping CPU but A, B and D. */
> + for (i = n = 0; i < ncpus - 3; i++)
> + n += snprintf(buf + n, sizeof(buf) - n, "%s%d", n ? "," : "",
> + online[i]);
> + if (make_partition_cpus(cs_q, buf)) {
> + ksft_print_msg("Q cannot be a partition root of CPUs %s\n", buf);
> + goto out;
> + }
> + ret = KSFT_FAIL;
> + err = cpu_online(d, 0);
> + if (err) {
> + ksft_print_msg("offline of CPU %d: %s\n", d, strerror(-err));
> + goto out;
> + }
> + bw = total_bw_of(a);
> + if (bw == -1 || pipe(fd)) {
> + ksft_print_msg("no total_bw of CPU %d, or no pipe\n", a);
> + goto out;
> + }
[Severity: Low]
Should rebuild_record_taken() return KSFT_SKIP instead of KSFT_FAIL when
total_bw_of(a) returns -1 because /sys/kernel/debug/sched/debug is not
available?
ret is set to KSFT_FAIL before total_bw_of(a) is checked, whereas the
other debugfs-dependent cases skip when debugfs files cannot be read.
> + mv_state[0] = 0;
> + mv_state[1] = 0;
> + pid = fork_child("a deadline task");
> + if (!pid) {
> + cpu_set_t set;
> + char ch;
> +
> + CPU_ZERO(&set);
> + CPU_SET(a, &set);
> + CPU_SET(b, &set);
> + if (sched_setaffinity(0, sizeof(set), &set) ||
> + set_deadline(0, 10 * NSEC_PER_MSEC, PERIOD)) {
[ ... ]
> + if (fallback_wake(fd[1], pid))
> + goto out;
> + /* Go on after a failure: the totals tell more. */
> + cpu = task_cpu_of(pid);
> + if (cpu != c)
> + ksft_print_msg("woken on CPU %d, not %d\n", cpu, c);
> + else if (!totals_check("woken", d, c, q, h))
> + ret = KSFT_PASS;
> + reap(pid);
> + pid = -1;
> + sleep_ns(3 * PERIOD);
> + if (totals_of(d, c, &q0, &h0)) {
> + ret = KSFT_FAIL;
> + goto out;
> + }
> + if (q != q0 || h != h0 + bw) {
> + ksft_print_msg("Q counts %lld of %lld, the root cgroup %lld\n",
> + q - q0, bw, h - h0);
> + ret = KSFT_FAIL;
> + }
[Severity: Medium]
Does rebuild_record_taken() fail when CPU 0 is isolated with
isolcpus=domain,0,1?
When CPU 0 is isolated, online[0] is the first housekeeping CPU (for
example, c = 2). The child in the root cgroup overwrites its inherited
housekeeping affinity with {a, b}. Once a and b are taken offline and
cs_q takes the remaining housekeeping CPUs except c,
dl_add_task_root_domain() queries cpuset_cpus_allowed_locked() for
top_cpuset, which includes the isolated CPUs 0 and 1, so
cpumask_first_and(cpu_active_mask, msk) picks CPU 0 and adds bw to CPU 0's
default root domain rather than CPU c's housekeeping root domain (leaving
h == h0 instead of h0 + bw). Likewise, waking pid via fallback_wake()
hits select_fallback_rq() -> cpuset_cpus_allowed_fallback(), which resets
pid's affinity to top_cpuset.cpus_allowed and wakes pid on CPU 0 instead
of CPU c.
--
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1795031605.git.mathieu.desnoyers@efficios.com?part=16
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 04/16] sched/deadline: Do not divide by zero active CPUs of a root domain
2026-11-18 20:23 ` sashiko-bot
@ 2026-11-18 21:08 ` Mathieu Desnoyers
0 siblings, 0 replies; 31+ messages in thread
From: Mathieu Desnoyers @ 2026-11-18 21:08 UTC (permalink / raw)
To: sashiko-reviews; +Cc: linux-rt-devel, tj, hannes, mkoutny, linux-kernel
> This problem wasn't introduced by this patch, but can casting tsk_bw to
> s32 in __dl_sub() and __dl_add() overflow and flip the sign [1]?
Yes, by reading. cpuset_can_attach() passes the sum of the bandwidth of
the tasks it moves to dl_bw_alloc(), which calls __dl_add() with it.
One CPU is 1 << 20, so the sum goes above S32_MAX past 2048 CPUs of
deadline bandwidth. That needs a machine with more than 2048 CPUs.
This series does not change the cast. I will fix it in a separate
patch, with a 64-bit division.
Thanks,
Mathieu
--
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 06/16] sched: Take cpuset_mutex for a deadline policy change only with pi
2026-11-18 20:17 ` sashiko-bot
@ 2026-11-18 21:09 ` Mathieu Desnoyers
0 siblings, 0 replies; 31+ messages in thread
From: Mathieu Desnoyers @ 2026-11-18 21:09 UTC (permalink / raw)
To: sashiko-reviews; +Cc: linux-rt-devel, tj, hannes, mkoutny, linux-kernel
> This problem wasn't introduced by this patch, but can
> __sched_setscheduler() leak deadline bandwidth and cpuset deadline task
> counters if called on a zombie task [1]?
Yes. Patch 11 of this series fixes it: "sched/deadline: Reject
SCHED_DEADLINE on exited tasks". sched_setattr() then refuses
SCHED_DEADLINE for a task that has exited.
Thanks,
Mathieu
--
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 07/16] sched/deadline: Do not lose a throttled task switched out and back
2026-11-18 20:31 ` sashiko-bot
@ 2026-11-18 21:09 ` Mathieu Desnoyers
0 siblings, 0 replies; 31+ messages in thread
From: Mathieu Desnoyers @ 2026-11-18 21:09 UTC (permalink / raw)
To: sashiko-reviews; +Cc: linux-rt-devel, tj, hannes, mkoutny, linux-kernel
> This problem wasn't introduced by this patch, but in the opposite order
> where the 0-lag time elapses before dl_se->dl_timer fires [1], does
> __dl_clear_params() leave a stale dl_se->dl_timer armed along with stale
> dl_se->deadline and dl_se->runtime values?
Yes. Two patches fix it:
- Patch 9 of this series resets the deadline and the runtime in
__dl_clear_params(). It is "sched/deadline: Reset the deadline and
the runtime with the parameters".
- "sched/deadline: Cancel the replenishment timer with the parameters"
cancels the timer there. It is in the second series, which applies on
top of this one.
Thanks,
Mathieu
--
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 08/16] sched/deadline: Do not replenish a boost from cleared parameters
2026-11-18 20:35 ` sashiko-bot
@ 2026-11-18 21:10 ` Mathieu Desnoyers
0 siblings, 0 replies; 31+ messages in thread
From: Mathieu Desnoyers @ 2026-11-18 21:10 UTC (permalink / raw)
To: sashiko-reviews; +Cc: linux-rt-devel, tj, hannes, mkoutny, linux-kernel
> This problem wasn't introduced by this patch, but can pi_se point to a
> freed task_struct after a nested rt_mutex wait is aborted [1]?
Yes. When W aborts its wait, remove_waiter() makes M its own donor
again. The sort keys of M stay equal, so the walk stops at M. "rtmutex:
Continue the chain walk when a waiter's deadline donor changes" fixes
it. It records the donor in the waiter with the sort keys, so the walk
goes on to O. It is in the second series, which applies on top of this
one. Its message describes this deboost case.
Thanks,
Mathieu
--
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 09/16] sched/deadline: Reset the deadline and the runtime with the parameters
2026-11-18 20:38 ` sashiko-bot
@ 2026-11-18 21:10 ` Mathieu Desnoyers
0 siblings, 0 replies; 31+ messages in thread
From: Mathieu Desnoyers @ 2026-11-18 21:10 UTC (permalink / raw)
To: sashiko-reviews; +Cc: linux-rt-devel, tj, hannes, mkoutny, linux-kernel
> Can a task still retain the donor's runtime and deadline after a PI
> deboost if p->dl.dl_runtime is non-zero?
Yes, in both cases. This patch only resets a task with no parameters
of its own. The second series, which applies on top of this one,
handles the others:
- "sched/deadline: Reset the donor's runtime at a deboost in the class"
covers a task that sched_setattr() makes SCHED_DEADLINE while
boosted. Its message names that case.
- "sched/deadline: Count the bandwidth a boosted task is given" adds the
new bandwidth to this_bw in __setparam_dl(). The warning goes away.
- For a task boosted before its 0-lag time, I answered on v7: the 0-lag
time computed from the donor's deadline is upstream behaviour. "Check
the runtime of a task that comes back runnable" checks the runtime
when the task comes back.
> If dl_se->dl_timer was armed before the task left SCHED_DEADLINE, can
> leaving it queued in __dl_clear_params() unthrottle the task's new period
> early?
Yes. "sched/deadline: Cancel the replenishment timer with the
parameters", in the second series, cancels it in __dl_clear_params().
Thanks,
Mathieu
--
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 12/16] sched/deadline: Fix this_bw when a task is changed during a remote wakeup
2026-11-18 20:46 ` sashiko-bot
@ 2026-11-18 21:11 ` Mathieu Desnoyers
0 siblings, 0 replies; 31+ messages in thread
From: Mathieu Desnoyers @ 2026-11-18 21:11 UTC (permalink / raw)
To: sashiko-reviews; +Cc: linux-rt-devel, tj, hannes, mkoutny, linux-kernel
> Can add_rq_bw() set dl_bw_oncpu = 1 while dl_se->dl_bw is 0 for a PI-boosted
> non-deadline task, causing dl_rq->this_bw to miss a later SCHED_DEADLINE
> bandwidth assignment?
Yes, but this patch does not cause it. Without it, the boost adds 0 to
this_bw too, and the same sched_setattr() leaves the new bandwidth out.
"sched/deadline: Count the bandwidth a boosted task is given", in the
second series, fixes it. __setparam_dl() changes this_bw for every task
that gains deadline parameters. With dl_bw_oncpu set, the old bandwidth
of 0 is replaced by the new one.
Thanks,
Mathieu
--
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 16/16] selftests/sched: Add a test of SCHED_DEADLINE bandwidth accounting
2026-11-18 21:04 ` sashiko-bot
@ 2026-11-18 21:12 ` Mathieu Desnoyers
0 siblings, 0 replies; 31+ messages in thread
From: Mathieu Desnoyers @ 2026-11-18 21:12 UTC (permalink / raw)
To: sashiko-reviews; +Cc: linux-rt-devel, tj, hannes, mkoutny, linux-kernel
> Does deboost_new_params_reboost() exercise reboosting after a deboost as
> its comment describes?
It reaches the check it was written for. It failed on v7, before the
split, with the bug that "sched/deadline: Check the runtime of a task
whose parameters change" fixes. I have not checked your
point on the deadline of 0 in the waiter node.
> fair_tid is set in pi_init_lock2() but never read, which triggers a
> -Wunused-but-set-variable warning under -Wall.
These come from splitting the test. They serve the cases that the second
series adds back. The test builds with no warning, as fair_tid is a
global.
> Does pi_chain_donor_leaves() fail because Z is never propagated past M to
> O as O's donor?
No. It passes in every run, in 25 guests with this series. The missed
propagation is what pi_chain_top_moves tests. That case comes with the
two rtmutex fixes, in the second series.
> What happens in move_cancelled() when deadline_bw_test is started from a
> non-root cgroup (such as a systemd session or service cgroup)?
Right. The case assumes the root cgroup, and it turns the cpu controller
off even if it was on before. I will fix both.
> In the same loop, late.done is a plain uint64_t written by late_setter()
> while move_cancelled() polls it in wait_for(5000, late.done), which is a
> data race.
Right. I will make it atomic, and join the thread after the reap.
> Does shm_lock_init() re-initialize shm_lock without destroying it across
> death_pi_waiter, death_pi_owner, and death_pi_deadline_owner?
Yes. I will destroy the mutex and its attributes.
> Can hotplug_server() pass when probe(NULL) fails and returns -1?
Yes. I will check the probes in the three cases.
> Should rebuild_record_taken() return KSFT_SKIP instead of KSFT_FAIL when
> total_bw_of(a) returns -1 because /sys/kernel/debug/sched/debug is not
> available?
Yes.
> Does rebuild_record_taken() fail when CPU 0 is isolated with
> isolcpus=domain,0,1?
Yes. With 7 CPUs and isolcpus=domain,0,1, it fails in 3 runs of 3:
the task wakes on CPU 0. With the second series, it passes in 3 of 3.
The case tests "sched/deadline: Make a rebuild count a task in the root
domain it runs in", of the second series. It belongs there. It passed
in our runs only because the guests with isolated CPUs had too few
CPUs to run it. I will move it to the second series.
Thanks,
Mathieu
--
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com
^ permalink raw reply [flat|nested] 31+ messages in thread
end of thread, other threads:[~2026-11-18 21:12 UTC | newest]
Thread overview: 31+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-18 20:06 [PATCH 00/16] sched/deadline: first bandwidth accounting fixes and selftests Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 01/16] sched: Refuse SCHED_FLAG_KEEP_PARAMS that would change the policy or the class Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 02/16] sched/deadline: Do not admit bandwidth for parameters that are kept Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 03/16] sched: Let a task boosted by a deadline waiter change its policy Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 04/16] sched/deadline: Do not divide by zero active CPUs of a root domain Mathieu Desnoyers
2026-11-18 20:23 ` sashiko-bot
2026-11-18 21:08 ` Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 05/16] sched/deadline: Refuse dl_server debugfs writes while the rq is offline Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 06/16] sched: Take cpuset_mutex for a deadline policy change only with pi Mathieu Desnoyers
2026-11-18 20:17 ` sashiko-bot
2026-11-18 21:09 ` Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 07/16] sched/deadline: Do not lose a throttled task switched out and back Mathieu Desnoyers
2026-11-18 20:31 ` sashiko-bot
2026-11-18 21:09 ` Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 08/16] sched/deadline: Do not replenish a boost from cleared parameters Mathieu Desnoyers
2026-11-18 20:35 ` sashiko-bot
2026-11-18 21:10 ` Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 09/16] sched/deadline: Reset the deadline and the runtime with the parameters Mathieu Desnoyers
2026-11-18 20:38 ` sashiko-bot
2026-11-18 21:10 ` Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 10/16] sched/deadline: Release the bandwidth of a task that leaves the class asleep Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 11/16] sched/deadline: Reject SCHED_DEADLINE on exited tasks Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 12/16] sched/deadline: Fix this_bw when a task is changed during a remote wakeup Mathieu Desnoyers
2026-11-18 20:46 ` sashiko-bot
2026-11-18 21:11 ` Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 13/16] sched/deadline: Check SCHED_DEADLINE affinity under the rq lock Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 14/16] sched/debug: Show the deadline bandwidth of each runqueue Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 15/16] selftests/sched: Add a test of SCHED_DEADLINE policy round trips Mathieu Desnoyers
2026-11-18 20:06 ` [PATCH 16/16] selftests/sched: Add a test of SCHED_DEADLINE bandwidth accounting Mathieu Desnoyers
2026-11-18 21:04 ` sashiko-bot
2026-11-18 21:12 ` Mathieu Desnoyers
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).