[PATCH 0/5] sched: Fix DL server starvation under core scheduling

[PATCH 0/5] sched: Fix DL server starvation under core scheduling

[Fernand Sieber]

Under core scheduling the class a DL server reserves CPU time for gets
almost nothing. With the default fair server (50 ms / 1 s), a CFS task
spinning on one SMT sibling beside a cookied SCHED_FIFO task on the
other measures 0.000% of its CPU instead of the reserved 5%. The ext
server starves sched_ext tasks the same way.

Two independent mechanisms are at fault. The core-wide selection
mishandles the rq->dl_server hand-off: a stale link is passed to idle
(which then forces the siblings idle and trips the forced-idle WARN), a
sibling's link is lost on the fast path, and picks are ranked before
p->dl_server is set (patches 1-3, Fixes: bd9bbc96e835). Independently,
a deferred server is charged its rq's forced-idle time, so its runtime
runs out while it is still throttled, replenish_dl_new_period() pushes
the activation a full period out, and the next period of forced idle
pushes it again (patch 4, Fixes: a110a81c52a9). Either fix alone leaves
the share at ~0%.

Patch 5 adds cs_dl_server_test to selftests/sched: a cookied FIFO
spinner on one sibling, an uncookied CFS spinner on the other, and a
check of the CFS share of that CPU against half the fair server's
reservation, 2.5% by default. Run under QEMU with -smp 4,cores=2,threads=2.

Over a 5 s window, plain tip fails 10/10 at 0.000%, and with the whole
series it passes 31/31 at 3.99-5.19%.

This series was prepared with the help of an AI coding assistant. I
have reviewed and tested all of it.

Fernand Sieber (5):
  sched/core: Clear stale rq->dl_server when core-sched replaces a pick
  sched/core: Keep a sibling's DL server link in the core-wide pick
  sched/core: Rank DL server picks as deadline in the core-wide pick
  sched/deadline: Don't charge core-sched forced idle to the DL servers
  selftests/sched: Add a core-sched forced-idle DL server test

 kernel/sched/core.c                           |  59 ++-
 kernel/sched/idle.c                           |   7 +
 kernel/sched/sched.h                          |  15 +
 tools/testing/selftests/sched/.gitignore      |   1 +
 tools/testing/selftests/sched/Makefile        |   4 +-
 tools/testing/selftests/sched/config          |   4 +-
 .../selftests/sched/cs_dl_server_test.c       | 402 ++++++++++++++++++
 7 files changed, 471 insertions(+), 21 deletions(-)
 create mode 100644 tools/testing/selftests/sched/cs_dl_server_test.c


base-commit: d7e670601af6c564ccf6d3b2ddf70ee4f41a6d1c
-- 
2.47.3




Amazon Development Centre (South Africa) (Proprietary) Limited
29 Gogosoa Street, Observatory, Cape Town, Western Cape, 7925, South Africa
Registration Number: 2004 / 034463 / 07

[PATCH 1/5] sched/core: Clear stale rq->dl_server when core-sched replaces a pick

[Fernand Sieber]

With core scheduling and the fair server, the forced-idle accounting
warns from an idle core:

  WARNING: kernel/sched/core_sched.c:263 at __sched_core_account_forceidle+0x1d6/0x1f0, CPU#0: swapper/0/0

The core is in forced idle with no sibling running a task. The reason
is a stale rq->dl_server causing the idle task to incorrectly be
considered as carried by the deadline server. The idle task then wins
over a runnable sibling task and the whole core is idle despite having
runnable tasks.

The timeline below describes the issue:

  Setup. CPU0 and CPU1 are SMT siblings.
  - CPU1 runs R, a SCHED_FIFO task with cookie X.
  - CPU0 has C, a CFS task with no cookie, which has been starved.
    So CPU0's fair server is due to step in.

  t0, CPU0 picks for the core.
  - pick_task(rq0) goes through the deadline class, which picks the
    fair server, which picks C. As a side effect it sets
    rq0->dl_server = &rq0->fair_server.
  - pick_task(rq1) picks R.

  t1, ranking. C->dl_server is only set once C is switched in, so C
  ranks as CFS. R (RT) beats C (CFS), so the core cookie becomes X.

  t2, matching. CPU0's pick C has cookie 0, not X, so it is replaced.
  There is no cookie-X task on CPU0, so the replacement is idle, and
  CPU0 is forced idle.
  - The code sets rq0->core_pick = idle and rq0->core_dl_server = NULL.
  - The bug: it leaves rq0->dl_server pointing at CPU0's fair server,
    even though idle didn't come through any server.

  t3, CPU0 switches. CPU0 is the selecting CPU, so next = idle, and
  put_prev_set_next_task() hands the stale rq0->dl_server to it. Now
  idle->dl_server is CPU0's fair server.

  t4, C leaves. Load balancing moves C to another core, so CPU0 has
  nothing runnable. The stale link stays on idle.

  t5, the next core-wide selection (when CPU1 reschedules, say on a
  wakeup).
  - CPU0's pick is idle, and the ranking sees idle->dl_server set, so
    it ranks idle as a deadline task.
  - idle now beats R, and the core cookie becomes idle's cookie, 0.
  - CPU1's R has cookie X, not 0, so CPU1 is forced idle.

  t6, result. The whole core is idle. R is runnable but can't run, and
  nothing runs on either CPU. That's the WARN in
  __sched_core_account_forceidle(): forced idle with no sibling running
  a task. It lasts until CPU0 goes through a full schedule that resets
  its server link.

With this patch, step t2 also does rq0->dl_server = NULL. At t3 idle
gets no server, and at t5 it ranks as plain idle, so R wins and keeps
running on CPU1.

Fixes: bd9bbc96e835 ("sched: Rework dl_server")
Signed-off-by: Fernand Sieber <sieberf@amazon.com>
---
 kernel/sched/core.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 175b52a7ea793..f7ac13186ee59 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -6453,6 +6453,12 @@ pick_next_task(struct rq *rq, struct rq_flags *rf)
 				p = sched_core_find(rq_i, cookie);
 			if (!p)
 				p = idle_sched_class.pick_task(rq_i, rf);
+
+			/*
+			 * Neither a core tree pick nor idle comes through a
+			 * DL server; drop the one the replaced pick left.
+			 */
+			rq_i->dl_server = NULL;
 		}
 
 		rq_i->core_pick = p;
-- 
2.47.3




Amazon Development Centre (South Africa) (Proprietary) Limited
29 Gogosoa Street, Observatory, Cape Town, Western Cape, 7925, South Africa
Registration Number: 2004 / 034463 / 07

[PATCH 2/5] sched/core: Keep a sibling's DL server link in the core-wide pick

[Fernand Sieber]

The core-wide selection records in rq_i->core_dl_server the DL server
each sibling's pick came through, then clears it for every sibling in
the cookie matching loop, including when the pick is kept. The sibling
takes the fast path, which sets rq->dl_server from rq->core_dl_server,
and runs the task its server picked with p->dl_server NULL. Only the
selecting CPU keeps the link, as its pick goes through rq->dl_server.

A running task is ranked for core scheduling by p->dl_server. While
such a task runs on a sibling, each following core-wide selection
ranks it as a plain fair or ext task rather than by its server's
deadline, so a higher class task on another sibling takes the core
back before the server's runtime is used. The RV nomiss monitor, which
follows the server through next->dl_server at sched_switch, does not
see the server run either.

Keep the server that pick_task() left in rq_i->dl_server. With the
previous patch it is NULL when the class pick was replaced. Drop the
assignment in the first loop, which the matching loop always
overwrites.

Fixes: bd9bbc96e835 ("sched: Rework dl_server")
Signed-off-by: Fernand Sieber <sieberf@amazon.com>
---
 kernel/sched/core.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index f7ac13186ee59..61d4e6e990bba 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -6431,7 +6431,6 @@ pick_next_task(struct rq *rq, struct rq_flags *rf)
 		}
 
 		rq_i->core_pick = p;
-		rq_i->core_dl_server = rq_i->dl_server;
 
 		if (!max || prio_less(max, p, fi_before))
 			max = p;
@@ -6462,7 +6461,7 @@ pick_next_task(struct rq *rq, struct rq_flags *rf)
 		}
 
 		rq_i->core_pick = p;
-		rq_i->core_dl_server = NULL;
+		rq_i->core_dl_server = rq_i->dl_server;
 
 		if (p == rq_i->idle) {
 			if (rq_i->nr_running) {
-- 
2.47.3




Amazon Development Centre (South Africa) (Proprietary) Limited
29 Gogosoa Street, Observatory, Cape Town, Western Cape, 7925, South Africa
Registration Number: 2004 / 034463 / 07

[PATCH 3/5] sched/core: Rank DL server picks as deadline in the core-wide pick

[Fernand Sieber]

Commit 4b26cfdd3956 ("sched/core: Fix priority checking for DL server
picks") made the core-wide selection rank a task picked through a DL
server as deadline, so that a fair task competing with RT tasks under
core scheduling still gets its server's bandwidth. It keys on
p->dl_server, which pick_task() used to set.

Since commit bd9bbc96e835 ("sched: Rework dl_server"), pick_task()
records the server in rq->dl_server and p->dl_server is only set once
the task is switched in. The core-wide selection still ranks each
sibling's pick with prio_less(), so a task the sibling's server has
just picked ranks as a plain fair or ext task and loses to an RT task
on another sibling. The server's CPU then runs a task with the RT
task's cookie or is forced idle, and the fair task gets none of the
server's bandwidth for as long as the RT task runs.

Rank each sibling's pick by the server it came through, rq_i->dl_server.
prio_less() keeps using p->dl_server for the core tree, whose tasks are
not being picked.

Fixes: bd9bbc96e835 ("sched: Rework dl_server")
Signed-off-by: Fernand Sieber <sieberf@amazon.com>
---
 kernel/sched/core.c | 50 ++++++++++++++++++++++++++++++---------------
 1 file changed, 34 insertions(+), 16 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 61d4e6e990bba..ffe8a41380d0e 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -196,12 +196,13 @@ __read_mostly int scheduler_running;
 DEFINE_STATIC_KEY_FALSE(__sched_core_enabled);
 
 /* kernel prio, less is more */
-static inline int __task_prio(const struct task_struct *p)
+static inline int __task_prio(const struct task_struct *p,
+			      const struct sched_dl_entity *dl_server)
 {
 	if (p->sched_class == &stop_sched_class) /* trumps deadline */
 		return -2;
 
-	if (p->dl_server)
+	if (dl_server)
 		return -1; /* deadline */
 
 	if (rt_or_dl_prio(p->prio))
@@ -223,12 +224,19 @@ static inline int __task_prio(const struct task_struct *p)
  * ge(a,b) := !l(a,b)
  */
 
-/* real prio, less is less */
-static inline bool prio_less(const struct task_struct *a,
-			     const struct task_struct *b, bool in_fi)
+/*
+ * real prio, less is less
+ *
+ * @a_dl_server and @b_dl_server are the DL servers @a and @b run on
+ * behalf of, if any.
+ */
+static inline bool __prio_less(const struct task_struct *a,
+			       const struct sched_dl_entity *a_dl_server,
+			       const struct task_struct *b,
+			       const struct sched_dl_entity *b_dl_server,
+			       bool in_fi)
 {
-
-	int pa = __task_prio(a), pb = __task_prio(b);
+	int pa = __task_prio(a, a_dl_server), pb = __task_prio(b, b_dl_server);
 
 	if (-pa < -pb)
 		return true;
@@ -239,18 +247,13 @@ static inline bool prio_less(const struct task_struct *a,
 	if (pa == -1) { /* dl_prio() doesn't work because of stop_class above */
 		const struct sched_dl_entity *a_dl, *b_dl;
 
-		a_dl = &a->dl;
 		/*
-		 * Since,'a' and 'b' can be CFS tasks served by DL server,
+		 * Since 'a' and 'b' can be tasks served by a DL server,
 		 * __task_prio() can return -1 (for DL) even for those. In that
 		 * case, get to the dl_server's DL entity.
 		 */
-		if (a->dl_server)
-			a_dl = a->dl_server;
-
-		b_dl = &b->dl;
-		if (b->dl_server)
-			b_dl = b->dl_server;
+		a_dl = a_dl_server ?: &a->dl;
+		b_dl = b_dl_server ?: &b->dl;
 
 		return !dl_time_before(a_dl->deadline, b_dl->deadline);
 	}
@@ -266,6 +269,12 @@ static inline bool prio_less(const struct task_struct *a,
 	return false;
 }
 
+static inline bool prio_less(const struct task_struct *a,
+			     const struct task_struct *b, bool in_fi)
+{
+	return __prio_less(a, a->dl_server, b, b->dl_server, in_fi);
+}
+
 static inline bool __sched_core_less(const struct task_struct *a,
 				     const struct task_struct *b)
 {
@@ -6297,6 +6306,7 @@ pick_next_task(struct rq *rq, struct rq_flags *rf)
 	__must_hold(__rq_lockp(rq))
 {
 	struct task_struct *next, *p, *max;
+	struct sched_dl_entity *max_dl_server;
 	bool fi_before = false;
 	bool core_clock_updated = (rq == rq->core);
 	unsigned long cookie;
@@ -6410,6 +6420,7 @@ pick_next_task(struct rq *rq, struct rq_flags *rf)
 	 * Tie-break prio towards the current CPU
 	 */
 	max = NULL;
+	max_dl_server = NULL;
 	for_each_core_cpu_wrap(i, rq) {
 		rq_i = cpu_rq(i);
 
@@ -6432,8 +6443,15 @@ pick_next_task(struct rq *rq, struct rq_flags *rf)
 
 		rq_i->core_pick = p;
 
-		if (!max || prio_less(max, p, fi_before))
+		/*
+		 * p->dl_server is only set once @p runs; rank a pick that
+		 * came through a DL server by that server.
+		 */
+		if (!max || __prio_less(max, max_dl_server, p, rq_i->dl_server,
+					fi_before)) {
 			max = p;
+			max_dl_server = rq_i->dl_server;
+		}
 	}
 
 	cookie = rq->core->core_cookie = max->core_cookie;
-- 
2.47.3




Amazon Development Centre (South Africa) (Proprietary) Limited
29 Gogosoa Street, Observatory, Cape Town, Western Cape, 7925, South Africa
Registration Number: 2004 / 034463 / 07

[PATCH 4/5] sched/deadline: Don't charge core-sched forced idle to the DL servers

[Fernand Sieber]

A deferred DL server is charged the idle time of its rq, on the premise
that the time was available to the server's class and went unused. Core
scheduling breaks that premise: a CPU is forced idle when a SMT sibling
runs a task with an incompatible cookie, so that time was available to
no class at all.

update_curr_dl_se() then runs the server's runtime out while it is
throttled and calls replenish_dl_new_period(), pushing the activation a
full period out. dl_defer_idle does not help, as idle_rq() is false on a
forced-idle CPU: nr_running is not 0. Every period of forced idle pushes
the activation again, the zero-laxity timer never fires, and the class
the server reserves for gets nothing.

Reproduction on 2 cores x 2 threads with the default fair server
(50 ms / 1 s): a SCHED_FIFO spinner with a core-sched cookie on CPU0 and
an uncookied CFS spinner on CPU1 force-idle CPU1, and the CFS task gets
under 0.1% of a CPU instead of the reserved 5%. The ext server starves
sched_ext tasks the same way, and there a plain CFS spinner on the
sibling is enough.

Don't charge forced idle. A server still only loses runtime to real
execution or real idle, and one whose class has nothing to run still
stops through dl_defer_idle once the rq is idle for real; with core
scheduling off nothing changes. Getting the reserved bandwidth back also
needs the server's pick to win the core-wide selection, which the
preceding sched/core patches fix.

Fixes: a110a81c52a9 ("sched/deadline: Deferrable dl server")
Signed-off-by: Fernand Sieber <sieberf@amazon.com>
---
 kernel/sched/idle.c  |  7 +++++++
 kernel/sched/sched.h | 15 +++++++++++++++
 2 files changed, 22 insertions(+)

diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c
index f7dd151f77920..451b4846219d5 100644
--- a/kernel/sched/idle.c
+++ b/kernel/sched/idle.c
@@ -573,6 +573,13 @@ static void update_curr_idle(struct rq *rq)
 
 	se->exec_start = now;
 
+	/*
+	 * Forced idle was not time available to any class on this rq, so
+	 * don't let the deferred servers consume their runtime on it.
+	 */
+	if (forced_idle_rq(rq))
+		return;
+
 	dl_server_update_idle(&rq->fair_server, delta_exec);
 #ifdef CONFIG_SCHED_CLASS_EXT
 	dl_server_update_idle(&rq->ext_server, delta_exec);
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index a4df59c9c9b60..f3b22c4bd6782 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -1525,6 +1525,16 @@ static inline bool sched_core_disabled(void)
 	return !static_branch_unlikely(&__sched_core_enabled);
 }
 
+/*
+ * True while core scheduling keeps @rq idle because a SMT sibling runs a task
+ * with an incompatible cookie, even though @rq has runnable tasks of its own.
+ */
+static inline bool forced_idle_rq(struct rq *rq)
+{
+	return sched_core_enabled(rq) && rq->core->core_forceidle_count &&
+	       rq->nr_running;
+}
+
 /*
  * Be careful with this function; not for general use. The return value isn't
  * stable unless you actually hold a relevant rq->__lock.
@@ -1659,6 +1669,11 @@ static inline bool sched_core_disabled(void)
 	return true;
 }
 
+static inline bool forced_idle_rq(struct rq *rq)
+{
+	return false;
+}
+
 static inline raw_spinlock_t *rq_lockp(struct rq *rq)
 {
 	return &rq->__lock;
-- 
2.47.3




Amazon Development Centre (South Africa) (Proprietary) Limited
29 Gogosoa Street, Observatory, Cape Town, Western Cape, 7925, South Africa
Registration Number: 2004 / 034463 / 07

[PATCH 5/5] selftests/sched: Add a core-sched forced-idle DL server test

[Fernand Sieber]

Nothing in selftests notices a deferred DL server that is charged for
core-sched forced idle and so never activates: the fair server keeps its
reservation in debugfs while the CFS tasks it reserves for get a few
hundredths of a percent of a CPU.

Add cs_dl_server_test. It pins a SCHED_FIFO 50 spinner with its own core
cookie to one SMT sibling and an uncookied CFS spinner to the other, so
the second sibling is forced idle whenever the FIFO task runs, then
checks the CFS task's share of that CPU over five seconds against the
fair server's reservation under /sys/kernel/debug/sched/. The floor is
half the reservation, 2.5% by default, since every slice the server gets
costs a forced-idle hand-over. It skips without SMT, without core
scheduling, or without the debugfs reservation.

At 2 cores x 2 threads the share measures 4.0-5.2% once both this idle
charge and the core-wide DL server pick are fixed, and 0.0% with either
missing, so it passes only with both.

Signed-off-by: Fernand Sieber <sieberf@amazon.com>
---
 tools/testing/selftests/sched/.gitignore      |   1 +
 tools/testing/selftests/sched/Makefile        |   4 +-
 tools/testing/selftests/sched/config          |   4 +-
 .../selftests/sched/cs_dl_server_test.c       | 402 ++++++++++++++++++
 4 files changed, 408 insertions(+), 3 deletions(-)
 create mode 100644 tools/testing/selftests/sched/cs_dl_server_test.c

diff --git a/tools/testing/selftests/sched/.gitignore b/tools/testing/selftests/sched/.gitignore
index 6996d4654d924..4e3a0254e0abc 100644
--- a/tools/testing/selftests/sched/.gitignore
+++ b/tools/testing/selftests/sched/.gitignore
@@ -1 +1,2 @@
 cs_prctl_test
+cs_dl_server_test
diff --git a/tools/testing/selftests/sched/Makefile b/tools/testing/selftests/sched/Makefile
index 099ee9213557a..fa50d1385bd68 100644
--- a/tools/testing/selftests/sched/Makefile
+++ b/tools/testing/selftests/sched/Makefile
@@ -8,7 +8,7 @@ CFLAGS += -O2 -Wall -g -I./ $(KHDR_INCLUDES) -Wl,-rpath=./ \
 	  $(CLANG_FLAGS)
 LDLIBS += -lpthread
 
-TEST_GEN_FILES := cs_prctl_test
-TEST_PROGS := cs_prctl_test
+TEST_GEN_FILES := cs_prctl_test cs_dl_server_test
+TEST_PROGS := cs_prctl_test cs_dl_server_test
 
 include ../lib.mk
diff --git a/tools/testing/selftests/sched/config b/tools/testing/selftests/sched/config
index 1bb8bf6d7fd4c..57b2439f75eed 100644
--- a/tools/testing/selftests/sched/config
+++ b/tools/testing/selftests/sched/config
@@ -1 +1,3 @@
-# empty
+CONFIG_SCHED_CORE=y
+CONFIG_SCHED_SMT=y
+CONFIG_DEBUG_FS=y
diff --git a/tools/testing/selftests/sched/cs_dl_server_test.c b/tools/testing/selftests/sched/cs_dl_server_test.c
new file mode 100644
index 0000000000000..38aa005080f1f
--- /dev/null
+++ b/tools/testing/selftests/sched/cs_dl_server_test.c
@@ -0,0 +1,402 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Verify that core-scheduling forced idle does not starve the fair deadline
+ * server.
+ *
+ * A CPU is forced idle when it has runnable tasks of its own but its SMT
+ * sibling runs a task with an incompatible core-sched cookie. If that time is
+ * charged to the rq's deferred DL servers, the fair server's runtime drains
+ * while it is throttled, its activation is pushed out every period, and the
+ * CFS tasks it reserves bandwidth for get next to nothing.
+ *
+ * The test pins a cookied SCHED_FIFO spinner to one SMT sibling and an
+ * uncookied CFS spinner to the other, so the second sibling is forced idle
+ * whenever the FIFO task runs, and checks that the CFS task still gets a
+ * sensible fraction of the fair server's reservation.
+ *
+ * Copyright (c) 2026 Amazon.com, Inc. or its affiliates.
+ */
+#define _GNU_SOURCE
+#include <errno.h>
+#include <limits.h>
+#include <sched.h>
+#include <signal.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/prctl.h>
+#include <sys/types.h>
+#include <sys/wait.h>
+#include <time.h>
+#include <unistd.h>
+
+#include "../kselftest.h"
+
+#ifndef PR_SCHED_CORE
+#define PR_SCHED_CORE			62
+#define PR_SCHED_CORE_GET		0
+#define PR_SCHED_CORE_CREATE		1 /* create unique core_sched cookie */
+#define PR_SCHED_CORE_SHARE_TO		2 /* push core_sched cookie to pid */
+#define PR_SCHED_CORE_SHARE_FROM	3 /* pull core_sched cookie to pid */
+#define PR_SCHED_CORE_MAX		4
+#endif
+
+enum pid_type {PIDTYPE_PID = 0, PIDTYPE_TGID, PIDTYPE_PGID};
+
+#define SIBLINGS_LIST	"/sys/devices/system/cpu/cpu0/topology/thread_siblings_list"
+#define FAIR_SERVER	"/sys/kernel/debug/sched/fair_server"
+
+#define RUN_TIME	5	/* measurement window, in seconds */
+#define FIFO_PRIO	50
+
+/*
+ * The reservation is only approached, not reached: every slice the fair server
+ * gets costs the sibling a forced-idle hand-over, and the core-wide pick runs
+ * at tick granularity. Ask for half of it, which is far above the few hundredths
+ * of a percent a starved server leaves.
+ */
+#define MIN_SHARE_OF_RESERVATION	0.5
+
+/* Status a spinner child reports to the parent over its readiness pipe. */
+#define CHILD_READY	'R'
+#define CHILD_EPERM	'P'
+#define CHILD_ERROR	'E'
+
+/* Tell the parent how setup went; it decides whether to run or to skip. */
+static void signal_status(int fd, char status)
+{
+	if (write(fd, &status, 1) != 1) {
+		ksft_perror("write to readiness pipe");
+		exit(1);
+	}
+	if (status != CHILD_READY)
+		exit(1);
+}
+
+/*
+ * Burn CPU until killed. A constant controlling expression keeps the compiler
+ * from removing the loop, so no volatile is needed to hold it in place.
+ */
+static void spin_forever(void)
+{
+	while (1)
+		;
+}
+
+static int set_affinity(int cpu)
+{
+	cpu_set_t mask;
+
+	CPU_ZERO(&mask);
+	CPU_SET(cpu, &mask);
+
+	return sched_setaffinity(0, sizeof(mask), &mask);
+}
+
+/*
+ * Read the first two distinct CPU ids out of a cpu list such as "0-1", "0,4"
+ * or "0,4-5". Returns 0 on success, -1 if the list holds fewer than two ids.
+ */
+static int parse_sibling_pair(const char *list, int *sib0, int *sib1)
+{
+	int found = 0, sib[2] = { -1, -1 };
+	const char *p = list;
+
+	while (*p && found < 2) {
+		int first, last, n = 0;
+
+		if (sscanf(p, "%d-%d%n", &first, &last, &n) != 2) {
+			if (sscanf(p, "%d%n", &first, &n) != 1)
+				return -1;
+			last = first;
+		}
+		for (; first <= last && found < 2; first++)
+			sib[found++] = first;
+		p += n;
+		if (*p != ',')
+			break;
+		p++;
+	}
+
+	if (found < 2)
+		return -1;
+
+	*sib0 = sib[0];
+	*sib1 = sib[1];
+
+	return 0;
+}
+
+static int read_sibling_pair(int *sib0, int *sib1)
+{
+	char list[256];
+	FILE *f;
+	char *s;
+
+	f = fopen(SIBLINGS_LIST, "r");
+	if (!f)
+		return -1;
+	s = fgets(list, sizeof(list), f);
+	fclose(f);
+	if (!s)
+		return -1;
+
+	return parse_sibling_pair(list, sib0, sib1);
+}
+
+static int read_u64_file(const char *path, unsigned long long *val)
+{
+	FILE *f;
+	int ret;
+
+	f = fopen(path, "r");
+	if (!f)
+		return -1;
+	ret = fscanf(f, "%llu", val);
+	fclose(f);
+
+	return ret == 1 ? 0 : -1;
+}
+
+/* Read the fair server reservation of @cpu, in nanoseconds. */
+static int read_fair_server(int cpu, unsigned long long *runtime,
+			    unsigned long long *period)
+{
+	char path[PATH_MAX];
+
+	snprintf(path, sizeof(path), FAIR_SERVER "/cpu%d/runtime", cpu);
+	if (read_u64_file(path, runtime))
+		return -1;
+
+	snprintf(path, sizeof(path), FAIR_SERVER "/cpu%d/period", cpu);
+	if (read_u64_file(path, period))
+		return -1;
+
+	return 0;
+}
+
+/* CPU time consumed by @pid so far, in seconds, from /proc/<pid>/stat. */
+static double process_runtime(pid_t pid)
+{
+	char path[PATH_MAX];
+	unsigned long utime, stime;
+	FILE *f;
+	int fields;
+
+	snprintf(path, sizeof(path), "/proc/%d/stat", pid);
+	f = fopen(path, "r");
+	if (!f)
+		return -1;
+
+	/* Skip the first 13 fields and read the 14th and 15th. */
+	fields = fscanf(f, "%*d %*s %*c %*d %*d %*d %*d %*d %*u %*u %*u %*u %*u %lu %lu",
+			&utime, &stime);
+	fclose(f);
+	if (fields != 2)
+		return -1;
+
+	return (double)(utime + stime) / sysconf(_SC_CLK_TCK);
+}
+
+static double monotonic_now(void)
+{
+	struct timespec ts;
+
+	if (clock_gettime(CLOCK_MONOTONIC, &ts))
+		return -1;
+
+	return ts.tv_sec + ts.tv_nsec / 1000000000.0;
+}
+
+/*
+ * Diagnostic only: how long the FIFO task's CPU was forced idle. Needs
+ * CONFIG_SCHEDSTATS and schedstats enabled, so it must not gate the result.
+ */
+static void print_forced_idle(pid_t pid)
+{
+	char path[PATH_MAX], line[256];
+	FILE *f;
+
+	snprintf(path, sizeof(path), "/proc/%d/sched", pid);
+	f = fopen(path, "r");
+	if (!f)
+		return;
+
+	while (fgets(line, sizeof(line), f)) {
+		if (strstr(line, "core_forceidle_sum")) {
+			line[strcspn(line, "\n")] = '\0';
+			ksft_print_msg("FIFO task %s\n", line);
+			break;
+		}
+	}
+	fclose(f);
+}
+
+/* Cookied SCHED_FIFO spinner. Only this task gets a cookie. */
+static void fifo_child(int cpu, int ready_fd)
+{
+	struct sched_param param = { .sched_priority = FIFO_PRIO };
+
+	if (set_affinity(cpu)) {
+		ksft_perror("FIFO child sched_setaffinity");
+		signal_status(ready_fd, CHILD_ERROR);
+	}
+	if (prctl(PR_SCHED_CORE, PR_SCHED_CORE_CREATE, 0, PIDTYPE_PID, 0)) {
+		ksft_perror("FIFO child PR_SCHED_CORE_CREATE");
+		signal_status(ready_fd, CHILD_ERROR);
+	}
+	if (sched_setscheduler(0, SCHED_FIFO, &param)) {
+		if (errno == EPERM)
+			signal_status(ready_fd, CHILD_EPERM);
+		ksft_perror("FIFO child sched_setscheduler");
+		signal_status(ready_fd, CHILD_ERROR);
+	}
+	signal_status(ready_fd, CHILD_READY);
+	spin_forever();
+}
+
+/* Uncookied CFS spinner on the FIFO task's SMT sibling. */
+static void cfs_child(int cpu, int ready_fd)
+{
+	if (set_affinity(cpu)) {
+		ksft_perror("CFS child sched_setaffinity");
+		signal_status(ready_fd, CHILD_ERROR);
+	}
+	signal_status(ready_fd, CHILD_READY);
+	spin_forever();
+}
+
+static char wait_ready(int fd)
+{
+	char status;
+
+	if (read(fd, &status, 1) != 1) {
+		ksft_perror("read from readiness pipe");
+		return CHILD_ERROR;
+	}
+
+	return status;
+}
+
+static void kill_child(pid_t *pid)
+{
+	if (*pid <= 0)
+		return;
+	kill(*pid, SIGKILL);
+	waitpid(*pid, NULL, 0);
+	*pid = -1;
+}
+
+int main(void)
+{
+	double fifo_runtime, cfs_start, cfs_end, t_start, t_end, share, want;
+	unsigned long long runtime, period;
+	pid_t fifo_pid = -1, cfs_pid = -1;
+	int fifo_ready[2], cfs_ready[2];
+	unsigned long long cookie;
+	int sib0, sib1;
+	char status;
+
+	ksft_print_header();
+	ksft_set_plan(1);
+
+	if (read_sibling_pair(&sib0, &sib1)) {
+		ksft_test_result_skip("CPU0 has no SMT sibling\n");
+		ksft_finished();
+	}
+	ksft_print_msg("SMT siblings: CPU%d (FIFO) and CPU%d (CFS)\n", sib0, sib1);
+
+	if (prctl(PR_SCHED_CORE, PR_SCHED_CORE_GET, 0, PIDTYPE_PID,
+		  (unsigned long)&cookie)) {
+		ksft_test_result_skip("core scheduling unavailable: %s\n",
+				      strerror(errno));
+		ksft_finished();
+	}
+
+	if (read_fair_server(sib1, &runtime, &period)) {
+		ksft_test_result_skip("cannot read " FAIR_SERVER "/cpu%d, debugfs not mounted?\n",
+				      sib1);
+		ksft_finished();
+	}
+	if (!runtime || !period) {
+		ksft_test_result_skip("fair server on CPU%d is disabled (%llu/%llu)\n",
+				      sib1, runtime, period);
+		ksft_finished();
+	}
+	ksft_print_msg("CPU%d fair server reservation: %llu/%llu ns = %.3f%%\n",
+		       sib1, runtime, period, 100.0 * runtime / period);
+
+	if (pipe(fifo_ready) || pipe(cfs_ready))
+		ksft_exit_fail_perror("pipe");
+
+	fifo_pid = fork();
+	if (!fifo_pid) {
+		close(fifo_ready[0]);
+		close(cfs_ready[0]);
+		close(cfs_ready[1]);
+		fifo_child(sib0, fifo_ready[1]);
+	} else if (fifo_pid < 0) {
+		ksft_exit_fail_perror("fork FIFO child");
+	}
+	close(fifo_ready[1]);
+
+	status = wait_ready(fifo_ready[0]);
+	close(fifo_ready[0]);
+	if (status != CHILD_READY) {
+		kill_child(&fifo_pid);
+		close(cfs_ready[0]);
+		close(cfs_ready[1]);
+		if (status == CHILD_EPERM) {
+			ksft_test_result_skip("need privileges for SCHED_FIFO\n");
+			ksft_finished();
+		}
+		ksft_exit_fail_msg("FIFO child setup failed\n");
+	}
+
+	cfs_pid = fork();
+	if (!cfs_pid) {
+		close(cfs_ready[0]);
+		cfs_child(sib1, cfs_ready[1]);
+	} else if (cfs_pid < 0) {
+		kill_child(&fifo_pid);
+		ksft_exit_fail_perror("fork CFS child");
+	}
+	close(cfs_ready[1]);
+
+	status = wait_ready(cfs_ready[0]);
+	close(cfs_ready[0]);
+	if (status != CHILD_READY) {
+		kill_child(&cfs_pid);
+		kill_child(&fifo_pid);
+		ksft_exit_fail_msg("CFS child setup failed\n");
+	}
+
+	cfs_start = process_runtime(cfs_pid);
+	t_start = monotonic_now();
+	sleep(RUN_TIME);
+	t_end = monotonic_now();
+	cfs_end = process_runtime(cfs_pid);
+	fifo_runtime = process_runtime(fifo_pid);
+
+	print_forced_idle(fifo_pid);
+	kill_child(&cfs_pid);
+	kill_child(&fifo_pid);
+
+	if (cfs_start < 0 || cfs_end < 0 || fifo_runtime < 0)
+		ksft_exit_fail_msg("cannot read /proc/<pid>/stat\n");
+	if (t_start < 0 || t_end <= t_start)
+		ksft_exit_fail_msg("cannot read CLOCK_MONOTONIC\n");
+
+	share = (cfs_end - cfs_start) / (t_end - t_start);
+	want = MIN_SHARE_OF_RESERVATION * runtime / period;
+
+	ksft_print_msg("FIFO task ran %.3fs, CFS task ran %.3fs over %.3fs\n",
+		       fifo_runtime, cfs_end - cfs_start, t_end - t_start);
+	ksft_print_msg("CFS share of CPU%d: %.3f%%, need %.3f%%\n",
+		       sib1, share * 100, want * 100);
+
+	ksft_test_result(share >= want,
+			 "CFS task gets the fair server reservation while forced idle\n");
+
+	ksft_finished();
+}
-- 
2.47.3




Amazon Development Centre (South Africa) (Proprietary) Limited
29 Gogosoa Street, Observatory, Cape Town, Western Cape, 7925, South Africa
Registration Number: 2004 / 034463 / 07