[PATCH v3 0/8] sched: RT throttling cleanup and a bit of documentation

[PATCH v3 0/8] sched: RT throttling cleanup and a bit of documentation

[Sebastian Andrzej Siewior]

People used to disable the RT throttling mechanism by writing -1 to
/proc/sys/kernel/sched_rt_runtime_us. Since v6.12 they can still do this
but they don't disable RT throttling anymore. Instead this throttling
mechanism has been taken over by the fair_server which can be configured
via debugfs.
This is not really documented by a hint appeared a few cycles later in
Documentation/scheduler/sched-rt-group.rst. This can be found once you
know what you are looking for because this document describes the RT
group scheduling (CONFIG_RT_GROUP_SCHED). The fair_server is used even
without it.
The then there is global DL resource control which also shares this
file proc file (which is stuck if CONFIG_RT_GROUP_SCHED is enabled).

This little series:
- Make sure the fair_serve configuration can only be changed while the
  CPU is active. Mathieu tries to fix this and allow it instead. That
  would be an alternative.

- Marks CONFIG_RT_GROUP_SCHED as broken because I was not allowed to
  remove it.

- Untangles DL usage of sched_rt_runtime_us and moves it to
    /sys/kernel/sched/dl_period_us
    /sys/kernel/sched/dl_runtime_us

  This keeps the _us sematic that we used to have /proc. It _could_ be
  changed to ns instead. Or replaced with a per-CPU/RQ setting if this
  is simpler/ preferred on the DL side.

- Moves rcutorture away from the procfs interface since it tries to
  shutdown the fair_server for its testing.

- Adds sysfs interface under /sys/kernel/sched/cpuX to represent the
  runqueues. The only knob for now is fair_server_{period|runtime}
  representing the fair_server parameters. If all goes well, debugfs
  could be removed. The ext server could be moved, too.

- I tried to correct/ update documentation while at it. I added the -doc
  folks on Cc so they are aware. I can drop them in the next posting, I
  expect that this gets routed via the sched/-tip.

v2…v3: https://patch.msgid.link/20261110-sched_dl-v2-0-cc9399f7aeb1@linutronix.de
  - Corrected in wording sched-rt-group.rst (Pointed out by Randy)
  - RCU-torture now disables the fair_server from its boost-thread. The
    code expects the CPU to be active which is after the CPU-hotplug
    notifier is run. An alternative would be to not start the
    fair_server (in the RCU-torture built-in case).
  - sched_dl_try_boundary() has now a private mutex to perform the
    update. sashiko complains that the values are read outside of
    sched_domains_mutex and written back while it is held, making it
    possible to write stale period/ runtime values back.
  - proper u64 casting for NSEC_PER_USEC multiplication.
  - sysfs_dl_runtime is not static and accessed in
    dl_bandwidth_enabled(). This was missed earlier.
    The remaining sysctl_sched_rt_{runtime|period} are now also hidden
    behind CONFIG_RT_GROUP_SCHED.

v1…v2: https://lore.kernel.org/all/20261105164724.yxONBd03@linutronix.de/
  - Tejun and PeterZ asked for marking CONFIG_RT_GROUP_SCHED broken
    first.
  - Everything after patch #1 that got queued up.

RFC…v1: https://lore.kernel.org/all/20261023161348.t3lpPLQZ@linutronix.de/
  - Try to remove RT-GROUP-sched after figuring out that it is useless.

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>

---
Sebastian Andrzej Siewior (8):
      sched/deadline: Reject debugfs dl_server writes for not-active CPUs.
      sched: Mark RT_GROUP_SCHED broken
      rcutorture: Don't try to disable to RT throttling.
      sched: rcutorture: Allow to disable DL fair_server
      sched: Move DL's admission control to sysfs
      sched: Limit sysctl sched_rt_{period|runtime}_us to RT_GROUP_SCHED
      sched: Add sysfs files for the per-CPU fair_server knobs
      Documentation/sched: Add fairserver to sched-deadline

 Documentation/arch/arm64/asymmetric-32bit.rst |   2 +-
 Documentation/scheduler/sched-deadline.rst    |  36 ++-
 Documentation/scheduler/sched-rt-group.rst    |   7 +-
 init/Kconfig                                  |   2 +-
 kernel/rcu/rcu.h                              |   2 -
 kernel/rcu/rcutorture.c                       |  37 +--
 kernel/sched/deadline.c                       | 346 +++++++++++++++++++++++++-
 kernel/sched/debug.c                          |  56 +----
 kernel/sched/features.h                       |   2 +
 kernel/sched/rt.c                             |  65 ++---
 kernel/sched/sched.h                          |  63 +++--
 11 files changed, 439 insertions(+), 179 deletions(-)
---
base-commit: fd9230005e85472b81c1c7c16afe93dd878996b1
change-id: 20261110-sched_dl-a9727787112f

Best regards,
--  
Sebastian Andrzej Siewior <bigeasy@linutronix.de>

[PATCH v3 1/8] sched/deadline: Reject debugfs dl_server writes for not-active CPUs.

[Sebastian Andrzej Siewior]

sched_server_write_common() checks if the CPU is online before allowing
fair_server updates. This is important because updating the fair_server
runtime/ period will use dl_bw_cpus() will return only the number of
CPUs which are also marked active.

The CPU is marked active as the last step during CPU-hotplug. If the CPU
has been marked "online" but not yet "active" or is stuck between those
two states then any update of period/ runtime will lead to a divide-by-0
error.

Let sched_server_write_common() check of active CPUs instead of online.
Additonaly, let dl_server_apply_params() return an error if dl_bw_cpus()
return 0 CPUs to avoid the divide error.

Fixes: 4043f54984161 ("sched/deadline: Reject debugfs dl_server writes for offline CPUs")
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 kernel/sched/deadline.c | 3 +++
 kernel/sched/debug.c    | 2 +-
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index dfae08625342e..10beb54281881 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -1920,6 +1920,9 @@ int dl_server_apply_params(struct sched_dl_entity *dl_se, u64 runtime, u64 perio
 	cpus = dl_bw_cpus(cpu);
 	cap = dl_bw_capacity(cpu);
 
+	if (!cpus)
+		return -EBUSY;
+
 	if (__dl_overflow(dl_b, cap, old_bw, new_bw))
 		return -EBUSY;
 
diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c
index e6a3b516c703b..1545139e52443 100644
--- a/kernel/sched/debug.c
+++ b/kernel/sched/debug.c
@@ -410,7 +410,7 @@ static ssize_t sched_server_write_common(struct file *filp, const char __user *u
 			return  -EINVAL;
 		}
 
-		if (!cpu_online(cpu_of(rq)))
+		if (!cpu_active(cpu_of(rq)))
 			return -EBUSY;
 
 		update_rq_clock(rq);

-- 
2.55.0

[PATCH v3 2/8] sched: Mark RT_GROUP_SCHED broken

[Sebastian Andrzej Siewior]

The RT_GROUP_SCHED is broken beyond repair and it is best to not enable
it during compile. If enabled it is recommended to disable it at boot
time with the rt_group_sched= option.
With it enabled it is not possible to set the policy of a task to
SCHED_{FIFO|RR} because its cgroup has no runtime assigned. The runtime
can not be assigned with cgroup v2. The root group has however all the
runtime assigned meaning the global sysctl runtime parameter
(sched_rt_runtime_us) can no be reduced because the new bandwidth will
be less than what is current available.
It is possible to schedule SCHED_DEADLINE tasks but the admission
control is stuck can only be disabled via -1 (same sysctl parameter).

As a result all major distors (Debian, Open Suse, Fedora, Archlinux)
have it disabled.

There is work on Hierarchical constant bandwidth server (HCBS) which aim
to group multiple real-time tasks within one deadline configuration.

Mark CONFIG_RT_GROUP_SCHED broken. Move code which is not under
RT_GROUP_SCHED but only used by it within its IFDEF section. This
includes also RT_RUNTIME_SHARE.
Update the documentation, to state that it is not related to real-time
tasks in general but only to RT-group scheduling. While at it, also
update the current defaults.

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 Documentation/scheduler/sched-rt-group.rst |  7 ++++++-
 init/Kconfig                               |  2 +-
 kernel/sched/features.h                    |  2 ++
 kernel/sched/rt.c                          |  6 ------
 kernel/sched/sched.h                       | 19 +++++++++++--------
 5 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/Documentation/scheduler/sched-rt-group.rst b/Documentation/scheduler/sched-rt-group.rst
index ab464335d3204..cd69c93baeb6a 100644
--- a/Documentation/scheduler/sched-rt-group.rst
+++ b/Documentation/scheduler/sched-rt-group.rst
@@ -18,6 +18,11 @@ Real-Time group scheduling
 0. WARNING
 ==========
 
+ This document applies to RT group scheduling provided by CONFIG_RT_GROUP_SCHED.
+ If you don't use RT-group scheduling, schedule real-time tasks and experience
+ throttling issues please look into Documentation/scheduler/sched-deadline.rst
+ for the fair_server.
+
  Fiddling with these settings can result in an unstable system, the knobs are
  root only and assumes root knows what he is doing.
 
@@ -111,7 +116,7 @@ lack an EDF scheduler to make non-uniform periods usable.
 ---------------------
 
 The default values for sched_rt_period_us (1000000 or 1s) and
-sched_rt_runtime_us (950000 or 0.95s).  This gives 0.05s to be used by
+sched_rt_runtime_us (1000000 or 1s).  This gives 0s to be used by
 SCHED_OTHER (non-RT tasks). These defaults were chosen so that a run-away
 real-time tasks will not lock up the machine but leave a little time to recover
 it.  By setting runtime to -1 you'd get the old behaviour back.
diff --git a/init/Kconfig b/init/Kconfig
index 4013781e0a029..7d5db38cc82c8 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -980,7 +980,7 @@ config CFS_BANDWIDTH
 
 config RT_GROUP_SCHED
 	bool "Group scheduling for SCHED_RR/FIFO"
-	depends on CGROUP_SCHED
+	depends on CGROUP_SCHED && BROKEN
 	default n
 	help
 	  This feature lets you explicitly allocate real CPU bandwidth
diff --git a/kernel/sched/features.h b/kernel/sched/features.h
index 8f0dee8fc4755..acc1c3fdfe2a2 100644
--- a/kernel/sched/features.h
+++ b/kernel/sched/features.h
@@ -122,7 +122,9 @@ SCHED_FEAT(RT_PUSH_IPI, false)
 # endif
 #endif
 
+#ifdef CONFIG_RT_GROUP_SCHED
 SCHED_FEAT(RT_RUNTIME_SHARE, false)
+#endif
 SCHED_FEAT(LB_MIN, false)
 SCHED_FEAT(ATTACH_AGE_LOAD, true)
 
diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
index 1535046a23ffa..cdeddd3fc9af2 100644
--- a/kernel/sched/rt.c
+++ b/kernel/sched/rt.c
@@ -944,12 +944,6 @@ static inline const struct cpumask *sched_rt_period_mask(void)
 	return cpu_online_mask;
 }
 
-static inline
-struct rt_rq *sched_rt_period_rt_rq(struct rt_bandwidth *rt_b, int cpu)
-{
-	return &cpu_rq(cpu)->rt;
-}
-
 static void __enable_runtime(struct rq *rq) { }
 static void __disable_runtime(struct rq *rq) { }
 
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 1d76950dbb0a1..a7964537bac75 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -313,6 +313,7 @@ struct rt_prio_array {
 	struct list_head queue[MAX_RT_PRIO];
 };
 
+#ifdef CONFIG_RT_GROUP_SCHED
 struct rt_bandwidth {
 	/* nests inside the rq lock: */
 	raw_spinlock_t		rt_runtime_lock;
@@ -321,6 +322,7 @@ struct rt_bandwidth {
 	struct hrtimer		rt_period_timer;
 	unsigned int		rt_period_active;
 };
+#endif
 
 static inline int dl_bandwidth_enabled(void)
 {
@@ -581,6 +583,7 @@ extern void start_cfs_bandwidth(struct cfs_bandwidth *cfs_b);
 extern void unthrottle_cfs_rq(struct cfs_rq *cfs_rq);
 extern bool cfs_task_bw_constrained(struct task_struct *p);
 
+#ifdef CONFIG_RT_GROUP_SCHED
 extern void init_tg_rt_entry(struct task_group *tg, struct rt_rq *rt_rq,
 		struct sched_rt_entity *rt_se, int cpu,
 		struct sched_rt_entity *parent);
@@ -589,6 +592,14 @@ extern int sched_group_set_rt_period(struct task_group *tg, u64 rt_period_us);
 extern long sched_group_rt_runtime(struct task_group *tg);
 extern long sched_group_rt_period(struct task_group *tg);
 extern int sched_rt_can_attach(struct task_group *tg, struct task_struct *tsk);
+extern void init_rt_bandwidth(struct rt_bandwidth *rt_b, u64 period, u64 runtime);
+extern bool sched_rt_bandwidth_account(struct rt_rq *rt_rq);
+
+static inline int rt_bandwidth_enabled(void)
+{
+	return sysctl_sched_rt_runtime >= 0;
+}
+#endif
 
 extern struct task_group *sched_create_group(struct task_group *parent);
 extern void sched_online_group(struct task_group *tg,
@@ -869,11 +880,6 @@ struct scx_rq {
 };
 #endif /* CONFIG_SCHED_CLASS_EXT */
 
-static inline int rt_bandwidth_enabled(void)
-{
-	return sysctl_sched_rt_runtime >= 0;
-}
-
 /* RT IPI pull logic requires IRQ_WORK */
 #if defined(CONFIG_IRQ_WORK) && defined(CONFIG_SMP)
 # define HAVE_RT_PUSH_IPI
@@ -3049,9 +3055,6 @@ extern void resched_curr(struct rq *rq);
 extern void resched_curr_lazy(struct rq *rq);
 extern void resched_cpu(int cpu);
 
-extern void init_rt_bandwidth(struct rt_bandwidth *rt_b, u64 period, u64 runtime);
-extern bool sched_rt_bandwidth_account(struct rt_rq *rt_rq);
-
 extern void init_dl_entity(struct sched_dl_entity *dl_se);
 
 extern void init_cfs_throttle_work(struct task_struct *p);

-- 
2.55.0

[PATCH v3 3/8] rcutorture: Don't try to disable to RT throttling.

[Sebastian Andrzej Siewior]

Commit 450efca7182a5 ("rcutorture: Disable RT throttling for boost
tests") added the ability to disable RT throttling while rcutorture was
run.

Since v6.12 the DL fair_server takes control of throttling and this
interface no longer serves this purpose. That sysctl control still
exists but serves a different cause.

There is no in-kernel interface to provide this feature, one needs to
echo 0 to each /sys/kernel/debug/sched/fair_server/cpu*/runtime file.

Since sysctl_sched_rt_runtime does not do what it once did, remove it
from rcutorture.

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 kernel/rcu/rcu.h        |  2 --
 kernel/rcu/rcutorture.c | 35 -----------------------------------
 2 files changed, 37 deletions(-)

diff --git a/kernel/rcu/rcu.h b/kernel/rcu/rcu.h
index d60444bf3a02d..6d1034e49d411 100644
--- a/kernel/rcu/rcu.h
+++ b/kernel/rcu/rcu.h
@@ -69,8 +69,6 @@
 /* A complete grace period count */
 #define RCU_SEQ_GP (RCU_SEQ_STATE_MASK + 1)
 
-extern int sysctl_sched_rt_runtime;
-
 /*
  * Return the counter portion of a sequence number previously returned
  * by rcu_seq_snap() or rcu_seq_current().
diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c
index 93ca7f06371e4..aecc4da7161bb 100644
--- a/kernel/rcu/rcutorture.c
+++ b/kernel/rcu/rcutorture.c
@@ -1333,39 +1333,6 @@ static unsigned long rcutorture_seq_diff(unsigned long new, unsigned long old)
 	return cur_ops->gp_diff(new, old);
 }
 
-/*
- * RCU torture priority-boost testing.  Runs one real-time thread per
- * CPU for moderate bursts, repeatedly starting grace periods and waiting
- * for them to complete.  If a given grace period takes too long, we assume
- * that priority inversion has occurred.
- */
-
-static int old_rt_runtime = -1;
-
-static void rcu_torture_disable_rt_throttle(void)
-{
-	/*
-	 * Disable RT throttling so that rcutorture's boost threads don't get
-	 * throttled. Only possible if rcutorture is built-in otherwise the
-	 * user should manually do this by setting the sched_rt_period_us and
-	 * sched_rt_runtime sysctls.
-	 */
-	if (!IS_BUILTIN(CONFIG_RCU_TORTURE_TEST) || old_rt_runtime != -1)
-		return;
-
-	old_rt_runtime = sysctl_sched_rt_runtime;
-	sysctl_sched_rt_runtime = -1;
-}
-
-static void rcu_torture_enable_rt_throttle(void)
-{
-	if (!IS_BUILTIN(CONFIG_RCU_TORTURE_TEST) || old_rt_runtime == -1)
-		return;
-
-	sysctl_sched_rt_runtime = old_rt_runtime;
-	old_rt_runtime = -1;
-}
-
 static bool rcu_torture_boost_failed(unsigned long gp_state, unsigned long *start)
 {
 	int cpu;
@@ -3390,7 +3357,6 @@ static int rcutorture_booster_cleanup(unsigned int cpu)
 	mutex_lock(&boost_mutex);
 	t = boost_tasks[cpu];
 	boost_tasks[cpu] = NULL;
-	rcu_torture_enable_rt_throttle();
 	mutex_unlock(&boost_mutex);
 
 	/* This must be outside of the mutex, otherwise deadlock! */
@@ -3428,7 +3394,6 @@ static int rcutorture_booster_init(unsigned int cpu)
 
 	/* Don't allow time recalculation while creating a new task. */
 	mutex_lock(&boost_mutex);
-	rcu_torture_disable_rt_throttle();
 	VERBOSE_TOROUT_STRING("Creating rcu_torture_boost task");
 	boost_tasks[cpu] = kthread_run_on_cpu(rcu_torture_boost, NULL,
 					      cpu, "rcu_torture_boost_%u");

-- 
2.55.0

[PATCH v3 4/8] sched: rcutorture: Allow to disable DL fair_server

[Sebastian Andrzej Siewior]

The rcutorture test used to disable RT throttling only in the built-in
mode and this functionality got lost while RT throttling was moved to
the fair_server.
Enabling this option (built-in) slows down the system considerably.
Therefore it makes little sense to enable the fair_server once the test
is done since a reboot comes next.

The fair server can not be disabled from the hotplug notifier because
the CPU must be marked active. Instead of this off-switch an alternative
would be to skip sched_init_dl_servers().

Add dl_server_fair_disable() do disable the fair_server which can only
be used by rcutorture. Use dl_server_fair_disable() in
rcu_torture_boost() which is invoked on the CPU which should have the
fair_server disabled.

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 kernel/rcu/rcutorture.c |  4 ++++
 kernel/sched/deadline.c | 28 ++++++++++++++++++++++++++++
 2 files changed, 32 insertions(+)

diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c
index aecc4da7161bb..5c434cb3ef732 100644
--- a/kernel/rcu/rcutorture.c
+++ b/kernel/rcu/rcutorture.c
@@ -1333,6 +1333,8 @@ static unsigned long rcutorture_seq_diff(unsigned long new, unsigned long old)
 	return cur_ops->gp_diff(new, old);
 }
 
+void dl_server_fair_disable(unsigned int cpu);
+
 static bool rcu_torture_boost_failed(unsigned long gp_state, unsigned long *start)
 {
 	int cpu;
@@ -1402,6 +1404,8 @@ static int rcu_torture_boost(void *arg)
 		}
 		VERBOSE_TOROUT_STRING("rcu_torture_boost finished holdoff period");
 	}
+	if (IS_BUILTIN(CONFIG_RCU_TORTURE_TEST))
+		dl_server_fair_disable(smp_processor_id());
 
 	/* Set real-time priority. */
 	sched_set_fifo_low(current);
diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index 10beb54281881..a713ad9dd6208 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -1950,6 +1950,34 @@ int dl_server_apply_params(struct sched_dl_entity *dl_se, u64 runtime, u64 perio
 	return 0;
 }
 
+#ifdef CONFIG_RCU_TORTURE_TEST
+/*
+ * The RCU-TORTURE built-in test will disable fair-server to avoid any
+ * throttling while testing RCU-boosting.
+ * If compiled as module then the user needs to disable it manually before
+ * loading the module.
+ */
+void dl_server_fair_disable(unsigned int cpu);
+
+void dl_server_fair_disable(unsigned int cpu)
+{
+	struct rq *rq = cpu_rq(cpu);
+	int retval;
+
+	scoped_guard (rq_lock_irqsave, rq) {
+		struct sched_dl_entity *dl_se = &rq->fair_server;
+
+		if (dl_se->dl_runtime == 0)
+			return;
+
+		update_rq_clock(rq);
+		dl_server_stop(dl_se);
+		retval = dl_server_apply_params(dl_se, 0, dl_se->dl_period, 0);
+	}
+	WARN_ON_ONCE(retval);
+}
+#endif
+
 /*
  * Add @dl_se's bw to the root-domain accounting.
  *

-- 
2.55.0

[PATCH v3 5/8] sched: Move DL's admission control to sysfs

[Sebastian Andrzej Siewior]

SCHED_DL uses the sysctl sched_rt_{period|runtime}_us to configure its
admission control. It is used to verify that it does not allocate more
time gobally to SCHED_DL tasks than it can theoretically handle.

This sysctl used to control RT throttling. With the introduction of the
DL server this knob controls RT throttling within a RT-cgroup if
CONFIG_RT_GROUP_SCHED is enabled. In this case the knob is also stuck
and can not be changed. So it can not be used for SCHED_DL control at
all. Unless to disable it by writing -1.

Split the DL bits into its knob at sysfs' location. Once the DL logic
is out, sched_domains_mutex.*() and rebuild_sched_domains() can be
removed from sched_rt_handler().

Move SCHED_DL usage of this sysctl to /sys/kernel/sched/ as
dl_runtime_us + dl_period_us. Replace global_rt_period()/
global_rt_runtime() with sysfs_dl_period/ sysfs_dl_runtime. Store the
new attribues in ns. Move global_rt_period()/ global_rt_runtime() to
CONFIG_RT_GROUP_SCHED.

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 Documentation/arch/arm64/asymmetric-32bit.rst |   2 +-
 Documentation/scheduler/sched-deadline.rst    |  15 +--
 kernel/sched/deadline.c                       | 165 ++++++++++++++++++++++++--
 kernel/sched/rt.c                             |  25 +---
 kernel/sched/sched.h                          |  31 +++--
 5 files changed, 178 insertions(+), 60 deletions(-)

diff --git a/Documentation/arch/arm64/asymmetric-32bit.rst b/Documentation/arch/arm64/asymmetric-32bit.rst
index fc0c350c5e001..375958f8971e5 100644
--- a/Documentation/arch/arm64/asymmetric-32bit.rst
+++ b/Documentation/arch/arm64/asymmetric-32bit.rst
@@ -105,7 +105,7 @@ affinity for the task is updated and any saved mask from a prior
 Explicit admission of a 32-bit deadline task to the default root domain
 (e.g. by calling ``sched_setattr(2)``) is rejected on an asymmetric
 32-bit system unless admission control is disabled by writing -1 to
-``/proc/sys/kernel/sched_rt_runtime_us``.
+``/sys/kernel/sched/dl_runtime_us``.
 
 ``execve(2)`` of a 32-bit program from a 64-bit deadline task will
 return ``-ENOEXEC`` if the root domain for the task contains any
diff --git a/Documentation/scheduler/sched-deadline.rst b/Documentation/scheduler/sched-deadline.rst
index 9019b66f6a5ba..23ea88b2ed7dc 100644
--- a/Documentation/scheduler/sched-deadline.rst
+++ b/Documentation/scheduler/sched-deadline.rst
@@ -590,24 +590,19 @@ Deadline Task Scheduling
 4.1 System wide settings
 ------------------------
 
- The system wide settings are configured under the /proc virtual file system.
+ The system wide settings are configured under /sys/kernel/sched.
 
- For now the -rt knobs are used for -deadline admission control and with
- CONFIG_RT_GROUP_SCHED the -deadline runtime is accounted against the (root)
- -rt runtime. With !CONFIG_RT_GROUP_SCHED the knob only serves for the -dl
- admission control. We realize that this isn't entirely desirable; however, it
- is better to have a small interface for now, and be able to change it easily
- later. The ideal situation (see 5.) is to run -rt tasks from a -deadline
- server; in which case the -rt bandwidth is a direct subset of dl_bw.
+ The ideal situation (see 5.) is to run -rt tasks from a -deadline server; in
+ which case the -rt bandwidth is a direct subset of dl_bw.
 
  This means that, for a root_domain comprising M CPUs, -deadline tasks
  can be created while the sum of their bandwidths stays below:
 
-   M * (sched_rt_runtime_us / sched_rt_period_us)
+   M * (dl_runtime_us / dl_period_us)
 
  It is also possible to disable this bandwidth management logic, and
  be thus free of oversubscribing the system up to any arbitrary level.
- This is done by writing -1 in /proc/sys/kernel/sched_rt_runtime_us.
+ This is done by writing -1 in /sys/kernel/sched/dl_runtime_us.
 
 
 4.2 Task interface
diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index a713ad9dd6208..839ae9a6acf63 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -23,6 +23,14 @@
 #include "sched.h"
 #include "pelt.h"
 
+/*
+ * Admission control for DL tasks. For a root_domain comprising M CPUs,
+ * -deadline tasks can be created while the sum of their bandwidths stays
+ *  below:  M * (dl_runtime / dl_period)
+ */
+static u64 sysfs_dl_period  = 1000000 * NSEC_PER_USEC;
+u64 sysfs_dl_runtime = 1000000 * NSEC_PER_USEC;
+
 /*
  * Default limits for DL period; on the top end we guard against small util
  * tasks still getting ridiculously long effective runtimes, on the bottom end we
@@ -509,10 +517,10 @@ static void init_dl_rq_bw_ratio(struct dl_rq *dl_rq);
 void init_dl_bw(struct dl_bw *dl_b)
 {
 	raw_spin_lock_init(&dl_b->lock);
-	if (global_rt_runtime() == RUNTIME_INF)
+	if (sysfs_dl_runtime == RUNTIME_INF)
 		dl_b->bw = -1;
 	else
-		dl_b->bw = to_ratio(global_rt_period(), global_rt_runtime());
+		dl_b->bw = to_ratio(sysfs_dl_period, sysfs_dl_runtime);
 	dl_b->total_bw = 0;
 }
 
@@ -3718,10 +3726,8 @@ DEFINE_SCHED_CLASS(dl) = {
  */
 u64 dl_cookie;
 
-int sched_dl_global_validate(void)
+static int sched_dl_global_validate(u64 runtime, u64 period)
 {
-	u64 runtime = global_rt_runtime();
-	u64 period = global_rt_period();
 	u64 new_bw = to_ratio(period, runtime);
 	u64 cookie = ++dl_cookie;
 	struct dl_bw *dl_b;
@@ -3759,18 +3765,18 @@ int sched_dl_global_validate(void)
 
 static void init_dl_rq_bw_ratio(struct dl_rq *dl_rq)
 {
-	if (global_rt_runtime() == RUNTIME_INF) {
+	if (sysfs_dl_runtime == RUNTIME_INF) {
 		dl_rq->bw_ratio = 1 << RATIO_SHIFT;
 		dl_rq->max_bw = dl_rq->extra_bw = 1 << BW_SHIFT;
 	} else {
-		dl_rq->bw_ratio = to_ratio(global_rt_runtime(),
-			  global_rt_period()) >> (BW_SHIFT - RATIO_SHIFT);
+		dl_rq->bw_ratio = to_ratio(sysfs_dl_runtime,
+			  sysfs_dl_period) >> (BW_SHIFT - RATIO_SHIFT);
 		dl_rq->max_bw = dl_rq->extra_bw =
-			to_ratio(global_rt_period(), global_rt_runtime());
+			to_ratio(sysfs_dl_period, sysfs_dl_runtime);
 	}
 }
 
-void sched_dl_do_global(void)
+static void sched_dl_do_global(void)
 {
 	u64 new_bw = -1;
 	u64 cookie = ++dl_cookie;
@@ -3778,8 +3784,8 @@ void sched_dl_do_global(void)
 	int cpu;
 	unsigned long flags;
 
-	if (global_rt_runtime() != RUNTIME_INF)
-		new_bw = to_ratio(global_rt_period(), global_rt_runtime());
+	if (sysfs_dl_runtime != RUNTIME_INF)
+		new_bw = to_ratio(sysfs_dl_period, sysfs_dl_runtime);
 
 	for_each_possible_cpu(cpu)
 		init_dl_rq_bw_ratio(&cpu_rq(cpu)->dl);
@@ -4130,3 +4136,138 @@ void print_dl_stats(struct seq_file *m, int cpu)
 {
 	print_dl_rq(m, cpu, &cpu_rq(cpu)->dl);
 }
+
+#define SCHED_ATTR(_name)					\
+	static struct kobj_attribute sched_attr_## _name =	\
+	__ATTR(_name, 0644, sched_attr_##_name##_show, sched_attr_##_name##_store)
+
+static int sched_dl_try_boundary(int runtime, int period)
+{
+	static DEFINE_MUTEX(sched_dl_update_lock);
+	int ret;
+	u64 runtime_ns;
+	u64 period_ns;
+
+	guard(mutex)(&sched_dl_update_lock);
+	if (runtime) {
+		if (runtime < 0)
+			runtime_ns = RUNTIME_INF;
+		else
+			runtime_ns = (u64)runtime * NSEC_PER_USEC;
+	} else {
+		runtime_ns = sysfs_dl_runtime;
+	}
+
+	if (period)
+		period_ns = (u64)period * NSEC_PER_USEC;
+	else
+		period_ns = sysfs_dl_period;
+
+	if (runtime_ns != RUNTIME_INF) {
+		if (runtime_ns > period_ns || runtime_ns > MAX_BW)
+			return -EINVAL;
+	}
+
+	sched_domains_mutex_lock();
+	ret = sched_dl_global_validate(runtime_ns, period_ns);
+	if (ret) {
+		sched_domains_mutex_unlock();
+		return ret;
+	}
+
+	sysfs_dl_period = period_ns;
+	sysfs_dl_runtime = runtime_ns;
+
+	sched_dl_do_global();
+	sched_domains_mutex_unlock();
+	/*
+	 * After changing maximum available bandwidth for DEADLINE, we need to
+	 * recompute per root domain and per cpus variables accordingly.
+	 */
+	rebuild_sched_domains();
+	return 0;
+}
+
+static ssize_t sched_attr_dl_runtime_us_show(struct kobject *kobj,
+					     struct kobj_attribute *attr, char *buf)
+{
+	u64 val = sysfs_dl_runtime;
+
+	if (val == RUNTIME_INF)
+		val = -1;
+	else
+		do_div(val, NSEC_PER_USEC);
+
+	return sysfs_emit(buf, "%lld\n", val);
+}
+
+static ssize_t sched_attr_dl_runtime_us_store(struct kobject *kobj,
+					      struct kobj_attribute *attr,
+					      const char *buf, size_t count)
+{
+	int val;
+	int ret;
+
+	ret = kstrtoint(buf, 0, &val);
+	if (ret)
+		return ret;
+
+	if (!(val == -1 || val > 0))
+		return -EINVAL;
+
+	ret = sched_dl_try_boundary(val, 0);
+	if (ret)
+		return ret;
+	return count;
+}
+SCHED_ATTR(dl_runtime_us);
+
+static ssize_t sched_attr_dl_period_us_show(struct kobject *kobj,
+					    struct kobj_attribute *attr, char *buf)
+{
+	u64 val = sysfs_dl_period;
+
+	do_div(val, NSEC_PER_USEC);
+	return sysfs_emit(buf, "%lld\n", val);
+}
+
+static ssize_t sched_attr_dl_period_us_store(struct kobject *kobj,
+					     struct kobj_attribute *attr,
+					     const char *buf, size_t count)
+{
+	int val;
+	int ret;
+
+	ret = kstrtoint(buf, 0, &val);
+	if (ret)
+		return ret;
+	if (val <= 0)
+		return -EINVAL;
+
+	ret = sched_dl_try_boundary(0, val);
+	if (ret)
+		return ret;
+	return count;
+}
+SCHED_ATTR(dl_period_us);
+
+static struct attribute *sched_global_attrs[] = {
+	&sched_attr_dl_runtime_us.attr,
+	&sched_attr_dl_period_us.attr,
+	NULL,
+};
+
+static const struct attribute_group sched_global_attrs_group = {
+	.attrs = sched_global_attrs,
+};
+
+static int __init sched_dl_init(void)
+{
+	struct kobject *sched_kobj;
+
+	sched_kobj = kobject_create_and_add("sched", kernel_kobj);
+	if (!sched_kobj)
+		return -ENOMEM;
+	return sysfs_create_group(sched_kobj, &sched_global_attrs_group);
+}
+late_initcall(sched_dl_init);
diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
index cdeddd3fc9af2..1a6450f91d567 100644
--- a/kernel/sched/rt.c
+++ b/kernel/sched/rt.c
@@ -2871,7 +2871,6 @@ static int sched_rt_handler(const struct ctl_table *table, int write, void *buff
 	int ret;
 
 	mutex_lock(&mutex);
-	sched_domains_mutex_lock();
 	old_period = sysctl_sched_rt_period;
 	old_runtime = sysctl_sched_rt_runtime;
 
@@ -2879,29 +2878,13 @@ static int sched_rt_handler(const struct ctl_table *table, int write, void *buff
 
 	if (!ret && write) {
 		ret = sched_rt_global_validate();
-		if (ret)
-			goto undo;
-
-		ret = sched_dl_global_validate();
-		if (ret)
-			goto undo;
-
-		sched_dl_do_global();
+		if (ret) {
+			sysctl_sched_rt_period = old_period;
+			sysctl_sched_rt_runtime = old_runtime;
+		}
 	}
-	if (0) {
-undo:
-		sysctl_sched_rt_period = old_period;
-		sysctl_sched_rt_runtime = old_runtime;
-	}
-	sched_domains_mutex_unlock();
 	mutex_unlock(&mutex);
 
-	/*
-	 * After changing maximum available bandwidth for DEADLINE, we need to
-	 * recompute per root domain and per cpus variables accordingly.
-	 */
-	rebuild_sched_domains();
-
 	return ret;
 }
 
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index a7964537bac75..00a3802d0ce04 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -109,6 +109,7 @@ extern long calc_load_fold_active(struct rq *this_rq, long adjust);
 
 extern void call_trace_sched_update_nr_running(struct rq *rq, int count);
 
+extern u64 sysfs_dl_runtime;
 extern int sysctl_sched_rt_period;
 extern int sysctl_sched_rt_runtime;
 extern int sched_rr_timeslice;
@@ -326,7 +327,7 @@ struct rt_bandwidth {
 
 static inline int dl_bandwidth_enabled(void)
 {
-	return sysctl_sched_rt_runtime >= 0;
+	return sysfs_dl_runtime >= 0 && sysfs_dl_runtime != RUNTIME_INF;
 }
 
 /*
@@ -354,8 +355,6 @@ struct dl_bw {
 };
 
 extern void init_dl_bw(struct dl_bw *dl_b);
-extern int  sched_dl_global_validate(void);
-extern void sched_dl_do_global(void);
 extern int  sched_dl_overflow(struct task_struct *p, int policy, const struct sched_attr *attr);
 extern void __setparam_dl(struct task_struct *p, const struct sched_attr *attr);
 extern void __getparam_dl(struct task_struct *p, struct sched_attr *attr, unsigned int flags);
@@ -599,6 +598,19 @@ static inline int rt_bandwidth_enabled(void)
 {
 	return sysctl_sched_rt_runtime >= 0;
 }
+
+static inline u64 global_rt_period(void)
+{
+	return (u64)sysctl_sched_rt_period * NSEC_PER_USEC;
+}
+
+static inline u64 global_rt_runtime(void)
+{
+	if (sysctl_sched_rt_runtime < 0)
+		return RUNTIME_INF;
+
+	return (u64)sysctl_sched_rt_runtime * NSEC_PER_USEC;
+}
 #endif
 
 extern struct task_group *sched_create_group(struct task_group *parent);
@@ -2505,19 +2517,6 @@ extern union sched_feat_key sched_feat_keys[__SCHED_FEAT_NR];
 extern struct static_key_false sched_numa_balancing;
 extern struct static_key_false sched_schedstats;
 
-static inline u64 global_rt_period(void)
-{
-	return (u64)sysctl_sched_rt_period * NSEC_PER_USEC;
-}
-
-static inline u64 global_rt_runtime(void)
-{
-	if (sysctl_sched_rt_runtime < 0)
-		return RUNTIME_INF;
-
-	return (u64)sysctl_sched_rt_runtime * NSEC_PER_USEC;
-}
-
 /*
  * Is p the current execution context?
  */

-- 
2.55.0

[PATCH v3 6/8] sched: Limit sysctl sched_rt_{period|runtime}_us to RT_GROUP_SCHED

[Sebastian Andrzej Siewior]

The admission control for SCHED_DL has been moved to sysfs and RT
throttling has been moved (long ago) to the fair_server and its debugfs
interface.

This file remains to have an impact only on RT_GROUP_SCHED. Keeping it
for ABI/ compatibility reasons makes no sense because its original
purpose (RT throttling) is no longer served. Anyone changing this file
and hoping to change the throttling behaviour will be surprised.

Limit sysctl sched_rt_{period|runtime}_us to RT_GROUP_SCHED so everyone
using this outdated file is aware.

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 kernel/sched/rt.c    | 34 ++++++++++++++++++++++------------
 kernel/sched/sched.h |  5 +++--
 2 files changed, 25 insertions(+), 14 deletions(-)

diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
index 1a6450f91d567..abe8e05942249 100644
--- a/kernel/sched/rt.c
+++ b/kernel/sched/rt.c
@@ -8,6 +8,8 @@
 #include "pelt.h"
 
 int sched_rr_timeslice = RR_TIMESLICE;
+
+#ifdef CONFIG_RT_GROUP_SCHED
 /* More than 4 hours if BW_SHIFT equals 20. */
 static const u64 max_rt_runtime = MAX_BW;
 
@@ -22,14 +24,26 @@ int sysctl_sched_rt_period = 1000000;
  * default: 1s
  */
 int sysctl_sched_rt_runtime = 1000000;
+#endif
 
 #ifdef CONFIG_SYSCTL
 static int sysctl_sched_rr_timeslice = (MSEC_PER_SEC * RR_TIMESLICE) / HZ;
-static int sched_rt_handler(const struct ctl_table *table, int write, void *buffer,
-		size_t *lenp, loff_t *ppos);
 static int sched_rr_handler(const struct ctl_table *table, int write, void *buffer,
 		size_t *lenp, loff_t *ppos);
 static const struct ctl_table sched_rt_sysctls[] = {
+	{
+		.procname       = "sched_rr_timeslice_ms",
+		.data           = &sysctl_sched_rr_timeslice,
+		.maxlen         = sizeof(int),
+		.mode           = 0644,
+		.proc_handler   = sched_rr_handler,
+	},
+};
+
+#ifdef CONFIG_RT_GROUP_SCHED
+static int sched_rt_handler(const struct ctl_table *table, int write, void *buffer,
+		size_t *lenp, loff_t *ppos);
+static const struct ctl_table sched_rt_group_sysctls[] = {
 	{
 		.procname       = "sched_rt_period_us",
 		.data           = &sysctl_sched_rt_period,
@@ -48,18 +62,15 @@ static const struct ctl_table sched_rt_sysctls[] = {
 		.extra1         = SYSCTL_NEG_ONE,
 		.extra2         = (void *)&sysctl_sched_rt_period,
 	},
-	{
-		.procname       = "sched_rr_timeslice_ms",
-		.data           = &sysctl_sched_rr_timeslice,
-		.maxlen         = sizeof(int),
-		.mode           = 0644,
-		.proc_handler   = sched_rr_handler,
-	},
 };
+#endif
 
 static int __init sched_rt_sysctl_init(void)
 {
 	register_sysctl_init("kernel", sched_rt_sysctls);
+#ifdef CONFIG_RT_GROUP_SCHED
+	register_sysctl_init("kernel", sched_rt_group_sysctls);
+#endif
 	return 0;
 }
 late_initcall(sched_rt_sysctl_init);
@@ -2845,6 +2856,7 @@ int sched_rt_can_attach(struct task_group *tg, struct task_struct *tsk)
 #endif /* !CONFIG_RT_GROUP_SCHED */
 
 #ifdef CONFIG_SYSCTL
+#ifdef CONFIG_RT_GROUP_SCHED
 static int sched_rt_global_validate(void)
 {
 	if ((sysctl_sched_rt_runtime != RUNTIME_INF) &&
@@ -2853,14 +2865,11 @@ static int sched_rt_global_validate(void)
 			NSEC_PER_USEC > max_rt_runtime)))
 		return -EINVAL;
 
-#ifdef CONFIG_RT_GROUP_SCHED
 	if (!rt_group_sched_enabled())
 		return 0;
 
 	scoped_guard(mutex, &rt_constraints_mutex)
 		return __rt_schedulable(NULL, 0, 0);
-#endif
-	return 0;
 }
 
 static int sched_rt_handler(const struct ctl_table *table, int write, void *buffer,
@@ -2887,6 +2896,7 @@ static int sched_rt_handler(const struct ctl_table *table, int write, void *buff
 
 	return ret;
 }
+#endif /* !CONFIG_RT_GROUP_SCHED */
 
 static int sched_rr_handler(const struct ctl_table *table, int write, void *buffer,
 		size_t *lenp, loff_t *ppos)
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 00a3802d0ce04..966fe46fb38e7 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -110,8 +110,6 @@ extern long calc_load_fold_active(struct rq *this_rq, long adjust);
 extern void call_trace_sched_update_nr_running(struct rq *rq, int count);
 
 extern u64 sysfs_dl_runtime;
-extern int sysctl_sched_rt_period;
-extern int sysctl_sched_rt_runtime;
 extern int sched_rr_timeslice;
 
 /*
@@ -583,6 +581,9 @@ extern void unthrottle_cfs_rq(struct cfs_rq *cfs_rq);
 extern bool cfs_task_bw_constrained(struct task_struct *p);
 
 #ifdef CONFIG_RT_GROUP_SCHED
+extern int sysctl_sched_rt_period;
+extern int sysctl_sched_rt_runtime;
+
 extern void init_tg_rt_entry(struct task_group *tg, struct rt_rq *rt_rq,
 		struct sched_rt_entity *rt_se, int cpu,
 		struct sched_rt_entity *parent);

-- 
2.55.0

[PATCH v3 7/8] sched: Add sysfs files for the per-CPU fair_server knobs

[Sebastian Andrzej Siewior]

The fair_server exposes the runtime and period knobs per-CPU in the
debugfs folder. The debugfs interface is not a stable ABI.
Make the /sys/kernel/debug/sched/fair_server/cpu*/{period|runtime} knobs
available as /sys/kernel/sched/cpuX/{fair_server_period|fair_server_runtime}.

It adds a struct kobject to the rq and exposes only those two. The
per-runqueue folder is named `cpu' rather than `rq' since it might be
more intuitive to the user than `rq' or `runqueue'.

The set logic is pulled from the debugfs folder but it is still shared.
That debugfs logic could be removed (after providing the same for
ext-server) or symlinked for a while.

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 kernel/sched/deadline.c | 152 +++++++++++++++++++++++++++++++++++++++++++++++-
 kernel/sched/debug.c    |  56 ++----------------
 kernel/sched/sched.h    |   8 +++
 3 files changed, 163 insertions(+), 53 deletions(-)

diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index 839ae9a6acf63..6cc1a5f53735f 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -4137,6 +4137,59 @@ void print_dl_stats(struct seq_file *m, int cpu)
 	print_dl_rq(m, cpu, &cpu_rq(cpu)->dl);
 }
 
+static unsigned long dl_server_period_max = (1UL << 22) * NSEC_PER_USEC; /* ~4 seconds */
+static unsigned long dl_server_period_min = (100) * NSEC_PER_USEC;     /* 100 us */
+
+int __sched_server_write_common(enum dl_param param, u64 value, struct rq *rq,
+				struct sched_dl_entity *dl_se, const char *sname)
+{
+	u64 old_runtime, runtime, period;
+	int retval = 0;
+
+	scoped_guard (rq_lock_irqsave, rq) {
+		old_runtime = runtime = dl_se->dl_runtime;
+		period = dl_se->dl_period;
+
+		switch (param) {
+		case DL_RUNTIME:
+			if (runtime == value)
+				break;
+			runtime = value;
+			break;
+		case DL_PERIOD:
+			if (value == period)
+				break;
+			period = value;
+			break;
+		}
+
+		if (runtime > period ||
+		    period > dl_server_period_max ||
+		    period < dl_server_period_min) {
+			return  -EINVAL;
+		}
+
+		if (!cpu_online(cpu_of(rq)))
+			return -EBUSY;
+
+		update_rq_clock(rq);
+		dl_server_stop(dl_se);
+		retval = dl_server_apply_params(dl_se, runtime, period, 0);
+		dl_server_start(dl_se);
+
+		if (retval < 0)
+			return retval;
+	}
+	if (!!old_runtime ^ !!runtime) {
+		pr_info("%s server %sabled on CPU %d%s.\n",
+			sname,
+			runtime ? "en" : "dis",
+			cpu_of(rq),
+			runtime ? "" : ", system may malfunction due to starvation");
+	}
+	return 0;
+}
+
 #define SCHED_ATTR(_name)					\
 	static struct kobj_attribute sched_attr_## _name =	\
 	__ATTR(_name, 0644, sched_attr_##_name##_show, sched_attr_##_name##_store)
@@ -4261,13 +4314,110 @@ static const struct attribute_group sched_global_attrs_group = {
 	.attrs = sched_global_attrs,
 };
 
+/* RQ */
+#define RQ_ATTR(_name)						\
+	static struct kobj_attribute rq_## _name =	\
+	__ATTR(_name, 0644, rq_##_name##_show, rq_##_name##_store)
+
+#define kobj_to_rq(__kobj) container_of(__kobj, struct rq, rq_kobj)
+
+static ssize_t rq_fair_server_runtime_show(struct kobject *kobj, struct kobj_attribute *attr,
+					   char *buf)
+{
+	struct rq *rq = kobj_to_rq(kobj);
+	struct sched_dl_entity *dl_se;
+
+	dl_se = &rq->fair_server;
+
+	return sysfs_emit(buf, "%llu\n", dl_se->dl_runtime);
+}
+
+static ssize_t rq_fair_server_runtime_store(struct kobject *kobj, struct kobj_attribute *attr,
+					    const char *buf, size_t count)
+{
+	struct rq *rq = kobj_to_rq(kobj);
+	u64 val;
+	int ret;
+
+	ret = kstrtoull(buf, 10, &val);
+	if (ret)
+		return ret;
+
+	ret =  __sched_server_write_common(DL_RUNTIME, val, rq, &rq->fair_server, "Fair");
+	if (ret)
+		return ret;
+	return count;
+}
+RQ_ATTR(fair_server_runtime);
+
+static ssize_t rq_fair_server_period_show(struct kobject *kobj, struct kobj_attribute *attr,
+					  char *buf)
+{
+	struct rq *rq = kobj_to_rq(kobj);
+	struct sched_dl_entity *dl_se;
+
+	dl_se = &rq->fair_server;
+
+	return sysfs_emit(buf, "%llu\n", dl_se->dl_period);
+}
+
+static ssize_t rq_fair_server_period_store(struct kobject *kobj, struct kobj_attribute *attr,
+					   const char *buf, size_t count)
+{
+	struct rq *rq = kobj_to_rq(kobj);
+	u64 val;
+	int ret;
+
+	ret = kstrtoull(buf, 10, &val);
+	if (ret)
+		return ret;
+
+	ret =  __sched_server_write_common(DL_PERIOD, val, rq, &rq->fair_server, "Fair");
+	if (ret)
+		return ret;
+	return count;
+}
+RQ_ATTR(fair_server_period);
+
+static struct attribute *rq_attrs[] = {
+	&rq_fair_server_runtime.attr,
+	&rq_fair_server_period.attr,
+	NULL
+};
+ATTRIBUTE_GROUPS(rq);
+
+static const struct kobj_type rq_ktype = {
+	.sysfs_ops      = &kobj_sysfs_ops,
+	.default_groups = rq_groups,
+};
+
+static int sched_create_rq_sysfs(struct kobject *sched_kobj)
+{
+	unsigned int cpu;
+	int ret;
+
+	for_each_possible_cpu(cpu) {
+		struct rq *rq = cpu_rq(cpu);
+
+		ret = kobject_init_and_add(&rq->rq_kobj, &rq_ktype, sched_kobj,
+					   "cpu%d", cpu);
+		if (ret)
+			return ret;
+	}
+	return 0;
+}
+
 static int __init sched_dl_init(void)
 {
 	struct kobject *sched_kobj;
+	int ret;
 
 	sched_kobj = kobject_create_and_add("sched", kernel_kobj);
 	if (!sched_kobj)
 		return -ENOMEM;
-	return sysfs_create_group(sched_kobj, &sched_global_attrs_group);
+	ret = sysfs_create_group(sched_kobj, &sched_global_attrs_group);
+	if (ret)
+		return ret;
+	return sched_create_rq_sysfs(sched_kobj);
 }
 late_initcall(sched_dl_init);
diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c
index 1545139e52443..0e0782603a99a 100644
--- a/kernel/sched/debug.c
+++ b/kernel/sched/debug.c
@@ -363,23 +363,13 @@ static const struct file_operations sched_debug_fops = {
 	.release	= seq_release,
 };
 
-enum dl_param {
-	DL_RUNTIME = 0,
-	DL_PERIOD,
-};
-
-static unsigned long dl_server_period_max = (1UL << 22) * NSEC_PER_USEC; /* ~4 seconds */
-static unsigned long dl_server_period_min = (100) * NSEC_PER_USEC;     /* 100 us */
-
 static ssize_t sched_server_write_common(struct file *filp, const char __user *ubuf,
 					 size_t cnt, loff_t *ppos, enum dl_param param,
 					 void *server)
 {
 	long cpu = (long) ((struct seq_file *) filp->private_data)->private;
 	struct sched_dl_entity *dl_se = (struct sched_dl_entity *)server;
-	u64 old_runtime, runtime, period;
 	struct rq *rq = cpu_rq(cpu);
-	int retval = 0;
 	size_t err;
 	u64 value;
 
@@ -387,48 +377,10 @@ static ssize_t sched_server_write_common(struct file *filp, const char __user *u
 	if (err)
 		return err;
 
-	scoped_guard (rq_lock_irqsave, rq) {
-		old_runtime = runtime = dl_se->dl_runtime;
-		period = dl_se->dl_period;
-
-		switch (param) {
-		case DL_RUNTIME:
-			if (runtime == value)
-				break;
-			runtime = value;
-			break;
-		case DL_PERIOD:
-			if (value == period)
-				break;
-			period = value;
-			break;
-		}
-
-		if (runtime > period ||
-		    period > dl_server_period_max ||
-		    period < dl_server_period_min) {
-			return  -EINVAL;
-		}
-
-		if (!cpu_active(cpu_of(rq)))
-			return -EBUSY;
-
-		update_rq_clock(rq);
-		dl_server_stop(dl_se);
-		retval = dl_server_apply_params(dl_se, runtime, period, 0);
-		dl_server_start(dl_se);
-
-		if (retval < 0)
-			return retval;
-	}
-
-	if (!!old_runtime ^ !!runtime) {
-		pr_info("%s server %sabled on CPU %d%s.\n",
-			server == &rq->fair_server ? "Fair" : "Ext",
-			runtime ? "en" : "dis",
-			cpu_of(rq),
-			runtime ? "" : ", system may malfunction due to starvation");
-	}
+	err = __sched_server_write_common(param, value, rq, dl_se,
+					  server == &rq->fair_server ? "Fair" : "Ext");
+	if (err)
+		return err;
 
 	*ppos += cnt;
 	return cnt;
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 966fe46fb38e7..661927a464c95 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -1434,6 +1434,7 @@ struct rq {
 #endif
 
 	atomic_t		nr_iowait;
+	struct kobject		rq_kobj;
 } __no_randomize_layout;
 
 #ifdef CONFIG_FAIR_GROUP_SCHED
@@ -3509,6 +3510,13 @@ extern void print_cfs_rq(struct seq_file *m, int cpu, struct cfs_rq *cfs_rq);
 extern void print_rt_rq(struct seq_file *m, int cpu, struct rt_rq *rt_rq);
 extern void print_dl_rq(struct seq_file *m, int cpu, struct dl_rq *dl_rq);
 
+enum dl_param {
+	DL_RUNTIME = 0,
+	DL_PERIOD,
+};
+extern int __sched_server_write_common(enum dl_param param, u64 value, struct rq *rq,
+				       struct sched_dl_entity *dl_se, const char *sname);
+
 extern void resched_latency_warn(int cpu, u64 latency);
 
 #ifdef CONFIG_NUMA_BALANCING

-- 
2.55.0

[PATCH v3 8/8] Documentation/sched: Add fairserver to sched-deadline

[Sebastian Andrzej Siewior]

The fair_server ensures the fair-class gets CPU run-time and throttles
even the rt-class if needed. Add this bit of wisdom and the recently
added sysfs knobs for configutiation.

Assisted-by: LLM
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 Documentation/scheduler/sched-deadline.rst | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)

diff --git a/Documentation/scheduler/sched-deadline.rst b/Documentation/scheduler/sched-deadline.rst
index 23ea88b2ed7dc..f0d667e09d10c 100644
--- a/Documentation/scheduler/sched-deadline.rst
+++ b/Documentation/scheduler/sched-deadline.rst
@@ -577,7 +577,8 @@ Deadline Task Scheduling
  level.
 
  A main difference between deadline bandwidth management and RT-throttling
- is that -deadline tasks have bandwidth on their own (while -rt ones don't!),
+ is that -deadline tasks have bandwidth on their own (while -rt have only
+ throttling via the fair_server),
  and thus we don't need a higher level throttling mechanism to enforce the
  desired bandwidth. In other words, this means that interface parameters are
  only used at admission control time (i.e., when the user calls
@@ -604,6 +605,24 @@ Deadline Task Scheduling
  be thus free of oversubscribing the system up to any arbitrary level.
  This is done by writing -1 in /sys/kernel/sched/dl_runtime_us.
 
+4.1.1 Fair server settings
+--------------------------
+
+ The fair server ensures that fair-class tasks (SCHED_OTHER) are not starved
+ and can be scheduled even when tasks in the RT scheduling class
+ (SCHED_FIFO and SCHED_RR) are consuming all CPU time. It replaces the
+ legacy RT throttling mechanism, which was configured through the procfs
+ interface.
+
+ The fair server executes fair-class tasks through a deadline-scheduling
+ server with a runtime budget. It guarantees that the runtime configured by
+ fair_server_runtime is provided within the interval defined by
+ fair_server_period.
+
+ Both values are specified in nanoseconds and can be configured on a
+ per-CPU basis through /sys/kernel/sched/cpuX. Setting
+ fair_server_runtime to 0 disables the fair server on the corresponding
+ CPU.
 
 4.2 Task interface
 ------------------

-- 
2.55.0

Re: [PATCH v3 1/8] sched/deadline: Reject debugfs dl_server writes for not-active CPUs.

[Juri Lelli]

Hello,

On 13/11/26 11:50, Sebastian Andrzej Siewior wrote:
> sched_server_write_common() checks if the CPU is online before allowing
> fair_server updates. This is important because updating the fair_server
> runtime/ period will use dl_bw_cpus() will return only the number of
> CPUs which are also marked active.
> 
> The CPU is marked active as the last step during CPU-hotplug. If the CPU
> has been marked "online" but not yet "active" or is stuck between those
> two states then any update of period/ runtime will lead to a divide-by-0
> error.
> 
> Let sched_server_write_common() check of active CPUs instead of online.
> Additonaly, let dl_server_apply_params() return an error if dl_bw_cpus()
> return 0 CPUs to avoid the divide error.
> 
> Fixes: 4043f54984161 ("sched/deadline: Reject debugfs dl_server writes for offline CPUs")
> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> ---
>  kernel/sched/deadline.c | 3 +++
>  kernel/sched/debug.c    | 2 +-
>  2 files changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
> index dfae08625342e..10beb54281881 100644
> --- a/kernel/sched/deadline.c
> +++ b/kernel/sched/deadline.c
> @@ -1920,6 +1920,9 @@ int dl_server_apply_params(struct sched_dl_entity *dl_se, u64 runtime, u64 perio
>  	cpus = dl_bw_cpus(cpu);
>  	cap = dl_bw_capacity(cpu);
>  
> +	if (!cpus)
> +		return -EBUSY;
> +
>  	if (__dl_overflow(dl_b, cap, old_bw, new_bw))
>  		return -EBUSY;
>  
> diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c
> index e6a3b516c703b..1545139e52443 100644
> --- a/kernel/sched/debug.c
> +++ b/kernel/sched/debug.c
> @@ -410,7 +410,7 @@ static ssize_t sched_server_write_common(struct file *filp, const char __user *u
>  			return  -EINVAL;
>  		}
>  
> -		if (!cpu_online(cpu_of(rq)))
> +		if (!cpu_active(cpu_of(rq)))
>  			return -EBUSY;

Mathieu's approach [1] looks safer to me. sched_cpu_deactivate() calls
set_cpu_active(false) without rq->lock, so a write that passed the
cpu_active() check can still be in flight until sched_set_rq_offline()
(which the !cpus check then has to cover). On the online side,
set_cpu_active(true) runs before the domains rebuild, so the check lets
the write through against the def rd's accounting. rq->online is only
changed under the rq->lock this write already holds.

Thanks,
Juri

1 - 20261108185406.1549770-6-mathieu.desnoyers@efficios.com