[PATCH 0/2] sched: issues with sched_rt_runtime_us

[PATCH 0/2] sched: issues with sched_rt_runtime_us

[Shrikanth Hegde]

ftrace selftest reported below failure in one of the systems.
[5] Test tracefs GID mount option       [FAIL]

On inspection this happens since writing -1 to sched_rt_runtime_us 
fails. This happened because the possible cpus were more than the 
active cpus. 

Fix this issue by considering only the online CPUs for validating the 
deadline entities. This is minimal impact compared to changing the 
dl_bw_cpus use possible mask. If that is better, we can follow that
approach. 

There is also another issue that is observed when working on it. Writing
any value less than 5% of period results in -EBUSY. This happens since 
fair_server dl_server allocates 5% of the bandwidth and that is part of
the bandwidth of the root domain. Lets document that behaviour since
the file allows writing -1 as the minimum. 

Shrikanth Hegde (2):
  sched/deadline: use online cpus for validating runtime
  sched/rt: update limit of sched_rt sysctl in documentation

 Documentation/scheduler/sched-rt-group.rst | 3 +++
 kernel/sched/deadline.c                    | 2 +-
 2 files changed, 4 insertions(+), 1 deletion(-)

-- 
2.39.3

[PATCH 1/2] sched/deadline: use online cpus for validating runtime

[Shrikanth Hegde]

ftrace selftest reported failure since writing -1 to sched_rt_runtime_us
return -EBUSY. This happens when the possible CPUs is different than
the active CPUS.

Active CPUs are part of one root domain, while remaining CPUs are part
of def_root_domain. Since active cpumask is being used, this results in
cpus=0 when a non active CPUs is used in the loop.

Fix it by looping over the online CPUs instead for validation the
bandwidth calculations.

Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
---
 kernel/sched/deadline.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index 2d0f571f95e9..7ea9c040c72f 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -3187,7 +3187,7 @@ int sched_dl_global_validate(void)
 	 * value smaller than the currently allocated bandwidth in
 	 * any of the root_domains.
 	 */
-	for_each_possible_cpu(cpu) {
+	for_each_online_cpu(cpu) {
 		rcu_read_lock_sched();
 
 		if (dl_bw_visited(cpu, gen))
-- 
2.39.3

[PATCH 2/2] sched/rt: update limit of sched_rt sysctl in documentation

[Shrikanth Hegde]

By default fair_server dl_server allocates 5% of the bandwidth to the root
domain. Due to this writing any value less than 5% fails due to -EBUSY.

cat /proc/sys/kernel/sched_rt_period_us
1000000
$ echo 49999 > /proc/sys/kernel/sched_rt_runtime_us
-bash: echo: write error: Device or resource busy
$ echo 50000 > /proc/sys/kernel/sched_rt_runtime_us
$

Since the sched_rt_runtime_us allows -1 as the minimum, put this
restriction in the documentation.

One should check average of runtime/period in 
/sys/kernel/debug/sched/fair_server/cpuX/* for exact value.

Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
---
 Documentation/scheduler/sched-rt-group.rst | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/Documentation/scheduler/sched-rt-group.rst b/Documentation/scheduler/sched-rt-group.rst
index 80b05a3009ea..ab464335d320 100644
--- a/Documentation/scheduler/sched-rt-group.rst
+++ b/Documentation/scheduler/sched-rt-group.rst
@@ -102,6 +102,9 @@ The system wide settings are configured under the /proc virtual file system:
   * sched_rt_period_us takes values from 1 to INT_MAX.
   * sched_rt_runtime_us takes values from -1 to sched_rt_period_us.
   * A run time of -1 specifies runtime == period, ie. no limit.
+  * sched_rt_runtime_us/sched_rt_period_us > 0.05 inorder to preserve
+    bandwidth for fair dl_server. For accurate value check average of
+    runtime/period in /sys/kernel/debug/sched/fair_server/cpuX/
 
 
 2.2 Default behaviour
-- 
2.39.3

Re: [PATCH 1/2] sched/deadline: use online cpus for validating runtime

[Juri Lelli]

Hi!

On 06/03/25 10:59, Shrikanth Hegde wrote:
> ftrace selftest reported failure since writing -1 to sched_rt_runtime_us
> return -EBUSY. This happens when the possible CPUs is different than
> the active CPUS.
> 
> Active CPUs are part of one root domain, while remaining CPUs are part
> of def_root_domain. Since active cpumask is being used, this results in
> cpus=0 when a non active CPUs is used in the loop.
> 
> Fix it by looping over the online CPUs instead for validation the
> bandwidth calculations.
> 
> Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
> ---
>  kernel/sched/deadline.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
> index 2d0f571f95e9..7ea9c040c72f 100644
> --- a/kernel/sched/deadline.c
> +++ b/kernel/sched/deadline.c
> @@ -3187,7 +3187,7 @@ int sched_dl_global_validate(void)
>  	 * value smaller than the currently allocated bandwidth in
>  	 * any of the root_domains.
>  	 */
> -	for_each_possible_cpu(cpu) {
> +	for_each_online_cpu(cpu) {
>  		rcu_read_lock_sched();
>  
>  		if (dl_bw_visited(cpu, gen))
> -- 

Makes sense to me.

Acked-by: Juri Lelli <juri.lelli@redhat.com>

Thanks,
Juri

Re: [PATCH 2/2] sched/rt: update limit of sched_rt sysctl in documentation

[Juri Lelli]

Hi,

On 06/03/25 10:59, Shrikanth Hegde wrote:
> By default fair_server dl_server allocates 5% of the bandwidth to the root
> domain. Due to this writing any value less than 5% fails due to -EBUSY.
> 
> cat /proc/sys/kernel/sched_rt_period_us
> 1000000
> $ echo 49999 > /proc/sys/kernel/sched_rt_runtime_us
> -bash: echo: write error: Device or resource busy
> $ echo 50000 > /proc/sys/kernel/sched_rt_runtime_us
> $
> 
> Since the sched_rt_runtime_us allows -1 as the minimum, put this
> restriction in the documentation.
> 
> One should check average of runtime/period in 
> /sys/kernel/debug/sched/fair_server/cpuX/* for exact value.
> 
> Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
> ---
>  Documentation/scheduler/sched-rt-group.rst | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/Documentation/scheduler/sched-rt-group.rst b/Documentation/scheduler/sched-rt-group.rst
> index 80b05a3009ea..ab464335d320 100644
> --- a/Documentation/scheduler/sched-rt-group.rst
> +++ b/Documentation/scheduler/sched-rt-group.rst
> @@ -102,6 +102,9 @@ The system wide settings are configured under the /proc virtual file system:
>    * sched_rt_period_us takes values from 1 to INT_MAX.
>    * sched_rt_runtime_us takes values from -1 to sched_rt_period_us.
>    * A run time of -1 specifies runtime == period, ie. no limit.
> +  * sched_rt_runtime_us/sched_rt_period_us > 0.05 inorder to preserve
> +    bandwidth for fair dl_server. For accurate value check average of
> +    runtime/period in /sys/kernel/debug/sched/fair_server/cpuX/

Looks good to me.

Reviewed-by: Juri Lelli <juri.lelli@redhat.com>

Thanks!
Juri

[tip: sched/urgent] sched/rt: Update limit of sched_rt sysctl in documentation

[tip-bot2 for Shrikanth Hegde]

The following commit has been merged into the sched/urgent branch of tip:

Commit-ID:     b1536481c81fb604074da799e4f2d2038a1663f7
Gitweb:        https://git.kernel.org/tip/b1536481c81fb604074da799e4f2d2038a1663f7
Author:        Shrikanth Hegde <sshegde@linux.ibm.com>
AuthorDate:    Thu, 06 Mar 2025 10:59:54 +05:30
Committer:     Ingo Molnar <mingo@kernel.org>
CommitterDate: Thu, 06 Mar 2025 10:21:31 +01:00

sched/rt: Update limit of sched_rt sysctl in documentation

By default fair_server dl_server allocates 5% of the bandwidth to the root
domain. Due to this writing any value less than 5% fails due to -EBUSY:

  $ cat /proc/sys/kernel/sched_rt_period_us
  1000000

  $ echo 49999 > /proc/sys/kernel/sched_rt_runtime_us
  -bash: echo: write error: Device or resource busy

  $ echo 50000 > /proc/sys/kernel/sched_rt_runtime_us
  $

Since the sched_rt_runtime_us allows -1 as the minimum, put this
restriction in the documentation.

One should check average of runtime/period in
/sys/kernel/debug/sched/fair_server/cpuX/* for exact value.

Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Juri Lelli <juri.lelli@redhat.com>
Link: https://lore.kernel.org/r/20250306052954.452005-3-sshegde@linux.ibm.com
---
 Documentation/scheduler/sched-rt-group.rst | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/Documentation/scheduler/sched-rt-group.rst b/Documentation/scheduler/sched-rt-group.rst
index 80b05a3..ab46433 100644
--- a/Documentation/scheduler/sched-rt-group.rst
+++ b/Documentation/scheduler/sched-rt-group.rst
@@ -102,6 +102,9 @@ The system wide settings are configured under the /proc virtual file system:
   * sched_rt_period_us takes values from 1 to INT_MAX.
   * sched_rt_runtime_us takes values from -1 to sched_rt_period_us.
   * A run time of -1 specifies runtime == period, ie. no limit.
+  * sched_rt_runtime_us/sched_rt_period_us > 0.05 inorder to preserve
+    bandwidth for fair dl_server. For accurate value check average of
+    runtime/period in /sys/kernel/debug/sched/fair_server/cpuX/
 
 
 2.2 Default behaviour

[tip: sched/urgent] sched/deadline: Use online cpus for validating runtime

[tip-bot2 for Shrikanth Hegde]

The following commit has been merged into the sched/urgent branch of tip:

Commit-ID:     14672f059d83f591afb2ee1fff56858efe055e5a
Gitweb:        https://git.kernel.org/tip/14672f059d83f591afb2ee1fff56858efe055e5a
Author:        Shrikanth Hegde <sshegde@linux.ibm.com>
AuthorDate:    Thu, 06 Mar 2025 10:59:53 +05:30
Committer:     Ingo Molnar <mingo@kernel.org>
CommitterDate: Thu, 06 Mar 2025 10:21:31 +01:00

sched/deadline: Use online cpus for validating runtime

The ftrace selftest reported a failure because writing -1 to
sched_rt_runtime_us returns -EBUSY. This happens when the possible
CPUs are different from active CPUs.

Active CPUs are part of one root domain, while remaining CPUs are part
of def_root_domain. Since active cpumask is being used, this results in
cpus=0 when a non active CPUs is used in the loop.

Fix it by looping over the online CPUs instead for validating the
bandwidth calculations.

Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Juri Lelli <juri.lelli@redhat.com>
Link: https://lore.kernel.org/r/20250306052954.452005-2-sshegde@linux.ibm.com
---
 kernel/sched/deadline.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index 38e4537..ff4df16 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -3189,7 +3189,7 @@ int sched_dl_global_validate(void)
 	 * value smaller than the currently allocated bandwidth in
 	 * any of the root_domains.
 	 */
-	for_each_possible_cpu(cpu) {
+	for_each_online_cpu(cpu) {
 		rcu_read_lock_sched();
 
 		if (dl_bw_visited(cpu, gen))