relax_domain_level boot parameter has no effect

relax_domain_level boot parameter has no effect

[Dimitri Sivanich]

I noticed (and verified) that the relax_domain_level boot parameter does not
get processed because sched_domain_level_max is 0 at the time that
setup_relax_domain_level() is run.

int sched_domain_level_max;

static int __init setup_relax_domain_level(char *str)
{
        unsigned long val;

        val = simple_strtoul(str, NULL, 0);
        if (val < sched_domain_level_max)
                default_relax_domain_level = val;

        return 1;
}
__setup("relax_domain_level=", setup_relax_domain_level);


struct sched_domain *build_sched_domain(struct sched_domain_topology_level *tl,
..
                sched_domain_level_max = max(sched_domain_level_max, sd->level);
..

Re: relax_domain_level boot parameter has no effect

[Peter Zijlstra]

On Fri, 2012-06-01 at 16:03 -0500, Dimitri Sivanich wrote:
> I noticed (and verified) that the relax_domain_level boot parameter does not
> get processed because sched_domain_level_max is 0 at the time that
> setup_relax_domain_level() is run.
> 
> int sched_domain_level_max;
> 
> static int __init setup_relax_domain_level(char *str)
> {
>         unsigned long val;
> 
>         val = simple_strtoul(str, NULL, 0);
>         if (val < sched_domain_level_max)
>                 default_relax_domain_level = val;
> 
>         return 1;
> }
> __setup("relax_domain_level=", setup_relax_domain_level);

Ah indeed.. this has been so for a while I guess.

What are you using this knob for?

Re: relax_domain_level boot parameter has no effect

[Dimitri Sivanich]

On Mon, Jun 04, 2012 at 05:37:42PM +0200, Peter Zijlstra wrote:
> On Fri, 2012-06-01 at 16:03 -0500, Dimitri Sivanich wrote:
> > I noticed (and verified) that the relax_domain_level boot parameter does not
> > get processed because sched_domain_level_max is 0 at the time that
> > setup_relax_domain_level() is run.
> > 
> > int sched_domain_level_max;
> > 
> > static int __init setup_relax_domain_level(char *str)
> > {
> >         unsigned long val;
> > 
> >         val = simple_strtoul(str, NULL, 0);
> >         if (val < sched_domain_level_max)
> >                 default_relax_domain_level = val;
> > 
> >         return 1;
> > }
> > __setup("relax_domain_level=", setup_relax_domain_level);
> 
> Ah indeed.. this has been so for a while I guess.

On a very related note, the sched_relax_domain_level setting in cpuset
doesn't appear to work correctly either (if I'm interpreting all of this
correctly).

The reason is that the build_sched_domain() routine calls the
set_domain_attribute() routine prior to setting the sd->level, however,
the set_domain_attribute() routine relies on the sd->level to decide whether
idle load balancing will be off/on.

static void set_domain_attribute(struct sched_domain *sd,
..
==>     if (request < sd->level) {
                /* turn off idle balance on this domain */
..
..
struct sched_domain *build_sched_domain(struct sched_domain_topology_level *tl,
..
==>     set_domain_attribute(sd, attr);
        cpumask_and(sched_domain_span(sd), cpu_map, tl->mask(cpu));
        if (child) {
==>             sd->level = child->level + 1;
                sched_domain_level_max = max(sched_domain_level_max, sd->level);


> What are you using this knob for?

I was poking around for different ways to turn off idle load balancing when
I ran into all of this.

Re: relax_domain_level boot parameter has no effect

[Dimitri Sivanich]

On Mon, Jun 04, 2012 at 01:16:14PM -0500, Dimitri Sivanich wrote:
> On Mon, Jun 04, 2012 at 05:37:42PM +0200, Peter Zijlstra wrote:
> > On Fri, 2012-06-01 at 16:03 -0500, Dimitri Sivanich wrote:
> > > I noticed (and verified) that the relax_domain_level boot parameter does not
> > > get processed because sched_domain_level_max is 0 at the time that
> > > setup_relax_domain_level() is run.
> > > 
> > > int sched_domain_level_max;
> > > 
> > > static int __init setup_relax_domain_level(char *str)
> > > {
> > >         unsigned long val;
> > > 
> > >         val = simple_strtoul(str, NULL, 0);
> > >         if (val < sched_domain_level_max)
> > >                 default_relax_domain_level = val;
> > > 
> > >         return 1;
> > > }
> > > __setup("relax_domain_level=", setup_relax_domain_level);
> > 
> > Ah indeed.. this has been so for a while I guess.
> 
> On a very related note, the sched_relax_domain_level setting in cpuset
> doesn't appear to work correctly either (if I'm interpreting all of this
> correctly).
> 
> The reason is that the build_sched_domain() routine calls the
> set_domain_attribute() routine prior to setting the sd->level, however,
> the set_domain_attribute() routine relies on the sd->level to decide whether
> idle load balancing will be off/on.
> 
> static void set_domain_attribute(struct sched_domain *sd,
> ..
> ==>     if (request < sd->level) {
>                 /* turn off idle balance on this domain */
> ..
> ..
> struct sched_domain *build_sched_domain(struct sched_domain_topology_level *tl,
> ..
> ==>     set_domain_attribute(sd, attr);
>         cpumask_and(sched_domain_span(sd), cpu_map, tl->mask(cpu));
>         if (child) {
> ==>             sd->level = child->level + 1;
>                 sched_domain_level_max = max(sched_domain_level_max, sd->level);
> 
> 
> > What are you using this knob for?
> 
> I was poking around for different ways to turn off idle load balancing when
> I ran into all of this.

I'll submit a patch shortly that should return these back to what appears to
be their intended functionality.

Re: relax_domain_level boot parameter has no effect

[Peter Zijlstra]

On Tue, 2012-06-05 at 13:34 -0500, Dimitri Sivanich wrote:
> 
> I'll submit a patch shortly that should return these back to what appears to
> be their intended functionality. 

I guess removing the max check in the paramater parsing is the easiest
thing.

[PATCH] Fix relax_domain_level interface

[Dimitri Sivanich]

On Tue, Jun 05, 2012 at 08:36:59PM +0200, Peter Zijlstra wrote:
> On Tue, 2012-06-05 at 13:34 -0500, Dimitri Sivanich wrote:
> > 
> > I'll submit a patch shortly that should return these back to what appears to
> > be their intended functionality. 
> 
> I guess removing the max check in the paramater parsing is the easiest
> thing.
My thoughts exactly.



Fix the relax_domain_level boot parameter. It does not get processed because
sched_domain_level_max is 0 at the time that setup_relax_domain_level()
is run.  Simply accept the value as it is, as we don't know the value
of sched_domain_level_max until sched domain construction is completed.

Fix sched_relax_domain_level in cpuset.  The build_sched_domain() routine calls
the set_domain_attribute() routine prior to setting the sd->level, however,
the set_domain_attribute() routine relies on the sd->level to decide whether
idle load balancing will be off/on.

Signed-off-by: Dimitri Sivanich <sivanich@sgi.com>
---
 kernel/sched/core.c |    9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

Index: linux/kernel/sched/core.c
===================================================================
--- linux.orig/kernel/sched/core.c
+++ linux/kernel/sched/core.c
@@ -6165,11 +6165,8 @@ int sched_domain_level_max;
 
 static int __init setup_relax_domain_level(char *str)
 {
-	unsigned long val;
-
-	val = simple_strtoul(str, NULL, 0);
-	if (val < sched_domain_level_max)
-		default_relax_domain_level = val;
+	if (kstrtoint(str, 0, &default_relax_domain_level))
+		pr_warn("Unable to set relax_domain_level\n");
 
 	return 1;
 }
@@ -6543,7 +6540,6 @@ struct sched_domain *build_sched_domain(
 	if (!sd)
 		return child;
 
-	set_domain_attribute(sd, attr);
 	cpumask_and(sched_domain_span(sd), cpu_map, tl->mask(cpu));
 	if (child) {
 		sd->level = child->level + 1;
@@ -6551,6 +6547,7 @@ struct sched_domain *build_sched_domain(
 		child->parent = sd;
 	}
 	sd->child = child;
+	set_domain_attribute(sd, attr);
 
 	return sd;
 }

[tip:sched/urgent] sched: Fix the relax_domain_level boot parameter

[tip-bot for Dimitri Sivanich]

Commit-ID:  a841f8cef4bb124f0f5563314d0beaf2e1249d72
Gitweb:     http://git.kernel.org/tip/a841f8cef4bb124f0f5563314d0beaf2e1249d72
Author:     Dimitri Sivanich <sivanich@sgi.com>
AuthorDate: Tue, 5 Jun 2012 13:44:36 -0500
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Wed, 6 Jun 2012 17:07:41 +0200

sched: Fix the relax_domain_level boot parameter

It does not get processed because sched_domain_level_max is 0 at the
time that setup_relax_domain_level() is run.

Simply accept the value as it is, as we don't know the value of
sched_domain_level_max until sched domain construction is completed.

Fix sched_relax_domain_level in cpuset.  The build_sched_domain() routine calls
the set_domain_attribute() routine prior to setting the sd->level, however,
the set_domain_attribute() routine relies on the sd->level to decide whether
idle load balancing will be off/on.

Signed-off-by: Dimitri Sivanich <sivanich@sgi.com>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/20120605184436.GA15668@sgi.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 kernel/sched/core.c |    9 +++------
 1 files changed, 3 insertions(+), 6 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 2bdd176..d5594a4 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -6268,11 +6268,8 @@ int sched_domain_level_max;
 
 static int __init setup_relax_domain_level(char *str)
 {
-	unsigned long val;
-
-	val = simple_strtoul(str, NULL, 0);
-	if (val < sched_domain_level_max)
-		default_relax_domain_level = val;
+	if (kstrtoint(str, 0, &default_relax_domain_level))
+		pr_warn("Unable to set relax_domain_level\n");
 
 	return 1;
 }
@@ -6698,7 +6695,6 @@ struct sched_domain *build_sched_domain(struct sched_domain_topology_level *tl,
 	if (!sd)
 		return child;
 
-	set_domain_attribute(sd, attr);
 	cpumask_and(sched_domain_span(sd), cpu_map, tl->mask(cpu));
 	if (child) {
 		sd->level = child->level + 1;
@@ -6706,6 +6702,7 @@ struct sched_domain *build_sched_domain(struct sched_domain_topology_level *tl,
 		child->parent = sd;
 	}
 	sd->child = child;
+	set_domain_attribute(sd, attr);
 
 	return sd;
 }