Re: relax_domain_level boot parameter has no effect

[Dimitri Sivanich <sivanich@sgi.com>]

On Mon, Jun 04, 2012 at 05:37:42PM +0200, Peter Zijlstra wrote:
> On Fri, 2012-06-01 at 16:03 -0500, Dimitri Sivanich wrote:
> > I noticed (and verified) that the relax_domain_level boot parameter does not
> > get processed because sched_domain_level_max is 0 at the time that
> > setup_relax_domain_level() is run.
> > 
> > int sched_domain_level_max;
> > 
> > static int __init setup_relax_domain_level(char *str)
> > {
> >         unsigned long val;
> > 
> >         val = simple_strtoul(str, NULL, 0);
> >         if (val < sched_domain_level_max)
> >                 default_relax_domain_level = val;
> > 
> >         return 1;
> > }
> > __setup("relax_domain_level=", setup_relax_domain_level);
> 
> Ah indeed.. this has been so for a while I guess.

On a very related note, the sched_relax_domain_level setting in cpuset
doesn't appear to work correctly either (if I'm interpreting all of this
correctly).

The reason is that the build_sched_domain() routine calls the
set_domain_attribute() routine prior to setting the sd->level, however,
the set_domain_attribute() routine relies on the sd->level to decide whether
idle load balancing will be off/on.

static void set_domain_attribute(struct sched_domain *sd,
..
==>     if (request < sd->level) {
                /* turn off idle balance on this domain */
..
..
struct sched_domain *build_sched_domain(struct sched_domain_topology_level *tl,
..
==>     set_domain_attribute(sd, attr);
        cpumask_and(sched_domain_span(sd), cpu_map, tl->mask(cpu));
        if (child) {
==>             sd->level = child->level + 1;
                sched_domain_level_max = max(sched_domain_level_max, sd->level);


> What are you using this knob for?

I was poking around for different ways to turn off idle load balancing when
I ran into all of this.