Re: scheduler scalability - cgroups, cpusets and load-balancing

[Paul Jackson <pj@sgi.com>]

Peter, responding to Paul:
> > I really doubt we'd want to have such systems triggering the hard RT
> > scheduler on whatever CPUs were in the batch schedulers big cpuset
> > that didn't happened to have an active job currently assigned to them.
> 
> My turn to be confused..
> 
> If SD_LOAD_BALANCE is only set on the smaller, per-job, sets, how will
> the RT balancer trigger on the large set?

What 'sched_load_balance' does now is help you setup a -partial-
covering of non-overlappping sched domains.  In the batch scheduler
example, those CPUs that were:
 1) being managed by the batch scheduler, but
 2) were not assigned to any active job at the moment
would -not- be in any sched domain.

It's not a question of the SC_LOAD_BALANCE flag.  It's a question
of whether a given CPU is even included in any sched domain.

If we did as you are suggesting (if I understand) then instead of
leaving these CPUs out of any sched domain, rather we'd setup a new
kind of sched domain for these CPUs, marked for hard real time load
balancing, rather than the somewhat more scalable, but softer normal
load balancing.

We want no load balancing on those CPUs, not realtime load balancing.
Indeed, I suspect, we especially do not want realtime load balancing
on those CPUs as that kind of load balancing is (I'm suspecting) more
expensive and less scalable than normal load balancing.

-- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@sgi.com> 1.940.382.4214