Re: [PATCH] cpuset and sched domains: sched_load_balance flag

[Nick Piggin <nickpiggin@yahoo.com.au>]

On Wednesday 03 October 2007 19:55, Paul Jackson wrote:
> > > Yeah -- cpusets are hierarchical.  And some of the use cases for
> > > which cpusets are designed are hierarchical.
> >
> > But partitioning isn't.
>
> Yup.  We've got a square peg and a round hole.  An impedance mismatch.
> That's the root cause of this entire wibbling session, in my view.

Basically, yeah.


> > > To repeat myself, in some cases, such as batch schedulers running in a
> > > subset of the CPUs on a large system, the code that knows some of the
> > > needs for load balancing does not have system wide control to mandate
> > > hard partitioning.  The batch scheduler can state where it is depending
> > > on load balancing being present, and the system administrator can
> > > choose or not to turn off load balancing in the top cpuset, thereby
> > > granting or not control over load balancing on the CPUs controlled by
> > > the batch scheduler to the batch scheduler.
> >
> > Why isn't that possible with my approach?
>
> If I understand your approach to the kernel-to-user interface correctly
> (sometimes I doubt I do) then your approach expected some user space code
> or person or semi-intelligent equivalent to define a flat partition,
> which will then be used to determine the sched domains.
>
> In the batch scheduler case, running on a large shared system used
> perhaps by several departments, no one entity can do that.  One person,
> perhaps the system admin, knows if they want to give complete control
> of some big chunk of CPUs to a batch scheduler.  The batch scheduler,
> written by someone else far away and long ago, knows which jobs are
> actively running on which subsets of the CPUs the batch scheduler is
> using.
>
> There is no single monolithic entity on such systems who knows all and
> can dictate all details of a single, flat, system-wide partitioning.

OK, so I don't exactly understand you either. To make it simple, can
you give a concrete example of a cpuset hierarchy that wouldn't
work?


> The partitioning has to be sythesized from the combined requests of
> several user space entities.  That's ok -- this is bread and butter
> work for cpusets.

OK, so to really do anything different (from a non-partitioned setup),
you would need to set sched_load_balance=0 for the root cpuset?
Suppose you do that to hard partition the machine, what happens to
newly created tasks like kernel threads or things that aren't in a
cpuset?