Re: [RFC PATCH] Dynamic sched domains aka Isolated cpusets

[Paul Jackson <pj@sgi.com>]

Nick wrote:
> It doesn't work if you have *most* jobs bound to either
> {0, 1, 2, 3} or {4, 5, 6, 7} but one which should be allowed
> to use any CPU from 0-7.

How bad does it not work?

My understanding is that Dinakar's patch did _not_ drive tasks out of
larger cpusets that included two or more of what he called isolated
cpusets, I call cpuset domains.

For example:

	System starts up with 8 CPUs and all tasks (except for
	a few kernel per-cpu daemons) in the root cpuset, able
	to run on CPUs 0-7.

	Two cpusets, Alpha and Beta are created, where Alpha
	has CPUs 0-3, and Beta has CPUs 4-7.

	Anytime someone logs in, their login shell and all
	they run from it are placed in one of Alpha or Beta.
	The main spawning daemons, such as inetd and cron,
	are placed in one of Alpha or Beta.

	Only a few daemons that don't do much are left in the
	root cpuset, able to run across 0-7.

If we tried to partition the sched domains with Alpha and Beta as
separate domains, what kind of pain do these few daemons in
the root cpuset, on CPUs 0-7, cause?

If the pain is too intolerable, then I'd guess not only do we have to
purge any cpusets superior to the ones determining the domain
partitioning of _all_ tasks, but we'd also have to invent yet one more
boolean flag attribute for any such superior cpusets, to mark them as
_not_ able to allow a task to be attached to them.  And we'd have to
refine the hotplug co-existance logic in cpusets, which currently bumps
a task up to its parent cpuset when all the cpus in its current cpuset
are hot unplugged, to also rebuild the sched domains to some legal
configuration, if the parent cpuset was not allowed to have any tasks
attached.

I'd rather not go there, unless push comes to shove.  How hard are
you pushing?

-- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@engr.sgi.com> 1.650.933.1373, 1.925.600.0401