Re: [RFC/PATCH] cpuset: cpuset irq affinities

Paul Jackson
2008-03-03T18:11:59Z

> But as long as nobody does CS_CPU_EXCLUSIVE they may overlap, right?

It's a bit stronger than that:

1) They need non-overlapping cpusets at this level to control
the sched_domain setup, if they want to avoid load balancing
across almost all CPUs in the system. Depending on the kernel
version, sched_domain partitioning is controlled either by the
cpuset flag cpu_exclusive, or the cpuset flag sched_load_balance.

2) They need non-overlapping cpusets at this level to control
memory placement of some kernel allocations, which are allowed
outside the current tasks cpuset, to be confined by the nearest
ancestor cpuset marked 'mem_exclusive'

3) Some sysadmin tools are likely coded to expect a /dev/cpuset/boot
cpuset, not a /dev/cpuset/system/boot cpuset, as that has been
customary for a long time.

(1) and (2) would break the major batch schedulers. They typically
mark their top cpuset, /dev/cpuset/pbs or /dev/cpuset/lfs or whatever
batch scheduler it is, as cpu_exclusive and mem_exclusive, by way of
expressing their intention to pretty much own those CPUs and memory
nodes. If we fired them up on a system where that wasn't allowed due
to overlap with /dev/cpuset/system, they'd croak. Such changes as that
are costly and unappreciated.

--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <pj@sgi.com> 1.940.382.4214