Re: [RFC/PATCH] cpuset: cpuset irq affinities

[Peter Zijlstra <a.p.zijlstra@chello.nl>]

On Mon, 2008-03-03 at 12:10 -0600, Paul Jackson wrote:
> > But as long as nobody does CS_CPU_EXCLUSIVE they may overlap, right?
> 
> It's a bit stronger than that:
> 
>  1) They need non-overlapping cpusets at this level to control
>     the sched_domain setup, if they want to avoid load balancing
>     across almost all CPUs in the system.  Depending on the kernel
>     version, sched_domain partitioning is controlled either by the
>     cpuset flag cpu_exclusive, or the cpuset flag sched_load_balance.
> 
>  2) They need non-overlapping cpusets at this level to control
>     memory placement of some kernel allocations, which are allowed
>     outside the current tasks cpuset, to be confined by the nearest
>     ancestor cpuset marked 'mem_exclusive'
> 
>  3) Some sysadmin tools are likely coded to expect a /dev/cpuset/boot
>     cpuset, not a /dev/cpuset/system/boot cpuset, as that has been
>     customary for a long time.
> 
> (1) and (2) would break the major batch schedulers.  They typically
> mark their top cpuset, /dev/cpuset/pbs or /dev/cpuset/lfs or whatever
> batch scheduler it is, as cpu_exclusive and mem_exclusive, by way of
> expressing their intention to pretty much own those CPUs and memory
> nodes.  If we fired them up on a system where that wasn't allowed due
> to overlap with /dev/cpuset/system, they'd croak.  Such changes as that
> are costly and unappreciated.

OK, understood, I'll try and come up with yet another scheme :-)