Re: [RFC/PATCH] cpuset: cpuset irq affinities

Peter Zijlstra
2008-03-03T18:18:55Z

On Mon, 2008-03-03 at 12:10 -0600, Paul Jackson wrote:
> > But as long as nobody does CS_CPU_EXCLUSIVE they may overlap, right?
>
> It's a bit stronger than that:
>
> 1) They need non-overlapping cpusets at this level to control
> the sched_domain setup, if they want to avoid load balancing
> across almost all CPUs in the system. Depending on the kernel
> version, sched_domain partitioning is controlled either by the
> cpuset flag cpu_exclusive, or the cpuset flag sched_load_balance.
>
> 2) They need non-overlapping cpusets at this level to control
> memory placement of some kernel allocations, which are allowed
> outside the current tasks cpuset, to be confined by the nearest
> ancestor cpuset marked 'mem_exclusive'
>
> 3) Some sysadmin tools are likely coded to expect a /dev/cpuset/boot
> cpuset, not a /dev/cpuset/system/boot cpuset, as that has been
> customary for a long time.
>
> (1) and (2) would break the major batch schedulers. They typically
> mark their top cpuset, /dev/cpuset/pbs or /dev/cpuset/lfs or whatever
> batch scheduler it is, as cpu_exclusive and mem_exclusive, by way of
> expressing their intention to pretty much own those CPUs and memory
> nodes. If we fired them up on a system where that wasn't allowed due
> to overlap with /dev/cpuset/system, they'd croak. Such changes as that
> are costly and unappreciated.

OK, understood, I'll try and come up with yet another scheme :-)