Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation

Ingo Molnar
2008-02-28T10:47:22Z

* Paul Jackson <pj@sgi.com> wrote:

> But your words sound alot like what we at SGI call a 'boot' cpuset.
>
> Our big honkin NUMA customers, who are managing most of the system
> either for a few dedicated, very-important jobs, and/or under a batch
> scheduler, need to leave a few nodes to run the classic Unix load such
> as init, cron, assorted daemons and the admins login shell.
>
> So we provide them some init script mechanisms that make it easy to
> set this up, which includes moving every task (not many at the low
> numbered init script time this runs) that isn't pinned (doesn't have a
> restricted Cpus_allowed) into the boot cpuset, conventionally named
> /dev/cpuset/boot.

yes. Ideally Peter's patchset should turn into something equivalent and
i very much agree with Peter's arguments. There was never any design
level problem with cpusets, and the parallel cpu_isolated_map approach
was misdirected IMO.

There was indeed a problem with the _manageability_ of cpusets in
certain (rather new) usecases like real-time or virtualization, and how
they are connected to other system resources like IRQs and how easy it
is to manage these resources. IRQs should probably be tied to specific
cpusets and should migrate together with them, were the span of that
cpuset be changed. (by default they'd be tied to the boot cpuset)

IMO Peter's patchset is a good first step in that it removes the
cpu_isolated_map API hack, and i think we should try to go the whole way
and just offer a /dev/cpuset/boot/ default set that can then be
restricted to isolate the default workloads away from other CPUs.

( an initscripts approach, while i'm sure it works, would always be a
bit fragile in that it requires precise knowledge about which task is
what. I think we should make this a turn-key in-kernel solution that
both the big-honking NUMA-box guys and the real-time guys would be
happy with. )

Ingo