Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation

[Max Krasnyansky <maxk@qualcomm.com>]

Ingo Molnar wrote:
> * Paul Jackson <pj@sgi.com> wrote:
> 
>> But your words sound alot like what we at SGI call a 'boot' cpuset.
>>
>> Our big honkin NUMA customers, who are managing most of the system 
>> either for a few dedicated, very-important jobs, and/or under a batch 
>> scheduler, need to leave a few nodes to run the classic Unix load such 
>> as init, cron, assorted daemons and the admins login shell.
>>
>> So we provide them some init script mechanisms that make it easy to 
>> set this up, which includes moving every task (not many at the low 
>> numbered init script time this runs) that isn't pinned (doesn't have a 
>> restricted Cpus_allowed) into the boot cpuset, conventionally named 
>> /dev/cpuset/boot.
> 
> yes. Ideally Peter's patchset should turn into something equivalent and 
> i very much agree with Peter's arguments. There was never any design 
> level problem with cpusets, and the parallel cpu_isolated_map approach 
> was misdirected IMO.
> 
> There was indeed a problem with the _manageability_ of cpusets in 
> certain (rather new) usecases like real-time or virtualization, and how 
> they are connected to other system resources like IRQs and how easy it 
> is to manage these resources. IRQs should probably be tied to specific 
> cpusets and should migrate together with them, were the span of that 
> cpuset be changed. (by default they'd be tied to the boot cpuset)
> 
> IMO Peter's patchset is a good first step in that it removes the 
> cpu_isolated_map API hack, and i think we should try to go the whole way 
> and just offer a /dev/cpuset/boot/ default set that can then be 
> restricted to isolate the default workloads away from other CPUs.

I like the concept of the "boot" set. But we still need a separate "system"
flag. Users should not have to move cpu back into the "boot" set to allow for
kernel (irqs, etc) activity on it. And it's just more explicit and clear that way.

Max