Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation

[Max Krasnyanskiy <maxk@qualcomm.com>]

Peter Zijlstra wrote:
> My vision on the direction we should take wrt cpu isolation.
General impressions:
- "cpu_system_map" is %100 identical to the "~cpu_isolated_map" as in my 
patches. It's updated from different place but functionally wise it's very 
much the same. I guess you did not like the 'isolated' name ;-). As I 
mentioned before I'm not hung up on the name so it's cool :).

- Updating cpu_system_map from cpusets
There are a couple of things that I do not like about this approach:
1. We lost the ability to isolate CPUs at boot. Which means slower boot times 
for me (ie before I can start my apps I need to create cpuset, etc). Not a big 
deal, I can live with it.

2. We now need another notification mechanism to propagate the updates to the 
cpu_system_map. That by itself is not a big deal. The big deal is that now we 
need to basically audit the kernel and make sure that everything affected must
have proper notifier and react to the mask changes.
For example your current patch does not move the timers and I do not think it 
makes sense to go and add notifier for the timers. I think the better approach 
is to use CPU hotplug here. ie Enforce the rule that cpu_system_map is updated 
  only when CPU is off-line.
By bringing CPU down first we get a lot of features for free. All the kernel 
threads, timers, softirqs, HW irqs, workqueues, etc are properly 
terminated/moved/canceled/etc. This gives us a very clean state when we bring 
the CPU back online with "system" bit cleared (or "isolated" bit set like in 
my patches). I do not see a good reason for reimplementing that functionality 
via system_map notifiers.

I'll comment more on the individual patches.

> Next on the list would be figuring out a nice solution to the workqueue
> flush issue.
Do not forget the "stop machine", or more specifically module loading/unloading.

Max