Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation

Max Krasnyanskiy
2008-02-27T23:38:53Z

Peter Zijlstra wrote:
> My vision on the direction we should take wrt cpu isolation.
General impressions:
- "cpu_system_map" is %100 identical to the "~cpu_isolated_map" as in my
patches. It's updated from different place but functionally wise it's very
much the same. I guess you did not like the 'isolated' name ;-). As I
mentioned before I'm not hung up on the name so it's cool :).

- Updating cpu_system_map from cpusets
There are a couple of things that I do not like about this approach:
1. We lost the ability to isolate CPUs at boot. Which means slower boot times
for me (ie before I can start my apps I need to create cpuset, etc). Not a big
deal, I can live with it.

2. We now need another notification mechanism to propagate the updates to the
cpu_system_map. That by itself is not a big deal. The big deal is that now we
need to basically audit the kernel and make sure that everything affected must
have proper notifier and react to the mask changes.
For example your current patch does not move the timers and I do not think it
makes sense to go and add notifier for the timers. I think the better approach
is to use CPU hotplug here. ie Enforce the rule that cpu_system_map is updated
only when CPU is off-line.
By bringing CPU down first we get a lot of features for free. All the kernel
threads, timers, softirqs, HW irqs, workqueues, etc are properly
terminated/moved/canceled/etc. This gives us a very clean state when we bring
the CPU back online with "system" bit cleared (or "isolated" bit set like in
my patches). I do not see a good reason for reimplementing that functionality
via system_map notifiers.

I'll comment more on the individual patches.

> Next on the list would be figuring out a nice solution to the workqueue
> flush issue.
Do not forget the "stop machine", or more specifically module loading/unloading.

Max