Re: [PATCH v2 0/7] CPU hotplug, cpusets: Fix issues with cpusets handling upon CPU hotplug

["Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com>]

On 05/05/2012 12:54 AM, Peter Zijlstra wrote:

> 
>>   Documentation/cgroups/cpusets.txt |   43 +++--
>>  include/linux/cpuset.h            |    4 
>>  kernel/cpuset.c                   |  317 ++++++++++++++++++++++++++++---------
>>  kernel/sched/core.c               |    4 
>>  4 files changed, 274 insertions(+), 94 deletions(-)
> 
> Bah, I really hate this complexity you've created for a problem that
> really doesn't exist.
> 


Doesn't exist? Well, I believe we do have a problem and a serious one
at that too!

The heart of the problem can be summarized in 2 sentences:

o	During a CPU hotplug, tasks can move between cpusets, and never
	come back to their original cpuset.
o	Tasks might get pinned to lesser number of cpus, unreasonably.

Both these are undesirable from a system-admin point of view.
Moreover, having workarounds for this from userspace is way too messy and
ugly, if not impossible.

> So why not fix the active mask crap?


Because I doubt if that is the right way to approach this problem.

An updated cpu_active_mask not being the necessary and sufficient condition
for all scheduler related activities, is a different problem altogether, IMHO.

(Btw, Ingo had also suggested reworking this whole cpuset thing, while
reviewing the previous version of this fix.
http://thread.gmane.org/gmane.linux.kernel/1250097/focus=1252133)

Also, we need to fix this problem at the CPU Hotplug level itself, and
not just for the suspend/resume case. Because, we have had numerous bug
reports and people complaining about this issue, in various scenarios,
including those that didn't involve suspend/resume.

I am sure some of the people in Cc will have more to add to this, but in
general, when the CPU hotplug (maybe even cpu offline + online) and the
cpuset administration are done asynchronously, it leads to nasty surprises.
In fact, there have been reports where people spent inordinate amounts of
time before they figured out that a long-forgotten cpu hotplug operation
which was performed, was the root-cause of a low-performing workload!.

All these only suggest that it is time that we cleaned this up thoroughly,
and at the root cause level itself.

Btw, though there are 7 patches in this series, I don't think this patchset
increases the complexity of the code.. In fact, it makes many things simpler
and saner/cleaner, IMHO.

Regards,
Srivatsa S. Bhat