Re: [PATCH 1/1] V3: hotplug cpu: migrate a task within its cpuset

Andrew Morton
2007-08-27T16:25:42Z

On Mon, 27 Aug 2007 11:07:03 -0500 Cliff Wickman <cpw@sgi.com> wrote:

>
> Version 3 adds a missing task_rq_lock()/task_rq_unlock() pair. (Oleg found)
>
> There was discussion about this patch among:
> Andrew Morton, Oleg Nesterov, Gautham Shenoy, Rusty Russell
> regarding other approaches:
> refusing to offline a cpu with tasks pinned to it, or
> providing an administrator the ability to assign such tasks to other cpus
>
> There is indeed an "assumption" in my patch that the cpuset containing a
> pinned task's cpu is a better choice than any online cpu. I think that is
> a reasonable assumption given the typical construction of a cpuset and the
> reason a task is running in a cpuset.
>
> And there will be coming cases, at least on some architectures, where a
> cpu is offlined as a kernel reaction to a hardware error. In that case
> would it not be preferrable to re-pin such tasks and let them proceed?
>
>
>
> When a cpu is disabled, move_task_off_dead_cpu() is called for tasks
> that have been running on that cpu.
>
> Currently, such a task is migrated:
> 1) to any cpu on the same node as the disabled cpu, which is both online
> and among that task's cpus_allowed
> 2) to any cpu which is both online and among that task's cpus_allowed
>
> It is typical of a multithreaded application running on a large NUMA system
> to have its tasks confined to a cpuset so as to cluster them near the
> memory that they share. Furthermore, it is typical to explicitly place such
> a task on a specific cpu in that cpuset. And in that case the task's
> cpus_allowed includes only a single cpu.
>
> This patch would insert a preference to migrate such a task to some cpu within
> its cpuset (and set its cpus_allowed to its entire cpuset).
>
> With this patch, migrate the task to:
> 1) to any cpu on the same node as the disabled cpu, which is both online
> and among that task's cpus_allowed
> 2) to any online cpu within the task's cpuset
> 3) to any cpu which is both online and among that task's cpus_allowed
>

<looks at the No more Mr. Nice Guy. code>

OK, so we're no worse than we used to be, really.

> include/linux/cpuset.h | 5 +++++
> kernel/cpuset.c | 15 ++++++++++++++-
> kernel/sched.c | 16 ++++++++++++++++

How do we communicate this new design/feature to our users?
Documentation/cpusets.txt, perhaps? Documentation/cpu-hotplug.txt?
git-log? ;)