Re: [rfc][patch] sched: remove smpnice

[Paul Jackson <pj@sgi.com>]

Peter wrote:
> In these circumstances, moving the task 
> to an idle CPU should be a "good thing" unless the time taken for the 
> move is longer than the time that will pass before the task becomes the 
> running task on its current CPU.

Even then, it's not always a "good thing".

The less of the cache-memory hierarchy the two CPUs share, the greater
the penalty to the task for memory accesses after the move.

At one extreme, two hyperthreads on the same core share essentially all
the memory hierarchy, so have no such penalty.

At the other extreme, two CPUs at opposite ends of a big NUMA box have,
so far as performance is concerned, quite different views of the memory
hierarchy.  A task moved to a far away CPU will be cache cold for
several layers of core, package, board, and perhaps router hierarchy,
and have slower access to its main memory pages.


-- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@sgi.com> 1.925.600.0401