Re: [RFC][PATCH 00/26] sched/numa

Ingo Molnar
2012-03-19T20:29:05Z

* Christoph Lameter <cl@linux.com> wrote:

> On Mon, 19 Mar 2012, Andrea Arcangeli wrote:
>
> > Yeah I'll try to fix that but it's massively complex and
> > frankly benchmarking wise it won't help much fixing that...
> > so it's beyond the end of my todo list.
>
> Well a word of caution here: SGI tried to implement automatic
> migration schemes back in the 90's but they were never able to
> show a general benefit of migration. The overhead added
> because of auto migration often was not made up by true
> acceleration of the applications running on the system. They
> were able to tune the automatic migration to work on
> particular classes of applications but it never turned out to
> be generally advantageous.

Obviously any such scheme must be a win in general for it to be
default on. We don't have the numbers to justify that - and I'm
sceptical whether it will be possible, but I'm willing to be
surprised.

I'm especially sceptical since most mainstream NUMA systems tend
to have a low NUMA factor. Thus the actual cost of being NUMA is
pretty low.

That having said PeterZ's numbers showed some pretty good
improvement for the streams workload:

before: 512.8M
after: 615.7M

i.e. a +20% improvement on a not very heavily NUMA box.

That kind of raw speedup of a CPU execution workload like
streams is definitely not something to ignore out of hand. *IF*
there is a good automatism that can activate it for the apps
that are very likely to benefit from it then we can possibly do
it.

But a lot more measurements have to be done, and I'd be also
very interested in the areas that regress.

Otherwise, if no robust automation is possible, it will have to
be opt-in, on a per app basis, with both programmatic and
sysadmin knobs available. (who will hopefully make use if it...)

That's the best we can do I think.

> I wonder how we can verify that the automatic migration
> schemes are a real benefit to the application? We have a
> history of developing a kernel that decreases in performance
> as development proceeds. How can we make sure that these
> schemes are actually beneficial overall for all loads and do
> not cause regressions elsewhere? [...]

The usual way?

Thanks,

Ingo