Re: Page Colouring (was: 2.6.0 Huge pages not working as expected)

Linus Torvalds
2003-12-28T04:54:59Z

On Sat, 27 Dec 2003, Eric W. Biederman wrote:
> Linus Torvalds <torvalds@osdl.org> writes:
> >
> > Basically: prove me wrong. People have tried before. They have failed.
> > Maybe you'll succeed. I doubt it, but hey, I'm not stopping you.
>
> For anyone taking you up on this I'd like to suggest two possible
> directions.
>
> 1) Increasing PAGE_SIZE in the kernel.

Yes. This is something I actually want to do anyway for 2.7.x. Dan
Phillips had some patches for this six months ago.

You have to be careful, since you have to be able to mmap "partial pages",
which is what makes it less than trivial, but there are tons of reasons to
want to do this, and cache coloring is actually very much a secondary
concern.

> 2) Creating zones for the different colors. Zones were not
> implemented last time, this was tried.

Hey, I can tell you that you _will_ fail.

Zones are actually a wonderful example of the kinds of problems you get
into when you have pages of different types aka "colors". We've had
nothing but trouble trying to balance different zones against each other,
and those problems were in fact _the_ reason for 99% of all the VM
problems in 2.4.x.

Trying to use them for cache colors would be "interesting".

Not to mention that it's impossible to coalesce pages across zones.

> Both of those should be minimal impact to the complexity
> of the current kernel.

Minimal? I don't think so. Zones are basically impossible, and page size
changes will hopefully happen during 2.7.x, but not due to page coloring.

> I don't know where we will wind up but the performance variation's
> caused by cache conflicts in today's applications are real, and easily
> measurable. Giving the growing increase in performance difference
> between CPUs and memory Amdahl's Law shows this will only grow
> so I think this is worth looking at.

Absolutely wrong.

Why? Because the fact is, that as memory gets further and further away
from CPU's, caches have gotten further and further away from being direct
mapped.

Cache coloring is already a very questionable win for four-way
set-associative caches. I doubt you can even _see_ it for eight-way or
higher associativity caches.

In other words: the pressures you mention clearly do exist, but they are
all driving direct-mapped caches out of the market, and thus making page
coloring _less_ interesting rather than more.

Linus