Re: regarding the x86_64 zero-based percpu patches

[Rusty Russell <rusty@rustcorp.com.au>]

On Wednesday 07 January 2009 22:43:25 Tejun Heo wrote:
> IIUC, Rusty is somewhat leaning toward limiting per-cpu area and using
> static allocator. (right?)

Not quite.  Six years ago I didn't do "proper" dynamic per-cpu because of
this lack-of-expanding problem.  I expected (myself or someone else) would
fix that and the current temporary solution would be replaced.

But Christoph showed that even in a limited form it can be used for more
than static per-cpu vars and such vars in modules.  (It's also in dire need
of a cleanup, since there have been several abortive changes made in the last
few years).

> As I was trying to do more stuff per-cpu
> (not putting a lot of stuff into per-cpu area but even with small
> things limited per-cpu area poses scalability problems), cpu_alloc
> seems to fit the bill better.

Unfortunately cpu_alloc didn't solve this problem either.

We need to grow the areas, but for NUMA layouts it's non-trivial.  I don't
like the idea of remapping: one TLB entry per page per cpu is going to suck.
Finding pages which are "congruent" with the original percpu pages is more
promising, but it will almost certainly need to elbow pages out the way to
have a chance of succeeding on a real system.

> Anyways, I think it's worthwhile to listen what people have on mind
> regarding how per-cpu stuff should proceed.

Absolutely.

Thanks,
Rusty.