Re: [rfc 08/45] cpu alloc: x86 support

[Andi Kleen <ak@suse.de>]

On Tuesday 20 November 2007 04:50, Christoph Lameter wrote:
> On Tue, 20 Nov 2007, Andi Kleen wrote:
> > I might be pointing out the obvious, but on x86-64 there is definitely
> > not 256TB of VM available for this.
>
> Well maybe in the future.

That would either require more than 4 levels or larger pages
in page tables.

> One of the issues that I ran into is that I had to place the cpu area
> in between to make the offsets link right.

Above -2GB, otherwise you cannot address them

If you can move all the other CPUs somewhere else it might work.

But even then 16MB/cpu max is unrealistic. Perhaps 1M/CPU 
max -- then 16k CPU would be 128GB which could still fit into the existing
vmalloc area.

>
> However, it would be best if the cpuarea came *after* the modules area. We
> only need linking that covers the per cpu area of processor 0.
>
> So I think we have a 2GB area right?

For everything that needs the -31bit offsets; that is everything linked

> 1GB kernel
> 1GB - 1x per cpu area (128M?) modules?
> cpu aree 0
> ---- 2GB limit
> cpu area 1
> cpu area 2
> ....
>
> For that we would need to move the kernel down a bit. Can we do that?

The kernel model requires kernel and modules and everything else
linked be in negative -31bit space. That is how the kernel code model is 
defined.

You could in theory move the modules, but then you would need to implement
a full PIC dynamic linker for them  first and also increase runtime overhead
for them because they would need to use a GOT/PLT.

Or you could switch kernel over to the large model, which is very costly
and has toolkit problems.

Or use the UML trick and run the kernel PIC but again that causes
overhead.

I suspect all of this  would cause far more overhead all over the kernel than 
you could ever save with the per cpu data in your fast paths.

-Andi