Re: pthread_create() slow for many threads; also time to revisit 64b context switch optimization?

[Andrew Morton <akpm@linux-foundation.org>]

On Wed, 13 Aug 2008 22:42:48 +0200
Andi Kleen <andi@firstfloor.org> wrote:

> Ingo Molnar <mingo@elte.hu> writes:
> >
> > i find it pretty unacceptable these days that we limit any aspect of 
> > pure 64-bit apps in any way to 4GB (or any other 32-bit-ish limit). 
> 
> It's not limited to 2GB, there's a fallback to >4GB of course. Ok
> admittedly the fallback is slow, but it's there.
> 
> I would prefer to not slow down the P4s. There are **lots** of them in
> field. And they ran 64bit still quite well. Also back then I
> benchmarked on early K8 and it also made a difference there (but I
> admit I forgot the numbers)
> 
> I think it would be better to fix the VM because there are
> other use cases of applications who prefer to allocate in a lower area.
> For example Java JVMs now widely use a technique called pointer
> compression where they dynamically adjust the pointer size based
> on how much memory the process uses. For that you have to get
> low memory in the 47bit VM too. The VM should deal with that gracefully.
> 
> To be honest I always thought the linear search in the VMA list
> was a little dumb. I'm sure there are other cases where it hurts
> too. Perhaps this would be really an opportunity  to do something about it :)
> 

Yes, the free_area_cache is always going to have failure modes - I
think we've been kind of waiting for it to explode.

I do think that we need an O(log(n)) search in there.  It could still
be on the fallback path, so we retain the mostly-O(1) benefits of
free_area_cache.