Re: [PATCH] Use numa policy API for boot time policy

[Andi Kleen <ak@suse.de>]

On Thu, 10 Jun 2004 01:44:29 +1000
Anton Blanchard <anton@samba.org> wrote:

>  
> > It would be a one liner change to allow process policy interleaving 
> > for orders > 0 in mempolicy. But I'm not sure how useful it is, since
> > the granuality would be really bad.
> 
> OK. Id like to take a quick look at order > 0 allocations during boot
> to see if its worth it. The ppc64 page size is small and we might be
> doing a significant number of order 1 allocations.

For what? 

> > Have you ever tried to switch to implement a vmalloc_interleave() for these
> > tables instead? My bet is that it will perform better.
> 
> Im warming to this idea. We would need a per arch override, since there
> is a trade off here between interleaving and TLB usage.

Actually just standard vmalloc is enough. The interleave policy in alloc_pages
will transparently interleave the order 0 pages allocated by vmalloc.

When I find some time I will try that on Opteron too.

> 
> We also have a problem in 2.6 on our bigger machines where our dcache
> hash and inode hash cache are limited to MAX_ORDER (16MB on ppc64). By
> using vmalloc would allow us to interleave the memory and allocate more
> than 16MB for those hashes.

IMHO 16MB hash table for a kernel structure is madness. A different data
structure is probably needed if it's really a problem
(is your dcache that big?). Or maybe just limit the dcache more aggressively
to keep the max number of entries smaller.

-Andi