Re: [PATCH 01/02] cpuset memory spread slab cache filesys

[Andi Kleen <ak@suse.de>]

On Wednesday 01 March 2006 23:20, Christoph Lameter wrote:

> Interleave is only beneficial for special applications that use a common 
> pool of data and that implement no other means of locality control. At 
> that point we sacrifice the performance benefit that comes with node locality
> in order not to overload a single node. 

My rationale is the locality is only important for cache lines that
are very frequently accessed.   By far the most frequently accessed
items is user space data, system calls are still relatively rare
compare to that and system calls that touch files even less
so 

(often in my measurements 40% and more of all syscalls are gettimeofday 
actually)

Also we don't have very good balancing control on dcaches.

The problem is when one node runs updatedb or similar it will
end up allocating a lot of these objects, and then later when
a process ends up on that node it can have trouble allocating
node local memory (and a user process missing local memory is
typically much worse than a kernel object)

I guess your remote claim changes will help, but I'm not 
convinced they are the best solution.

[hmm, actually didn't we discuss this once at length anyways.
Apparently I failed to convince you back then @:]

> 
> Kernels before 2.6.16 suffer from special overload situations that are due 
> to not having the ability to reclaim the pagecache and the slab cache. 

Reclaiming is slow. Better not dig into this hole in the first place.

> > I would be in favour of it
> 
> Please run performance tests with single threaded processes if you 
> do not believe me before doing any of this.

Sure. But the motivation is less the single thread performance
anyways, but more the degradation under extreme loads.

-Andi