Re: [PATCH 1/5] cpuset memory spread basic implementation

[Ingo Molnar <mingo@elte.hu>]

* Andi Kleen <ak@suse.de> wrote:

> I still don't really think it will make much difference if the file 
> cache is local or global. Compare to disk IO it is still infinitely 
> faster, so a relatively small slowdown from going off node is not that 
> big an issue.

well, maybe the SGI folks can give us some numbers?

> > another thing: on NUMA, are the pagecache portions of readonly files 
> > (such as /usr binaries, etc.) duplicated across nodes in current 
> > kernels, or is it still random which node gets it? 
> 
> Random.
> 
> > This too could be an  
> > EA caching attribute: whether to create per-node caches for file 
> > content.
> 
> There were (ugly) patches floating around for text duplication, but 
> iirc the benchmarkers were still trying to figure out if it's even a 
> good idea. My guess it is not because CPUs tend to have very 
> aggressive prefetching for code streams which can deal with latency 
> well.

you are a bit biased towards low-latency NUMA setups i guess (read: 
Opterons) :-) Obviously with a low NUMA factor, we dont have to deal 
with memory access assymetries all that much.

But i think we should expand our file caching architecture into those 
caching details nevertheless: it's directly applicable to software 
driven clusters as well. There pagecache replication on nodes is a must, 
and obviously there it makes a big difference whether files are cached 
locally or remotely.

	Ingo