Re: Minutes from Feb 21 LSE Call

[Larry McVoy <lm@bitmover.com>]

> Userspace owns the cache; using
> cache for the kernel is "cache pollution", which should be minimized.
> Going too far out on the space end of time/space tradeoff curves is
> every bit as bad for SMP as UP, and really horrible for NUMA.

Cool, I agree 100% with this.

> > So "measuring" doesn't mean "it's not slower on XYZ microbenchmark".
> > It means "under the following work loads the cache misses went down or
> > stayed the same for before and after tests".
> 
> This kind of measurement is actually relatively unusual. I'm definitely
> interested in it, as there appear to be some deficits wrt. locality of
> reference that show up as big profile spikes on NUMA boxen. With care
> exercised good solutions should also trim down cache misses on UP also.
> Cache and TLB miss profile driven development sounds very attractive.

Again, I'm with you all the way on this.  If the scale up guys can adopt
this as a mantra, I'm a lot less concerned that anything bad will happen.

Tim at OSDL and I have been talking about trying to work out some benchmarks
to test for this.  I came up with the idea of adding a "-s XXX" which means
"touch XXX bytes between each iteration" to each LMbench test.  One problem
is the lack of page coloring will make the numbers bounce around too much.
We talked that over with Linus and he suggested using the big TLB hack to
get around that.  Assuming we can deal with the page coloring, do you think
that there is any merit in taking microbenchmarks, adding an artificial
working set, and running those?

> Let me put it this way: IBM sells tiny boxen too, from 4x, to UP, to
> whatever. And people are simultaneously actively trying to scale
> downward to embedded bacteria or whatever. 

That's really great, I know it's a lot less sexy but it's important.
I'd love to see as much attention on making Linux work on tiny embedded
platforms as there is on making it work on big iron.  Small is cool too.
-- 
---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm