Re: [PATCH] reduce inter-node balancing frequency

[Jesse Barnes <jbarnes@engr.sgi.com>]

On Friday, July 16, 2004 1:53 am, Nick Piggin wrote:
> I'm going to cut a patch to consolidate the arch setup code in a
> bit. It would be easy to just not define SD_NODE_INIT at all in
> generic code. I expect that won't go down too well at this stage
> though ;)

It's fine with me, as long as you move it to an arch specific header at the 
same time (i.e. don't functionally change it while you move code around).

> Well I think the main thing is that you do not want a global domain
> with 512 CPUs in it, even if it is very rarely balanced.
>
> Apart from having to pull hot cachelines off 511 CPUs while in interrupt
> context, it just doesn't make sense: It could easily move a task to a
> node that is  x (=large) hops away.
>
> It is probably way too much complexity to try to model your topology in
> any amount of detail at this stage, but it could be made smarter.
>
> Instead of a top level domain spanning all CPUs, have each CPU's top level
> domain just span all CPUs within a couple of hops (enough to get, say 16 to
> 64 CPUs into each top level domain). I could give you a hand with this if
> you need.

Yeah, that's what I had in mind.  I'll wait for the patch you mentioned above 
and hack on top of that...

For sn2 at least, there are quite a few ways we could dice up the topology.  
We'll have to experiment with things a bit to find some good defaults.

Thanks,
Jesse