Re: Scaling noise

[Larry McVoy <lm@bitmover.com>]

Here's a thought.  Maybe the next kernel summit needs to have a CC cluster
BOF or whatever.  I'd be happy to show up, describe what it is that I see
and have you all try and poke holes in it.  If the net result was that you
walked away with the same picture in your head that I have that would be
cool.  Heck, I'll sponser it and buy beer and food if you like.

On Wed, Sep 03, 2003 at 02:46:06PM -0700, Martin J. Bligh wrote:
> >> That's where I disagree - it's much easier for the USER because an SSI
> >> cluster works out all the load balancing shit for itself, instead of
> >> pushing the problem out to userspace. It's much harder for the KERNEL
> >> programmer, sure ... but we're smart ;-) And I'd rather solve it once,
> >> properly, in the right place where all the right data is about all
> >> the apps running on the system, and the data about the machine hardware.
> > 
> > This is only truly feasible when the nodes are homogeneous. They will
> > not be as there will be physical locality (esp. bits like device
> > proximity) concerns.
> 
> Same problem as a traditonal set up of a NUMA system - the scheduler
> needs to try to move the process closer to the resources it's using.
> 
> > It's vaguely possible some kind of punting out
> > of the kernel of the solutions to these concerns is possible, but upon
> > the assumption it will appear, we descend further toward science fiction.
> 
> Nah, punting to userspace is crap - they have no more ability to solve
> this than we do on any sort of dynamic worseload, and in most cases,
> much worse - they don't have the information that the kernel has available,
> at least not on a timely basis. The scheduler belongs in the kernel,
> where it can balance decisions across all of userspace, and we have
> all the info we need rapidly and efficiently available.
>  
> > Some of these proposals also beg the question of "who's going to write
> > the rest of the hypervisor supporting this stuff?", which is ominous.
> 
> Yeah, it needs lots of hard work by bright people. It's not easy.
> 
> M.

-- 
---
Larry McVoy              lm at bitmover.com          http://www.bitmover.com/lm