Re: Scaling noise

Martin J. Bligh
2003-09-03T21:58:46Z

>> That's where I disagree - it's much easier for the USER because an SSI
>> cluster works out all the load balancing shit for itself, instead of
>> pushing the problem out to userspace. It's much harder for the KERNEL
>> programmer, sure ... but we're smart ;-) And I'd rather solve it once,
>> properly, in the right place where all the right data is about all
>> the apps running on the system, and the data about the machine hardware.
>
> This is only truly feasible when the nodes are homogeneous. They will
> not be as there will be physical locality (esp. bits like device
> proximity) concerns.

Same problem as a traditonal set up of a NUMA system - the scheduler
needs to try to move the process closer to the resources it's using.

> It's vaguely possible some kind of punting out
> of the kernel of the solutions to these concerns is possible, but upon
> the assumption it will appear, we descend further toward science fiction.

Nah, punting to userspace is crap - they have no more ability to solve
this than we do on any sort of dynamic worseload, and in most cases,
much worse - they don't have the information that the kernel has available,
at least not on a timely basis. The scheduler belongs in the kernel,
where it can balance decisions across all of userspace, and we have
all the info we need rapidly and efficiently available.

> Some of these proposals also beg the question of "who's going to write
> the rest of the hypervisor supporting this stuff?", which is ominous.

Yeah, it needs lots of hard work by bright people. It's not easy.

M.