Re: [rfc] balance-on-fork NUMA placement

Christoph Lameter
2007-08-03T05:47:13Z

On Fri, 3 Aug 2007, Nick Piggin wrote:

> Well what's wrong with it? It seems to use memory policies for exactly
> what they are intended (aside from it being kernel directed...).

Sure I think you could do it with some effort. They were primarily
designed for user space. Lots of little side issues where surprises await
you. I think Lee documented many of them. See the recent mm commits.

> > start the new thread and can the original processor wait on some flag
> > until that is complete?
>
> I guess you could, but that is going to add a context switch to fork
> (although it usually already has one in single-CPU situation because we
> run child first)... I bet it will slow something down, but it would be
> interesting to see.

The context switch is needed at some point anyways to get the new process
running on the new CPU? Just do it before allocating structures. That way
the potential memory policy and cpuset context is preserved and followed.

> I don't know the fork path well enough off the top of my head to know if
> it will be that simple (with error handling etc). But I think it could
> be done.

I would think that the forking process has to wait on completion anyways
and get an error code.

> > Forking off from there not only places the data correctly but it also
> > warms up the caches for the new process and avoids evicting cacheline on
> > the original processor.
>
> Yeah, you might be right there. If the numbers say that approach is
> better, then I'd not be against it. But we'd still need the simpler
> mpol approach to compare it with.

Lets hope that the simpler process is really simpler after all the corner
cases have been dealt with.