Re: [Patch 4/4] cpusets top mask just online, not all possible

[Dave Hansen <haveblue@us.ibm.com>]

On Sat, 2004-09-11 at 10:07, Paul Jackson wrote:
> I'm adding Andi Kleen, Iwamoto-san and Dave Hansen to the cc list, since
> the numa code and other hotplug work might have similar considerations.
> 
> Anton asks:
> > How does this change interact with CPU hotplug?
> 
> You beat my estimate ;).  I figured it would be a day before someone
> asked this question.  It only took you six hours.  Good.
> 
> Cpusets and hotplug (CPU or Memory) aren't friends, yet.

Heh.  I assumed that cpusets didn't deal with memory, and I ignored the
code when I saw it.  Oh well... :)

> It wouldn't surprise me if Andi Kleen's numa code, especially the
> MPOL_BIND which builds up special restricted zonelists holding only the
> bound Memory Nodes, has the same sorts of interactions with Memory
> hotplug.  However, I have not given this suspicion any careful thought,
> so could easily be wrong.

It shouldn't interact yet because we don't add any NUMA nodes yet, we
only grow existing ones.  The only case that would be a problem is if an
entire node's memory was removed, but that would probably be similar to
trying to bind to an empty node now, or what happens during CPU hotplug
with cpus_allowed set to a now-unplugged CPU.

>  2) Someday soon, the cpuset (and numa?) placement code needs to add an
>     internal kernel call that the hotplug code can call to inform the
>     placement code that a CPU or Memory resource has gone on or offline,
>     so that the placement code can "deal with it", somehow.

See kernel/cpu.c:25
/* Need to know about CPUs going up/down? */
int register_cpu_notifier(struct notifier_block *nb)

We'll do the same for memory, I promise.

>  3) The way that I anticipate the cpuset code will "deal with it"
>     will be:

Other than providing the notifiers, I don't think there's a whole lot
else the hotplug code can do.  That's left for the poor souls working on
the binding APIs.  Memory will probably need some slightly more
informative things that the CPU notifiers because we might have some
more properties that the handlers might care about, but it shouldn't be
fundamentally different. 

-- Dave