Re: NUMA aware slab allocator V2

[Matthew Dobson <colpatch@us.ibm.com>]

Excuse for the week-late response...

Andrew Morton wrote:
> Christoph Lameter <clameter@engr.sgi.com> wrote:
> 
>> Could we boot the box without quiet so that we can get better debug
>> messages?
> 
> 
> It didn't produce anything interesting.  For some reason the console output
> stops when start_kernel() runs console_init() (I guess it all comes out
> later) so the machine is running blind when we run kmem_cache_init(). 
> Irritating.  I just moved the console_init() call to happen later on.
> 
> It's going BUG() in kmem_cache_init()->set_up_list3s->is_node_online
> because for some reason the !CONFIG_NUMA ppc build has MAX_NUMNODES=16,
> even though there's only one node.
> 
> Doing
> 
> #define is_node_online(node) node_online(node)

As Dave Hansen mentioned elsewhere in this thread, there is no need to
define this is_node_online() macro, as node_online() does EXACTLY the same
thing (minus the BUG() which is probably overkill).


> unconditionally fixes that up (your patch shuld be using
> for_each_online_node() everywhere?) but it oopses later - I think it's the
> first time kmem_cache_alloc() is called.

Christoph should replace all the for (i = 0; i < MAX_NUMNODES; i++) loops
with for_each_node(i) and the one loop that does this:
for (i = 0; i < MAX_NUMNODES; i++) {
	if (!node_online(i))
		continue;
(or something similar) with for_each_online_node(i)

Also, there is a similar loop for CPUs which should be replaced with
for_each_online_cpu(i).

These for_each_FOO macros are cleaner and less likely to break in the
future, since we can simply modify the one definition if the way to
itterate over nodes/cpus changes, rather than auditing 100 open coded
implementations and trying to determine the intent of the loop's author.

-Matt