Re: Nodes with no memory

Dave Hansen
2008-11-22T01:17:27Z

On Fri, 2008-11-21 at 18:49 -0600, Nathan Lynch wrote:
> Dave Hansen wrote:
> > I was handed off a bug report about a blade not booting with a, um
> > "newer" kernel.
>
> If you're unable to provide basic information such as the kernel
> version then perhaps this isn't the best forum for discussing this. :)

Let's just say a derivative of 2.6.27.5. I will, of course be trying to reproduce on mainline. I'm just going with the kernel closest to the bug report as I can get for now.

> > I'm thinking that we need to at least fix careful_allocation() to oops
> > and not return NULL, or check to make sure all it callers check its
> > return code.
>
> Well, careful_allocation() in current mainline tries pretty hard to
> panic if it can't satisfy the request. Why isn't that happening?

I added some random debugging to careful_alloc() to find out.

careful_allocation(1, 7680, 80, 0)
careful_allocation() ret1: 00000001dffe4100
careful_allocation() ret2: 00000001dffe4100
careful_allocation() ret3: 00000001dffe4100
careful_allocation() ret4: c000000000000000
careful_allocation() ret5: 0000000000000000

It looks to me like it is hitting 'the memory came from a previously
allocated node' check. So, the __lmb_alloc_base() appears to get
something worthwhile, but that gets overwritten later.

I'm still not quite sure what this comment means. Are we just trying to
get node locality from the allocation?

I also need to go look at how __alloc_bootmem_node() ends up returning
c000000000000000. It should be returning NULL, and panic'ing, in
careful_alloc(). This probably has to do with the fact that NODE_DATA()
isn't set up, yet, but I'll double check.

/*
* If the memory came from a previously allocated node, we must
* retry with the bootmem allocator.
*/
new_nid = early_pfn_to_nid(ret >> PAGE_SHIFT);
if (new_nid < nid) {
ret = (unsigned long)__alloc_bootmem_node(NODE_DATA(new_nid),
size, align, 0);
dbg("careful_allocation() ret4: %016lx\n", ret);

if (!ret)
panic("numa.c: cannot allocate %lu bytes on node %d",
size, new_nid);

ret = __pa(ret);
dbg("careful_allocation() ret5: %016lx\n", ret);

dbg("alloc_bootmem %lx %lx\n", ret, size);
}

-- Dave