Problem with nr_nodes on large memory NUMA machine

[beth kon <eak@us.ibm.com>]

We've run into an issue with an 8 node x3950 where xm info is showing 
only 6 nodes. I've traced the problem to the clip_to_limit function in 
arch/x86/e820.c.

#ifdef __x86_64__
   clip_to_limit((uint64_t)(MACH2PHYS_COMPAT_VIRT_END -
                            __HYPERVISOR_COMPAT_VIRT_START) << 10,
                 "Only the first %u GB of the physical memory map "
                 "can be accessed by 32-on-64 guests.");
#endif

Boot messages....
(XEN) WARNING: Only the first 166 GB of the physical memory map can be 
accessed by 32-on-64 guests. (XEN) Truncating memory map to 174063616kB

After the memory is clipped, acpi_scan_nodes runs cutoff_node, which 
limits the memory associated with each node according to the cutoff 
values. Then, acpi_scan_nodes calls unparse_node to "remove" nodes that 
don't have the minimum amount of memory, due to the clipping of the 
memory range.

Can someone explain what this is all about and why it might be necessary?

-- 
Elizabeth Kon (Beth)
IBM Linux Technology Center
Open Hypervisor Team
email: eak@us.ibm.com