Re: NUMA problem with AMD G34 system

[Andre Przywara <andre.przywara@amd.com>]

Brian Marcotte wrote:
>> The SRAT CPU table seems sane, but the memory table seems to say
>> that there is no memory on node 1 and 3, thus Xen will simply put
>> those CPUs to node 0 as it cannot cope with memory-less nodes.
> 
> Ah. I think that's what was causing my confusion. Since the cpu-to-node
> mapping was unbalanced and Xen wasn't complaining, I thought for sure
> this was a problem with Xen and newer CPUs and I'd need to help with
> debugging the problem.
I prepared Xen to handle this situation (nodes without memory) before 
the M-C launch, before that it simply crashed. I agree that it is not 
obvious that you need to populate at least 4 DIMMs per CPU.

> In this case, I suppose I'll remove a CPU and rearrange the memory
> until it's time to add more memory.
Do you have only 4 DIMMs? If yes, you could do so if you can cope with 
half of the CPU power.
If you don't want to sacrifice the second CPU, you could go ahead and 
use the system with this kind of strange NUMA setup. Depending on your 
workload you will benefit more from this setup, as the impact of remote 
memory on most application is not that huge (compared to less cores, at 
least).
But if you have 8 DIMMs, you should keep the second CPU and put all 
DIMMs in the blue slots.

Regards,
Andre.

-- 
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany