From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andre Przywara Subject: Re: NUMA problem with AMD G34 system Date: Mon, 21 Feb 2011 12:51:36 +0100 Message-ID: <4D6251C8.1020809@amd.com> References: <20110219085932.E08B624258@panix5.panix.com> <4D6226F0.1080007@amd.com> <20110221112958.GA25000@panix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20110221112958.GA25000@panix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Brian Marcotte Cc: "xen-devel@lists.xensource.com" List-Id: xen-devel@lists.xenproject.org Brian Marcotte wrote: >> The SRAT CPU table seems sane, but the memory table seems to say >> that there is no memory on node 1 and 3, thus Xen will simply put >> those CPUs to node 0 as it cannot cope with memory-less nodes. > > Ah. I think that's what was causing my confusion. Since the cpu-to-node > mapping was unbalanced and Xen wasn't complaining, I thought for sure > this was a problem with Xen and newer CPUs and I'd need to help with > debugging the problem. I prepared Xen to handle this situation (nodes without memory) before the M-C launch, before that it simply crashed. I agree that it is not obvious that you need to populate at least 4 DIMMs per CPU. > In this case, I suppose I'll remove a CPU and rearrange the memory > until it's time to add more memory. Do you have only 4 DIMMs? If yes, you could do so if you can cope with half of the CPU power. If you don't want to sacrifice the second CPU, you could go ahead and use the system with this kind of strange NUMA setup. Depending on your workload you will benefit more from this setup, as the impact of remote memory on most application is not that huge (compared to less cores, at least). But if you have 8 DIMMs, you should keep the second CPU and put all DIMMs in the blue slots. Regards, Andre. -- Andre Przywara AMD-Operating System Research Center (OSRC), Dresden, Germany