From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andre Przywara Subject: Re: Xen 3.4.1 NUMA support Date: Mon, 9 Nov 2009 23:51:50 +0100 Message-ID: <4AF89D06.9010204@amd.com> References: <4AF82F12.6040400@amd.com> <4AF82FD8.6020409@eu.citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <4AF82FD8.6020409@eu.citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: George Dunlap Cc: Dan Magenheimer , "xen-devel@lists.xensource.com" , Keir Fraser , Papagiannis Anastasios List-Id: xen-devel@lists.xenproject.org George Dunlap wrote: > Andre Przywara wrote: >> BTW: Shouldn't we set finally numa=on as the default value? >> > Is there any data to support the idea that this helps significantly on > common systems? I don't have any numbers handy, but I will try if I can generate some. Looking from a high level perspective it is a shame that it's not the default: With numa=off the Xen domain loader will allocate physical memory from some node (maybe even from several nodes) and will schedule the guest on some other (even rapidly changing) nodes. According to Murphy's law you will end up with _all_ the memory access of a guest to be remote. But in fact a NUMA architecture is really beneficial for virtualization: As there are close to zero cross domain memory accesses (except for Dom0), each node is more or less self contained and each guest can use the node's memory controller almost exclusively. But this is all spoiled as most people don't know about Xen's NUMA capabilities and don't set numa=on. Using this as a default would solve this. Regards, Andre. -- Andre Przywara AMD-Operating System Research Center (OSRC), Dresden, Germany Tel: +49 351 488-3567-12 ----to satisfy European Law for business letters: Advanced Micro Devices GmbH Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen Geschaeftsfuehrer: Jochen Polster; Thomas M. McCoy; Giuliano Meroni Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen Registergericht Muenchen, HRB Nr. 43632