From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andre Przywara Subject: [Patch] tools: improve NUMA guest placement when ballooning Date: Thu, 10 Dec 2009 15:07:55 +0100 Message-ID: <4B2100BB.70807@amd.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------070201090907040702040207" Return-path: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Keir Fraser , "xen-devel@lists.xensource.com" List-Id: xen-devel@lists.xenproject.org --------------070201090907040702040207 Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit Hi, the "guest to a single NUMA node" constrain algorithm does not work well when we do ballooning. Ballooning and NUMA don't play together anyway, as Dom0 and thus ballooning is not NUMA aware, I am working on this but it will not be ready for the Xen 4.0 release window. The usual ballooning situation will result in an empty candidate list, as no node has enough free memory to host the guest. In this case the code will simply pick the first node: again and again, because all nodes without enough memory will be ultimately penalized with the same maxint value (regardless of the actual load). The attached patch will change this to use a relative penalty in case of not-enough memory, so that low-load low-memory nodes will be used at one point. A half loaded node has shown to be a good value, as an unbalanced system is much worse than non-local memory access for guests. Regardless of that you should restrict the Dom0 on a NUMA system to a reasonable memory size, so that ballooning is not necessary most of the time. In this case the guest's memory will be NUMA local. Signed-off-by: Andre Przywara Regards, Andre. -- Andre Przywara AMD-Operating System Research Center (OSRC), Dresden, Germany Tel: +49 351 448 3567 12 ----to satisfy European Law for business letters: Advanced Micro Devices GmbH Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen Geschaeftsfuehrer: Andrew Bowd; Thomas M. McCoy; Giuliano Meroni Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen Registergericht Muenchen, HRB Nr. 43632 --------------070201090907040702040207 Content-Type: text/x-patch; name="improve_NUMA_balloon.patch" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="improve_NUMA_balloon.patch" diff -r 8f304c003af4 tools/python/xen/xend/XendDomainInfo.py --- a/tools/python/xen/xend/XendDomainInfo.py Wed Dec 09 10:59:31 2009 +0000 +++ b/tools/python/xen/xend/XendDomainInfo.py Thu Dec 10 15:04:41 2009 +0100 @@ -2639,10 +2639,9 @@ nodeload[i] += 1 break for i in range(0, nr_nodes): - if len(info['node_to_cpu'][i]) > 0 and i in node_list: - nodeload[i] = int(nodeload[i] * 16 / len(info['node_to_cpu'][i])) - else: - nodeload[i] = sys.maxint + nodeload[i] = int(nodeload[i] * 16 / len(info['node_to_cpu'][i])) + if len(info['node_to_cpu'][i]) == 0 or i not in node_list: + nodelist[i] += 8 return map(lambda x: x[0], sorted(enumerate(nodeload), key=lambda x:x[1])) info = xc.physinfo() --------------070201090907040702040207 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --------------070201090907040702040207--