From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andre Przywara Subject: [PATCH] NUMA: Fix computation of needed nodes Date: Wed, 13 Jan 2010 23:10:54 +0100 Message-ID: <4B4E44EE.70608@amd.com> References: <4B4D8F1C02000078000299EC@vpn.id2.novell.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------060706020909020605060805" Return-path: In-Reply-To: <4B4D8F1C02000078000299EC@vpn.id2.novell.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Jan Beulich , Keir Fraser Cc: xen-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org --------------060706020909020605060805 Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit Hi, as Jan Beulich pointed out: > I'm afraid this change isn't really correct: > >> + cores_per_node = info['nr_cpus'] / info['nr_nodes'] >> + nodes_required = (self.info['VCPUs_max'] + cores_per_node - 1) / cores_per_node > > Simply using cores_per_node (as calculated here) as a divisor is bound > to cause division-by-zero issues, namely when limiting the number of > CPUs on the Xen command line (maxcpus=). Actually Jan's proposed method of getting additional nodes is more elegant, so I implemented: enumerating the best nodes and adding CPU affinity until all VCPUs can be backed by at least on physical core. This should fix problems with asymmetric NUMA configurations and cropped number of CPUs in Xen. Regards, Andre. Signed-off-by: Andre Przywara -- Andre Przywara AMD-Operating System Research Center (OSRC), Dresden, Germany Tel: +49 351 488-3567-12 ----to satisfy European Law for business letters: Advanced Micro Devices GmbH Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen Geschaeftsfuehrer: Andrew Bowd; Thomas M. McCoy; Giuliano Meroni Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen Registergericht Muenchen, HRB Nr. 43632 --------------060706020909020605060805 Content-Type: text/plain; name="numa_div_by_zero.patch" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="numa_div_by_zero.patch" diff -r 13d4e78ede97 tools/python/xen/xend/XendDomainInfo.py --- a/tools/python/xen/xend/XendDomainInfo.py Wed Jan 13 08:33:34 2010 +0000 +++ b/tools/python/xen/xend/XendDomainInfo.py Wed Jan 13 22:55:56 2010 +0100 @@ -2724,13 +2724,12 @@ candidate_node_list.append(i) best_node = find_relaxed_node(candidate_node_list)[0] cpumask = info['node_to_cpu'][best_node] - cores_per_node = info['nr_cpus'] / info['nr_nodes'] - nodes_required = (self.info['VCPUs_max'] + cores_per_node - 1) / cores_per_node - if nodes_required > 1: - log.debug("allocating %d NUMA nodes", nodes_required) - best_nodes = find_relaxed_node(filter(lambda x: x != best_node, range(0,info['nr_nodes']))) - for i in best_nodes[:nodes_required - 1]: - cpumask = cpumask + info['node_to_cpu'][i] + best_nodes = find_relaxed_node(filter(lambda x: x != best_node, range(0,info['nr_nodes']))) + for node_idx in best_nodes: + if len(cpumask) >= self.info['VCPUs_max']: + break + cpumask = cpumask + info['node_to_cpu'][node_idx] + log.debug("allocating additional NUMA node %d", node_idx) for v in range(0, self.info['VCPUs_max']): xc.vcpu_setaffinity(self.domid, v, cpumask) return index --------------060706020909020605060805 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --------------060706020909020605060805--