From mboxrd@z Thu Jan  1 00:00:00 1970
From: beth kon <eak@us.ibm.com>
Subject: Re: [PATCH] nr_cpus calculation problem due to
	incorrectsockets_per_node
Date: Fri, 19 Oct 2007 14:27:09 -0400
Message-ID: <4718F6FD.2030103@us.ibm.com>
References: <4718C36F.5090607@us.ibm.com>
	<4C02909CDA7BF14384D8A3F2FA01E89807A25B69@USEA-EXCH2.na.uis.unisys.com>
Reply-To: eak@us.ibm.com
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Return-path: <xen-devel-bounces@lists.xensource.com>
In-Reply-To: <4C02909CDA7BF14384D8A3F2FA01E89807A25B69@USEA-EXCH2.na.uis.unisys.com>
List-Unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xensource.com>
List-Help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-Subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
Sender: xen-devel-bounces@lists.xensource.com
Errors-To: xen-devel-bounces@lists.xensource.com
To: "Puthiyaparambil, Aravindh" <aravindh.puthiyaparambil@unisys.com>
Cc: Xen-devel@lists.xensource.com
List-Id: xen-devel@lists.xenproject.org

This patch does not address how many CPUs are recognized by Xen, it just 
addresses the number of CPUs reported via xm info. Without this patch, 
on asymmetric NUMA machines, xm info would report an incorrect nr_cpus 
and sockets_per_node.

The patch does not address building with max_phys_cpus=128, but I have 
only been able to build and boot with max_phys_cpus=64. With 128 the 
build breaks, so apparently other changes are required, but I haven't 
investigated what.


Puthiyaparambil, Aravindh wrote:

>Beth,
>
>With this patch you are able to come up with 128 logical CPUs on x86_64
>Xen? I.E you did a compile with max_phys_cpus=128 and successfully
>booted? Don't you need to also increase NR_RESERVED_GDT_PAGES based on
>number of CPUs among other things?
>
>Thanks,
>
>Aravindh Puthiyaparambil 
>Virtualization Engineer
>Virtual Systems Development
>Unisys, Tredyffrin PA
>
>-----Original Message-----
>From: xen-devel-bounces@lists.xensource.com
>[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of beth kon
>Sent: Friday, October 19, 2007 10:47 AM
>To: Xen-devel@lists.xensource.com; xen-ppc-devel@lists.xensource.com;
>xen-ia64-devel@lists.xensource.com
>Subject: [Xen-devel] [PATCH] nr_cpus calculation problem due to
>incorrectsockets_per_node
>
>Testing on an 8-node 128-way NUMA machine has exposed a problem with 
>Xen's nr_cpus calculation. In this case, since Xen cuts off recognized 
>CPUs at 32, the machine appears to have 16 CPUs on the first and second 
>nodes and none on the remaining nodes. Given this asymmetry, the 
>calculation of sockets_per_node (which is later used to calculate 
>nr_cpus) is incorrect:
>
>pi->sockets_per_node = num_online_cpus() /(num_online_nodes() * 
>pi->cores_per_socket * pi->threads_per_core);
>
>The most straightforward solution is to remove sockets_per_node, and 
>instead determine nr_cpus directly from num_online_cpus.
>
>This patch has been tested on x86_64 NUMA machines.
>
>  
>


-- 
Elizabeth Kon (Beth)
IBM Linux Technology Center
Open Hypervisor Team
email: eak@us.ibm.com