From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37376) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WmXZv-00006V-W5 for qemu-devel@nongnu.org; Mon, 19 May 2014 20:06:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WmXZk-0003yQ-7t for qemu-devel@nongnu.org; Mon, 19 May 2014 20:06:47 -0400 Received: from e39.co.us.ibm.com ([32.97.110.160]:47464) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WmXZj-0003yE-UT for qemu-devel@nongnu.org; Mon, 19 May 2014 20:06:36 -0400 Received: from /spool/local by e39.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 19 May 2014 18:06:34 -0600 Date: Mon, 19 May 2014 17:06:18 -0700 From: Nishanth Aravamudan Message-ID: <20140520000617.GA29892@linux.vnet.ibm.com> References: <20140519223752.GA23182@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140519223752.GA23182@linux.vnet.ibm.com> Subject: Re: [Qemu-devel] qemu-ppc and NUMA topology List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Alexey Kardashevskiy Cc: qemu-ppc@nongnu.org, anton@samba.org, qemu-devel@nongnu.org On 19.05.2014 [15:37:52 -0700], Nishanth Aravamudan wrote: > Hi Alexey, > > I've been looking at hw/ppc/spapr.c::spapr_populate_memory() and ran > into a few questions: > > 1) The values from 1 to nb_numa_nodes are used as indices into the > node_mem array, but that is not populated, necessarily, linearly. > vl.c::add_node() uses the nodeid parameter as the index into node_mem, > if it is specified. > > 2) The node ID is based upon the index into the array, but it seems like > it should actually be based upon the nodeid specified, if any. That is, > we set the value at index 4 (which is statically the reference point in > 'ibm,associativity-reference-points') of 'ibm,associativty' for each > 'ibm,memory@....' node to the index we are currently at. But as > mentioned in 1) above that index isn't necessarily currently the nodeid > specified on the command-line. > > What this all means, is that if I specify something like: > > -numa node,nodeid=1,cpus=0-7,mem=2048 -numa > node,nodeid=5,cpus=8-15,mem=0 -numa node,nodeid=9,mem=2048 > > Linux sees: > > numactl --hardware > available: 3 nodes (0-2) > node 0 cpus: 8 9 10 11 12 13 14 15 > node 0 size: 0 MB > node 0 free: 0 MB > node 1 cpus: 0 1 2 3 4 5 6 7 > node 1 size: 2024 MB > node 1 free: 1560 MB > node 2 cpus: > node 2 size: 0 MB > node 2 free: 0 MB > > Maybe we don't really care about this, but I just noticed it when trying > to reproduce some really weird topologies from PowerVM. Upon further investigation into node_mem, it seems like this assumption is present throughout the qemu code, e.g, the qemu monitor 'info numa' command. Will just document it for myself as a weird way to make memoryless nodes show up :) Thanks, Nish