From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50540) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WmWCp-00018I-0u for qemu-devel@nongnu.org; Mon, 19 May 2014 18:39:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WmWCg-0005EK-0B for qemu-devel@nongnu.org; Mon, 19 May 2014 18:38:50 -0400 Received: from e33.co.us.ibm.com ([32.97.110.151]:47353) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WmWCf-0005E4-NI for qemu-devel@nongnu.org; Mon, 19 May 2014 18:38:41 -0400 Received: from /spool/local by e33.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 19 May 2014 16:38:38 -0600 Date: Mon, 19 May 2014 15:37:52 -0700 From: Nishanth Aravamudan Message-ID: <20140519223752.GA23182@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Subject: [Qemu-devel] qemu-ppc and NUMA topology List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Alexey Kardashevskiy Cc: qemu-ppc@nongnu.org, anton@samba.org, qemu-devel@nongnu.org Hi Alexey, I've been looking at hw/ppc/spapr.c::spapr_populate_memory() and ran into a few questions: 1) The values from 1 to nb_numa_nodes are used as indices into the node_mem array, but that is not populated, necessarily, linearly. vl.c::add_node() uses the nodeid parameter as the index into node_mem, if it is specified. 2) The node ID is based upon the index into the array, but it seems like it should actually be based upon the nodeid specified, if any. That is, we set the value at index 4 (which is statically the reference point in 'ibm,associativity-reference-points') of 'ibm,associativty' for each 'ibm,memory@....' node to the index we are currently at. But as mentioned in 1) above that index isn't necessarily currently the nodeid specified on the command-line. What this all means, is that if I specify something like: -numa node,nodeid=1,cpus=0-7,mem=2048 -numa node,nodeid=5,cpus=8-15,mem=0 -numa node,nodeid=9,mem=2048 Linux sees: numactl --hardware available: 3 nodes (0-2) node 0 cpus: 8 9 10 11 12 13 14 15 node 0 size: 0 MB node 0 free: 0 MB node 1 cpus: 0 1 2 3 4 5 6 7 node 1 size: 2024 MB node 1 free: 1560 MB node 2 cpus: node 2 size: 0 MB node 2 free: 0 MB Maybe we don't really care about this, but I just noticed it when trying to reproduce some really weird topologies from PowerVM. Thanks, Nish