linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH] x86: use near online node instead of round bin for numa
@ 2009-10-03 18:26 Yinghai Lu
  2009-10-05 14:44 ` Andi Kleen
  0 siblings, 1 reply; 6+ messages in thread
From: Yinghai Lu @ 2009-10-03 18:26 UTC (permalink / raw)
  To: Ingo Molnar, Thomas Gleixner, H. Peter Anvin, Suresh Siddha,
	Tejun Heo
  Cc: linux-kernel@vger.kernel.org


cpu to node mapping is set in following sequence:
1. numa_init_array: set up roundbin from cpu to online node
2. init_cpu_to_node: set that according to apicid_to_node[] according to srat
		  only handle that node is online, and leave other cpu on node
		  without ram (aka not online) to still round-bin
3. later srat_detect_node for intel/amd, will use first_online node or near by
   node.

problem is that setup_per_cpu_areas() is called between  2 and 3. the per_cpu
for cpu on node with ram is on different node. and could put that on node with
two hops away.

so try add find_near_online_node() and call int init_cpu_to_node()

Signed-off-by: Yinghai Lu <yinghai@kernel.org>

---
 arch/x86/kernel/cpu/intel.c |    6 +++++-
 arch/x86/mm/numa_64.c       |   21 ++++++++++++++++++++-
 2 files changed, 25 insertions(+), 2 deletions(-)

Index: linux-2.6/arch/x86/mm/numa_64.c
===================================================================
--- linux-2.6.orig/arch/x86/mm/numa_64.c
+++ linux-2.6/arch/x86/mm/numa_64.c
@@ -601,6 +601,25 @@ static __init int numa_setup(char *opt)
 early_param("numa", numa_setup);
 
 #ifdef CONFIG_NUMA
+
+static __init int find_near_online_node(int node)
+{
+	int n, val;
+	int min_val = INT_MAX;
+	int best_node = -1;
+
+	for_each_online_node(n) {
+		val = node_distance(node, n);
+
+		if (val < min_val) {
+			min_val = val;
+			best_node = n;
+		}
+	}
+
+	return best_node;
+}
+
 /*
  * Setup early cpu_to_node.
  *
@@ -632,7 +651,7 @@ void __init init_cpu_to_node(void)
 		if (node == NUMA_NO_NODE)
 			continue;
 		if (!node_online(node))
-			continue;
+			node = find_near_online_node(node);
 		numa_set_node(cpu, node);
 	}
 }
Index: linux-2.6/arch/x86/kernel/cpu/intel.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/cpu/intel.c
+++ linux-2.6/arch/x86/kernel/cpu/intel.c
@@ -263,8 +263,12 @@ static void __cpuinit srat_detect_node(s
 	/* Don't do the funky fallback heuristics the AMD version employs
 	   for now. */
 	node = apicid_to_node[apicid];
-	if (node == NUMA_NO_NODE || !node_online(node))
+	if (node == NUMA_NO_NODE)
 		node = first_node(node_online_map);
+	else if (!node_online(node)) {
+		/* reuse the value from init_cpu_to_node() */
+		node = cpu_to_node(cpu);
+	}
 	numa_set_node(cpu, node);
 
 	printk(KERN_INFO "CPU %d/0x%x -> Node %d\n", cpu, apicid, node);

^ permalink raw reply	[flat|nested] 6+ messages in thread
end of thread, other threads:[~2009-10-05 18:51 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2009-10-03 18:26 [PATCH] x86: use near online node instead of round bin for numa Yinghai Lu
2009-10-05 14:44 ` Andi Kleen
2009-10-05 18:09   ` Yinghai Lu
2009-10-05 18:35     ` Andi Kleen
2009-10-05 18:40       ` Yinghai Lu
2009-10-05 18:50         ` Andi Kleen

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).