[PATCH] - Memoryless nodes

[PATCH] - Memoryless nodes

[Jack Steiner]

Add support for nodes that have cpus but no memory.
The current code was failing to add these nodes
to the nodes_present_map.

Signed-off-by: Jack Steiner <steiner@sgi.com>

---
 arch/x86/kernel/tlb_uv.c |   12 +++++++++++-
 arch/x86/mm/srat_64.c    |    5 ++++-
 2 files changed, 15 insertions(+), 2 deletions(-)

Index: linux/arch/x86/mm/srat_64.c
===================================================================
--- linux.orig/arch/x86/mm/srat_64.c	2009-04-12 10:38:26.000000000 -0500
+++ linux/arch/x86/mm/srat_64.c	2009-04-12 10:38:47.000000000 -0500
@@ -28,6 +28,7 @@ int acpi_numa __initdata;
 static struct acpi_table_slit *acpi_slit;
 
 static nodemask_t nodes_parsed __initdata;
+static nodemask_t cpu_nodes_parsed __initdata;
 static struct bootnode nodes[MAX_NUMNODES] __initdata;
 static struct bootnode nodes_add[MAX_NUMNODES];
 static int found_add_area __initdata;
@@ -174,6 +175,7 @@ acpi_numa_processor_affinity_init(struct
 	else
 		apic_id = pa->apic_id;
 	apicid_to_node[apic_id] = node;
+	node_set(node, cpu_nodes_parsed);
 	acpi_numa = 1;
 	printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
 	       pxm, apic_id, node);
@@ -402,7 +404,8 @@ int __init acpi_scan_nodes(unsigned long
 		return -1;
 	}
 
-	node_possible_map = nodes_parsed;
+	/* Account for nodes with cpus and no memory */
+	nodes_or(node_possible_map, nodes_parsed, cpu_nodes_parsed);
 
 	/* Finally register nodes */
 	for_each_node_mask(i, node_possible_map)

[tip:x86/urgent] x86/uv: fix init of memory-less nodes

[tip-bot for Jack Steiner]

Commit-ID:  28468050e2f5410c04fd737c922f2bea73ddd690
Gitweb:     http://git.kernel.org/tip/28468050e2f5410c04fd737c922f2bea73ddd690
Author:     Jack Steiner <steiner@sgi.com>
AuthorDate: Fri, 17 Apr 2009 09:22:42 -0500
Committer:  Ingo Molnar <mingo@elte.hu>
CommitDate: Fri, 17 Apr 2009 16:45:29 +0200

x86/uv: fix init of memory-less nodes

Add support for nodes that have cpus but no memory.
The current code was failing to add these nodes
to the nodes_present_map.

[ Impact: fix potential boot crash on memory-less UV nodes. ]

Signed-off-by: Jack Steiner <steiner@sgi.com>
LKML-Reference: <20090417142242.GA23743@sgi.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>


---
 arch/x86/mm/srat_64.c |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/arch/x86/mm/srat_64.c b/arch/x86/mm/srat_64.c
index c7d272b..1619026 100644
--- a/arch/x86/mm/srat_64.c
+++ b/arch/x86/mm/srat_64.c
@@ -28,6 +28,7 @@ int acpi_numa __initdata;
 static struct acpi_table_slit *acpi_slit;
 
 static nodemask_t nodes_parsed __initdata;
+static nodemask_t cpu_nodes_parsed __initdata;
 static struct bootnode nodes[MAX_NUMNODES] __initdata;
 static struct bootnode nodes_add[MAX_NUMNODES];
 static int found_add_area __initdata;
@@ -174,6 +175,7 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
 	else
 		apic_id = pa->apic_id;
 	apicid_to_node[apic_id] = node;
+	node_set(node, cpu_nodes_parsed);
 	acpi_numa = 1;
 	printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
 	       pxm, apic_id, node);
@@ -402,7 +404,8 @@ int __init acpi_scan_nodes(unsigned long start, unsigned long end)
 		return -1;
 	}
 
-	node_possible_map = nodes_parsed;
+	/* Account for nodes with cpus and no memory */
+	nodes_or(node_possible_map, nodes_parsed, cpu_nodes_parsed);
 
 	/* Finally register nodes */
 	for_each_node_mask(i, node_possible_map)

Re: [tip:x86/urgent] x86/uv: fix init of memory-less nodes

[David Rientjes]

On Fri, 17 Apr 2009, tip-bot for Jack Steiner wrote:

> Commit-ID:  28468050e2f5410c04fd737c922f2bea73ddd690
> Gitweb:     http://git.kernel.org/tip/28468050e2f5410c04fd737c922f2bea73ddd690
> Author:     Jack Steiner <steiner@sgi.com>
> AuthorDate: Fri, 17 Apr 2009 09:22:42 -0500
> Committer:  Ingo Molnar <mingo@elte.hu>
> CommitDate: Fri, 17 Apr 2009 16:45:29 +0200
> 
> x86/uv: fix init of memory-less nodes
> 
> Add support for nodes that have cpus but no memory.
> The current code was failing to add these nodes
> to the nodes_present_map.
> 
> [ Impact: fix potential boot crash on memory-less UV nodes. ]
> 

This is incomplete, x2apic maps cpus to pxms with a type 2 srat entry and 
that pxm may not have affinity to any type 1 memory structure.

Re: [tip:x86/urgent] x86/uv: fix init of memory-less nodes

[Jack Steiner]

x86/uv: fix init of memory-less nodes

Add support for nodes that have cpus but no memory.
The current code was failing to add these nodes
to the nodes_present_map.

[ Impact: fix potential boot crash on memory-less nodes. ]

Signed-off-by: Jack Steiner <steiner@sgi.com>



---

Fixes case caught by David - missed support for the x2apic SRAT table.



 arch/x86/mm/srat_64.c |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

Index: linux/arch/x86/mm/srat_64.c
===================================================================
--- linux.orig/arch/x86/mm/srat_64.c	2009-04-17 15:12:13.000000000 -0500
+++ linux/arch/x86/mm/srat_64.c	2009-04-17 15:15:15.000000000 -0500
@@ -28,6 +28,7 @@ int acpi_numa __initdata;
 static struct acpi_table_slit *acpi_slit;
 
 static nodemask_t nodes_parsed __initdata;
+static nodemask_t cpu_nodes_parsed __initdata;
 static struct bootnode nodes[MAX_NUMNODES] __initdata;
 static struct bootnode nodes_add[MAX_NUMNODES];
 static int found_add_area __initdata;
@@ -141,6 +142,7 @@ acpi_numa_x2apic_affinity_init(struct ac
 
 	apic_id = pa->apic_id;
 	apicid_to_node[apic_id] = node;
+	node_set(node, cpu_nodes_parsed);
 	acpi_numa = 1;
 	printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
 	       pxm, apic_id, node);
@@ -174,6 +176,7 @@ acpi_numa_processor_affinity_init(struct
 	else
 		apic_id = pa->apic_id;
 	apicid_to_node[apic_id] = node;
+	node_set(node, cpu_nodes_parsed);
 	acpi_numa = 1;
 	printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
 	       pxm, apic_id, node);
@@ -402,7 +405,8 @@ int __init acpi_scan_nodes(unsigned long
 		return -1;
 	}
 
-	node_possible_map = nodes_parsed;
+	/* Account for nodes with cpus and no memory */
+	nodes_or(node_possible_map, nodes_parsed, cpu_nodes_parsed);
 
 	/* Finally register nodes */
 	for_each_node_mask(i, node_possible_map)

[tip:x86/urgent] x86/uv: fix init of memory-less nodes

[tip-bot for Jack Steiner]

Commit-ID:  dc098551918093901d8ac8936e9d1a1b891b56ed
Gitweb:     http://git.kernel.org/tip/dc098551918093901d8ac8936e9d1a1b891b56ed
Author:     Jack Steiner <steiner@sgi.com>
AuthorDate: Fri, 17 Apr 2009 09:22:42 -0500
Committer:  Ingo Molnar <mingo@elte.hu>
CommitDate: Fri, 17 Apr 2009 22:42:12 +0200

x86/uv: fix init of memory-less nodes

Add support for nodes that have cpus but no memory.
The current code was failing to add these nodes
to the nodes_present_map.

v2: Fixes case caught by David Rientjes - missed support
    for the x2apic SRAT table.

[ Impact: fix potential boot crash on memory-less UV nodes. ]

Reported-by: David Rientjes <rientjes@google.com>
Signed-off-by: Jack Steiner <steiner@sgi.com>
LKML-Reference: <20090417142242.GA23743@sgi.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>


---
 arch/x86/mm/srat_64.c |    6 +++++-
 1 files changed, 5 insertions(+), 1 deletions(-)

diff --git a/arch/x86/mm/srat_64.c b/arch/x86/mm/srat_64.c
index c7d272b..33c5fa5 100644
--- a/arch/x86/mm/srat_64.c
+++ b/arch/x86/mm/srat_64.c
@@ -28,6 +28,7 @@ int acpi_numa __initdata;
 static struct acpi_table_slit *acpi_slit;
 
 static nodemask_t nodes_parsed __initdata;
+static nodemask_t cpu_nodes_parsed __initdata;
 static struct bootnode nodes[MAX_NUMNODES] __initdata;
 static struct bootnode nodes_add[MAX_NUMNODES];
 static int found_add_area __initdata;
@@ -141,6 +142,7 @@ acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
 
 	apic_id = pa->apic_id;
 	apicid_to_node[apic_id] = node;
+	node_set(node, cpu_nodes_parsed);
 	acpi_numa = 1;
 	printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
 	       pxm, apic_id, node);
@@ -174,6 +176,7 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
 	else
 		apic_id = pa->apic_id;
 	apicid_to_node[apic_id] = node;
+	node_set(node, cpu_nodes_parsed);
 	acpi_numa = 1;
 	printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
 	       pxm, apic_id, node);
@@ -402,7 +405,8 @@ int __init acpi_scan_nodes(unsigned long start, unsigned long end)
 		return -1;
 	}
 
-	node_possible_map = nodes_parsed;
+	/* Account for nodes with cpus and no memory */
+	nodes_or(node_possible_map, nodes_parsed, cpu_nodes_parsed);
 
 	/* Finally register nodes */
 	for_each_node_mask(i, node_possible_map)

Re: [PATCH] - Memoryless nodes

[Andi Kleen]

Jack Steiner <steiner@sgi.com> writes:

> Add support for nodes that have cpus but no memory.
> The current code was failing to add these nodes
> to the nodes_present_map.

That was intentional because a node is defined as an area of memory
and an area of memory without memory didn't seem to make sense.
I still don't think it does. Instead the CPUs are assigned
to nearby nodes. Back then the generic code also couldn't deal
with them at all, so you might well run into more problems.

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.

Re: [PATCH] - Memoryless nodes

[Jack Steiner]

On Sat, Apr 18, 2009 at 11:00:46AM +0200, Andi Kleen wrote:
> Jack Steiner <steiner@sgi.com> writes:
> 
> > Add support for nodes that have cpus but no memory.
> > The current code was failing to add these nodes
> > to the nodes_present_map.
> 
> That was intentional because a node is defined as an area of memory
> and an area of memory without memory didn't seem to make sense.
> I still don't think it does. Instead the CPUs are assigned
> to nearby nodes. Back then the generic code also couldn't deal
> with them at all, so you might well run into more problems.


Nodes can also be viewed as a collection of cpus and memory that are
located "close together" - ie., same PXM in the SRAT. Normally nodes will
have both memory & cpus but in theory either cpus or memory could be
missing.

I have run a number of tests of nodes with cpus only and so far have not
seen any issues. I certainly won't rule out problems but have not seen
them yet.

If there are any corner cases that you are aware of, let me know & I can
do some more targeted testing.

(Note: this is not a bizarre end case that I'm trying to fix. UV actually
has blades with cpus and no memory available to the OS.)

--- jack