[PATCH] you have how many nodes??

[PATCH] you have how many nodes??

[Jesse Barnes]

Needed this for booting on a 128 node system.

Thanks,
Jesse


diff -Nru a/include/linux/mm.h b/include/linux/mm.h
--- a/include/linux/mm.h	Wed Sep 10 14:31:09 2003
+++ b/include/linux/mm.h	Wed Sep 10 14:31:09 2003
@@ -324,7 +324,7 @@
  * sets it, so none of the operations on it need to be atomic.
  */
 #define NODE_SHIFT 4
-#define ZONE_SHIFT (BITS_PER_LONG - 8)
+#define ZONE_SHIFT (BITS_PER_LONG - 10)
 
 struct zone;
 extern struct zone *zone_table[];
diff -Nru a/include/linux/mmzone.h b/include/linux/mmzone.h
--- a/include/linux/mmzone.h	Wed Sep 10 14:31:09 2003
+++ b/include/linux/mmzone.h	Wed Sep 10 14:31:09 2003
@@ -311,8 +311,8 @@
 
 #include <asm/mmzone.h>
 
-/* page->zone is currently 8 bits ... */
-#define MAX_NR_NODES		(255 / MAX_NR_ZONES)
+/* page->zone is currently 10 bits ... */
+#define MAX_NR_NODES		NR_NODES
 
 #endif /* !CONFIG_DISCONTIGMEM */
 

Re: [PATCH] you have how many nodes??

[Andrew Morton]

jbarnes@sgi.com (Jesse Barnes) wrote:
>
> Needed this for booting on a 128 node system.
>
> -#define ZONE_SHIFT (BITS_PER_LONG - 8)
> +#define ZONE_SHIFT (BITS_PER_LONG - 10)

eeek, ia32 just lost another two page flags.

This stuff needs to be controlled by per-arch and per-subarch header files.

Instead of going backwards like this we'd like to actually free up _more_
bits in page->flags.  The worst (and controlling) case is on 32-bit NUMA:
eight nodes, three zones per node.  That's five bits, leaving us 27 page
flags.

So we'd need

	include/asm-foo/zonestuff.h:
	
	#define ARCH_MAX_NODES_SHIFT	3	/* Up to 8 nodes */
	#define ARCH_MAX_ZONES_SHIFT	2	/* Up to 4 zones per node */


and all the mm.h/mmzone.h constants use those two.


I think.  We could just say "dang numaq needs five bits", so:


	#if BITS_PER_LONG == 32
	#define ZONE_SHIFT 5
	#else
	#define ZONE_SHIFT 10
	#endif


Bit sleazy, but I think that would suffice.

Re: [PATCH] you have how many nodes??

[Jesse Barnes]

On Wed, Sep 10, 2003 at 03:12:54PM -0700, Andrew Morton wrote:
> I think.  We could just say "dang numaq needs five bits", so:
> 
> 
> 	#if BITS_PER_LONG == 32
> 	#define ZONE_SHIFT 5
> 	#else
> 	#define ZONE_SHIFT 10
> 	#endif

That's fine with me, do you want me to rediff and send a new patch?

Thanks,
Jesse

Re: [PATCH] you have how many nodes??

[Andrew Morton]

jbarnes@sgi.com (Jesse Barnes) wrote:
>
> On Wed, Sep 10, 2003 at 03:12:54PM -0700, Andrew Morton wrote:
> > I think.  We could just say "dang numaq needs five bits", so:
> > 
> > 
> > 	#if BITS_PER_LONG == 32
> > 	#define ZONE_SHIFT 5
> > 	#else
> > 	#define ZONE_SHIFT 10
> > 	#endif
> 
> That's fine with me, do you want me to rediff and send a new patch?
> 

Well your patch as it stands would appear to break NUMAQ builds, due to
NUMAQ setting MAX_NUMNODES directly in the arch code.  ia64 is using
another layer of macroification via NR_NODES instead.

MAX_NUMNODES, NR_NODES and MAX_NR_NODES appear to be a bit of a mess, and
they should all be replaced with shift distances anyway.

Could you please get together with Martin Bligh, come up with something
which works on NUMAQ and your 128 CPU PDA and also cast an eye across the
other architectures (sparc64, sh, ...)?  It all needs a bit of thought and
a spring clean.

Thanks.

Re: [PATCH] you have how many nodes??

[Martin J. Bligh]

>> > I think.  We could just say "dang numaq needs five bits", so:
>> > 
>> > 
>> > 	# if BITS_PER_LONG == 32
>> > 	# define ZONE_SHIFT 5
>> > 	# else
>> > 	# define ZONE_SHIFT 10
>> > 	# endif
>> 
>> That's fine with me, do you want me to rediff and send a new patch?
> 
> Well your patch as it stands would appear to break NUMAQ builds, due to
> NUMAQ setting MAX_NUMNODES directly in the arch code.  ia64 is using
> another layer of macroification via NR_NODES instead.
> 
> MAX_NUMNODES, NR_NODES and MAX_NR_NODES appear to be a bit of a mess, and
> they should all be replaced with shift distances anyway.

;-)

Yes, it's a turgid mess.

I'd prefer to define things in terms of MAX_NUMNODES, and derive the shifts
from that if possible - much more intuitive to maintain.
But other than that I agree completely with you.
 
> Could you please get together with Martin Bligh, come up with something
> which works on NUMAQ and your 128 CPU PDA and also cast an eye across the
> other architectures (sparc64, sh, ...)?  It all needs a bit of thought and
> a spring clean.

I'll have a look, I'm sure we can come up with something between us.

M.

Re: [PATCH] you have how many nodes??

[Jesse Barnes]

On Wed, Sep 10, 2003 at 04:46:40PM -0700, Martin J. Bligh wrote:
> Yes, it's a turgid mess.
> 
> I'd prefer to define things in terms of MAX_NUMNODES, and derive the shifts
> from that if possible - much more intuitive to maintain.
> But other than that I agree completely with you.

Yeah, I don't mind switching, should just be a search and replace.

> > Could you please get together with Martin Bligh, come up with something
> > which works on NUMAQ and your 128 CPU PDA and also cast an eye across the
> > other architectures (sparc64, sh, ...)?  It all needs a bit of thought and
> > a spring clean.
> 
> I'll have a look, I'm sure we can come up with something between us.

Cool, thanks.

Jesse

Re: [PATCH] you have how many nodes??

[William Lee Irwin III]

On Wed, Sep 10, 2003 at 03:12:54PM -0700, Andrew Morton wrote:
> Instead of going backwards like this we'd like to actually free up _more_
> bits in page->flags.  The worst (and controlling) case is on 32-bit NUMA:
> eight nodes, three zones per node.  That's five bits, leaving us 27 page
> flags.

The worst case for i386 NUMA is actually 16 nodes.


-- wli

Re: [PATCH] you have how many nodes??

[Matthew Dobson]

Ok, I made an attempt to clean up this mess quite a while ago (2.5.47), 
but that patch is utterly useless now.  At Martin's urging I've created 
a new series of patches to resolve this.

01 - Make sure MAX_NUMNODES is defined in one and only one place. 
Remove superfluous definitions.  Instead of defining MAX_NUMNODES in 
asm/numnodes.h, we define NODES_SHIFT there.  Then in linux/mmzone.h we 
turn that NODES_SHIFT value into MAX_NUMNODES.

02 - Remove MAX_NR_NODES.  This value is only used in a couple of 
places, and it's incorrectly used in all those places as far as I can 
tell.  Replace with MAX_NUMNODES.  Create MAX_NODES_SHIFT and use this 
value to check NODES_SHIFT is appropriate.  A possible future patch 
should make MAX_NODES_SHIFT vary based on 32 vs. 64 bit archs.

03 - Fix up the sh arch.  sh defined NR_NODES, change sh to use standard 
MAX_NUMNODES instead.

04 - Fix up the arm arch.  This needs to be reviewed.  Relatively 
straightforward replacement of NR_NODES with standard MAX_NUMNODES.

05 - Fix up the ia64 arch.  This *definitely* needs to be reviewed. 
This code made my head hurt.  I think I may have gotten it right. 
Totally untested.

Cheers!

-Matt

Jesse Barnes wrote:
> On Wed, Sep 10, 2003 at 04:46:40PM -0700, Martin J. Bligh wrote:
> 
>>Yes, it's a turgid mess.
>>
>>I'd prefer to define things in terms of MAX_NUMNODES, and derive the shifts
>>from that if possible - much more intuitive to maintain.
>>But other than that I agree completely with you.
> 
> 
> Yeah, I don't mind switching, should just be a search and replace.
> 
> 
>>>Could you please get together with Martin Bligh, come up with something
>>>which works on NUMAQ and your 128 CPU PDA and also cast an eye across the
>>>other architectures (sparc64, sh, ...)?  It all needs a bit of thought and
>>>a spring clean.
>>
>>I'll have a look, I'm sure we can come up with something between us.
> 
> 
> Cool, thanks.
> 
> Jesse
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

[PATCH] Clean up MAX_NR_NODES/NUMNODES/etc. [1/5]

[Matthew Dobson]

[-- Attachment #1: Type: text/plain, Size: 1262 bytes --]

Matthew Dobson wrote:
> Ok, I made an attempt to clean up this mess quite a while ago (2.5.47), 
> but that patch is utterly useless now.  At Martin's urging I've created 
> a new series of patches to resolve this.
> 
> 01 - Make sure MAX_NUMNODES is defined in one and only one place. Remove 
> superfluous definitions.  Instead of defining MAX_NUMNODES in 
> asm/numnodes.h, we define NODES_SHIFT there.  Then in linux/mmzone.h we 
> turn that NODES_SHIFT value into MAX_NUMNODES.
> 
> 02 - Remove MAX_NR_NODES.  This value is only used in a couple of 
> places, and it's incorrectly used in all those places as far as I can 
> tell.  Replace with MAX_NUMNODES.  Create MAX_NODES_SHIFT and use this 
> value to check NODES_SHIFT is appropriate.  A possible future patch 
> should make MAX_NODES_SHIFT vary based on 32 vs. 64 bit archs.
> 
> 03 - Fix up the sh arch.  sh defined NR_NODES, change sh to use standard 
> MAX_NUMNODES instead.
> 
> 04 - Fix up the arm arch.  This needs to be reviewed.  Relatively 
> straightforward replacement of NR_NODES with standard MAX_NUMNODES.
> 
> 05 - Fix up the ia64 arch.  This *definitely* needs to be reviewed. This 
> code made my head hurt.  I think I may have gotten it right. Totally 
> untested.

Cheers!

-Matt

[-- Attachment #2: 01-max_numnodes2nodes_shift.patch --]
[-- Type: text/plain, Size: 5045 bytes --]

diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5/include/asm-alpha/numnodes.h linux-2.6.0-test5-max_numnodes2nodes_shift/include/asm-alpha/numnodes.h
--- linux-2.6.0-test5/include/asm-alpha/numnodes.h	Mon Sep  8 12:49:53 2003
+++ linux-2.6.0-test5-max_numnodes2nodes_shift/include/asm-alpha/numnodes.h	Mon Sep 15 13:28:10 2003
@@ -1,6 +1,7 @@
 #ifndef _ASM_MAX_NUMNODES_H
 #define _ASM_MAX_NUMNODES_H
 
-#define MAX_NUMNODES		128 /* Marvel */
+/* Max 128 Nodes - Marvel */
+#define NODES_SHIFT	7
 
 #endif /* _ASM_MAX_NUMNODES_H */
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5/include/asm-i386/numaq.h linux-2.6.0-test5-max_numnodes2nodes_shift/include/asm-i386/numaq.h
--- linux-2.6.0-test5/include/asm-i386/numaq.h	Mon Sep  8 12:50:06 2003
+++ linux-2.6.0-test5-max_numnodes2nodes_shift/include/asm-i386/numaq.h	Fri Sep 12 16:58:19 2003
@@ -28,7 +28,6 @@
 
 #ifdef CONFIG_X86_NUMAQ
 
-#define MAX_NUMNODES		16
 extern void get_memcfg_numaq(void);
 #define get_memcfg_numa() get_memcfg_numaq()
 
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5/include/asm-i386/numnodes.h linux-2.6.0-test5-max_numnodes2nodes_shift/include/asm-i386/numnodes.h
--- linux-2.6.0-test5/include/asm-i386/numnodes.h	Mon Sep  8 12:49:54 2003
+++ linux-2.6.0-test5-max_numnodes2nodes_shift/include/asm-i386/numnodes.h	Fri Sep 12 16:58:19 2003
@@ -4,11 +4,15 @@
 #include <linux/config.h>
 
 #ifdef CONFIG_X86_NUMAQ
-#include <asm/numaq.h>
+
+/* Max 16 Nodes */
+#define NODES_SHIFT	4
+
 #elif CONFIG_NUMA
-#include <asm/srat.h>
-#else
-#define MAX_NUMNODES	1
+
+/* Max 8 Nodes */
+#define NODES_SHIFT	3
+
 #endif /* CONFIG_X86_NUMAQ */
 
 #endif /* _ASM_MAX_NUMNODES_H */
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5/include/asm-i386/srat.h linux-2.6.0-test5-max_numnodes2nodes_shift/include/asm-i386/srat.h
--- linux-2.6.0-test5/include/asm-i386/srat.h	Mon Sep  8 12:49:58 2003
+++ linux-2.6.0-test5-max_numnodes2nodes_shift/include/asm-i386/srat.h	Fri Sep 12 16:37:51 2003
@@ -27,7 +27,6 @@
 #ifndef _ASM_SRAT_H_
 #define _ASM_SRAT_H_
 
-#define MAX_NUMNODES		8
 extern void get_memcfg_from_srat(void);
 extern unsigned long *get_zholes_size(int);
 #define get_memcfg_numa() get_memcfg_from_srat()
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5/include/asm-ppc64/numnodes.h linux-2.6.0-test5-max_numnodes2nodes_shift/include/asm-ppc64/numnodes.h
--- linux-2.6.0-test5/include/asm-ppc64/numnodes.h	Mon Sep  8 12:50:22 2003
+++ linux-2.6.0-test5-max_numnodes2nodes_shift/include/asm-ppc64/numnodes.h	Fri Sep 12 16:58:19 2003
@@ -1,6 +1,7 @@
 #ifndef _ASM_MAX_NUMNODES_H
 #define _ASM_MAX_NUMNODES_H
 
-#define MAX_NUMNODES 16
+/* Max 16 Nodes */
+#define NODES_SHIFT	4
 
 #endif /* _ASM_MAX_NUMNODES_H */
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5/include/asm-x86_64/mmzone.h linux-2.6.0-test5-max_numnodes2nodes_shift/include/asm-x86_64/mmzone.h
--- linux-2.6.0-test5/include/asm-x86_64/mmzone.h	Mon Sep  8 12:50:22 2003
+++ linux-2.6.0-test5-max_numnodes2nodes_shift/include/asm-x86_64/mmzone.h	Fri Sep 12 16:37:51 2003
@@ -10,7 +10,6 @@
 
 #define VIRTUAL_BUG_ON(x) 
 
-#include <asm/numnodes.h>
 #include <asm/smp.h>
 
 #define MAXNODE 8 
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5/include/asm-x86_64/numnodes.h linux-2.6.0-test5-max_numnodes2nodes_shift/include/asm-x86_64/numnodes.h
--- linux-2.6.0-test5/include/asm-x86_64/numnodes.h	Mon Sep  8 12:50:03 2003
+++ linux-2.6.0-test5-max_numnodes2nodes_shift/include/asm-x86_64/numnodes.h	Fri Sep 12 16:37:51 2003
@@ -3,10 +3,7 @@
 
 #include <linux/config.h>
 
-#ifdef CONFIG_DISCONTIGMEM
-#define MAX_NUMNODES 8	/* APIC limit currently */
-#else
-#define MAX_NUMNODES 1
-#endif
+/* Max 8 Nodes - APIC limit currently */
+#define NODES_SHIFT	3
 
 #endif
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5/include/linux/mmzone.h linux-2.6.0-test5-max_numnodes2nodes_shift/include/linux/mmzone.h
--- linux-2.6.0-test5/include/linux/mmzone.h	Mon Sep  8 12:50:07 2003
+++ linux-2.6.0-test5-max_numnodes2nodes_shift/include/linux/mmzone.h	Mon Sep 15 13:29:53 2003
@@ -14,9 +14,10 @@
 #ifdef CONFIG_DISCONTIGMEM
 #include <asm/numnodes.h>
 #endif
-#ifndef MAX_NUMNODES
-#define MAX_NUMNODES 1
+#ifndef NODES_SHIFT
+#define NODES_SHIFT	0
 #endif
+#define MAX_NUMNODES	(1 << NODES_SHIFT)
 
 /* Free memory management - zoned buddy allocator.  */
 #ifndef CONFIG_FORCE_MAX_ZONEORDER
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5/mm/slab.c linux-2.6.0-test5-nr_nodes/mm/slab.c
--- linux-2.6.0-test5/mm/slab.c	Mon Sep  8 12:50:21 2003
+++ linux-2.6.0-test5-nr_nodes/mm/slab.c	Fri Sep 12 17:10:01 2003
@@ -249,7 +249,7 @@ struct kmem_cache_s {
 	unsigned int		limit;
 /* 2) touched by every alloc & free from the backend */
 	struct kmem_list3	lists;
-	/* NUMA: kmem_3list_t	*nodelists[NR_NODES] */
+	/* NUMA: kmem_3list_t	*nodelists[MAX_NUMNODES] */
 	unsigned int		objsize;
 	unsigned int	 	flags;	/* constant flags */
 	unsigned int		num;	/* # of objs per slab */

[PATCH[ Clean up MAX_NR_NODES/NUMNODES/etc. [2/5]

[Matthew Dobson]

[-- Attachment #1: Type: text/plain, Size: 1262 bytes --]

Matthew Dobson wrote:
> Ok, I made an attempt to clean up this mess quite a while ago (2.5.47), 
> but that patch is utterly useless now.  At Martin's urging I've created 
> a new series of patches to resolve this.
> 
> 01 - Make sure MAX_NUMNODES is defined in one and only one place. Remove 
> superfluous definitions.  Instead of defining MAX_NUMNODES in 
> asm/numnodes.h, we define NODES_SHIFT there.  Then in linux/mmzone.h we 
> turn that NODES_SHIFT value into MAX_NUMNODES.
> 
> 02 - Remove MAX_NR_NODES.  This value is only used in a couple of 
> places, and it's incorrectly used in all those places as far as I can 
> tell.  Replace with MAX_NUMNODES.  Create MAX_NODES_SHIFT and use this 
> value to check NODES_SHIFT is appropriate.  A possible future patch 
> should make MAX_NODES_SHIFT vary based on 32 vs. 64 bit archs.
> 
> 03 - Fix up the sh arch.  sh defined NR_NODES, change sh to use standard 
> MAX_NUMNODES instead.
> 
> 04 - Fix up the arm arch.  This needs to be reviewed.  Relatively 
> straightforward replacement of NR_NODES with standard MAX_NUMNODES.
> 
> 05 - Fix up the ia64 arch.  This *definitely* needs to be reviewed. This 
> code made my head hurt.  I think I may have gotten it right. Totally 
> untested.

Cheers!

-Matt

[-- Attachment #2: 02-remove-max_nr_nodes.patch --]
[-- Type: text/plain, Size: 3005 bytes --]

diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-max_numnodes2nodes_shift/arch/i386/kernel/smpboot.c linux-2.6.0-test5-remove-max_nr_nodes/arch/i386/kernel/smpboot.c
--- linux-2.6.0-test5-max_numnodes2nodes_shift/arch/i386/kernel/smpboot.c	Mon Sep  8 12:50:03 2003
+++ linux-2.6.0-test5-remove-max_nr_nodes/arch/i386/kernel/smpboot.c	Mon Sep 15 13:44:57 2003
@@ -499,8 +499,8 @@ static struct task_struct * __init fork_
 #ifdef CONFIG_NUMA
 
 /* which logical CPUs are on which nodes */
-cpumask_t node_2_cpu_mask[MAX_NR_NODES] =
-				{ [0 ... MAX_NR_NODES-1] = CPU_MASK_NONE };
+cpumask_t node_2_cpu_mask[MAX_NUMNODES] =
+				{ [0 ... MAX_NUMNODES-1] = CPU_MASK_NONE };
 /* which node each logical CPU is on */
 int cpu_2_node[NR_CPUS] = { [0 ... NR_CPUS-1] = 0 };
 
@@ -518,7 +518,7 @@ static inline void unmap_cpu_to_node(int
 	int node;
 
 	printk("Unmapping cpu %d from all nodes\n", cpu);
-	for (node = 0; node < MAX_NR_NODES; node ++)
+	for (node = 0; node < MAX_NUMNODES; node ++)
 		cpu_clear(cpu, node_2_cpu_mask[node]);
 	cpu_2_node[cpu] = -1;
 }
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-max_numnodes2nodes_shift/include/linux/mmzone.h linux-2.6.0-test5-remove-max_nr_nodes/include/linux/mmzone.h
--- linux-2.6.0-test5-max_numnodes2nodes_shift/include/linux/mmzone.h	Mon Sep 15 13:44:27 2003
+++ linux-2.6.0-test5-remove-max_nr_nodes/include/linux/mmzone.h	Mon Sep 15 13:44:57 2003
@@ -304,19 +304,27 @@ extern void setup_per_zone_pages_min(voi
 #define numa_node_id()		(cpu_to_node(smp_processor_id()))
 
 #ifndef CONFIG_DISCONTIGMEM
+
 extern struct pglist_data contig_page_data;
 #define NODE_DATA(nid)		(&contig_page_data)
 #define NODE_MEM_MAP(nid)	mem_map
-#define MAX_NR_NODES		1
+#define MAX_NODES_SHIFT		0
+
 #else /* CONFIG_DISCONTIGMEM */
 
 #include <asm/mmzone.h>
-
-/* page->zone is currently 8 bits ... */
-#define MAX_NR_NODES		(255 / MAX_NR_ZONES)
+/*
+ * page->zone is currently 8 bits
+ * there are 3 zones (2 bits)
+ * this leaves 8-2=6 bits for nodes
+ */
+#define MAX_NODES_SHIFT		6
 
 #endif /* !CONFIG_DISCONTIGMEM */
 
+#if NODES_SHIFT > MAX_NODES_SHIFT
+#error NODES_SHIFT > MAX_NODES_SHIFT
+#endif
 
 extern DECLARE_BITMAP(node_online_map, MAX_NUMNODES);
 extern DECLARE_BITMAP(memblk_online_map, MAX_NR_MEMBLKS);
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-max_numnodes2nodes_shift/mm/page_alloc.c linux-2.6.0-test5-remove-max_nr_nodes/mm/page_alloc.c
--- linux-2.6.0-test5-max_numnodes2nodes_shift/mm/page_alloc.c	Mon Sep  8 12:49:52 2003
+++ linux-2.6.0-test5-remove-max_nr_nodes/mm/page_alloc.c	Mon Sep 15 13:44:57 2003
@@ -50,7 +50,7 @@ EXPORT_SYMBOL(nr_swap_pages);
  * Used by page_zone() to look up the address of the struct zone whose
  * id is encoded in the upper bits of page->flags
  */
-struct zone *zone_table[MAX_NR_ZONES*MAX_NR_NODES];
+struct zone *zone_table[MAX_NR_ZONES*MAX_NUMNODES];
 EXPORT_SYMBOL(zone_table);
 
 static char *zone_names[MAX_NR_ZONES] = { "DMA", "Normal", "HighMem" };

[PATCH] Clean up MAX_NR_NODES/NUMNODES/etc. [3/5]

[Matthew Dobson]

[-- Attachment #1: Type: text/plain, Size: 1262 bytes --]

Matthew Dobson wrote:
> Ok, I made an attempt to clean up this mess quite a while ago (2.5.47), 
> but that patch is utterly useless now.  At Martin's urging I've created 
> a new series of patches to resolve this.
> 
> 01 - Make sure MAX_NUMNODES is defined in one and only one place. Remove 
> superfluous definitions.  Instead of defining MAX_NUMNODES in 
> asm/numnodes.h, we define NODES_SHIFT there.  Then in linux/mmzone.h we 
> turn that NODES_SHIFT value into MAX_NUMNODES.
> 
> 02 - Remove MAX_NR_NODES.  This value is only used in a couple of 
> places, and it's incorrectly used in all those places as far as I can 
> tell.  Replace with MAX_NUMNODES.  Create MAX_NODES_SHIFT and use this 
> value to check NODES_SHIFT is appropriate.  A possible future patch 
> should make MAX_NODES_SHIFT vary based on 32 vs. 64 bit archs.
> 
> 03 - Fix up the sh arch.  sh defined NR_NODES, change sh to use standard 
> MAX_NUMNODES instead.
> 
> 04 - Fix up the arm arch.  This needs to be reviewed.  Relatively 
> straightforward replacement of NR_NODES with standard MAX_NUMNODES.
> 
> 05 - Fix up the ia64 arch.  This *definitely* needs to be reviewed. This 
> code made my head hurt.  I think I may have gotten it right. Totally 
> untested.

Cheers!

-Matt

[-- Attachment #2: 03-fix-sh.patch --]
[-- Type: text/plain, Size: 2154 bytes --]

diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5/include/asm-sh/numnodes.h linux-2.6.0-test5-max_numnodes2nodes_shift/include/asm-sh/numnodes.h
--- linux-2.6.0-test5/include/asm-sh/numnodes.h	Wed Dec 31 16:00:00 1969
+++ linux-2.6.0-test5-max_numnodes2nodes_shift/include/asm-sh/numnodes.h	Fri Sep 12 17:26:31 2003
@@ -0,0 +1,7 @@
+#ifndef _ASM_MAX_NUMNODES_H
+#define _ASM_MAX_NUMNODES_H
+
+/* Max 2 Nodes */
+#define NODES_SHIFT	1
+
+#endif /* _ASM_MAX_NUMNODES_H */
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5/arch/sh/mm/init.c linux-2.6.0-test5-nr_nodes/arch/sh/mm/init.c
--- linux-2.6.0-test5/arch/sh/mm/init.c	Mon Sep  8 12:50:21 2003
+++ linux-2.6.0-test5-nr_nodes/arch/sh/mm/init.c	Fri Sep 12 17:28:43 2003
@@ -51,8 +51,8 @@ unsigned long mmu_context_cache = NO_CON
 #endif
 
 #ifdef CONFIG_DISCONTIGMEM
-pg_data_t discontig_page_data[NR_NODES];
-bootmem_data_t discontig_node_bdata[NR_NODES];
+pg_data_t discontig_page_data[MAX_NUMNODES];
+bootmem_data_t discontig_node_bdata[MAX_NUMNODES];
 #endif
 
 void show_mem(void)
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5/include/asm-sh/mmzone.h linux-2.6.0-test5-nr_nodes/include/asm-sh/mmzone.h
--- linux-2.6.0-test5/include/asm-sh/mmzone.h	Mon Sep  8 12:50:27 2003
+++ linux-2.6.0-test5-nr_nodes/include/asm-sh/mmzone.h	Fri Sep 12 17:28:08 2003
@@ -10,14 +10,14 @@
 
 #include <linux/config.h>
 
+#ifdef CONFIG_DISCONTIGMEM
+
 /* Currently, just for HP690 */
 #define PHYSADDR_TO_NID(phys)	((((phys) - __MEMORY_START) >= 0x01000000)?1:0)
-#define NR_NODES 2
 
-extern pg_data_t discontig_page_data[NR_NODES];
-extern bootmem_data_t discontig_node_bdata[NR_NODES];
+extern pg_data_t discontig_page_data[MAX_NUMNODES];
+extern bootmem_data_t discontig_node_bdata[MAX_NUMNODES];
 
-#ifdef CONFIG_DISCONTIGMEM
 /*
  * Following are macros that each numa implmentation must define.
  */
@@ -46,7 +46,7 @@ static inline int is_valid_page(struct p
 {
 	unsigned int i;
 
-	for (i = 0; i < NR_NODES; i++) {
+	for (i = 0; i < MAX_NUMNODES; i++) {
 		if (page >= NODE_MEM_MAP(i) &&
 		    page < NODE_MEM_MAP(i) + NODE_DATA(i)->node_size)
 			return 1;

[PATCH] Clean up MAX_NR_NODES/NUMNODES/etc. [4/5]

[Matthew Dobson]

[-- Attachment #1: Type: text/plain, Size: 1262 bytes --]

Matthew Dobson wrote:
> Ok, I made an attempt to clean up this mess quite a while ago (2.5.47), 
> but that patch is utterly useless now.  At Martin's urging I've created 
> a new series of patches to resolve this.
> 
> 01 - Make sure MAX_NUMNODES is defined in one and only one place. Remove 
> superfluous definitions.  Instead of defining MAX_NUMNODES in 
> asm/numnodes.h, we define NODES_SHIFT there.  Then in linux/mmzone.h we 
> turn that NODES_SHIFT value into MAX_NUMNODES.
> 
> 02 - Remove MAX_NR_NODES.  This value is only used in a couple of 
> places, and it's incorrectly used in all those places as far as I can 
> tell.  Replace with MAX_NUMNODES.  Create MAX_NODES_SHIFT and use this 
> value to check NODES_SHIFT is appropriate.  A possible future patch 
> should make MAX_NODES_SHIFT vary based on 32 vs. 64 bit archs.
> 
> 03 - Fix up the sh arch.  sh defined NR_NODES, change sh to use standard 
> MAX_NUMNODES instead.
> 
> 04 - Fix up the arm arch.  This needs to be reviewed.  Relatively 
> straightforward replacement of NR_NODES with standard MAX_NUMNODES.
> 
> 05 - Fix up the ia64 arch.  This *definitely* needs to be reviewed. This 
> code made my head hurt.  I think I may have gotten it right. Totally 
> untested.

Cheers!

-Matt

[-- Attachment #2: 04-fix-arm.patch --]
[-- Type: text/plain, Size: 4983 bytes --]

diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5/arch/arm/mm/discontig.c linux-2.6.0-test5-nr_nodes/arch/arm/mm/discontig.c
--- linux-2.6.0-test5/arch/arm/mm/discontig.c	Mon Sep  8 12:50:22 2003
+++ linux-2.6.0-test5-nr_nodes/arch/arm/mm/discontig.c	Mon Sep 15 11:58:03 2003
@@ -15,7 +15,7 @@
 #include <linux/init.h>
 #include <linux/bootmem.h>
 
-#if NR_NODES != 4
+#if MAX_NUMNODES != 4
 #error Fix Me Please
 #endif
 
@@ -23,9 +23,9 @@
  * Our node_data structure for discontiguous memory.
  */
 
-static bootmem_data_t node_bootmem_data[NR_NODES];
+static bootmem_data_t node_bootmem_data[MAX_NUMNODES];
 
-pg_data_t discontig_node_data[NR_NODES] = {
+pg_data_t discontig_node_data[MAX_NUMNODES] = {
   { .bdata = &node_bootmem_data[0] },
   { .bdata = &node_bootmem_data[1] },
   { .bdata = &node_bootmem_data[2] },
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5/arch/arm/mm/init.c linux-2.6.0-test5-nr_nodes/arch/arm/mm/init.c
--- linux-2.6.0-test5/arch/arm/mm/init.c	Mon Sep  8 12:49:52 2003
+++ linux-2.6.0-test5-nr_nodes/arch/arm/mm/init.c	Mon Sep 15 11:56:36 2003
@@ -33,12 +33,6 @@
 #include <asm/mach/arch.h>
 #include <asm/mach/map.h>
 
-#ifndef CONFIG_DISCONTIGMEM
-#define NR_NODES	1
-#else
-#define NR_NODES	4
-#endif
-
 #ifdef CONFIG_CPU_32
 #define TABLE_OFFSET	(PTRS_PER_PTE)
 #else
@@ -178,7 +172,7 @@ find_memend_and_nodes(struct meminfo *mi
 {
 	unsigned int i, bootmem_pages = 0, memend_pfn = 0;
 
-	for (i = 0; i < NR_NODES; i++) {
+	for (i = 0; i < MAX_NUMNODES; i++) {
 		np[i].start = -1U;
 		np[i].end = 0;
 		np[i].bootmap_pages = 0;
@@ -207,7 +201,7 @@ find_memend_and_nodes(struct meminfo *mi
 			 * we have, we're in trouble.  (maybe we ought to
 			 * limit, instead of bugging?)
 			 */
-			if (numnodes > NR_NODES)
+			if (numnodes > MAX_NUMNODES)
 				BUG();
 		}
 
@@ -365,7 +359,7 @@ static inline void free_bootmem_node_ban
  */
 void __init bootmem_init(struct meminfo *mi)
 {
-	struct node_info node_info[NR_NODES], *np = node_info;
+	struct node_info node_info[MAX_NUMNODES], *np = node_info;
 	unsigned int bootmap_pages, bootmap_pfn, map_pg;
 	int node, initrd_node;
 
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5/include/asm-arm/arch-clps711x/memory.h linux-2.6.0-test5-nr_nodes/include/asm-arm/arch-clps711x/memory.h
--- linux-2.6.0-test5/include/asm-arm/arch-clps711x/memory.h	Mon Sep  8 12:49:51 2003
+++ linux-2.6.0-test5-nr_nodes/include/asm-arm/arch-clps711x/memory.h	Mon Sep 15 13:12:20 2003
@@ -109,8 +109,6 @@
  * 	node 3:  0xd8000000 - 0xdfffffff
  */
 
-#define NR_NODES	4
-
 /*
  * Given a kernel address, find the home node of the underlying memory.
  */
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5/include/asm-arm/arch-sa1100/memory.h linux-2.6.0-test5-nr_nodes/include/asm-arm/arch-sa1100/memory.h
--- linux-2.6.0-test5/include/asm-arm/arch-sa1100/memory.h	Mon Sep  8 12:49:51 2003
+++ linux-2.6.0-test5-nr_nodes/include/asm-arm/arch-sa1100/memory.h	Mon Sep 15 13:12:26 2003
@@ -74,8 +74,6 @@
  * 	node 3:  0xd8000000 - 0xdfffffff
  */
 
-#define NR_NODES	4
-
 /*
  * Given a kernel address, find the home node of the underlying memory.
  */
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5/include/asm-arm/memory.h linux-2.6.0-test5-nr_nodes/include/asm-arm/memory.h
--- linux-2.6.0-test5/include/asm-arm/memory.h	Mon Sep  8 12:50:28 2003
+++ linux-2.6.0-test5-nr_nodes/include/asm-arm/memory.h	Mon Sep 15 13:19:21 2003
@@ -89,6 +89,9 @@ static inline void *phys_to_virt(unsigne
  * This is more complex.  We have a set of mem_map arrays spread
  * around in memory.
  */
+#include <asm/numnodes.h>
+#define NUM_NODES	(1 << NODES_SHIFT)
+
 #define page_to_pfn(page)					\
 	(( (page) - page_zone(page)->zone_mem_map)		\
 	  + page_zone(page)->zone_start_pfn)
@@ -96,12 +99,12 @@ static inline void *phys_to_virt(unsigne
 #define pfn_to_page(pfn)					\
 	(PFN_TO_MAPBASE(pfn) + LOCAL_MAP_NR((pfn) << PAGE_SHIFT))
 
-#define pfn_valid(pfn)		(PFN_TO_NID(pfn) < NR_NODES)
+#define pfn_valid(pfn)		(PFN_TO_NID(pfn) < NUM_NODES)
 
 #define virt_to_page(kaddr)					\
 	(ADDR_TO_MAPBASE(kaddr) + LOCAL_MAP_NR(kaddr))
 
-#define virt_addr_valid(kaddr)	(KVADDR_TO_NID(kaddr) < NR_NODES)
+#define virt_addr_valid(kaddr)	(KVADDR_TO_NID(kaddr) < NUM_NODES)
 
 /*
  * Common discontigmem stuff.
@@ -109,6 +112,8 @@ static inline void *phys_to_virt(unsigne
  */
 #define PHYS_TO_NID(addr)	PFN_TO_NID((addr) >> PAGE_SHIFT)
 
+#undef NUM_NODES
+
 #endif
 
 /*
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5/include/asm-arm/numnodes.h linux-2.6.0-test5-nr_nodes/include/asm-arm/numnodes.h
--- linux-2.6.0-test5/include/asm-arm/numnodes.h	Mon Sep  8 12:50:02 2003
+++ linux-2.6.0-test5-nr_nodes/include/asm-arm/numnodes.h	Mon Sep 15 13:19:46 2003
@@ -10,8 +10,7 @@
 #ifndef __ASM_ARM_NUMNODES_H
 #define __ASM_ARM_NUMNODES_H
 
-#include <asm/memory.h>
-
-#define MAX_NUMNODES	NR_NODES
+/* Max 4 Nodes */
+#define NODES_SHIFT	2
 
 #endif

[PATCH] Clean up MAX_NR_NODES/NUMNODES/etc. [5/5]

[Matthew Dobson]

[-- Attachment #1: Type: text/plain, Size: 1262 bytes --]

Matthew Dobson wrote:
> Ok, I made an attempt to clean up this mess quite a while ago (2.5.47), 
> but that patch is utterly useless now.  At Martin's urging I've created 
> a new series of patches to resolve this.
> 
> 01 - Make sure MAX_NUMNODES is defined in one and only one place. Remove 
> superfluous definitions.  Instead of defining MAX_NUMNODES in 
> asm/numnodes.h, we define NODES_SHIFT there.  Then in linux/mmzone.h we 
> turn that NODES_SHIFT value into MAX_NUMNODES.
> 
> 02 - Remove MAX_NR_NODES.  This value is only used in a couple of 
> places, and it's incorrectly used in all those places as far as I can 
> tell.  Replace with MAX_NUMNODES.  Create MAX_NODES_SHIFT and use this 
> value to check NODES_SHIFT is appropriate.  A possible future patch 
> should make MAX_NODES_SHIFT vary based on 32 vs. 64 bit archs.
> 
> 03 - Fix up the sh arch.  sh defined NR_NODES, change sh to use standard 
> MAX_NUMNODES instead.
> 
> 04 - Fix up the arm arch.  This needs to be reviewed.  Relatively 
> straightforward replacement of NR_NODES with standard MAX_NUMNODES.
> 
> 05 - Fix up the ia64 arch.  This *definitely* needs to be reviewed. This 
> code made my head hurt.  I think I may have gotten it right. Totally 
> untested.

Cheers!

-Matt

[-- Attachment #2: 05-fix-ia64.patch --]
[-- Type: text/plain, Size: 9672 bytes --]

diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-arm/arch/ia64/kernel/acpi.c linux-2.6.0-test5-ia64/arch/ia64/kernel/acpi.c
--- linux-2.6.0-test5-arm/arch/ia64/kernel/acpi.c	Mon Sep  8 12:50:04 2003
+++ linux-2.6.0-test5-ia64/arch/ia64/kernel/acpi.c	Mon Sep 15 14:25:47 2003
@@ -41,6 +41,7 @@
 #include <linux/irq.h>
 #include <linux/acpi.h>
 #include <linux/efi.h>
+#include <linux/mmzone.h>
 #include <asm/io.h>
 #include <asm/iosapic.h>
 #include <asm/machvec.h>
@@ -341,7 +342,7 @@ static u32 __initdata pxm_flag[PXM_FLAG_
 #define pxm_bit_test(bit)	(test_bit(bit,(void *)pxm_flag))
 /* maps to convert between proximity domain and logical node ID */
 int __initdata pxm_to_nid_map[MAX_PXM_DOMAINS];
-int __initdata nid_to_pxm_map[NR_NODES];
+int __initdata nid_to_pxm_map[MAX_NUMNODES];
 static struct acpi_table_slit __initdata *slit_table;
 
 /*
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-arm/arch/ia64/mm/discontig.c linux-2.6.0-test5-ia64/arch/ia64/mm/discontig.c
--- linux-2.6.0-test5-arm/arch/ia64/mm/discontig.c	Mon Sep  8 12:50:22 2003
+++ linux-2.6.0-test5-ia64/arch/ia64/mm/discontig.c	Mon Sep 15 14:26:52 2003
@@ -13,7 +13,6 @@
 #include <linux/kernel.h>
 #include <linux/mm.h>
 #include <linux/bootmem.h>
-#include <linux/mmzone.h>
 #include <linux/acpi.h>
 #include <linux/efi.h>
 
@@ -23,10 +22,10 @@
  */
 #define GRANULEROUNDUP(n) (((n)+IA64_GRANULE_SIZE-1) & ~(IA64_GRANULE_SIZE-1))
 
-static struct ia64_node_data	*node_data[NR_NODES];
-static long			boot_pg_data[8*NR_NODES+sizeof(pg_data_t)]  __initdata;
-static pg_data_t		*pg_data_ptr[NR_NODES] __initdata;
-static bootmem_data_t		bdata[NR_NODES][NR_BANKS_PER_NODE+1] __initdata;
+static struct ia64_node_data	*node_data[MAX_NUMNODES];
+static long			boot_pg_data[8*MAX_NUMNODES+sizeof(pg_data_t)]  __initdata;
+static pg_data_t		*pg_data_ptr[MAX_NUMNODES] __initdata;
+static bootmem_data_t		bdata[MAX_NUMNODES][NR_BANKS_PER_NODE+1] __initdata;
 
 extern int  filter_rsvd_memory (unsigned long start, unsigned long end, void *arg);
 
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-arm/arch/ia64/mm/numa.c linux-2.6.0-test5-ia64/arch/ia64/mm/numa.c
--- linux-2.6.0-test5-arm/arch/ia64/mm/numa.c	Mon Sep  8 12:49:58 2003
+++ linux-2.6.0-test5-ia64/arch/ia64/mm/numa.c	Mon Sep 15 14:27:39 2003
@@ -15,7 +15,6 @@
 #include <linux/mm.h>
 #include <linux/init.h>
 #include <linux/bootmem.h>
-#include <linux/mmzone.h>
 #include <asm/numa.h>
 
 /*
@@ -29,7 +28,7 @@ struct node_cpuid_s node_cpuid[NR_CPUS];
  * This is a matrix with "distances" between nodes, they should be
  * proportional to the memory access latency ratios.
  */
-u8 numa_slit[NR_NODES * NR_NODES];
+u8 numa_slit[MAX_NUMNODES * MAX_NUMNODES];
 
 /* Identify which cnode a physical address resides on */
 int
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-arm/include/asm-ia64/acpi.h linux-2.6.0-test5-ia64/include/asm-ia64/acpi.h
--- linux-2.6.0-test5-arm/include/asm-ia64/acpi.h	Mon Sep  8 12:50:23 2003
+++ linux-2.6.0-test5-ia64/include/asm-ia64/acpi.h	Mon Sep 15 16:09:52 2003
@@ -109,7 +109,7 @@ int acpi_get_addr_space (void *obj, u8 t
 /* Proximity bitmap length; _PXM is at most 255 (8 bit)*/
 #define MAX_PXM_DOMAINS (256)
 extern int __initdata pxm_to_nid_map[MAX_PXM_DOMAINS];
-extern int __initdata nid_to_pxm_map[NR_NODES];
+extern int __initdata nid_to_pxm_map[MAX_NUMNODES];
 #endif
 
 #endif /*__KERNEL__*/
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-arm/include/asm-ia64/mmzone.h linux-2.6.0-test5-ia64/include/asm-ia64/mmzone.h
--- linux-2.6.0-test5-arm/include/asm-ia64/mmzone.h	Mon Sep  8 12:50:59 2003
+++ linux-2.6.0-test5-ia64/include/asm-ia64/mmzone.h	Mon Sep 15 16:24:33 2003
@@ -92,14 +92,12 @@
 extern unsigned long max_low_pfn;
 
 
-#ifdef CONFIG_IA64_DIG
+#if defined(CONFIG_IA64_DIG)
 
 /*
  * Platform definitions for DIG platform with contiguous memory.
  */
-#define MAX_PHYSNODE_ID	8	/* Maximum node number +1 */
-#define NR_NODES	8	/* Maximum number of nodes in SSI */
-
+#define MAX_PHYSNODE_ID	8		/* Maximum node number +1 */
 #define MAX_PHYS_MEMORY	(1UL << 40)	/* 1 TB */
 
 /*
@@ -119,37 +117,34 @@ extern unsigned long max_low_pfn;
 # error Unsupported bank and nodesize!
 #endif
 #define BANKSIZE		(1UL << BANKSHIFT)
-#define BANK_OFFSET(addr)	((unsigned long)(addr) & (BANKSIZE-1))
-#define NR_BANKS		(NR_BANKS_PER_NODE * NR_NODES)
-
-/*
- * VALID_MEM_KADDR returns a boolean to indicate if a kaddr is
- * potentially a valid cacheable identity mapped RAM memory address.
- * Note that the RAM may or may not actually be present!!
- */
-#define VALID_MEM_KADDR(kaddr)	1
-
-/*
- * Given a nodeid & a bank number, find the address of the mem_map
- * entry for the first page of the bank.
- */
-#define BANK_MEM_MAP_INDEX(kaddr) \
-	(((unsigned long)(kaddr) & (MAX_PHYS_MEMORY-1)) >> BANKSHIFT)
 
 #elif defined(CONFIG_IA64_SGI_SN2)
+
 /*
  * SGI SN2 discontig definitions
  */
 #define MAX_PHYSNODE_ID	2048	/* 2048 node ids (also called nasid) */
-#define NR_NODES	128	/* Maximum number of nodes in SSI */
 #define MAX_PHYS_MEMORY	(1UL << 49)
 
-#define BANKSHIFT		38
 #define NR_BANKS_PER_NODE	4
+#define BANKSHIFT		38
 #define SN2_NODE_SIZE		(64UL*1024*1024*1024)	/* 64GB per node */
 #define BANKSIZE		(SN2_NODE_SIZE/NR_BANKS_PER_NODE)
+
+#endif /* CONFIG_IA64_DIG */
+
+#if defined(CONFIG_IA64_DIG) || defined (CONFIG_IA64_SGI_SN2)
+/* Common defines for both platforms */
+#include <asm/numnodes.h>
 #define BANK_OFFSET(addr)	((unsigned long)(addr) & (BANKSIZE-1))
-#define NR_BANKS		(NR_BANKS_PER_NODE * NR_NODES)
+#define NR_BANKS		(NR_BANKS_PER_NODE * (1 << NODES_SHIFT))
+#define NR_MEMBLKS		(NR_BANKS)
+
+/*
+ * VALID_MEM_KADDR returns a boolean to indicate if a kaddr is
+ * potentially a valid cacheable identity mapped RAM memory address.
+ * Note that the RAM may or may not actually be present!!
+ */
 #define VALID_MEM_KADDR(kaddr)	1
 
 /*
@@ -159,5 +154,6 @@ extern unsigned long max_low_pfn;
 #define BANK_MEM_MAP_INDEX(kaddr) \
 	(((unsigned long)(kaddr) & (MAX_PHYS_MEMORY-1)) >> BANKSHIFT)
 
-#endif /* CONFIG_IA64_DIG */
+#endif /* CONFIG_IA64_DIG || CONFIG_IA64_SGI_SN2 */
+
 #endif /* _ASM_IA64_MMZONE_H */
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-arm/include/asm-ia64/nodedata.h linux-2.6.0-test5-ia64/include/asm-ia64/nodedata.h
--- linux-2.6.0-test5-arm/include/asm-ia64/nodedata.h	Mon Sep  8 12:50:18 2003
+++ linux-2.6.0-test5-ia64/include/asm-ia64/nodedata.h	Mon Sep 15 16:07:59 2003
@@ -14,7 +14,7 @@
 #define _ASM_IA64_NODEDATA_H
 
 
-#include <asm/mmzone.h>
+#include <linux/mmzone.h>
 
 /*
  * Node Data. One of these structures is located on each node of a NUMA system.
@@ -24,9 +24,9 @@ struct pglist_data;
 struct ia64_node_data {
 	short			active_cpu_count;
 	short			node;
-        struct pglist_data	*pg_data_ptrs[NR_NODES];
+        struct pglist_data	*pg_data_ptrs[MAX_NUMNODES];
 	struct page		*bank_mem_map_base[NR_BANKS];
-	struct ia64_node_data	*node_data_ptrs[NR_NODES];
+	struct ia64_node_data	*node_data_ptrs[MAX_NUMNODES];
 	short			node_id_map[NR_BANKS];
 };
 
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-arm/include/asm-ia64/numa.h linux-2.6.0-test5-ia64/include/asm-ia64/numa.h
--- linux-2.6.0-test5-arm/include/asm-ia64/numa.h	Mon Sep  8 12:50:01 2003
+++ linux-2.6.0-test5-ia64/include/asm-ia64/numa.h	Mon Sep 15 16:07:24 2003
@@ -13,17 +13,11 @@
 
 #ifdef CONFIG_NUMA
 
-#ifdef CONFIG_DISCONTIGMEM
-# include <asm/mmzone.h>
-# define NR_MEMBLKS   (NR_BANKS)
-#else
-# define NR_NODES     (8)
-# define NR_MEMBLKS   (NR_NODES * 8)
-#endif
+#include <linux/mmzone.h>
 
 #include <linux/cache.h>
 extern volatile char cpu_to_node_map[NR_CPUS] __cacheline_aligned;
-extern volatile unsigned long node_to_cpu_mask[NR_NODES] __cacheline_aligned;
+extern volatile unsigned long node_to_cpu_mask[MAX_NUMNODES] __cacheline_aligned;
 
 /* Stuff below this line could be architecture independent */
 
@@ -57,7 +51,7 @@ extern struct node_cpuid_s node_cpuid[NR
  * proportional to the memory access latency ratios.
  */
 
-extern u8 numa_slit[NR_NODES * NR_NODES];
+extern u8 numa_slit[MAX_NUMNODES * MAX_NUMNODES];
 #define node_distance(from,to) (numa_slit[from * numnodes + to])
 
 extern int paddr_to_nid(unsigned long paddr);
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-arm/include/asm-ia64/numnodes.h linux-2.6.0-test5-ia64/include/asm-ia64/numnodes.h
--- linux-2.6.0-test5-arm/include/asm-ia64/numnodes.h	Mon Sep  8 12:50:58 2003
+++ linux-2.6.0-test5-ia64/include/asm-ia64/numnodes.h	Mon Sep 15 14:37:20 2003
@@ -1,7 +1,12 @@
 #ifndef _ASM_MAX_NUMNODES_H
 #define _ASM_MAX_NUMNODES_H
 
-#include <asm/mmzone.h>
-#define MAX_NUMNODES	NR_NODES
+#ifdef CONFIG_IA64_DIG
+/* Max 8 Nodes */
+#define NODES_SHIFT	3
+#elif defined(CONFIG_IA64_SGI_SN2)
+/* Max 128 Nodes */
+#define NODES_SHIFT	7
+#endif
 
 #endif /* _ASM_MAX_NUMNODES_H */
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-arm/include/asm-ia64/sn/pda.h linux-2.6.0-test5-ia64/include/asm-ia64/sn/pda.h
--- linux-2.6.0-test5-arm/include/asm-ia64/sn/pda.h	Mon Sep  8 12:50:28 2003
+++ linux-2.6.0-test5-ia64/include/asm-ia64/sn/pda.h	Mon Sep 15 16:10:47 2003
@@ -10,6 +10,7 @@
 
 #include <linux/config.h>
 #include <linux/cache.h>
+#include <linux/mmzone.h>
 #include <asm/percpu.h>
 #include <asm/system.h>
 #include <asm/processor.h>
@@ -56,7 +57,7 @@ typedef struct pda_s {
 
 	unsigned long	sn_soft_irr[4];
 	unsigned long	sn_in_service_ivecs[4];
-	short		cnodeid_to_nasid_table[NR_NODES];	
+	short		cnodeid_to_nasid_table[MAX_NUMNODES];	
 	int		sn_lb_int_war_ticks;
 	int		sn_last_irq;
 	int		sn_first_irq;

Re: [PATCH] Clean up MAX_NR_NODES/NUMNODES/etc. [1/5]

[Matthew Dobson]

[-- Attachment #1: Type: text/plain, Size: 151 bytes --]

Here's a version rediffed against test5-mm2.  The other patches, except 
for a little fuzz on 5/5, apply fine to either test5 or -mm2.

Cheers!

-Matt

[-- Attachment #2: 01-max_numnodes2nodes_shift-mm2.patch --]
[-- Type: text/plain, Size: 4704 bytes --]

diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-mm2/include/asm-alpha/numnodes.h linux-2.6.0-test5-01/include/asm-alpha/numnodes.h
--- linux-2.6.0-test5-mm2/include/asm-alpha/numnodes.h	Mon Sep  8 12:49:53 2003
+++ linux-2.6.0-test5-01/include/asm-alpha/numnodes.h	Tue Sep 16 05:19:42 2003
@@ -1,6 +1,7 @@
 #ifndef _ASM_MAX_NUMNODES_H
 #define _ASM_MAX_NUMNODES_H
 
-#define MAX_NUMNODES		128 /* Marvel */
+/* Max 128 Nodes - Marvel */
+#define NODES_SHIFT	7
 
 #endif /* _ASM_MAX_NUMNODES_H */
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-mm2/include/asm-i386/numaq.h linux-2.6.0-test5-01/include/asm-i386/numaq.h
--- linux-2.6.0-test5-mm2/include/asm-i386/numaq.h	Tue Sep 16 05:18:10 2003
+++ linux-2.6.0-test5-01/include/asm-i386/numaq.h	Tue Sep 16 05:20:47 2003
@@ -28,7 +28,6 @@
 
 #ifdef CONFIG_X86_NUMAQ
 
-#define MAX_NUMNODES		16
 extern int get_memcfg_numaq(void);
 
 /*
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-mm2/include/asm-i386/numnodes.h linux-2.6.0-test5-01/include/asm-i386/numnodes.h
--- linux-2.6.0-test5-mm2/include/asm-i386/numnodes.h	Tue Sep 16 05:18:10 2003
+++ linux-2.6.0-test5-01/include/asm-i386/numnodes.h	Tue Sep 16 05:22:21 2003
@@ -4,11 +4,15 @@
 #include <linux/config.h>
 
 #ifdef CONFIG_X86_NUMAQ
-#include <asm/numaq.h>
+
+/* Max 16 Nodes */
+#define NODES_SHIFT	4
+
 #elif CONFIG_ACPI_SRAT
-#include <asm/srat.h>
-#else
-#define MAX_NUMNODES	1
+
+/* Max 8 Nodes */
+#define NODES_SHIFT	3
+
 #endif /* CONFIG_X86_NUMAQ */
 
 #endif /* _ASM_MAX_NUMNODES_H */
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-mm2/include/asm-i386/srat.h linux-2.6.0-test5-01/include/asm-i386/srat.h
--- linux-2.6.0-test5-mm2/include/asm-i386/srat.h	Tue Sep 16 05:18:11 2003
+++ linux-2.6.0-test5-01/include/asm-i386/srat.h	Tue Sep 16 05:22:59 2003
@@ -31,7 +31,6 @@
 #error CONFIG_ACPI_SRAT not defined, and srat.h header has been included
 #endif
 
-#define MAX_NUMNODES		8
 extern int get_memcfg_from_srat(void);
 extern unsigned long *get_zholes_size(int);
 
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-mm2/include/asm-ppc64/numnodes.h linux-2.6.0-test5-01/include/asm-ppc64/numnodes.h
--- linux-2.6.0-test5-mm2/include/asm-ppc64/numnodes.h	Mon Sep  8 12:50:22 2003
+++ linux-2.6.0-test5-01/include/asm-ppc64/numnodes.h	Tue Sep 16 05:19:42 2003
@@ -1,6 +1,7 @@
 #ifndef _ASM_MAX_NUMNODES_H
 #define _ASM_MAX_NUMNODES_H
 
-#define MAX_NUMNODES 16
+/* Max 16 Nodes */
+#define NODES_SHIFT	4
 
 #endif /* _ASM_MAX_NUMNODES_H */
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-mm2/include/asm-x86_64/mmzone.h linux-2.6.0-test5-01/include/asm-x86_64/mmzone.h
--- linux-2.6.0-test5-mm2/include/asm-x86_64/mmzone.h	Mon Sep  8 12:50:22 2003
+++ linux-2.6.0-test5-01/include/asm-x86_64/mmzone.h	Tue Sep 16 05:19:42 2003
@@ -10,7 +10,6 @@
 
 #define VIRTUAL_BUG_ON(x) 
 
-#include <asm/numnodes.h>
 #include <asm/smp.h>
 
 #define MAXNODE 8 
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-mm2/include/asm-x86_64/numnodes.h linux-2.6.0-test5-01/include/asm-x86_64/numnodes.h
--- linux-2.6.0-test5-mm2/include/asm-x86_64/numnodes.h	Mon Sep  8 12:50:03 2003
+++ linux-2.6.0-test5-01/include/asm-x86_64/numnodes.h	Tue Sep 16 05:19:42 2003
@@ -3,10 +3,7 @@
 
 #include <linux/config.h>
 
-#ifdef CONFIG_DISCONTIGMEM
-#define MAX_NUMNODES 8	/* APIC limit currently */
-#else
-#define MAX_NUMNODES 1
-#endif
+/* Max 8 Nodes - APIC limit currently */
+#define NODES_SHIFT	3
 
 #endif
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-mm2/include/linux/mmzone.h linux-2.6.0-test5-01/include/linux/mmzone.h
--- linux-2.6.0-test5-mm2/include/linux/mmzone.h	Mon Sep  8 12:50:07 2003
+++ linux-2.6.0-test5-01/include/linux/mmzone.h	Tue Sep 16 05:19:42 2003
@@ -14,9 +14,10 @@
 #ifdef CONFIG_DISCONTIGMEM
 #include <asm/numnodes.h>
 #endif
-#ifndef MAX_NUMNODES
-#define MAX_NUMNODES 1
+#ifndef NODES_SHIFT
+#define NODES_SHIFT	0
 #endif
+#define MAX_NUMNODES	(1 << NODES_SHIFT)
 
 /* Free memory management - zoned buddy allocator.  */
 #ifndef CONFIG_FORCE_MAX_ZONEORDER
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-mm2/mm/slab.c linux-2.6.0-test5-01/mm/slab.c
--- linux-2.6.0-test5-mm2/mm/slab.c	Tue Sep 16 05:18:13 2003
+++ linux-2.6.0-test5-01/mm/slab.c	Tue Sep 16 05:19:42 2003
@@ -250,7 +250,7 @@ struct kmem_cache_s {
 	unsigned int		limit;
 /* 2) touched by every alloc & free from the backend */
 	struct kmem_list3	lists;
-	/* NUMA: kmem_3list_t	*nodelists[NR_NODES] */
+	/* NUMA: kmem_3list_t	*nodelists[MAX_NUMNODES] */
 	unsigned int		objsize;
 	unsigned int	 	flags;	/* constant flags */
 	unsigned int		num;	/* # of objs per slab */

Re: [PATCH] Clean up MAX_NR_NODES/NUMNODES/etc. [5/5]

[Jesse Barnes]

On Mon, Sep 15, 2003 at 05:37:47PM -0700, Matthew Dobson wrote:
> Matthew Dobson wrote:
> >Ok, I made an attempt to clean up this mess quite a while ago (2.5.47), 
> >but that patch is utterly useless now.  At Martin's urging I've created 
> >a new series of patches to resolve this.
> >
> >01 - Make sure MAX_NUMNODES is defined in one and only one place. Remove 
> >superfluous definitions.  Instead of defining MAX_NUMNODES in 
> >asm/numnodes.h, we define NODES_SHIFT there.  Then in linux/mmzone.h we 
> >turn that NODES_SHIFT value into MAX_NUMNODES.
> >
> >02 - Remove MAX_NR_NODES.  This value is only used in a couple of 
> >places, and it's incorrectly used in all those places as far as I can 
> >tell.  Replace with MAX_NUMNODES.  Create MAX_NODES_SHIFT and use this 
> >value to check NODES_SHIFT is appropriate.  A possible future patch 
> >should make MAX_NODES_SHIFT vary based on 32 vs. 64 bit archs.
> >
> >03 - Fix up the sh arch.  sh defined NR_NODES, change sh to use standard 
> >MAX_NUMNODES instead.
> >
> >04 - Fix up the arm arch.  This needs to be reviewed.  Relatively 
> >straightforward replacement of NR_NODES with standard MAX_NUMNODES.
> >
> >05 - Fix up the ia64 arch.  This *definitely* needs to be reviewed. This 
> >code made my head hurt.  I think I may have gotten it right. Totally 
> >untested.

Can you send me a patch that contains everything (or just the generic
code plus the ia64 stuff)?  The stuff you posted looks good, and I'd
like to test ia64, but I have to merge your patch into the latest
discontig stuff I've been working on to do so.

Thanks,
Jesse

Re: [PATCH] you have how many nodes??

[Matthew Dobson]

[-- Attachment #1: Type: text/plain, Size: 1326 bytes --]

Andrew,
	Here's a small update to the numnodes fix that went into -mm3.  The 
biggest changes are:
	1) move the actual NODES_SHIFT and MAX_NUMNODES definitions into 
linux/numa.h and include this in linux/mmzone.h, instead of being 
directly in linux/mmzone.h.  This allows other files to include *just* 
the NUMNODES stuff w/out grabbing all of mmzone.h.
	2) pull NODE_SHIFT out of linux/mm.h.  This isn't used anywhere in the 
kernel, and it will only get confused with NODES_SHIFT.
	3) Fix the IA64 patch.  The original patch I had sent out hadn't been 
tested on IA64.  It was mostly right, but there were circular 
dependencies.  All better now, and acked by Jesse.
	4) In linux/mmzone.h, insert code to define MAX_NODES_SHIFT based on 
the size of unsigned long.  For 64-bit arches, we can have a much larger 
value.  This allows IA64 to have 100's or 1000's of nodes. 
MAX_NODES_SHIFT is defined as 10 (ie: 1024 nodes) for 64-bit for now, 
although it could likely be much larger.  For 32-bit it is 6 (ie: 64 nodes).
	5) Small cleanup in include/asm-arm/memory.h.  Mostly the result of the 
new linux/numa.h file.  Much cleaner and more readable now.

Russell, if you get a chance, I'd really appreciate a sanity check on 
the arm code.  It really hasn't been tested, but the changes are pretty 
small.

Cheers!

-Matt

[-- Attachment #2: numnodes_update.patch --]
[-- Type: text/plain, Size: 6117 bytes --]

diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-mm3/include/asm-arm/memory.h linux-2.6.0-test5-numnodes_update/include/asm-arm/memory.h
--- linux-2.6.0-test5-mm3/include/asm-arm/memory.h	Fri Sep 19 14:05:33 2003
+++ linux-2.6.0-test5-numnodes_update/include/asm-arm/memory.h	Fri Sep 19 14:52:11 2003
@@ -84,27 +84,24 @@ static inline void *phys_to_virt(unsigne
 
 #define PHYS_TO_NID(addr)	(0)
 
-#else
+#else /* CONFIG_DISCONTIGMEM */
+
 /*
  * This is more complex.  We have a set of mem_map arrays spread
  * around in memory.
  */
-#include <asm/numnodes.h>
-#define NUM_NODES	(1 << NODES_SHIFT)
+#include <linux/numa.h>
 
 #define page_to_pfn(page)					\
 	(( (page) - page_zone(page)->zone_mem_map)		\
 	  + page_zone(page)->zone_start_pfn)
-
 #define pfn_to_page(pfn)					\
 	(PFN_TO_MAPBASE(pfn) + LOCAL_MAP_NR((pfn) << PAGE_SHIFT))
-
-#define pfn_valid(pfn)		(PFN_TO_NID(pfn) < NUM_NODES)
+#define pfn_valid(pfn)		(PFN_TO_NID(pfn) < MAX_NUMNODES)
 
 #define virt_to_page(kaddr)					\
 	(ADDR_TO_MAPBASE(kaddr) + LOCAL_MAP_NR(kaddr))
-
-#define virt_addr_valid(kaddr)	(KVADDR_TO_NID(kaddr) < NUM_NODES)
+#define virt_addr_valid(kaddr)	(KVADDR_TO_NID(kaddr) < MAX_NUMNODES)
 
 /*
  * Common discontigmem stuff.
@@ -112,9 +109,7 @@ static inline void *phys_to_virt(unsigne
  */
 #define PHYS_TO_NID(addr)	PFN_TO_NID((addr) >> PAGE_SHIFT)
 
-#undef NUM_NODES
-
-#endif
+#endif /* !CONFIG_DISCONTIGMEM */
 
 /*
  * For BIO.  "will die".  Kill me when bio_to_phys() and bvec_to_phys() die.
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-mm3/include/asm-ia64/nodedata.h linux-2.6.0-test5-numnodes_update/include/asm-ia64/nodedata.h
--- linux-2.6.0-test5-mm3/include/asm-ia64/nodedata.h	Fri Sep 19 14:05:33 2003
+++ linux-2.6.0-test5-numnodes_update/include/asm-ia64/nodedata.h	Fri Sep 19 14:52:13 2003
@@ -8,13 +8,11 @@
  * Copyright (c) 2002 Erich Focht <efocht@ess.nec.de>
  * Copyright (c) 2002 Kimio Suganuma <k-suganuma@da.jp.nec.com>
  */
-
-
 #ifndef _ASM_IA64_NODEDATA_H
 #define _ASM_IA64_NODEDATA_H
 
-
-#include <linux/mmzone.h>
+#include <linux/numa.h>
+#include <asm/mmzone.h>
 
 /*
  * Node Data. One of these structures is located on each node of a NUMA system.
@@ -24,7 +22,7 @@ struct pglist_data;
 struct ia64_node_data {
 	short			active_cpu_count;
 	short			node;
-        struct pglist_data	*pg_data_ptrs[MAX_NUMNODES];
+	struct pglist_data	*pg_data_ptrs[MAX_NUMNODES];
 	struct page		*bank_mem_map_base[NR_BANKS];
 	struct ia64_node_data	*node_data_ptrs[MAX_NUMNODES];
 	short			node_id_map[NR_BANKS];
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-mm3/include/asm-ia64/numa.h linux-2.6.0-test5-numnodes_update/include/asm-ia64/numa.h
--- linux-2.6.0-test5-mm3/include/asm-ia64/numa.h	Fri Sep 19 14:05:33 2003
+++ linux-2.6.0-test5-numnodes_update/include/asm-ia64/numa.h	Fri Sep 19 14:52:13 2003
@@ -13,9 +13,9 @@
 
 #ifdef CONFIG_NUMA
 
-#include <linux/mmzone.h>
-
+#include <linux/numa.h>
 #include <linux/cache.h>
+
 extern volatile char cpu_to_node_map[NR_CPUS] __cacheline_aligned;
 extern volatile unsigned long node_to_cpu_mask[MAX_NUMNODES] __cacheline_aligned;
 
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-mm3/include/asm-ia64/sn/pda.h linux-2.6.0-test5-numnodes_update/include/asm-ia64/sn/pda.h
--- linux-2.6.0-test5-mm3/include/asm-ia64/sn/pda.h	Fri Sep 19 14:05:33 2003
+++ linux-2.6.0-test5-numnodes_update/include/asm-ia64/sn/pda.h	Fri Sep 19 14:52:13 2003
@@ -10,7 +10,7 @@
 
 #include <linux/config.h>
 #include <linux/cache.h>
-#include <linux/mmzone.h>
+#include <linux/numa.h>
 #include <asm/percpu.h>
 #include <asm/system.h>
 #include <asm/processor.h>
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-mm3/include/linux/mm.h linux-2.6.0-test5-numnodes_update/include/linux/mm.h
--- linux-2.6.0-test5-mm3/include/linux/mm.h	Fri Sep 19 14:05:37 2003
+++ linux-2.6.0-test5-numnodes_update/include/linux/mm.h	Fri Sep 19 14:52:04 2003
@@ -323,7 +323,6 @@ static inline void put_page(struct page 
  * The zone field is never updated after free_area_init_core()
  * sets it, so none of the operations on it need to be atomic.
  */
-#define NODE_SHIFT 4
 #define ZONE_SHIFT (BITS_PER_LONG - 8)
 
 struct zone;
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-mm3/include/linux/mmzone.h linux-2.6.0-test5-numnodes_update/include/linux/mmzone.h
--- linux-2.6.0-test5-mm3/include/linux/mmzone.h	Fri Sep 19 14:05:37 2003
+++ linux-2.6.0-test5-numnodes_update/include/linux/mmzone.h	Fri Sep 19 14:52:06 2003
@@ -10,14 +10,8 @@
 #include <linux/wait.h>
 #include <linux/cache.h>
 #include <linux/threads.h>
+#include <linux/numa.h>
 #include <asm/atomic.h>
-#ifdef CONFIG_DISCONTIGMEM
-#include <asm/numnodes.h>
-#endif
-#ifndef NODES_SHIFT
-#define NODES_SHIFT	0
-#endif
-#define MAX_NUMNODES	(1 << NODES_SHIFT)
 
 /* Free memory management - zoned buddy allocator.  */
 #ifndef CONFIG_FORCE_MAX_ZONEORDER
@@ -313,12 +307,19 @@ extern struct pglist_data contig_page_da
 #else /* CONFIG_DISCONTIGMEM */
 
 #include <asm/mmzone.h>
+
+#if BITS_PER_LONG == 32
 /*
- * page->zone is currently 8 bits
- * there are 3 zones (2 bits)
- * this leaves 8-2=6 bits for nodes
+ * with 32 bit flags field, page->zone is currently 8 bits.
+ * there are 3 zones (2 bits) and this leaves 8-2=6 bits for nodes.
  */
 #define MAX_NODES_SHIFT		6
+#elif BITS_PER_LONG == 64
+/*
+ * with 64 bit flags field, there's plenty of room.
+ */
+#define MAX_NODES_SHIFT		10
+#endif
 
 #endif /* !CONFIG_DISCONTIGMEM */
 
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.0-test5-mm3/include/linux/numa.h linux-2.6.0-test5-numnodes_update/include/linux/numa.h
--- linux-2.6.0-test5-mm3/include/linux/numa.h	Wed Dec 31 16:00:00 1969
+++ linux-2.6.0-test5-numnodes_update/include/linux/numa.h	Fri Sep 19 14:52:04 2003
@@ -0,0 +1,16 @@
+#ifndef _LINUX_NUMA_H
+#define _LINUX_NUMA_H
+
+#include <linux/config.h>
+
+#ifdef CONFIG_DISCONTIGMEM
+#include <asm/numnodes.h>
+#endif
+
+#ifndef NODES_SHIFT
+#define NODES_SHIFT     0
+#endif
+
+#define MAX_NUMNODES    (1 << NODES_SHIFT)
+
+#endif /* _LINUX_NUMA_H */

Re: [PATCH] you have how many nodes??

[Jesse Barnes]

I just tried -test8-mm1 on a 256p/128 node machine and these changes
almost work.  I still needed to change ZONE_SHIFT from BITS_PER_LONG - 8
to BITS_PER_LONG - 10 because of the way set_page_zone() is called from
memmap_init_zone() (set_page_zone(page, nid * MAX_NR_ZONES + zone);).
So shouldn't ZONE_SHIFT be BITS_PER_LONG -  NODES_SHIFT or something?

Jesse

On Fri, Sep 19, 2003 at 03:01:46PM -0700, Matthew Dobson wrote:
> Andrew,
> 	Here's a small update to the numnodes fix that went into -mm3.  The 
> biggest changes are:
> 	1) move the actual NODES_SHIFT and MAX_NUMNODES definitions into 
> linux/numa.h and include this in linux/mmzone.h, instead of being 
> directly in linux/mmzone.h.  This allows other files to include *just* 
> the NUMNODES stuff w/out grabbing all of mmzone.h.
> 	2) pull NODE_SHIFT out of linux/mm.h.  This isn't used anywhere in 
> 	the kernel, and it will only get confused with NODES_SHIFT.
> 	3) Fix the IA64 patch.  The original patch I had sent out hadn't 
> 	been tested on IA64.  It was mostly right, but there were circular 
> dependencies.  All better now, and acked by Jesse.
> 	4) In linux/mmzone.h, insert code to define MAX_NODES_SHIFT based on 
> the size of unsigned long.  For 64-bit arches, we can have a much larger 
> value.  This allows IA64 to have 100's or 1000's of nodes. 
> MAX_NODES_SHIFT is defined as 10 (ie: 1024 nodes) for 64-bit for now, 
> although it could likely be much larger.  For 32-bit it is 6 (ie: 64 nodes).
> 	5) Small cleanup in include/asm-arm/memory.h.  Mostly the result of 
> 	the new linux/numa.h file.  Much cleaner and more readable now.
> 
> Russell, if you get a chance, I'd really appreciate a sanity check on 
> the arm code.  It really hasn't been tested, but the changes are pretty 
> small.