[PATCH] - Allow NODES_SHIFT to be a config option on x86_64

[PATCH] - Allow NODES_SHIFT to be a config option on x86_64

[Jack Steiner]

Allow the maximum number of nodes in an x86_64 system to
be configurable. This patch does NOT change the default value
but allows the value to be a config option.


	Signed-off-by: Jack Steiner <steiner@sgi.com>

---
 arch/x86/Kconfig |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Index: linux/arch/x86/Kconfig
===================================================================
--- linux.orig/arch/x86/Kconfig	2008-03-18 08:11:19.000000000 -0500
+++ linux/arch/x86/Kconfig	2008-03-18 08:23:35.000000000 -0500
@@ -905,7 +905,7 @@ config NUMA_EMU
 	  number of nodes. This is only useful for debugging.
 
 config NODES_SHIFT
-	int
+	int "Max num nodes shift(1-15)"
 	range 1 15  if X86_64
 	default "6" if X86_64
 	default "4" if X86_NUMAQ

Re: [PATCH] - Allow NODES_SHIFT to be a config option on x86_64

[Ingo Molnar]

* Jack Steiner <steiner@sgi.com> wrote:

> Allow the maximum number of nodes in an x86_64 system to be 
> configurable. This patch does NOT change the default value but allows 
> the value to be a config option.

i've applied your patch - but i'm wondering, shouldnt we auto-scale the 
default according to max number of CPUs? (with some sensible scaling 
that happens to meet your expected large-system needs as well ;-) All 
the current manual configuration of nodes shift is ugly.

	Ingo

Re: [PATCH] - Allow NODES_SHIFT to be a config option on x86_64

[Jack Steiner]

On Fri, Mar 21, 2008 at 03:26:49PM +0100, Ingo Molnar wrote:
> 
> * Jack Steiner <steiner@sgi.com> wrote:
> 
> > Allow the maximum number of nodes in an x86_64 system to be 
> > configurable. This patch does NOT change the default value but allows 
> > the value to be a config option.
> 
> i've applied your patch - but i'm wondering, shouldnt we auto-scale the 
> default according to max number of CPUs? (with some sensible scaling 
> that happens to meet your expected large-system needs as well ;-) All 
> the current manual configuration of nodes shift is ugly.

I would prefer to auto-scale, too, but our hardware platform allows
too many options to make it easy. The current system configs will support
from 0 to 32 cpus per node (some nodes have memory only).

However, I'm open to suggestions if you have any ideas....


--- jack

Re: [PATCH] - Allow NODES_SHIFT to be a config option on x86_64

[Ingo Molnar]

* Jack Steiner <steiner@sgi.com> wrote:

> On Fri, Mar 21, 2008 at 03:26:49PM +0100, Ingo Molnar wrote:
> > 
> > * Jack Steiner <steiner@sgi.com> wrote:
> > 
> > > Allow the maximum number of nodes in an x86_64 system to be 
> > > configurable. This patch does NOT change the default value but allows 
> > > the value to be a config option.
> > 
> > i've applied your patch - but i'm wondering, shouldnt we auto-scale the 
> > default according to max number of CPUs? (with some sensible scaling 
> > that happens to meet your expected large-system needs as well ;-) All 
> > the current manual configuration of nodes shift is ugly.
> 
> I would prefer to auto-scale, too, but our hardware platform allows 
> too many options to make it easy. The current system configs will 
> support from 0 to 32 cpus per node (some nodes have memory only).
> 
> However, I'm open to suggestions if you have any ideas....

how about scaling for the worst case, and allowing distros to tune down 
if they really want? What's the cost of a too large NODE_SHIFT?

	Ingo

Re: [PATCH] - Allow NODES_SHIFT to be a config option on x86_64

[Jack Steiner]

On Fri, Mar 21, 2008 at 03:57:36PM +0100, Ingo Molnar wrote:
> 
> * Jack Steiner <steiner@sgi.com> wrote:
> 
> > On Fri, Mar 21, 2008 at 03:26:49PM +0100, Ingo Molnar wrote:
> > > 
> > > * Jack Steiner <steiner@sgi.com> wrote:
> > > 
> > > > Allow the maximum number of nodes in an x86_64 system to be 
> > > > configurable. This patch does NOT change the default value but allows 
> > > > the value to be a config option.
> > > 
> > > i've applied your patch - but i'm wondering, shouldnt we auto-scale the 
> > > default according to max number of CPUs? (with some sensible scaling 
> > > that happens to meet your expected large-system needs as well ;-) All 
> > > the current manual configuration of nodes shift is ugly.
> > 
> > I would prefer to auto-scale, too, but our hardware platform allows 
> > too many options to make it easy. The current system configs will 
> > support from 0 to 32 cpus per node (some nodes have memory only).
> > 
> > However, I'm open to suggestions if you have any ideas....
> 
> how about scaling for the worst case, and allowing distros to tune down 
> if they really want? What's the cost of a too large NODE_SHIFT?
> 


NODE_SHIFT affects a number of nodemasks so you probably don't want it
a lot larger than needed - especially if you are going over 64 nodes. Our
target value is currently 512 although it is possible that it may
change to 256.

Let me do some experiments on auto-config. I'll get back to you....

--- jack