[patch] CONFIG_NR_CPUS

[patch] CONFIG_NR_CPUS

[Andrew Morton]

Reducing NR_CPUS from 32 to 2 reduces the kernel footprint by
approximately 240 kilobytes.

Before:
   text    data     bss     dec     hex filename
2120633  283268  251572 2655473  2884f1 vmlinux

After:
   text    data     bss     dec     hex filename
2120777  390116  384308 2895201  2c2d61 vmlinux




--- 2.5.20/arch/i386/config.in~config_nr_cpus	Thu Jun  6 02:47:35 2002
+++ 2.5.20-akpm/arch/i386/config.in	Thu Jun  6 02:47:35 2002
@@ -185,8 +185,8 @@ fi
 
 bool 'Math emulation' CONFIG_MATH_EMULATION
 bool 'MTRR (Memory Type Range Register) support' CONFIG_MTRR
-bool 'Symmetric multi-processing support' CONFIG_SMP
 bool 'Preemptible Kernel' CONFIG_PREEMPT
+bool 'Symmetric multi-processing support' CONFIG_SMP
 if [ "$CONFIG_SMP" != "y" ]; then
    bool 'Local APIC support on uniprocessors' CONFIG_X86_UP_APIC
    dep_bool 'IO-APIC support on uniprocessors' CONFIG_X86_UP_IOAPIC $CONFIG_X86_UP_APIC
@@ -197,6 +197,7 @@ if [ "$CONFIG_SMP" != "y" ]; then
       define_bool CONFIG_X86_IO_APIC y
    fi
 else
+   int  'Maximum number of CPUs (2-32)' CONFIG_NR_CPUS 32
    bool 'Multiquad NUMA system' CONFIG_MULTIQUAD
 fi
 
--- 2.5.20/include/linux/threads.h~config_nr_cpus	Thu Jun  6 02:47:35 2002
+++ 2.5.20-akpm/include/linux/threads.h	Thu Jun  6 02:47:35 2002
@@ -9,7 +9,13 @@
  */
  
 #ifdef CONFIG_SMP
+
+#ifdef CONFIG_NR_CPUS
+#define NR_CPUS CONFIG_NR_CPUS
+#else
 #define NR_CPUS	32		/* Max processors that can be running in SMP */
+#endif
+
 #else
 #define NR_CPUS 1
 #endif
--- 2.5.20/arch/i386/Config.help~config_nr_cpus	Thu Jun  6 02:47:35 2002
+++ 2.5.20-akpm/arch/i386/Config.help	Thu Jun  6 02:47:35 2002
@@ -25,6 +25,14 @@ CONFIG_SMP
 
   If you don't know what to do here, say N.
 
+CONFIG_NR_CPUS
+  This allows you to specify the maximum number of CPUs which this
+  kernel will support.  The maximum supported value is 32 and the
+  mimimum value which makes sense is 2.
+
+  This is purely to save memory - each supported CPU adds
+  approximately eight kilobytes to the kernel image.
+
 CONFIG_PREEMPT
   This option reduces the latency of the kernel when reacting to
   real-time or interactive events by allowing a low priority process to
--- 2.5.20/arch/i386/kernel/smpboot.c~config_nr_cpus	Thu Jun  6 02:53:20 2002
+++ 2.5.20-akpm/arch/i386/kernel/smpboot.c	Thu Jun  6 02:57:35 2002
@@ -54,7 +54,7 @@
 static int smp_b_stepping;
 
 /* Setup configured maximum number of CPUs to activate */
-static int max_cpus = -1;
+static int max_cpus = NR_CPUS;
 
 /* Total count of live CPUs */
 int smp_num_cpus = 1;
@@ -1145,7 +1145,7 @@ void __init smp_boot_cpus(void)
 
 		if (!(phys_cpu_present_map & (1 << bit)))
 			continue;
-		if ((max_cpus >= 0) && (max_cpus <= cpucount+1))
+		if (max_cpus <= cpucount+1)
 			continue;
 
 		do_boot_cpu(apicid);

-

Re: [patch] CONFIG_NR_CPUS

[David S. Miller]

   From: Andrew Morton <akpm@zip.com.au>
   Date: Thu, 06 Jun 2002 03:10:12 -0700

   Reducing NR_CPUS from 32 to 2 reduces the kernel footprint by
   approximately 240 kilobytes.
   
Nice.  While you're at it can you fix the value on 64-bit
platforms when CONFIG_NR_CPUS is not specified?  (it should
be 64, not 32)

Re: [patch] CONFIG_NR_CPUS

[Robert Love]

On Thu, 2002-06-06 at 03:15, David S. Miller wrote:
 
> Nice.  While you're at it can you fix the value on 64-bit
> platforms when CONFIG_NR_CPUS is not specified?  (it should
> be 64, not 32)

I agree, this is good.  I often am toying with some debugging aid that
is an array of NR_CPUS and waste a lot of memory with NR_CPUS stuck at
32... no reason my kernels should not be set to 2 or whatever I need.

I have attached a patch that is Andrew's + your request, Dave.  Since
what really determines the maximum number of CPUs is the size of
unsigned long, I used that.  Cool?

	Robert Love

diff -urN linux-2.5.20/arch/i386/Config.help linux/arch/i386/Config.help
--- linux-2.5.20/arch/i386/Config.help	Sun Jun  2 18:44:41 2002
+++ linux/arch/i386/Config.help	Thu Jun  6 07:58:58 2002
@@ -25,6 +25,14 @@
 
   If you don't know what to do here, say N.
 
+CONFIG_NR_CPUS
+  This allows you to specify the maximum number of CPUs which this
+  kernel will support.  The maximum supported value is 32 and the
+  mimimum value which makes sense is 2.
+
+  This is purely to save memory - each supported CPU adds
+  approximately eight kilobytes to the kernel image.
+
 CONFIG_PREEMPT
   This option reduces the latency of the kernel when reacting to
   real-time or interactive events by allowing a low priority process to
diff -urN linux-2.5.20/arch/i386/config.in linux/arch/i386/config.in
--- linux-2.5.20/arch/i386/config.in	Sun Jun  2 18:44:46 2002
+++ linux/arch/i386/config.in	Thu Jun  6 07:58:58 2002
@@ -185,8 +185,8 @@
 
 bool 'Math emulation' CONFIG_MATH_EMULATION
 bool 'MTRR (Memory Type Range Register) support' CONFIG_MTRR
-bool 'Symmetric multi-processing support' CONFIG_SMP
 bool 'Preemptible Kernel' CONFIG_PREEMPT
+bool 'Symmetric multi-processing support' CONFIG_SMP
 if [ "$CONFIG_SMP" != "y" ]; then
    bool 'Local APIC support on uniprocessors' CONFIG_X86_UP_APIC
    dep_bool 'IO-APIC support on uniprocessors' CONFIG_X86_UP_IOAPIC $CONFIG_X86_UP_APIC
@@ -197,6 +197,7 @@
       define_bool CONFIG_X86_IO_APIC y
    fi
 else
+   int  'Maximum number of CPUs (2-32)' CONFIG_NR_CPUS 32
    bool 'Multiquad NUMA system' CONFIG_MULTIQUAD
 fi
 
diff -urN linux-2.5.20/arch/i386/kernel/smpboot.c linux/arch/i386/kernel/smpboot.c
--- linux-2.5.20/arch/i386/kernel/smpboot.c	Sun Jun  2 18:44:49 2002
+++ linux/arch/i386/kernel/smpboot.c	Thu Jun  6 07:58:58 2002
@@ -54,7 +54,7 @@
 static int smp_b_stepping;
 
 /* Setup configured maximum number of CPUs to activate */
-static int max_cpus = -1;
+static int max_cpus = NR_CPUS;
 
 /* Total count of live CPUs */
 int smp_num_cpus = 1;
@@ -1145,7 +1145,7 @@
 
 		if (!(phys_cpu_present_map & (1 << bit)))
 			continue;
-		if ((max_cpus >= 0) && (max_cpus <= cpucount+1))
+		if (max_cpus <= cpucount+1)
 			continue;
 
 		do_boot_cpu(apicid);
diff -urN linux-2.5.20/include/linux/threads.h linux/include/linux/threads.h
--- linux-2.5.20/include/linux/threads.h	Sun Jun  2 18:44:39 2002
+++ linux/include/linux/threads.h	Thu Jun  6 08:01:29 2002
@@ -7,11 +7,18 @@
  * The default limit for the nr of threads is now in
  * /proc/sys/kernel/threads-max.
  */
- 
+
+/* Max processors that can be running in SMP */
 #ifdef CONFIG_SMP
-#define NR_CPUS	32		/* Max processors that can be running in SMP */
+
+#ifdef CONFIG_NR_CPUS
+#define NR_CPUS	CONFIG_NR_CPUS
+#else
+#define NR_CPUS	(sizeof(unsigned long) * 8)
+#endif
+
 #else
-#define NR_CPUS 1
+#define NR_CPUS	1
 #endif
 
 #define MIN_THREADS_LEFT_FOR_ROOT 4

Re: [patch] CONFIG_NR_CPUS

[Tom Rini]

On Thu, Jun 06, 2002 at 08:26:52AM -0700, Robert Love wrote:

> On Thu, 2002-06-06 at 03:15, David S. Miller wrote:
>  
> > Nice.  While you're at it can you fix the value on 64-bit
> > platforms when CONFIG_NR_CPUS is not specified?  (it should
> > be 64, not 32)
> 
> I agree, this is good.  I often am toying with some debugging aid that
> is an array of NR_CPUS and waste a lot of memory with NR_CPUS stuck at
> 32... no reason my kernels should not be set to 2 or whatever I need.
> 
> I have attached a patch that is Andrew's + your request, Dave.  Since
> what really determines the maximum number of CPUs is the size of
> unsigned long, I used that.  Cool?

Here's a (compile) tested version for PPC.  arch/ppc/kernel/smp.c makes
much less use of max_cpus, so this should be all that's needed. BTW, on
x86 max_cpus could become __initdata if someone cares..

-- 
Tom Rini (TR1265)
http://gate.crashing.org/~trini/


===== arch/ppc/config.in 1.36 vs edited =====
--- 1.36/arch/ppc/config.in	Fri May 24 04:15:43 2002
+++ edited/arch/ppc/config.in	Thu Jun  6 09:30:39 2002
@@ -172,6 +172,7 @@
 bool 'Symmetric multi-processing support' CONFIG_SMP
 if [ "$CONFIG_SMP" = "y" ]; then
    bool '  Distribute interrupts on all CPUs by default' CONFIG_IRQ_ALL_CPUS
+   int  '  Maximum number of CPUs (2-32)' CONFIG_NR_CPUS 32
 fi
 if [ "$CONFIG_SMP" != "y" ]; then
    bool 'Preemptible Kernel' CONFIG_PREEMPT
===== arch/ppc/Config.help 1.10 vs edited =====
--- 1.10/arch/ppc/Config.help	Fri May 24 03:38:05 2002
+++ edited/arch/ppc/Config.help	Thu Jun  6 09:31:04 2002
@@ -14,6 +14,14 @@
 
   If you don't know what to do here, say N.
 
+CONFIG_NR_CPUS
+  This allows you to specify the maximum number of CPUs which this
+  kernel will support.  The maximum supported value is 32 and the
+  mimimum value which makes sense is 2.
+
+  This is purely to save memory - each supported CPU adds
+  approximately eight kilobytes to the kernel image.
+
 CONFIG_PREEMPT
   This option reduces the latency of the kernel when reacting to
   real-time or interactive events by allowing a low priority process to

Re: [patch] CONFIG_NR_CPUS

[Adam Kropelin]

On Thu, Jun 06, 2002 at 08:26:52AM -0700, Robert Love wrote:
> I have attached a patch that is Andrew's + your request, Dave.  Since
> what really determines the maximum number of CPUs is the size of
> unsigned long, I used that.  Cool?
> 
> 	Robert Love
> 
> diff -urN linux-2.5.20/arch/i386/Config.help linux/arch/i386/Config.help
> --- linux-2.5.20/arch/i386/Config.help	Sun Jun  2 18:44:41 2002
> +++ linux/arch/i386/Config.help	Thu Jun  6 07:58:58 2002
> @@ -25,6 +25,14 @@
>  
>    If you don't know what to do here, say N.
>  
> +CONFIG_NR_CPUS
> +  This allows you to specify the maximum number of CPUs which this
> +  kernel will support.  The maximum supported value is 32 and the
                                                          ^^
This isn't quite true now...

--Adam

Re: [patch] CONFIG_NR_CPUS

[Robert Love]

On Thu, 2002-06-06 at 10:09, Adam Kropelin wrote:

> This isn't quite true now...

Ah, but it is on i386 which is what that configure entry is... since
this is in architecture-specific configure options, each arch must
maintain its only Config.help and Configure.in code.

Arches with 64-bit longs can s/32/64/ there..

	Robert Love

Re: [patch] CONFIG_NR_CPUS

[Andrew Morton]

Robert Love wrote:
> 
> On Thu, 2002-06-06 at 03:15, David S. Miller wrote:
> 
> > Nice.  While you're at it can you fix the value on 64-bit
> > platforms when CONFIG_NR_CPUS is not specified?  (it should
> > be 64, not 32)
> 
> I agree, this is good.  I often am toying with some debugging aid that
> is an array of NR_CPUS and waste a lot of memory with NR_CPUS stuck at
> 32... no reason my kernels should not be set to 2 or whatever I need.
> 
> I have attached a patch that is Andrew's + your request, Dave.  Since
> what really determines the maximum number of CPUs is the size of
> unsigned long, I used that.  Cool?
> 
> ...
> +#define NR_CPUS        (sizeof(unsigned long) * 8)

OK.  What I'll do is:

#ifdef CONFIG_SMP
#define NR_CPUS CONFIG_NR_CPUS
#else
#define NR_CPUS 1
#endif

and then go edit every SMP-capable arch's config.in/Config.help
files.  But the arch maintainers should test one case please - x86
was locking up at boot on quad CPU with NR_CPUS=2.  Others may do
the same.

About a quarter of the bloat is runqueues.  If we could dynamically
allocate those in sched_init() it would be good, because presumably
vendor kernels will be configured for the maximum number of CPUs.

-

Re: [patch] CONFIG_NR_CPUS

[Andreas Dilger]

On Jun 06, 2002  12:48 -0700, Andrew Morton wrote:
> But the arch maintainers should test one case please - x86
> was locking up at boot on quad CPU with NR_CPUS=2.  Others may do
> the same.

Just a guess, but this could be because the two CPUs chosen for the
boot sequence are not physically numbered 0 and 1, so they are
overwriting the bounds of the per-CPU arrays.

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/

Re: [patch] CONFIG_NR_CPUS

[Andrew Morton]

Andreas Dilger wrote:
> 
> On Jun 06, 2002  12:48 -0700, Andrew Morton wrote:
> > But the arch maintainers should test one case please - x86
> > was locking up at boot on quad CPU with NR_CPUS=2.  Others may do
> > the same.
> 
> Just a guess, but this could be because the two CPUs chosen for the
> boot sequence are not physically numbered 0 and 1, so they are
> overwriting the bounds of the per-CPU arrays.

Well the code was assuming that the number of physical
CPUs was always <= NR_CPUS unless max_cpus had been
specified.  I fixed that in the patch.

-

Re: [patch] CONFIG_NR_CPUS

[Thomas 'Dent' Mirlacher]

--snip/snip

> OK.  What I'll do is:
> 
> #ifdef CONFIG_SMP
> #define NR_CPUS CONFIG_NR_CPUS
> #else
> #define NR_CPUS 1
> #endif
> 
> and then go edit every SMP-capable arch's config.in/Config.help
> files.  But the arch maintainers should test one case please - x86
> was locking up at boot on quad CPU with NR_CPUS=2.  Others may do
> the same.

well, what you need to do is make sure smp_num_cpu <= NR_CPUS,
otherwise the kernel will go ballistic on several places within
the code.

- at least in the network system there is the assumption, that
NR_CPUS is the upper limit of cpus. after initialization, everything 
uses smp_num_cpus, which is a nice thing for cpu-hotplugging :)

	tm
-- 
in some way i do, and in some way i don't.

Re: [patch] CONFIG_NR_CPUS

[Dave Jones]

On Thu, Jun 06, 2002 at 11:23:11PM +0200, Thomas 'Dent' Mirlacher wrote:
 > > and then go edit every SMP-capable arch's config.in/Config.help
 > > files.  But the arch maintainers should test one case please - x86
 > > was locking up at boot on quad CPU with NR_CPUS=2.  Others may do
 > > the same.
 > 
 > well, what you need to do is make sure smp_num_cpu <= NR_CPUS,
 > otherwise the kernel will go ballistic on several places within
 > the code.

*nod*, relying on the user to get it right may not be as simple
as it seems. Quite a few people were pleasantly surprised when
their 2-way P4 Xeon become a 4-way HT P4 box.

        Dave

-- 
| Dave Jones.        http://www.codemonkey.org.uk
| SuSE Labs

Re: [patch] CONFIG_NR_CPUS

[Thomas 'Dent' Mirlacher]

On Thu, 6 Jun 2002, Dave Jones wrote:

> On Thu, Jun 06, 2002 at 11:23:11PM +0200, Thomas 'Dent' Mirlacher wrote:
>  > > and then go edit every SMP-capable arch's config.in/Config.help
>  > > files.  But the arch maintainers should test one case please - x86
>  > > was locking up at boot on quad CPU with NR_CPUS=2.  Others may do
>  > > the same.
>  > 
>  > well, what you need to do is make sure smp_num_cpu <= NR_CPUS,
>  > otherwise the kernel will go ballistic on several places within
>  > the code.
> 
> *nod*, relying on the user to get it right may not be as simple
> as it seems. Quite a few people were pleasantly surprised when
> their 2-way P4 Xeon become a 4-way HT P4 box.

:)

you could allocate all the cpu-nr dependend structures during startup,
but then you're limited to the nr of cpus present at startup time
(and use smp_num_cpus instead of NR_CPUS)

- which means:

	1) no recompilation when you upgrade your (already smp) box
	2) no chance for cpu hotplugging

or the other choice, as it seems to be vafoured right now:

let the user pick, and blame the user if he/she did something wrong. 

and a question: is current_thread_info()->cpu a logical cpu#, or 
	a harwired one.

if it's a logical one, the easiest solution for not crashing the
kernel (on bootup - not talking about adding a cpu during runtime),
would be to restrict mac_cpus to NR_CPUS.
if it's a harwired one, well, the user has to live with the consequences
of a compilation time error.

just my $0.02

	tm

-- 
in some way i do, and in some way i don't.