[PATCH 1/2 v4] Make iSeries spin on __secondary_hold_spinloop, like pSeries.

[PATCH 1/2 v4] Make iSeries spin on __secondary_hold_spinloop, like pSeries.

[Tony Breeds]

Currently all iSeries secondary CPU's spin directly on the cpu_start in thier
paca.  Make them spin on the global __secondary_hold_spinloop, until after the
pacas have been initialised.

As sfr points out this works because __secondary_hold_spinloop is being set
already, but iSeries isn't looking at it :)

Signed-off-by: Tony Breeds <tony@bakeyournoodle.com>
Acked-by: Stephen Rothwell <sfr@canb.auug.org.au>
---
Changes since v1:
 - Updated commit message.
Changes since v2:
 - Fixed my brain damage, and split out the code that should be in 3/3
Changes since v3:
 - moved in patch series, no funcational changes.


 arch/powerpc/platforms/iseries/exception.S |   27 ++++++++++++++++++++++++++-
 1 files changed, 26 insertions(+), 1 deletions(-)

diff --git a/arch/powerpc/platforms/iseries/exception.S b/arch/powerpc/platforms/iseries/exception.S
index c775cd4..8ff330d 100644
--- a/arch/powerpc/platforms/iseries/exception.S
+++ b/arch/powerpc/platforms/iseries/exception.S
@@ -59,8 +59,33 @@ system_reset_iSeries:
 	andc	r4,r4,r5
 	mtspr	SPRN_CTRLT,r4
 
+/* Spin on __secondary_hold_spinloop until it is updated by the boot cpu. */
+/* In the UP case we'll yeild() later, and we will not access the paca anyway */
+#ifdef CONFIG_SMP
 1:
 	HMT_LOW
+	LOAD_REG_IMMEDIATE(r23, __secondary_hold_spinloop)
+	ld	r23,0(r23)
+	sync
+	LOAD_REG_IMMEDIATE(r3,current_set)
+	sldi	r28,r24,3		/* get current_set[cpu#] */
+	ldx	r3,r3,r28
+	addi	r1,r3,THREAD_SIZE
+	subi	r1,r1,STACK_FRAME_OVERHEAD
+
+	cmpwi	0,r23,0			/* Keep poking the Hypervisor until */
+	bne	2f			/* we're released */
+	/* Let the Hypervisor know we are alive */
+	/* 8002 is a call to HvCallCfg::getLps, a harmless Hypervisor function */
+	lis	r3,0x8002
+	rldicr	r3,r3,32,15		/* r0 = (r3 << 32) & 0xffff000000000000 */
+	li	r0,-1			/* r0=-1 indicates a Hypervisor call */
+	sc				/* Invoke the hypervisor via a system call */
+	b	1b
+#endif
+
+2:
+	HMT_LOW
 #ifdef CONFIG_SMP
 	lbz	r23,PACAPROCSTART(r13)	/* Test if this processor
 					 * should start */
@@ -91,7 +116,7 @@ iSeries_secondary_smp_loop:
 	li	r0,-1			/* r0=-1 indicates a Hypervisor call */
 	sc				/* Invoke the hypervisor via a system call */
 	mfspr	r13,SPRN_SPRG3		/* Put r13 back ???? */
-	b	1b			/* If SMP not configured, secondaries
+	b	2b			/* If SMP not configured, secondaries
 					 * loop forever */
 
 /***  ISeries-LPAR interrupt handlers ***/
-- 
1.5.5.1

[PATCH 2/2] Raise the upper limit of NR_CPUS and move the pacas into the BSS.

[Tony Breeds]

This patch adds the required functionality to fill in all pacas at runtime.

With NR_CPUS=1024
text    data     bss     dec     hex filename
 137 1704032       0 1704169  1a00e9 arch/powerpc/kernel/paca.o :Before
 121 1179744  524288 1704153  1a00d9 arch/powerpc/kernel/paca.o :After

Also remove unneeded #includes from arch/powerpc/kernel/paca.c

Signed-off-by: Tony Breeds <tony@bakeyournoodle.com>
---
Now merged, with patch 1 fomr previous series.

 arch/powerpc/kernel/paca.c             |   87 +++++++++----------------------
 arch/powerpc/kernel/setup_64.c         |    3 +
 arch/powerpc/platforms/Kconfig.cputype |    4 +-
 include/asm-powerpc/paca.h             |    1 +
 4 files changed, 31 insertions(+), 64 deletions(-)

diff --git a/arch/powerpc/kernel/paca.c b/arch/powerpc/kernel/paca.c
index ac163bd..c9bf17e 100644
--- a/arch/powerpc/kernel/paca.c
+++ b/arch/powerpc/kernel/paca.c
@@ -7,17 +7,11 @@
  *      2 of the License, or (at your option) any later version.
  */
 
-#include <linux/types.h>
 #include <linux/threads.h>
 #include <linux/module.h>
 
-#include <asm/processor.h>
-#include <asm/ptrace.h>
-#include <asm/page.h>
 #include <asm/lppaca.h>
 #include <asm/paca.h>
-#include <asm/mmu.h>
-
 
 /* This symbol is provided by the linker - let it fill in the paca
  * field correctly */
@@ -65,60 +59,29 @@ struct slb_shadow slb_shadow[] __cacheline_aligned = {
  * processors.  The processor VPD array needs one entry per physical
  * processor (not thread).
  */
-#define PACA_INIT(number)						    \
-{									    \
-	.lppaca_ptr = &lppaca[number],					    \
-	.lock_token = 0x8000,						    \
-	.paca_index = (number),		/* Paca Index */		    \
-	.kernel_toc = (unsigned long)(&__toc_start) + 0x8000UL,		    \
-	.hw_cpu_id = 0xffff,						    \
-	.slb_shadow_ptr = &slb_shadow[number],				    \
-	.__current = &init_task,					    \
-}
-
-struct paca_struct paca[] = {
-	PACA_INIT(0),
-#if NR_CPUS > 1
-	PACA_INIT(  1), PACA_INIT(  2), PACA_INIT(  3),
-#if NR_CPUS > 4
-	PACA_INIT(  4), PACA_INIT(  5), PACA_INIT(  6), PACA_INIT(  7),
-#if NR_CPUS > 8
-	PACA_INIT(  8), PACA_INIT(  9), PACA_INIT( 10), PACA_INIT( 11),
-	PACA_INIT( 12), PACA_INIT( 13), PACA_INIT( 14), PACA_INIT( 15),
-	PACA_INIT( 16), PACA_INIT( 17), PACA_INIT( 18), PACA_INIT( 19),
-	PACA_INIT( 20), PACA_INIT( 21), PACA_INIT( 22), PACA_INIT( 23),
-	PACA_INIT( 24), PACA_INIT( 25), PACA_INIT( 26), PACA_INIT( 27),
-	PACA_INIT( 28), PACA_INIT( 29), PACA_INIT( 30), PACA_INIT( 31),
-#if NR_CPUS > 32
-	PACA_INIT( 32), PACA_INIT( 33), PACA_INIT( 34), PACA_INIT( 35),
-	PACA_INIT( 36), PACA_INIT( 37), PACA_INIT( 38), PACA_INIT( 39),
-	PACA_INIT( 40), PACA_INIT( 41), PACA_INIT( 42), PACA_INIT( 43),
-	PACA_INIT( 44), PACA_INIT( 45), PACA_INIT( 46), PACA_INIT( 47),
-	PACA_INIT( 48), PACA_INIT( 49), PACA_INIT( 50), PACA_INIT( 51),
-	PACA_INIT( 52), PACA_INIT( 53), PACA_INIT( 54), PACA_INIT( 55),
-	PACA_INIT( 56), PACA_INIT( 57), PACA_INIT( 58), PACA_INIT( 59),
-	PACA_INIT( 60), PACA_INIT( 61), PACA_INIT( 62), PACA_INIT( 63),
-#if NR_CPUS > 64
-	PACA_INIT( 64), PACA_INIT( 65), PACA_INIT( 66), PACA_INIT( 67),
-	PACA_INIT( 68), PACA_INIT( 69), PACA_INIT( 70), PACA_INIT( 71),
-	PACA_INIT( 72), PACA_INIT( 73), PACA_INIT( 74), PACA_INIT( 75),
-	PACA_INIT( 76), PACA_INIT( 77), PACA_INIT( 78), PACA_INIT( 79),
-	PACA_INIT( 80), PACA_INIT( 81), PACA_INIT( 82), PACA_INIT( 83),
-	PACA_INIT( 84), PACA_INIT( 85), PACA_INIT( 86), PACA_INIT( 87),
-	PACA_INIT( 88), PACA_INIT( 89), PACA_INIT( 90), PACA_INIT( 91),
-	PACA_INIT( 92), PACA_INIT( 93), PACA_INIT( 94), PACA_INIT( 95),
-	PACA_INIT( 96), PACA_INIT( 97), PACA_INIT( 98), PACA_INIT( 99),
-	PACA_INIT(100), PACA_INIT(101), PACA_INIT(102), PACA_INIT(103),
-	PACA_INIT(104), PACA_INIT(105), PACA_INIT(106), PACA_INIT(107),
-	PACA_INIT(108), PACA_INIT(109), PACA_INIT(110), PACA_INIT(111),
-	PACA_INIT(112), PACA_INIT(113), PACA_INIT(114), PACA_INIT(115),
-	PACA_INIT(116), PACA_INIT(117), PACA_INIT(118), PACA_INIT(119),
-	PACA_INIT(120), PACA_INIT(121), PACA_INIT(122), PACA_INIT(123),
-	PACA_INIT(124), PACA_INIT(125), PACA_INIT(126), PACA_INIT(127),
-#endif
-#endif
-#endif
-#endif
-#endif
-};
+struct paca_struct paca[NR_CPUS];
 EXPORT_SYMBOL(paca);
+
+void __init initialise_pacas(void)
+{
+	int cpu;
+
+	/* The TOC register (GPR2) points 32kB into the TOC, so that 64kB
+	 * of the TOC can be addressed using a single machine instruction.
+	 */
+	unsigned long kernel_toc = (unsigned long)(&__toc_start) + 0x8000UL;
+
+	/* Can't use for_each_*_cpu, as they aren't functional yet */
+	for (cpu = 0; cpu < NR_CPUS; cpu++) {
+		struct paca_struct *new_paca = &paca[cpu];
+
+		new_paca->lppaca_ptr = &lppaca[cpu];
+		new_paca->lock_token = 0x8000;
+		new_paca->paca_index = cpu;
+		new_paca->kernel_toc = kernel_toc;
+		new_paca->hw_cpu_id = 0xffff;
+		new_paca->slb_shadow_ptr = &slb_shadow[cpu];
+		new_paca->__current = &init_task;
+
+	}
+}
diff --git a/arch/powerpc/kernel/setup_64.c b/arch/powerpc/kernel/setup_64.c
index 31ada9f..5e382ac 100644
--- a/arch/powerpc/kernel/setup_64.c
+++ b/arch/powerpc/kernel/setup_64.c
@@ -170,6 +170,9 @@ void __init setup_paca(int cpu)
 
 void __init early_setup(unsigned long dt_ptr)
 {
+	/* Fill in any unititialised pacas */
+	initialise_pacas();
+
 	/* Identify CPU type */
 	identify_cpu(0, mfspr(SPRN_PVR));
 
diff --git a/arch/powerpc/platforms/Kconfig.cputype b/arch/powerpc/platforms/Kconfig.cputype
index 5fc7fac..f7efaa9 100644
--- a/arch/powerpc/platforms/Kconfig.cputype
+++ b/arch/powerpc/platforms/Kconfig.cputype
@@ -220,8 +220,8 @@ config SMP
 	  If you don't know what to do here, say N.
 
 config NR_CPUS
-	int "Maximum number of CPUs (2-128)"
-	range 2 128
+	int "Maximum number of CPUs (2-1024)"
+	range 2 1024
 	depends on SMP
 	default "32" if PPC64
 	default "4"
diff --git a/include/asm-powerpc/paca.h b/include/asm-powerpc/paca.h
index eb61b9c..7b56444 100644
--- a/include/asm-powerpc/paca.h
+++ b/include/asm-powerpc/paca.h
@@ -108,6 +108,7 @@ struct paca_struct {
 };
 
 extern struct paca_struct paca[];
+extern void initialise_pacas(void);
 
 #endif /* __KERNEL__ */
 #endif /* _ASM_POWERPC_PACA_H */
-- 
1.5.5.1

Re: [PATCH 2/2] Raise the upper limit of NR_CPUS and move the pacas into the BSS.

[Michael Ellerman]

[-- Attachment #1: Type: text/plain, Size: 631 bytes --]

On Thu, 2008-04-24 at 13:43 +1000, Tony Breeds wrote:
> This patch adds the required functionality to fill in all pacas at runtime.
> 
> With NR_CPUS=1024
> text    data     bss     dec     hex filename
>  137 1704032       0 1704169  1a00e9 arch/powerpc/kernel/paca.o :Before
>  121 1179744  524288 1704153  1a00d9 arch/powerpc/kernel/paca.o :After

Niiiiice.

cheers

-- 
Michael Ellerman
OzLabs, IBM Australia Development Lab

wwweb: http://michael.ellerman.id.au
phone: +61 2 6212 1183 (tie line 70 21183)

We do not inherit the earth from our ancestors,
we borrow it from our children. - S.M.A.R.T Person

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH 2/2] Raise the upper limit of NR_CPUS and move the pacas into the BSS.

[Arnd Bergmann]

On Thursday 24 April 2008, Tony Breeds wrote:
> This patch adds the required functionality to fill in all pacas at runtim=
e.
>=20
> With NR_CPUS=3D1024
> text =A0 =A0data =A0 =A0 bss =A0 =A0 dec =A0 =A0 hex filename
> =A0137 1704032 =A0 =A0 =A0 0 1704169 =A01a00e9 arch/powerpc/kernel/paca.o=
 :Before
> =A0121 1179744 =A0524288 1704153 =A01a00d9 arch/powerpc/kernel/paca.o :Af=
ter
>=20
> Also remove unneeded #includes from arch/powerpc/kernel/paca.c
>=20
> Signed-off-by: Tony Breeds <tony@bakeyournoodle.com>

Maybe we can go even further than this: Since it is now a substantial amount
of .bss, maybe the unused parts can be returned to the buddy allocator
using free_bootmem or similar once you know how many CPUs there are?

I know that this would be an evil hack, but it may still be worth it.
Your patch should certainly go in first though, freeing the memory would
be an additional extension.

	Arnd <><