[PATCH 0/2] MIPS: Netlogic: Fixes for 3.6

[PATCH 0/2] MIPS: Netlogic: Fixes for 3.6

[Jayachandran C]

Two patches for Netlogic processors for 3.6, the second patch is critical
and fixes a hang on boot in 3.6, and the first patch is optional and fixes
a BUG() when preempt is enabled.

I had send these two separately earlier today, but the patches had a
mixture of netlogic and broadcom mail addresses.

JC.

Jayachandran C (2):
  MIPS: Netlogic: define cpu_has_fpu macro
  MIPS: Synchronize MIPS count one CPU at a time

 .../asm/mach-netlogic/cpu-feature-overrides.h      |    2 ++
 arch/mips/include/asm/r4k-timer.h                  |    8 +++---
 arch/mips/kernel/smp.c                             |    4 +--
 arch/mips/kernel/sync-r4k.c                        |   26 +++++++++-----------
 4 files changed, 19 insertions(+), 21 deletions(-)

-- 
1.7.9.5

[PATCH 1/2] MIPS: Netlogic: define cpu_has_fpu macro

[Jayachandran C]

The default implementation of 'cpu_has_fpu' macro calls
smp_processor_id() which causes this warning when preemption is enabled:

[    4.664000] Algorithmics/MIPS FPU Emulator v1.5
[    4.676000] BUG: using smp_processor_id() in preemptible [00000000] code: init/1
[    4.700000] caller is fpu_emulator_cop1Handler+0x434/0x27b8

Work around this by defining cpu_has_fpu for XLR and XLP in
mach-netlogic/cpu-feature-overrides.h

Signed-off-by: Jayachandran C <jchandra@broadcom.com>
---
 .../asm/mach-netlogic/cpu-feature-overrides.h      |    2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/mips/include/asm/mach-netlogic/cpu-feature-overrides.h b/arch/mips/include/asm/mach-netlogic/cpu-feature-overrides.h
index 966db4b..4f5907f 100644
--- a/arch/mips/include/asm/mach-netlogic/cpu-feature-overrides.h
+++ b/arch/mips/include/asm/mach-netlogic/cpu-feature-overrides.h
@@ -44,10 +44,12 @@
 #define cpu_has_dc_aliases	0
 #define cpu_has_mips32r2	0
 #define cpu_has_mips64r2	0
+#define cpu_has_fpu		0
 #elif defined(CONFIG_CPU_XLP)
 #define cpu_has_userlocal	1
 #define cpu_has_mips32r2	1
 #define cpu_has_mips64r2	1
+#define cpu_has_fpu		1
 #else
 #error "Unknown Netlogic CPU"
 #endif
-- 
1.7.9.5

Re: [PATCH 1/2] MIPS: Netlogic: define cpu_has_fpu macro

[Ralf Baechle]

On Tue, Aug 14, 2012 at 06:56:12PM +0530, Jayachandran C wrote:

> The default implementation of 'cpu_has_fpu' macro calls
> smp_processor_id() which causes this warning when preemption is enabled:
> 
> [    4.664000] Algorithmics/MIPS FPU Emulator v1.5
> [    4.676000] BUG: using smp_processor_id() in preemptible [00000000] code: init/1
> [    4.700000] caller is fpu_emulator_cop1Handler+0x434/0x27b8
> 
> Work around this by defining cpu_has_fpu for XLR and XLP in
> mach-netlogic/cpu-feature-overrides.h

Where is cpu_has_fpu being invoked from?  For exactly the scenario you're
running into there is raw_cpu_has_fpu and I wonder if the caller should be
switched to that instead.

  Ralf

[PATCH] MIPS: Fix for warning from FPU emulation code

[Jayachandran C]

The default implementation of 'cpu_has_fpu' macro calls
smp_processor_id() which causes this warning to be printed when
preemption is enabled:

[    4.664000] Algorithmics/MIPS FPU Emulator v1.5
[    4.676000] BUG: using smp_processor_id() in preemptible [00000000] code: ini
[    4.700000] caller is fpu_emulator_cop1Handler+0x434/0x27b8

Use 'raw_cpu_has_fpu' macro in cop1_64bit() instead of 'cpu_has_fpu'
to fix this. Fix suggested by Ralf Baechle <ralf@linux-mips.org>

Signed-off-by: Jayachandran C <jchandra@broadcom.com>
---
 arch/mips/math-emu/cp1emu.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/mips/math-emu/cp1emu.c b/arch/mips/math-emu/cp1emu.c
index a03bf00..663bfb9 100644
--- a/arch/mips/math-emu/cp1emu.c
+++ b/arch/mips/math-emu/cp1emu.c
@@ -178,7 +178,7 @@ static int isBranchInstr(mips_instruction * i)
  */
 static inline int cop1_64bit(struct pt_regs *xcp)
 {
-	if (cpu_has_fpu)
+	if (raw_cpu_has_fpu)
 		return xcp->cp0_status & ST0_FR;
 #ifdef CONFIG_64BIT
 	return !test_thread_flag(TIF_32BIT_REGS);
-- 
1.7.9.5

Re: [PATCH] MIPS: Fix for warning from FPU emulation code

[Ralf Baechle]

On Tue, Aug 14, 2012 at 09:33:40PM +0530, Jayachandran C wrote:

> The default implementation of 'cpu_has_fpu' macro calls
> smp_processor_id() which causes this warning to be printed when
> preemption is enabled:
> 
> [    4.664000] Algorithmics/MIPS FPU Emulator v1.5
> [    4.676000] BUG: using smp_processor_id() in preemptible [00000000] code: ini
> [    4.700000] caller is fpu_emulator_cop1Handler+0x434/0x27b8
> 
> Use 'raw_cpu_has_fpu' macro in cop1_64bit() instead of 'cpu_has_fpu'
> to fix this. Fix suggested by Ralf Baechle <ralf@linux-mips.org>

I think your patch was 34K-ly correct but I never managed to convince
myself that it really was and anyway, cpu_has_fpu expands into too much
code for the runtime detection case.  With below patch all platforms I
tested have either unchanged or smaller code size depending on the
kernel configuration.

The whole problem got introduced in November 2009 by
af1d2af877ef6c36990671bc86a5b9c5bb50b1da (lmo) [MIPS: Fix emulation of
64-bit FPU on 64-bit CPUs.] rsp.  da0bac33413b2888d3623dad3ad19ce76b688f07
(kernel.org) [MIPS: Fix emulation of 64-bit FPU on FPU-less 64-bit CPUs.]
in 2.6.32.

  Ralf

 arch/mips/math-emu/cp1emu.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/arch/mips/math-emu/cp1emu.c b/arch/mips/math-emu/cp1emu.c
index a03bf00..47c77e7 100644
--- a/arch/mips/math-emu/cp1emu.c
+++ b/arch/mips/math-emu/cp1emu.c
@@ -171,16 +171,17 @@ static int isBranchInstr(mips_instruction * i)
  * In the Linux kernel, we support selection of FPR format on the
  * basis of the Status.FR bit.  If an FPU is not present, the FR bit
  * is hardwired to zero, which would imply a 32-bit FPU even for
- * 64-bit CPUs.  For 64-bit kernels with no FPU we use TIF_32BIT_REGS
- * as a proxy for the FR bit so that a 64-bit FPU is emulated.  In any
- * case, for a 32-bit kernel which uses the O32 MIPS ABI, only the
- * even FPRs are used (Status.FR = 0).
+ * 64-bit CPUs so we rather look at TIF_32BIT_REGS.
+ * FPU emu is slow and bulky and optimizing this function offers fairly
+ * sizeable benefits so we try to be clever and make this function return
+ * a constant whenever possible, that is on 64-bit kernels without O32
+ * compatibility enabled and on 32-bit kernels.
  */
 static inline int cop1_64bit(struct pt_regs *xcp)
 {
-	if (cpu_has_fpu)
-		return xcp->cp0_status & ST0_FR;
-#ifdef CONFIG_64BIT
+#if defined(CONFIG_64BIT) && !defined(CONFIG_MIPS32_O32)
+	return 1;
+#elif defined(CONFIG_64BIT) && defined(CONFIG_MIPS32_O32)
 	return !test_thread_flag(TIF_32BIT_REGS);
 #else
 	return 0;

[PATCH 2/2] MIPS: Synchronize MIPS count one CPU at a time

[Jayachandran C]

The current implementation of synchronise_count_{master,slave} blocks
slave CPUs in early boot until all of them come up. This no longer
works because blocking a CPU with interrupts off after notifying the
CPU to be online causes problems with the current kernel.

Specifically, after the workqueue changes
(commit a08489c569dc1 "Pull workqueue changes from Tejun Heo")
the CPU_ONLINE notification callback workqueue_cpu_up_callback()
will hang on wait_for_completion(&idle_rebind.done), if the slave
CPUs are blocked for synchronize_count_slave().

The changes are to update synchronize_count_{master,slave}() to handle
one CPU at a time and to call synchronise_count_master() in __cpu_up()
so that the CPU_ONLINE notification goes out only after the COP0 COUNT
register is synchronized.

Signed-off-by: Jayachandran C <jchandra@broadcom.com>
---
 arch/mips/include/asm/r4k-timer.h |    8 ++++----
 arch/mips/kernel/smp.c            |    4 ++--
 arch/mips/kernel/sync-r4k.c       |   26 +++++++++++---------------
 3 files changed, 17 insertions(+), 21 deletions(-)

diff --git a/arch/mips/include/asm/r4k-timer.h b/arch/mips/include/asm/r4k-timer.h
index a37d12b..afe9e0e 100644
--- a/arch/mips/include/asm/r4k-timer.h
+++ b/arch/mips/include/asm/r4k-timer.h
@@ -12,16 +12,16 @@
 
 #ifdef CONFIG_SYNC_R4K
 
-extern void synchronise_count_master(void);
-extern void synchronise_count_slave(void);
+extern void synchronise_count_master(int cpu);
+extern void synchronise_count_slave(int cpu);
 
 #else
 
-static inline void synchronise_count_master(void)
+static inline void synchronise_count_master(int cpu)
 {
 }
 
-static inline void synchronise_count_slave(void)
+static inline void synchronise_count_slave(int cpu)
 {
 }
 
diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c
index 31637d8..9005bf9 100644
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -130,7 +130,7 @@ asmlinkage __cpuinit void start_secondary(void)
 
 	cpu_set(cpu, cpu_callin_map);
 
-	synchronise_count_slave();
+	synchronise_count_slave(cpu);
 
 	/*
 	 * irq will be enabled in ->smp_finish(), enabling it too early
@@ -173,7 +173,6 @@ void smp_send_stop(void)
 void __init smp_cpus_done(unsigned int max_cpus)
 {
 	mp_ops->cpus_done();
-	synchronise_count_master();
 }
 
 /* called from main before smp_init() */
@@ -206,6 +205,7 @@ int __cpuinit __cpu_up(unsigned int cpu, struct task_struct *tidle)
 	while (!cpu_isset(cpu, cpu_callin_map))
 		udelay(100);
 
+	synchronise_count_master(cpu);
 	return 0;
 }
 
diff --git a/arch/mips/kernel/sync-r4k.c b/arch/mips/kernel/sync-r4k.c
index 842d55e..7f1eca3 100644
--- a/arch/mips/kernel/sync-r4k.c
+++ b/arch/mips/kernel/sync-r4k.c
@@ -28,12 +28,11 @@ static atomic_t __cpuinitdata count_reference = ATOMIC_INIT(0);
 #define COUNTON	100
 #define NR_LOOPS 5
 
-void __cpuinit synchronise_count_master(void)
+void __cpuinit synchronise_count_master(int cpu)
 {
 	int i;
 	unsigned long flags;
 	unsigned int initcount;
-	int nslaves;
 
 #ifdef CONFIG_MIPS_MT_SMTC
 	/*
@@ -43,8 +42,7 @@ void __cpuinit synchronise_count_master(void)
 	return;
 #endif
 
-	printk(KERN_INFO "Synchronize counters across %u CPUs: ",
-	       num_online_cpus());
+	printk(KERN_INFO "Synchronize counters for CPU %u: ", cpu);
 
 	local_irq_save(flags);
 
@@ -52,7 +50,7 @@ void __cpuinit synchronise_count_master(void)
 	 * Notify the slaves that it's time to start
 	 */
 	atomic_set(&count_reference, read_c0_count());
-	atomic_set(&count_start_flag, 1);
+	atomic_set(&count_start_flag, cpu);
 	smp_wmb();
 
 	/* Count will be initialised to current timer for all CPU's */
@@ -69,10 +67,9 @@ void __cpuinit synchronise_count_master(void)
 	 * two CPUs.
 	 */
 
-	nslaves = num_online_cpus()-1;
 	for (i = 0; i < NR_LOOPS; i++) {
-		/* slaves loop on '!= ncpus' */
-		while (atomic_read(&count_count_start) != nslaves)
+		/* slaves loop on '!= 2' */
+		while (atomic_read(&count_count_start) != 1)
 			mb();
 		atomic_set(&count_count_stop, 0);
 		smp_wmb();
@@ -89,7 +86,7 @@ void __cpuinit synchronise_count_master(void)
 		/*
 		 * Wait for all slaves to leave the synchronization point:
 		 */
-		while (atomic_read(&count_count_stop) != nslaves)
+		while (atomic_read(&count_count_stop) != 1)
 			mb();
 		atomic_set(&count_count_start, 0);
 		smp_wmb();
@@ -97,6 +94,7 @@ void __cpuinit synchronise_count_master(void)
 	}
 	/* Arrange for an interrupt in a short while */
 	write_c0_compare(read_c0_count() + COUNTON);
+	atomic_set(&count_start_flag, 0);
 
 	local_irq_restore(flags);
 
@@ -108,11 +106,10 @@ void __cpuinit synchronise_count_master(void)
 	printk("done.\n");
 }
 
-void __cpuinit synchronise_count_slave(void)
+void __cpuinit synchronise_count_slave(int cpu)
 {
 	int i;
 	unsigned int initcount;
-	int ncpus;
 
 #ifdef CONFIG_MIPS_MT_SMTC
 	/*
@@ -127,16 +124,15 @@ void __cpuinit synchronise_count_slave(void)
 	 * so we first wait for the master to say everyone is ready
 	 */
 
-	while (!atomic_read(&count_start_flag))
+	while (atomic_read(&count_start_flag) != cpu)
 		mb();
 
 	/* Count will be initialised to next expire for all CPU's */
 	initcount = atomic_read(&count_reference);
 
-	ncpus = num_online_cpus();
 	for (i = 0; i < NR_LOOPS; i++) {
 		atomic_inc(&count_count_start);
-		while (atomic_read(&count_count_start) != ncpus)
+		while (atomic_read(&count_count_start) != 2)
 			mb();
 
 		/*
@@ -146,7 +142,7 @@ void __cpuinit synchronise_count_slave(void)
 			write_c0_count(initcount);
 
 		atomic_inc(&count_count_stop);
-		while (atomic_read(&count_count_stop) != ncpus)
+		while (atomic_read(&count_count_stop) != 2)
 			mb();
 	}
 	/* Arrange for an interrupt in a short while */
-- 
1.7.9.5

Re: [PATCH 2/2] MIPS: Synchronize MIPS count one CPU at a time

[Ralf Baechle]

On Tue, Aug 14, 2012 at 06:56:13PM +0530, Jayachandran C wrote:

Thanks, applied.

  Ralf