[PATCH RESEND 1/9] MIPS: Add local_flush_tlb_all_mm to clear all mm contexts on calling cpu

[PATCH RESEND 1/9] MIPS: Add local_flush_tlb_all_mm to clear all mm contexts on calling cpu

[Kevin Cernekee]

From: Maksim Rayskiy <maksim.rayskiy@gmail.com>

When hotplug removing a cpu, all mm context TLB entries must be cleared
to avoid ASID conflict when cpu is restarted.  New functions
local_flush_tlb_all_mm() and all-cpu version flush_tlb_all_mm() are
added.  To function properly, local_flush_tlb_all_mm() must be called
when mm_cpumask for all mm context on given cpu is cleared.

Signed-off-by: Maksim Rayskiy <maksim.rayskiy@gmail.com>
Tested-by: Kevin Cernekee <cernekee@gmail.com>
---
 arch/mips/include/asm/tlbflush.h |    4 ++++
 arch/mips/kernel/smp.c           |   10 ++++++++++
 arch/mips/mm/tlb-r4k.c           |   13 +++++++++++++
 3 files changed, 27 insertions(+), 0 deletions(-)

diff --git a/arch/mips/include/asm/tlbflush.h b/arch/mips/include/asm/tlbflush.h
index 86b21de..d7b75e6 100644
--- a/arch/mips/include/asm/tlbflush.h
+++ b/arch/mips/include/asm/tlbflush.h
@@ -8,12 +8,14 @@
  *
  *  - flush_tlb_all() flushes all processes TLB entries
  *  - flush_tlb_mm(mm) flushes the specified mm context TLB entries
+ *  - flush_tlb_all_mm() flushes all mm context TLB entries
  *  - flush_tlb_page(vma, vmaddr) flushes one page
  *  - flush_tlb_range(vma, start, end) flushes a range of pages
  *  - flush_tlb_kernel_range(start, end) flushes a range of kernel pages
  */
 extern void local_flush_tlb_all(void);
 extern void local_flush_tlb_mm(struct mm_struct *mm);
+extern void local_flush_tlb_all_mm(void);
 extern void local_flush_tlb_range(struct vm_area_struct *vma,
 	unsigned long start, unsigned long end);
 extern void local_flush_tlb_kernel_range(unsigned long start,
@@ -26,6 +28,7 @@ extern void local_flush_tlb_one(unsigned long vaddr);
 
 extern void flush_tlb_all(void);
 extern void flush_tlb_mm(struct mm_struct *);
+extern void flush_tlb_all_mm(void);
 extern void flush_tlb_range(struct vm_area_struct *vma, unsigned long,
 	unsigned long);
 extern void flush_tlb_kernel_range(unsigned long, unsigned long);
@@ -36,6 +39,7 @@ extern void flush_tlb_one(unsigned long vaddr);
 
 #define flush_tlb_all()			local_flush_tlb_all()
 #define flush_tlb_mm(mm)		local_flush_tlb_mm(mm)
+#define flush_tlb_all_mm()		local_flush_tlb_all_mm()
 #define flush_tlb_range(vma, vmaddr, end)	local_flush_tlb_range(vma, vmaddr, end)
 #define flush_tlb_kernel_range(vmaddr,end) \
 	local_flush_tlb_kernel_range(vmaddr, end)
diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c
index 32c1e95..78ca04c 100644
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -269,6 +269,16 @@ void flush_tlb_all(void)
 	on_each_cpu(flush_tlb_all_ipi, NULL, 1);
 }
 
+static void flush_tlb_all_mm_ipi(void *info)
+{
+	local_flush_tlb_all_mm();
+}
+
+void flush_tlb_all_mm(void)
+{
+	on_each_cpu(flush_tlb_all_mm_ipi, NULL, 1);
+}
+
 static void flush_tlb_mm_ipi(void *mm)
 {
 	local_flush_tlb_mm((struct mm_struct *)mm);
diff --git a/arch/mips/mm/tlb-r4k.c b/arch/mips/mm/tlb-r4k.c
index 0d394e0..908fbe9 100644
--- a/arch/mips/mm/tlb-r4k.c
+++ b/arch/mips/mm/tlb-r4k.c
@@ -66,6 +66,19 @@ extern void build_tlb_refill_handler(void);
 
 #endif
 
+/* This function will clear all mm contexts on calling cpu
+ * To produce desired effect it must be called
+ * when mm_cpumask for all mm contexts is cleared
+ */
+void local_flush_tlb_all_mm(void)
+{
+	struct task_struct *p;
+
+	for_each_process(p)
+		if (p->mm)
+			local_flush_tlb_mm(p->mm);
+}
+
 void local_flush_tlb_all(void)
 {
 	unsigned long flags;
-- 
1.7.6.3

[PATCH 2/9] MIPS: BMIPS: Fix up Kconfig settings

[Kevin Cernekee]

Factor out common BMIPS options into "CPU_BMIPS".  Add L2 cache for
BMIPS5000.  Add CPU_MIPS32 to satisfy checks in page.h, r4k_switch.S,
tlb-r4k.c, etc.

Signed-off-by: Kevin Cernekee <cernekee@gmail.com>
---
 arch/mips/Kconfig |   34 ++++++++++++++--------------------
 1 files changed, 14 insertions(+), 20 deletions(-)

diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
index 62b9677..8b231ba 100644
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -1417,51 +1417,36 @@ config CPU_CAVIUM_OCTEON
 config CPU_BMIPS3300
 	bool "BMIPS3300"
 	depends on SYS_HAS_CPU_BMIPS3300
-	select DMA_NONCOHERENT
-	select IRQ_CPU
-	select SWAP_IO_SPACE
-	select SYS_SUPPORTS_32BIT_KERNEL
-	select WEAK_ORDERING
+	select CPU_BMIPS
 	help
 	  Broadcom BMIPS3300 processors.
 
 config CPU_BMIPS4350
 	bool "BMIPS4350"
 	depends on SYS_HAS_CPU_BMIPS4350
-	select CPU_SUPPORTS_32BIT_KERNEL
-	select DMA_NONCOHERENT
-	select IRQ_CPU
-	select SWAP_IO_SPACE
+	select CPU_BMIPS
 	select SYS_SUPPORTS_SMP
 	select SYS_SUPPORTS_HOTPLUG_CPU
-	select WEAK_ORDERING
 	help
 	  Broadcom BMIPS4350 ("VIPER") processors.
 
 config CPU_BMIPS4380
 	bool "BMIPS4380"
 	depends on SYS_HAS_CPU_BMIPS4380
-	select CPU_SUPPORTS_32BIT_KERNEL
-	select DMA_NONCOHERENT
-	select IRQ_CPU
-	select SWAP_IO_SPACE
+	select CPU_BMIPS
 	select SYS_SUPPORTS_SMP
 	select SYS_SUPPORTS_HOTPLUG_CPU
-	select WEAK_ORDERING
 	help
 	  Broadcom BMIPS4380 processors.
 
 config CPU_BMIPS5000
 	bool "BMIPS5000"
 	depends on SYS_HAS_CPU_BMIPS5000
-	select CPU_SUPPORTS_32BIT_KERNEL
+	select CPU_BMIPS
 	select CPU_SUPPORTS_HIGHMEM
-	select DMA_NONCOHERENT
-	select IRQ_CPU
-	select SWAP_IO_SPACE
+	select MIPS_CPU_SCACHE
 	select SYS_SUPPORTS_SMP
 	select SYS_SUPPORTS_HOTPLUG_CPU
-	select WEAK_ORDERING
 	help
 	  Broadcom BMIPS5000 processors.
 
@@ -1522,6 +1507,15 @@ config CPU_LOONGSON2
 	select CPU_SUPPORTS_64BIT_KERNEL
 	select CPU_SUPPORTS_HIGHMEM
 
+config CPU_BMIPS
+	bool
+	select CPU_MIPS32
+	select CPU_SUPPORTS_32BIT_KERNEL
+	select DMA_NONCOHERENT
+	select IRQ_CPU
+	select SWAP_IO_SPACE
+	select WEAK_ORDERING
+
 config SYS_HAS_CPU_LOONGSON2E
 	bool
 
-- 
1.7.6.3

[PATCH 3/9] MIPS: BMIPS: Add XKS01 feature flag to Kconfig

[Kevin Cernekee]

Many BMIPS processors have the ability to extend kseg0 to 1024MB in
order to reclaim large amounts of kernel virtual address space.

Signed-off-by: Kevin Cernekee <cernekee@gmail.com>
---
 arch/mips/Kconfig |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
index 8b231ba..2983b5f 100644
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -1990,6 +1990,9 @@ config CPU_HAS_SMARTMIPS
 config CPU_HAS_WB
 	bool
 
+config XKS01
+	bool
+
 #
 # Vectored interrupt mode is an R2 feature
 #
-- 
1.7.6.3

Re: [PATCH 3/9] MIPS: BMIPS: Add XKS01 feature flag to Kconfig

[Ralf Baechle]

On Sat, Nov 05, 2011 at 02:21:12PM -0700, Kevin Cernekee wrote:

I suggest you fold this patch into patch 9/9.

  Ralf

[PATCH 4/9] MIPS: Clean up whitespace warning in hazards.h

[Kevin Cernekee]

Use a tab on second and subsequent lines of multiline #if's, for
consistency with the next commit.

Signed-off-by: Kevin Cernekee <cernekee@gmail.com>
---
 arch/mips/include/asm/hazards.h |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/mips/include/asm/hazards.h b/arch/mips/include/asm/hazards.h
index 4e33216..8f630e4 100644
--- a/arch/mips/include/asm/hazards.h
+++ b/arch/mips/include/asm/hazards.h
@@ -139,8 +139,8 @@ do {									\
 } while (0)
 
 #elif defined(CONFIG_MIPS_ALCHEMY) || defined(CONFIG_CPU_CAVIUM_OCTEON) || \
-      defined(CONFIG_CPU_LOONGSON2) || defined(CONFIG_CPU_R10000) || \
-      defined(CONFIG_CPU_R5500)
+	defined(CONFIG_CPU_LOONGSON2) || defined(CONFIG_CPU_R10000) || \
+	defined(CONFIG_CPU_R5500)
 
 /*
  * R10000 rocks - all hazards handled in hardware, so this becomes a nobrainer.
-- 
1.7.6.3

[PATCH 5/9] MIPS: BMIPS: Add CFLAGS, Makefile entries for BMIPS

[Kevin Cernekee]

Add CONFIG_CPU_BMIPS* in all of the right places, so that BMIPS kernel
images will compile and run.

Signed-off-by: Kevin Cernekee <cernekee@gmail.com>
---
 arch/mips/Makefile              |    1 +
 arch/mips/include/asm/hazards.h |    3 ++-
 arch/mips/include/asm/module.h  |    2 ++
 3 files changed, 5 insertions(+), 1 deletions(-)

diff --git a/arch/mips/Makefile b/arch/mips/Makefile
index 9b4cb00..ae5cfa7 100644
--- a/arch/mips/Makefile
+++ b/arch/mips/Makefile
@@ -157,6 +157,7 @@ ifeq (,$(findstring march=octeon, $(cflags-$(CONFIG_CPU_CAVIUM_OCTEON))))
 cflags-$(CONFIG_CPU_CAVIUM_OCTEON) += -Wa,-march=octeon
 endif
 cflags-$(CONFIG_CAVIUM_CN63XXP1) += -Wa,-mfix-cn63xxp1
+cflags-$(CONFIG_CPU_BMIPS)	+= -march=mips32 -Wa,-mips32 -Wa,--trap
 
 cflags-$(CONFIG_CPU_R4000_WORKAROUNDS)	+= $(call cc-option,-mfix-r4000,)
 cflags-$(CONFIG_CPU_R4400_WORKAROUNDS)	+= $(call cc-option,-mfix-r4400,)
diff --git a/arch/mips/include/asm/hazards.h b/arch/mips/include/asm/hazards.h
index 8f630e4..b4c20e4 100644
--- a/arch/mips/include/asm/hazards.h
+++ b/arch/mips/include/asm/hazards.h
@@ -87,7 +87,8 @@ do {									\
 	: "=r" (tmp));							\
 } while (0)
 
-#elif defined(CONFIG_CPU_MIPSR1) && !defined(CONFIG_MIPS_ALCHEMY)
+#elif (defined(CONFIG_CPU_MIPSR1) && !defined(CONFIG_MIPS_ALCHEMY)) || \
+	defined(CONFIG_CPU_BMIPS)
 
 /*
  * These are slightly complicated by the fact that we guarantee R1 kernels to
diff --git a/arch/mips/include/asm/module.h b/arch/mips/include/asm/module.h
index bc01a02..c392df2 100644
--- a/arch/mips/include/asm/module.h
+++ b/arch/mips/include/asm/module.h
@@ -120,6 +120,8 @@ search_module_dbetables(unsigned long addr)
 #define MODULE_PROC_FAMILY "OCTEON "
 #elif defined CONFIG_CPU_XLR
 #define MODULE_PROC_FAMILY "XLR "
+#elif defined CONFIG_CPU_BMIPS
+#define MODULE_PROC_FAMILY "BMIPS "
 #else
 #error MODULE_PROC_FAMILY undefined for your processor configuration
 #endif
-- 
1.7.6.3

[PATCH 6/9] MIPS: BMIPS: Add set/clear CP0 macros for BMIPS operations

[Kevin Cernekee]

Several BMIPS-specific CP0 registers are used for SMP boot and other
operations.

Signed-off-by: Kevin Cernekee <cernekee@gmail.com>
---
 arch/mips/include/asm/mipsregs.h |    9 ++++++++-
 1 files changed, 8 insertions(+), 1 deletions(-)

diff --git a/arch/mips/include/asm/mipsregs.h b/arch/mips/include/asm/mipsregs.h
index 2ea7b81..7f87d82 100644
--- a/arch/mips/include/asm/mipsregs.h
+++ b/arch/mips/include/asm/mipsregs.h
@@ -1106,7 +1106,7 @@ do {									\
 #define read_c0_brcm_reset()		__read_32bit_c0_register($22, 5)
 #define write_c0_brcm_reset(val)	__write_32bit_c0_register($22, 5, val)
 
-/* BMIPS4380 */
+/* BMIPS43xx */
 #define read_c0_brcm_cmt_intr()		__read_32bit_c0_register($22, 1)
 #define write_c0_brcm_cmt_intr(val)	__write_32bit_c0_register($22, 1, val)
 
@@ -1667,6 +1667,13 @@ __BUILD_SET_C0(config)
 __BUILD_SET_C0(intcontrol)
 __BUILD_SET_C0(intctl)
 __BUILD_SET_C0(srsmap)
+__BUILD_SET_C0(brcm_config_0)
+__BUILD_SET_C0(brcm_bus_pll)
+__BUILD_SET_C0(brcm_reset)
+__BUILD_SET_C0(brcm_cmt_intr)
+__BUILD_SET_C0(brcm_cmt_ctrl)
+__BUILD_SET_C0(brcm_config)
+__BUILD_SET_C0(brcm_mode)
 
 #endif /* !__ASSEMBLY__ */
 
-- 
1.7.6.3

[PATCH 7/9] MIPS: BMIPS: Introduce bmips.h

[Kevin Cernekee]

bmips.h contains BMIPS definitions that are useful for SMP, vector
relocation, performance counters, etc.

Signed-off-by: Kevin Cernekee <cernekee@gmail.com>
---
 arch/mips/include/asm/bmips.h |   95 +++++++++++++++++++++++++++++++++++++++++
 1 files changed, 95 insertions(+), 0 deletions(-)
 create mode 100644 arch/mips/include/asm/bmips.h

diff --git a/arch/mips/include/asm/bmips.h b/arch/mips/include/asm/bmips.h
new file mode 100644
index 0000000..1d9130a
--- /dev/null
+++ b/arch/mips/include/asm/bmips.h
@@ -0,0 +1,95 @@
+/*
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License.  See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * Copyright (C) 2011 by Kevin Cernekee (cernekee@gmail.com)
+ *
+ * Definitions for BMIPS processors
+ */
+#ifndef _ASM_BMIPS_H
+#define _ASM_BMIPS_H
+
+#include <linux/compiler.h>
+#include <linux/linkage.h>
+#include <asm/addrspace.h>
+#include <asm/mipsregs.h>
+
+/* NOTE: the CBR register returns a PA, and it can be above 0xff00_0000 */
+#define BMIPS_GET_CBR()			((void __iomem *)(CKSEG1 | \
+					 (unsigned long) \
+					 ((read_c0_brcm_cbr() >> 18) << 18)))
+
+#define BMIPS_RAC_CONFIG		0x00000000
+#define BMIPS_RAC_ADDRESS_RANGE		0x00000004
+#define BMIPS_RAC_CONFIG_1		0x00000008
+#define BMIPS_L2_CONFIG			0x0000000c
+#define BMIPS_LMB_CONTROL		0x0000001c
+#define BMIPS_SYSTEM_BASE		0x00000020
+#define BMIPS_PERF_GLOBAL_CONTROL	0x00020000
+#define BMIPS_PERF_CONTROL_0		0x00020004
+#define BMIPS_PERF_CONTROL_1		0x00020008
+#define BMIPS_PERF_COUNTER_0		0x00020010
+#define BMIPS_PERF_COUNTER_1		0x00020014
+#define BMIPS_PERF_COUNTER_2		0x00020018
+#define BMIPS_PERF_COUNTER_3		0x0002001c
+#define BMIPS_RELO_VECTOR_CONTROL_0	0x00030000
+#define BMIPS_RELO_VECTOR_CONTROL_1	0x00038000
+
+#define BMIPS_NMI_RESET_VEC		0x80000000
+#define BMIPS_WARM_RESTART_VEC		0x80000380
+
+#define ZSCM_REG_BASE			0x97000000
+
+#if !defined(__ASSEMBLY__)
+
+#include <linux/cpumask.h>
+#include <asm/r4kcache.h>
+
+extern struct plat_smp_ops bmips_smp_ops;
+extern char bmips_reset_nmi_vec;
+extern char bmips_reset_nmi_vec_end;
+extern char bmips_smp_movevec;
+extern char bmips_smp_int_vec;
+extern char bmips_smp_int_vec_end;
+extern int bmips_smp_enabled;
+extern cpumask_t bmips_booted_mask;
+
+extern void bmips_ebase_setup(void);
+extern asmlinkage void plat_wired_tlb_setup(void);
+
+static inline unsigned long bmips_read_zscm_reg(unsigned int offset)
+{
+	unsigned long ret;
+
+	cache_op(Index_Load_Tag_S, ZSCM_REG_BASE + offset);
+
+	__asm__ __volatile__(
+		".set push\n"
+		".set noreorder\n"
+		"sync\n"
+		"ssnop\n"
+		"ssnop\n"
+		"ssnop\n"
+		"ssnop\n"
+		"ssnop\n"
+		"ssnop\n"
+		"ssnop\n"
+		"mfc0 %0, $28, 3\n"
+		"ssnop\n"
+		".set pop\n"
+		: "=&r" (ret) : : "memory");
+	return ret;
+}
+
+static inline void bmips_write_zscm_reg(unsigned int offset, unsigned long data)
+{
+	__write_32bit_c0_register($28, 3, data);
+	back_to_back_c0_hazard();
+	cache_op(Index_Store_Tag_S, ZSCM_REG_BASE + offset);
+	back_to_back_c0_hazard();
+}
+
+#endif /* !defined(__ASSEMBLY__) */
+
+#endif /* _ASM_BMIPS_H */
-- 
1.7.6.3

Re: [PATCH 7/9] MIPS: BMIPS: Introduce bmips.h

[Ralf Baechle]

On Sat, Nov 05, 2011 at 02:21:16PM -0700, Kevin Cernekee wrote:

> +static inline unsigned long bmips_read_zscm_reg(unsigned int offset)
> +{
> +	unsigned long ret;
> +
> +	cache_op(Index_Load_Tag_S, ZSCM_REG_BASE + offset);
> +
> +	__asm__ __volatile__(
> +		".set push\n"
> +		".set noreorder\n"
> +		"sync\n"
> +		"ssnop\n"
> +		"ssnop\n"
> +		"ssnop\n"
> +		"ssnop\n"
> +		"ssnop\n"
> +		"ssnop\n"
> +		"ssnop\n"
> +		"mfc0 %0, $28, 3\n"
> +		"ssnop\n"
> +		".set pop\n"
> +		: "=&r" (ret) : : "memory");

Is it critical that the C compiler or assembler can't reorder anything to
in between the cache_op and the inline asm statement?  If so, I'd
recommend to put the cache instruction into the asm as well.

> +static inline void bmips_write_zscm_reg(unsigned int offset, unsigned long data)
> +{
> +	__write_32bit_c0_register($28, 3, data);
> +	back_to_back_c0_hazard();
> +	cache_op(Index_Store_Tag_S, ZSCM_REG_BASE + offset);
> +	back_to_back_c0_hazard();

back_to_back_c0_hazard() is meant as the hazard barrier between a write
followed immediately by a read from the same cp0 register:

	mtc0	$reg1, $12
	mfc0	$reg2, $12

On various MIPS processors this instruction sequence would result in
undefined operation such as the mfc0 instruction reading the value of
$12 before the mtc0 or even next week's lucky lottery numbers..

I think we don't really have a type of hazard barrier defined in hazard.h
and this seems a rather special purpose use so I suggest you just open
code whatever needs to be done.

  Ralf

[PATCH 8/9] MIPS: Add board_* hooks for ebase and NMI

[Kevin Cernekee]

Certain BMIPS platforms need to move the exception vectors and/or
intercept NMI events.  Add hooks to make this possible.

Signed-off-by: Kevin Cernekee <cernekee@gmail.com>
---
 arch/mips/include/asm/traps.h |    2 ++
 arch/mips/kernel/traps.c      |    6 ++++++
 2 files changed, 8 insertions(+), 0 deletions(-)

diff --git a/arch/mips/include/asm/traps.h b/arch/mips/include/asm/traps.h
index 90ff2f4..2954bd9 100644
--- a/arch/mips/include/asm/traps.h
+++ b/arch/mips/include/asm/traps.h
@@ -22,7 +22,9 @@ extern void (*board_be_init)(void);
 extern int (*board_be_handler)(struct pt_regs *regs, int is_fixup);
 
 extern void (*board_nmi_handler_setup)(void);
+extern void (*board_nmi_handler)(struct pt_regs *regs);
 extern void (*board_ejtag_handler_setup)(void);
 extern void (*board_bind_eic_interrupt)(int irq, int regset);
+extern void (*board_ebase_setup)(void);
 
 #endif /* _ASM_TRAPS_H */
diff --git a/arch/mips/kernel/traps.c b/arch/mips/kernel/traps.c
index cbea618..f98349c 100644
--- a/arch/mips/kernel/traps.c
+++ b/arch/mips/kernel/traps.c
@@ -90,8 +90,10 @@ extern int fpu_emulator_cop1Handler(struct pt_regs *xcp,
 void (*board_be_init)(void);
 int (*board_be_handler)(struct pt_regs *regs, int is_fixup);
 void (*board_nmi_handler_setup)(void);
+void (*board_nmi_handler)(struct pt_regs *regs);
 void (*board_ejtag_handler_setup)(void);
 void (*board_bind_eic_interrupt)(int irq, int regset);
+void (*board_ebase_setup)(void);
 
 
 static void show_raw_backtrace(unsigned long reg29)
@@ -1343,6 +1345,8 @@ void ejtag_exception_handler(struct pt_regs *regs)
  */
 NORET_TYPE void ATTRIB_NORET nmi_exception_handler(struct pt_regs *regs)
 {
+	if (board_nmi_handler)
+		board_nmi_handler(regs);
 	bust_spinlocks(1);
 	printk("NMI taken!!!!\n");
 	die("NMI", regs);
@@ -1682,6 +1686,8 @@ void __init trap_init(void)
 			ebase += (read_c0_ebase() & 0x3ffff000);
 	}
 
+	if (board_ebase_setup)
+		board_ebase_setup();
 	per_cpu_trap_init();
 
 	/*
-- 
1.7.6.3

Re: [PATCH 8/9] MIPS: Add board_* hooks for ebase and NMI

[Ralf Baechle]

On Sat, Nov 05, 2011 at 02:21:17PM -0700, Kevin Cernekee wrote:

> Certain BMIPS platforms need to move the exception vectors and/or
> intercept NMI events.  Add hooks to make this possible.

There are other potencial users of NMI hooks and exporting random
function pointers has never been terribly stylish.  I instead suggest
to use notifiers, see <linux/notifiers.h>.

Due to the locking constraints in an NMI board_nmi_handler should then
become a raw notifier and the board_ebase_setup() function could
just go away.

  Ralf

[PATCH 9/9] MIPS: BMIPS: Add SMP support code for BMIPS43xx/BMIPS5000

[Kevin Cernekee]

Initial commit of BMIPS SMP support code.  Smoke-tested on a variety of
BMIPS4350, BMIPS4380, and BMIPS5000 platforms.

Signed-off-by: Kevin Cernekee <cernekee@gmail.com>
---
 arch/mips/kernel/Makefile    |    1 +
 arch/mips/kernel/bmips_vec.S |  273 +++++++++++++++++++++++++
 arch/mips/kernel/smp-bmips.c |  458 ++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 732 insertions(+), 0 deletions(-)
 create mode 100644 arch/mips/kernel/bmips_vec.S
 create mode 100644 arch/mips/kernel/smp-bmips.c

diff --git a/arch/mips/kernel/Makefile b/arch/mips/kernel/Makefile
index 1a96618..0198321 100644
--- a/arch/mips/kernel/Makefile
+++ b/arch/mips/kernel/Makefile
@@ -58,6 +58,7 @@ obj-$(CONFIG_CPU_XLR)		+= r4k_fpu.o r4k_switch.o
 
 obj-$(CONFIG_SMP)		+= smp.o
 obj-$(CONFIG_SMP_UP)		+= smp-up.o
+obj-$(CONFIG_CPU_BMIPS)		+= smp-bmips.o bmips_vec.o
 
 obj-$(CONFIG_MIPS_MT)		+= mips-mt.o
 obj-$(CONFIG_MIPS_MT_FPAFF)	+= mips-mt-fpaff.o
diff --git a/arch/mips/kernel/bmips_vec.S b/arch/mips/kernel/bmips_vec.S
new file mode 100644
index 0000000..ddabc82d
--- /dev/null
+++ b/arch/mips/kernel/bmips_vec.S
@@ -0,0 +1,273 @@
+/*
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License.  See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * Copyright (C) 2011 by Kevin Cernekee (cernekee@gmail.com)
+ *
+ * Reset/NMI/re-entry vectors for BMIPS processors
+ */
+
+#include <linux/init.h>
+
+#include <asm/asm.h>
+#include <asm/asmmacro.h>
+#include <asm/cacheops.h>
+#include <asm/regdef.h>
+#include <asm/mipsregs.h>
+#include <asm/stackframe.h>
+#include <asm/addrspace.h>
+#include <asm/bmips.h>
+
+	.macro	CLR_FPR a b c d
+	mtc1	$0, $\a
+	mtc1	$0, $\b
+	mtc1	$0, $\c
+	mtc1	$0, $\d
+	.endm
+
+	.macro	BARRIER
+	.set	mips32
+	ssnop
+	ssnop
+	ssnop
+	.set	mips0
+	.endm
+
+	__CPUINIT
+
+/***********************************************************************
+ * Alternate CPU1 startup vector for BMIPS4350
+ *
+ * On some systems the bootloader has already started CPU1 and configured
+ * it to resume execution at 0x8000_0200 (!BEV IV vector) when it is
+ * triggered by the SW1 interrupt.  If that is the case we try to move
+ * it to a more convenient place: BMIPS_WARM_RESTART_VEC @ 0x8000_0380.
+ ***********************************************************************/
+
+LEAF(bmips_smp_movevec)
+	la	k0, 1f
+	li	k1, CKSEG1
+	or	k0, k1
+	jr	k0
+
+1:
+	/* clear IV, pending IPIs */
+	mtc0	zero, CP0_CAUSE
+
+	/* re-enable IRQs to wait for SW1 */
+	li	k0, ST0_IE | ST0_BEV | STATUSF_IP1
+	mtc0	k0, CP0_STATUS
+
+	/* set up CPU1 CBR; move BASE to 0xa000_0000 */
+	li	k0, 0xff400000
+	mtc0	k0, $22, 6
+	li	k1, CKSEG1 | BMIPS_RELO_VECTOR_CONTROL_1
+	or	k0, k1
+	li	k1, 0xa0080000
+	sw	k1, 0(k0)
+
+	/* wait here for SW1 interrupt from bmips_boot_secondary() */
+	wait
+
+	la	k0, bmips_reset_nmi_vec
+	li	k1, CKSEG1
+	or	k0, k1
+	jr	k0
+END(bmips_smp_movevec)
+
+/***********************************************************************
+ * Reset/NMI vector
+ * For BMIPS processors that can relocate their exception vectors, this
+ * entire function gets copied to 0x8000_0000.
+ ***********************************************************************/
+
+NESTED(bmips_reset_nmi_vec, PT_SIZE, sp)
+	.set	push
+	.set	noat
+	.align	4
+
+#ifdef CONFIG_SMP
+	/* if the NMI bit is clear, assume this is a CPU1 reset instead */
+	li	k1, (1 << 19)
+	mfc0	k0, CP0_STATUS
+	and	k0, k1
+	beqz	k0, bmips_smp_entry
+
+#if defined(CONFIG_CPU_BMIPS5000)
+	/* if we're not on core 0, this must be the SMP boot signal */
+	li	k1, (3 << 25)
+	mfc0	k0, $22
+	and	k0, k1
+	bnez	k0, bmips_smp_entry
+#endif
+#endif /* CONFIG_SMP */
+
+	/* nope, it's just a regular NMI */
+	SAVE_ALL
+	move	a0, sp
+
+	/* clear EXL, ERL, BEV so that TLB refills still work */
+	mfc0	k0, CP0_STATUS
+	li	k1, ST0_ERL | ST0_EXL | ST0_BEV | ST0_IE
+	or	k0, k1
+	xor	k0, k1
+	mtc0	k0, CP0_STATUS
+	BARRIER
+
+	/* call the NMI handler function (probably doesn't return) */
+	la	k0, nmi_exception_handler
+	jalr	k0
+
+	RESTORE_ALL
+	.set	mips3
+	eret
+
+/***********************************************************************
+ * CPU1 reset vector (used for the initial boot only)
+ * This is still part of bmips_reset_nmi_vec().
+ ***********************************************************************/
+
+#ifdef CONFIG_SMP
+
+bmips_smp_entry:
+
+	/* set up CP0 STATUS; enable FPU */
+	li	k0, 0x30000000
+	mtc0	k0, CP0_STATUS
+	BARRIER
+
+	/* set local CP0 CONFIG to make kseg0 cacheable, write-back */
+	mfc0	k0, CP0_CONFIG
+	ori	k0, 0x07
+	xori	k0, 0x04
+	mtc0	k0, CP0_CONFIG
+
+#if defined(CONFIG_CPU_BMIPS4380)
+	/* initialize FPU registers */
+	CLR_FPR	f0 f1 f2 f3
+	CLR_FPR	f4 f5 f6 f7
+	CLR_FPR	f8 f9 f10 f11
+	CLR_FPR	f12 f13 f14 f15
+	CLR_FPR	f16 f17 f18 f19
+	CLR_FPR	f20 f21 f22 f23
+	CLR_FPR	f24 f25 f26 f27
+	CLR_FPR	f28 f29 f30 f31
+#endif
+
+#if defined(CONFIG_CPU_BMIPS4350) || defined(CONFIG_CPU_BMIPS4380)
+	/* initialize CPU1's local I-cache */
+	li	k0, 0x80000000
+	li	k1, 0x80010000
+	mtc0	zero, $28
+	mtc0	zero, $28, 1
+	BARRIER
+
+1:	cache	Index_Store_Tag_I, 0(k0)
+	addiu	k0, 16
+	bne	k0, k1, 1b
+#elif defined(CONFIG_CPU_BMIPS5000)
+	/* set exception vector base */
+	la	k0, ebase
+	lw	k0, 0(k0)
+	mtc0	k0, $15, 1
+	BARRIER
+#endif
+
+	/* jump back to kseg0 in case we need to remap the kseg1 area */
+	la	k0, 1f
+	jr	k0
+1:
+	la	k0, bmips_enable_xks01
+	jalr	k0
+
+	/* use temporary stack to set up upper memory TLB */
+	li	sp, BMIPS_WARM_RESTART_VEC
+	la	k0, plat_wired_tlb_setup
+	jalr	k0
+
+	/* switch to permanent stack and continue booting */
+
+	.global	bmips_secondary_reentry
+bmips_secondary_reentry:
+	la	k0, bmips_smp_boot_sp
+	lw	sp, 0(k0)
+	la	k0, bmips_smp_boot_gp
+	lw	gp, 0(k0)
+	la	k0, start_secondary
+	jr	k0
+
+#endif /* CONFIG_SMP */
+
+	.align	4
+	.global	bmips_reset_nmi_vec_end
+bmips_reset_nmi_vec_end:
+
+END(bmips_reset_nmi_vec)
+
+	.set	pop
+	.previous
+
+/***********************************************************************
+ * CPU1 warm restart vector (used for second and subsequent boots).
+ * Also used for S2 standby recovery (PM).
+ * This entire function gets copied to (BMIPS_WARM_RESTART_VEC)
+ ***********************************************************************/
+
+LEAF(bmips_smp_int_vec)
+
+	.align	4
+	mfc0	k0, CP0_STATUS
+	ori	k0, 0x01
+	xori	k0, 0x01
+	mtc0	k0, CP0_STATUS
+	eret
+
+	.align	4
+	.global	bmips_smp_int_vec_end
+bmips_smp_int_vec_end:
+
+END(bmips_smp_int_vec)
+
+/***********************************************************************
+ * XKS01 support
+ * Certain CPUs support extending kseg0 to 1024MB.
+ ***********************************************************************/
+
+	__CPUINIT
+
+LEAF(bmips_enable_xks01)
+
+#if defined(CONFIG_XKS01)
+
+#if defined(CONFIG_CPU_BMIPS4380)
+	mfc0	t0, $22, 3
+	li	t1, 0x1ff0
+	li	t2, (1 << 12) | (1 << 9)
+	or	t0, t1
+	xor	t0, t1
+	or	t0, t2
+	mtc0	t0, $22, 3
+	BARRIER
+#elif defined(CONFIG_CPU_BMIPS5000)
+	mfc0	t0, $22, 5
+	li	t1, 0x01ff
+	li	t2, (1 << 8) | (1 << 5)
+	or	t0, t1
+	xor	t0, t1
+	or	t0, t2
+	mtc0	t0, $22, 5
+	BARRIER
+#else
+
+#error Missing XKS01 setup
+
+#endif
+
+#endif /* defined(CONFIG_XKS01) */
+
+	jr	ra
+
+END(bmips_enable_xks01)
+
+	.previous
diff --git a/arch/mips/kernel/smp-bmips.c b/arch/mips/kernel/smp-bmips.c
new file mode 100644
index 0000000..31c46ae
--- /dev/null
+++ b/arch/mips/kernel/smp-bmips.c
@@ -0,0 +1,458 @@
+/*
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License.  See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * Copyright (C) 2011 by Kevin Cernekee (cernekee@gmail.com)
+ *
+ * SMP support for BMIPS
+ */
+
+#include <linux/version.h>
+#include <linux/init.h>
+#include <linux/sched.h>
+#include <linux/mm.h>
+#include <linux/delay.h>
+#include <linux/smp.h>
+#include <linux/interrupt.h>
+#include <linux/spinlock.h>
+#include <linux/init.h>
+#include <linux/cpu.h>
+#include <linux/cpumask.h>
+#include <linux/reboot.h>
+#include <linux/io.h>
+#include <linux/compiler.h>
+#include <linux/linkage.h>
+#include <linux/bug.h>
+
+#include <asm/time.h>
+#include <asm/pgtable.h>
+#include <asm/processor.h>
+#include <asm/system.h>
+#include <asm/bootinfo.h>
+#include <asm/pmon.h>
+#include <asm/cacheflush.h>
+#include <asm/tlbflush.h>
+#include <asm/mipsregs.h>
+#include <asm/bmips.h>
+#include <asm/traps.h>
+#include <asm/barrier.h>
+
+static int __maybe_unused max_cpus = 1;
+
+cpumask_t bmips_booted_mask;
+
+#ifdef CONFIG_SMP
+
+/* initial $sp, $gp - used by arch/mips/kernel/bmips_vec.S */
+unsigned long bmips_smp_boot_sp;
+unsigned long bmips_smp_boot_gp;
+
+static void bmips_send_ipi_single(int cpu, unsigned int action);
+static irqreturn_t bmips_ipi_interrupt(int irq, void *dev_id);
+
+/* the platform code may forcibly disable SMP */
+int bmips_smp_enabled = 1;
+
+/* SW interrupts 0,1 are used for interprocessor signaling */
+#define IPI0_IRQ			(MIPS_CPU_IRQ_BASE + 0)
+#define IPI1_IRQ			(MIPS_CPU_IRQ_BASE + 1)
+
+#define ACTION_CLR_IPI(cpu, ipi)	(0x2000 | ((cpu) << 9) | ((ipi) << 8))
+#define ACTION_SET_IPI(cpu, ipi)	(0x3000 | ((cpu) << 9) | ((ipi) << 8))
+#define ACTION_BOOT_THREAD(cpu)		(0x08 | (cpu))
+
+static void __init bmips_smp_setup(void)
+{
+	int i;
+
+#if defined(CONFIG_CPU_BMIPS4350) || defined(CONFIG_CPU_BMIPS4380)
+	/* arbitration priority */
+	clear_c0_brcm_cmt_ctrl(0x30);
+
+	/* NBK and weak order flags */
+	set_c0_brcm_config_0(0x30000);
+
+	/*
+	 * MIPS interrupts 0,1 (SW INT 0,1) cross over to the other thread
+	 * MIPS interrupt 2 (HW INT 0) is the CPU0 L1 controller output
+	 * MIPS interrupt 3 (HW INT 1) is the CPU1 L1 controller output
+	 */
+	change_c0_brcm_cmt_intr(0xf8018000,
+		(0x02 << 27) | (0x03 << 15));
+
+	/* single core, 2 threads (2 pipelines) */
+	max_cpus = 2;
+#elif defined(CONFIG_CPU_BMIPS5000)
+	/* enable raceless SW interrupts */
+	set_c0_brcm_config(0x03 << 22);
+
+	/* route HW interrupt 0 to CPU0, HW interrupt 1 to CPU1 */
+	change_c0_brcm_mode(0x1f << 27, 0x02 << 27);
+
+	/* N cores, 2 threads per core */
+	max_cpus = (((read_c0_brcm_config() >> 6) & 0x03) + 1) << 1;
+
+	/* clear any pending SW interrupts */
+	for (i = 0; i < max_cpus; i++) {
+		write_c0_brcm_action(ACTION_CLR_IPI(i, 0));
+		write_c0_brcm_action(ACTION_CLR_IPI(i, 1));
+	}
+#endif
+
+	if (!bmips_smp_enabled)
+		max_cpus = 1;
+
+	/* this can be overridden by the BSP */
+	if (!board_ebase_setup)
+		board_ebase_setup = &bmips_ebase_setup;
+
+	for (i = 0; i < max_cpus; i++) {
+		__cpu_number_map[i] = 1;
+		__cpu_logical_map[i] = 1;
+		set_cpu_possible(i, 1);
+		set_cpu_present(i, 1);
+	}
+}
+
+/*
+ * IPI IRQ setup - runs on CPU0
+ */
+static void bmips_prepare_cpus(unsigned int max_cpus)
+{
+	if (request_irq(IPI0_IRQ, bmips_ipi_interrupt, IRQF_PERCPU,
+			"smp_ipi0", NULL))
+		panic("Can't request IPI0 interrupt\n");
+	if (request_irq(IPI1_IRQ, bmips_ipi_interrupt, IRQF_PERCPU,
+			"smp_ipi1", NULL))
+		panic("Can't request IPI1 interrupt\n");
+}
+
+/*
+ * Tell the hardware to boot CPUx - runs on CPU0
+ */
+static void bmips_boot_secondary(int cpu, struct task_struct *idle)
+{
+	bmips_smp_boot_sp = __KSTK_TOS(idle);
+	bmips_smp_boot_gp = (unsigned long)task_thread_info(idle);
+	mb();
+
+	/*
+	 * Initial boot sequence for secondary CPU:
+	 *   bmips_reset_nmi_vec @ a000_0000 ->
+	 *   bmips_smp_entry ->
+	 *   plat_wired_tlb_setup (cached function call; optional) ->
+	 *   start_secondary (cached jump)
+	 *
+	 * Warm restart sequence:
+	 *   play_dead WAIT loop ->
+	 *   bmips_smp_int_vec @ BMIPS_WARM_RESTART_VEC ->
+	 *   eret to play_dead ->
+	 *   bmips_secondary_reentry ->
+	 *   start_secondary
+	 */
+
+	printk(KERN_INFO "SMP: Booting CPU%d...\n", cpu);
+
+	if (cpumask_test_cpu(cpu, &bmips_booted_mask))
+		bmips_send_ipi_single(cpu, 0);
+	else {
+#if defined(CONFIG_CPU_BMIPS4350) || defined(CONFIG_CPU_BMIPS4380)
+		set_c0_brcm_cmt_ctrl(0x01);
+#elif defined(CONFIG_CPU_BMIPS5000)
+		if (cpu & 0x01)
+			write_c0_brcm_action(ACTION_BOOT_THREAD(cpu));
+		else {
+			/*
+			 * core N thread 0 was already booted; just
+			 * pulse the NMI line
+			 */
+			bmips_write_zscm_reg(0x210, 0xc0000000);
+			udelay(10);
+			bmips_write_zscm_reg(0x210, 0x00);
+		}
+#endif
+		cpumask_set_cpu(cpu, &bmips_booted_mask);
+	}
+}
+
+/*
+ * Early setup - runs on secondary CPU after cache probe
+ */
+static void bmips_init_secondary(void)
+{
+	/* move NMI vector to kseg0, in case XKS01 is enabled */
+
+#if defined(CONFIG_CPU_BMIPS4350) || defined(CONFIG_CPU_BMIPS4380)
+	void __iomem *cbr = BMIPS_GET_CBR();
+	unsigned long old_vec;
+
+	old_vec = __raw_readl(cbr + BMIPS_RELO_VECTOR_CONTROL_1);
+	__raw_writel(old_vec & ~0x20000000, cbr + BMIPS_RELO_VECTOR_CONTROL_1);
+
+	clear_c0_cause(smp_processor_id() ? C_SW1 : C_SW0);
+#elif defined(CONFIG_CPU_BMIPS5000)
+	write_c0_brcm_bootvec(read_c0_brcm_bootvec() &
+		(smp_processor_id() & 0x01 ? ~0x20000000 : ~0x2000));
+
+	write_c0_brcm_action(ACTION_CLR_IPI(smp_processor_id(), 0));
+#endif
+
+	/* make sure there won't be a timer interrupt for a little while */
+	write_c0_compare(read_c0_count() + mips_hpt_frequency / HZ);
+
+	irq_enable_hazard();
+	set_c0_status(IE_SW0 | IE_SW1 | IE_IRQ1 | IE_IRQ5 | ST0_IE);
+	irq_enable_hazard();
+}
+
+/*
+ * Late setup - runs on secondary CPU before entering the idle loop
+ */
+static void bmips_smp_finish(void)
+{
+	printk(KERN_INFO "SMP: CPU%d is running\n", smp_processor_id());
+}
+
+/*
+ * Runs on CPU0 after all CPUs have been booted
+ */
+static void bmips_cpus_done(void)
+{
+}
+
+#if defined(CONFIG_CPU_BMIPS5000)
+
+/*
+ * BMIPS5000 raceless IPIs
+ *
+ * Each CPU has two inbound SW IRQs which are independent of all other CPUs.
+ * IPI0 is used for SMP_RESCHEDULE_YOURSELF
+ * IPI1 is used for SMP_CALL_FUNCTION
+ */
+
+static void bmips_send_ipi_single(int cpu, unsigned int action)
+{
+	write_c0_brcm_action(ACTION_SET_IPI(cpu, action == SMP_CALL_FUNCTION));
+}
+
+static irqreturn_t bmips_ipi_interrupt(int irq, void *dev_id)
+{
+	int action = irq - IPI0_IRQ;
+
+	write_c0_brcm_action(ACTION_CLR_IPI(smp_processor_id(), action));
+
+	if (action == 0)
+		scheduler_ipi();
+	else
+		smp_call_function_interrupt();
+
+	return IRQ_HANDLED;
+}
+
+#else
+
+/*
+ * BMIPS43xx racey IPIs
+ *
+ * We use one inbound SW IRQ for each CPU.
+ *
+ * A spinlock must be held in order to keep CPUx from accidentally clearing
+ * an incoming IPI when it writes CP0 CAUSE to raise an IPI on CPUy.  The
+ * same spinlock is used to protect the action masks.
+ */
+
+static DEFINE_SPINLOCK(ipi_lock);
+static DEFINE_PER_CPU(int, ipi_action_mask);
+
+static void bmips_send_ipi_single(int cpu, unsigned int action)
+{
+	unsigned long flags;
+
+	spin_lock_irqsave(&ipi_lock, flags);
+	set_c0_cause(cpu ? C_SW1 : C_SW0);
+	per_cpu(ipi_action_mask, cpu) |= action;
+	irq_enable_hazard();
+	spin_unlock_irqrestore(&ipi_lock, flags);
+}
+
+static irqreturn_t bmips_ipi_interrupt(int irq, void *dev_id)
+{
+	unsigned long flags;
+	int action, cpu = irq - IPI0_IRQ;
+
+	spin_lock_irqsave(&ipi_lock, flags);
+	action = __get_cpu_var(ipi_action_mask);
+	per_cpu(ipi_action_mask, cpu) = 0;
+	clear_c0_cause(cpu ? C_SW1 : C_SW0);
+	spin_unlock_irqrestore(&ipi_lock, flags);
+
+	if (action & SMP_RESCHEDULE_YOURSELF)
+		scheduler_ipi();
+	if (action & SMP_CALL_FUNCTION)
+		smp_call_function_interrupt();
+
+	return IRQ_HANDLED;
+}
+
+#endif /* BMIPS type */
+
+static void bmips_send_ipi_mask(const struct cpumask *mask,
+	unsigned int action)
+{
+	unsigned int i;
+
+	for_each_cpu(i, mask)
+		bmips_send_ipi_single(i, action);
+}
+
+#ifdef CONFIG_HOTPLUG_CPU
+
+static int bmips_cpu_disable(void)
+{
+	unsigned int cpu = smp_processor_id();
+
+	if (cpu == 0)
+		return -EBUSY;
+
+	printk(KERN_INFO "SMP: CPU%d is offline\n", cpu);
+
+	cpu_clear(cpu, cpu_online_map);
+	cpu_clear(cpu, cpu_callin_map);
+
+	local_flush_tlb_all();
+	local_flush_icache_range(0, ~0);
+
+	return 0;
+}
+
+static void bmips_cpu_die(unsigned int cpu)
+{
+}
+
+void __ref play_dead(void)
+{
+	idle_task_exit();
+
+	/* Drop all mm context TLB entries on this cpu */
+	local_flush_tlb_all_mm();
+	/* flush data cache */
+	_dma_cache_wback_inv(0, ~0);
+
+	/*
+	 * Wakeup is on SW0 or SW1; disable everything else
+	 * Use BEV !IV (BMIPS_WARM_RESTART_VEC) to avoid the regular Linux
+	 * IRQ handlers; this clears ST0_IE and returns immediately.
+	 */
+	clear_c0_cause(CAUSEF_IV | C_SW0 | C_SW1);
+	change_c0_status(IE_IRQ5 | IE_IRQ1 | IE_SW0 | IE_SW1 | ST0_IE | ST0_BEV,
+		IE_SW0 | IE_SW1 | ST0_IE | ST0_BEV);
+	irq_disable_hazard();
+
+	/*
+	 * wait for SW interrupt from bmips_boot_secondary(), then jump
+	 * back to start_secondary()
+	 */
+	__asm__ __volatile__(
+	"	wait\n"
+	"	j	bmips_secondary_reentry\n"
+	: : : "memory");
+}
+
+#endif /* CONFIG_HOTPLUG_CPU */
+
+struct plat_smp_ops bmips_smp_ops = {
+	.smp_setup		= bmips_smp_setup,
+	.prepare_cpus		= bmips_prepare_cpus,
+	.boot_secondary		= bmips_boot_secondary,
+	.smp_finish		= bmips_smp_finish,
+	.init_secondary		= bmips_init_secondary,
+	.cpus_done		= bmips_cpus_done,
+	.send_ipi_single	= bmips_send_ipi_single,
+	.send_ipi_mask		= bmips_send_ipi_mask,
+#ifdef CONFIG_HOTPLUG_CPU
+	.cpu_disable		= bmips_cpu_disable,
+	.cpu_die		= bmips_cpu_die,
+#endif
+};
+
+#endif /* CONFIG_SMP */
+
+/***********************************************************************
+ * BMIPS vector relocation
+ * This is primarily used for SMP boot, but it is applicable to some
+ * UP BMIPS systems as well.
+ ***********************************************************************/
+
+static void __cpuinit bmips_wr_vec(unsigned long dst, char *start, char *end)
+{
+	memcpy((void *)dst, start, end - start);
+	dma_cache_wback((unsigned long)start, end - start);
+	local_flush_icache_range(dst, dst + (end - start));
+	instruction_hazard();
+}
+
+static inline void __cpuinit bmips_nmi_handler_setup(void)
+{
+	bmips_wr_vec(BMIPS_NMI_RESET_VEC, &bmips_reset_nmi_vec,
+		&bmips_reset_nmi_vec_end);
+	bmips_wr_vec(BMIPS_WARM_RESTART_VEC, &bmips_smp_int_vec,
+		&bmips_smp_int_vec_end);
+}
+
+void __cpuinit bmips_ebase_setup(void)
+{
+	unsigned long new_ebase = ebase;
+	void __iomem __maybe_unused *cbr;
+
+	BUG_ON(ebase != CKSEG0);
+
+#if defined(CONFIG_CPU_BMIPS4350)
+	/*
+	 * BMIPS4350 cannot relocate the normal vectors, but it
+	 * can relocate the BEV=1 vectors.  So CPU1 starts up at
+	 * the relocated BEV=1, IV=0 general exception vector @
+	 * 0xa000_0380.
+	 *
+	 * set_uncached_handler() is used here because:
+	 *  - CPU1 will run this from uncached space
+	 *  - None of the cacheflush functions are set up yet
+	 */
+	set_uncached_handler(BMIPS_WARM_RESTART_VEC - CKSEG0,
+		&bmips_smp_int_vec, 0x80);
+	__sync();
+	return;
+#elif defined(CONFIG_CPU_BMIPS4380)
+	/*
+	 * 0x8000_0000: reset/NMI (initially in kseg1)
+	 * 0x8000_0400: normal vectors
+	 */
+	new_ebase = 0x80000400;
+	cbr = BMIPS_GET_CBR();
+	__raw_writel(0x80080800, cbr + BMIPS_RELO_VECTOR_CONTROL_0);
+	__raw_writel(0xa0080800, cbr + BMIPS_RELO_VECTOR_CONTROL_1);
+#elif defined(CONFIG_CPU_BMIPS5000)
+	/*
+	 * 0x8000_0000: reset/NMI (initially in kseg1)
+	 * 0x8000_1000: normal vectors
+	 */
+	new_ebase = 0x80001000;
+	write_c0_brcm_bootvec(0xa0088008);
+	write_c0_ebase(new_ebase);
+	if (max_cpus > 2)
+		bmips_write_zscm_reg(0xa0, 0xa008a008);
+#else
+	return;
+#endif
+	board_nmi_handler_setup = &bmips_nmi_handler_setup;
+	ebase = new_ebase;
+}
+
+asmlinkage void __weak plat_wired_tlb_setup(void)
+{
+	/*
+	 * Called when starting/restarting a secondary CPU.
+	 * Kernel stacks and other important data might only be accessible
+	 * once the wired entries are present.
+	 */
+}
-- 
1.7.6.3

Re: [PATCH 9/9] MIPS: BMIPS: Add SMP support code for BMIPS43xx/BMIPS5000

[Ralf Baechle]

On Sat, Nov 05, 2011 at 02:21:18PM -0700, Kevin Cernekee wrote:

> Initial commit of BMIPS SMP support code.  Smoke-tested on a variety of
> BMIPS4350, BMIPS4380, and BMIPS5000 platforms.

How good I'm a fireman ...

> +#if defined(CONFIG_CPU_BMIPS4380)
> +	/* initialize FPU registers */
> +	CLR_FPR	f0 f1 f2 f3
> +	CLR_FPR	f4 f5 f6 f7
> +	CLR_FPR	f8 f9 f10 f11
> +	CLR_FPR	f12 f13 f14 f15
> +	CLR_FPR	f16 f17 f18 f19
> +	CLR_FPR	f20 f21 f22 f23
> +	CLR_FPR	f24 f25 f26 f27
> +	CLR_FPR	f28 f29 f30 f31
> +#endif

Why initialize the FPU registers at all?  The kernel doesn't use the FPU
and before userland gets to play with the FPU the kernel will zero the
registers anyway.

> + * This file is subject to the terms and conditions of the GNU General Public
> + * License.  See the file "COPYING" in the main directory of this archive
> + * for more details.
> + *
> + * Copyright (C) 2011 by Kevin Cernekee (cernekee@gmail.com)
> + *
> + * SMP support for BMIPS
> + */
> +
> +#include <linux/version.h>
> +#include <linux/init.h>
> +#include <linux/sched.h>
> +#include <linux/mm.h>
> +#include <linux/delay.h>
> +#include <linux/smp.h>
> +#include <linux/interrupt.h>
> +#include <linux/spinlock.h>
> +#include <linux/init.h>
> +#include <linux/cpu.h>
> +#include <linux/cpumask.h>
> +#include <linux/reboot.h>
> +#include <linux/io.h>
> +#include <linux/compiler.h>
> +#include <linux/linkage.h>
> +#include <linux/bug.h>
> +
> +#include <asm/time.h>
> +#include <asm/pgtable.h>
> +#include <asm/processor.h>
> +#include <asm/system.h>
> +#include <asm/bootinfo.h>
> +#include <asm/pmon.h>
> +#include <asm/cacheflush.h>
> +#include <asm/tlbflush.h>
> +#include <asm/mipsregs.h>
> +#include <asm/bmips.h>
> +#include <asm/traps.h>
> +#include <asm/barrier.h>
> +
> +static int __maybe_unused max_cpus = 1;
> +
> +cpumask_t bmips_booted_mask;
> +
> +#ifdef CONFIG_SMP
> +
> +/* initial $sp, $gp - used by arch/mips/kernel/bmips_vec.S */
> +unsigned long bmips_smp_boot_sp;
> +unsigned long bmips_smp_boot_gp;
> +
> +static void bmips_send_ipi_single(int cpu, unsigned int action);
> +static irqreturn_t bmips_ipi_interrupt(int irq, void *dev_id);
> +
> +/* the platform code may forcibly disable SMP */
> +int bmips_smp_enabled = 1;
> +
> +/* SW interrupts 0,1 are used for interprocessor signaling */
> +#define IPI0_IRQ			(MIPS_CPU_IRQ_BASE + 0)
> +#define IPI1_IRQ			(MIPS_CPU_IRQ_BASE + 1)
> +
> +#define ACTION_CLR_IPI(cpu, ipi)	(0x2000 | ((cpu) << 9) | ((ipi) << 8))
> +#define ACTION_SET_IPI(cpu, ipi)	(0x3000 | ((cpu) << 9) | ((ipi) << 8))
> +#define ACTION_BOOT_THREAD(cpu)		(0x08 | (cpu))
> +
> +static void __init bmips_smp_setup(void)
> +{
> +	int i;
> +
> +#if defined(CONFIG_CPU_BMIPS4350) || defined(CONFIG_CPU_BMIPS4380)
> +	/* arbitration priority */
> +	clear_c0_brcm_cmt_ctrl(0x30);
> +
> +	/* NBK and weak order flags */
> +	set_c0_brcm_config_0(0x30000);
> +
> +	/*
> +	 * MIPS interrupts 0,1 (SW INT 0,1) cross over to the other thread
> +	 * MIPS interrupt 2 (HW INT 0) is the CPU0 L1 controller output
> +	 * MIPS interrupt 3 (HW INT 1) is the CPU1 L1 controller output
> +	 */
> +	change_c0_brcm_cmt_intr(0xf8018000,
> +		(0x02 << 27) | (0x03 << 15));
> +
> +	/* single core, 2 threads (2 pipelines) */
> +	max_cpus = 2;
> +#elif defined(CONFIG_CPU_BMIPS5000)
> +	/* enable raceless SW interrupts */
> +	set_c0_brcm_config(0x03 << 22);
> +
> +	/* route HW interrupt 0 to CPU0, HW interrupt 1 to CPU1 */
> +	change_c0_brcm_mode(0x1f << 27, 0x02 << 27);
> +
> +	/* N cores, 2 threads per core */
> +	max_cpus = (((read_c0_brcm_config() >> 6) & 0x03) + 1) << 1;
> +
> +	/* clear any pending SW interrupts */
> +	for (i = 0; i < max_cpus; i++) {
> +		write_c0_brcm_action(ACTION_CLR_IPI(i, 0));
> +		write_c0_brcm_action(ACTION_CLR_IPI(i, 1));
> +	}
> +#endif
> +
> +	if (!bmips_smp_enabled)
> +		max_cpus = 1;
> +
> +	/* this can be overridden by the BSP */
> +	if (!board_ebase_setup)
> +		board_ebase_setup = &bmips_ebase_setup;
> +
> +	for (i = 0; i < max_cpus; i++) {
> +		__cpu_number_map[i] = 1;
> +		__cpu_logical_map[i] = 1;
> +		set_cpu_possible(i, 1);
> +		set_cpu_present(i, 1);
> +	}
> +}
> +
> +/*
> + * IPI IRQ setup - runs on CPU0
> + */
> +static void bmips_prepare_cpus(unsigned int max_cpus)
> +{
> +	if (request_irq(IPI0_IRQ, bmips_ipi_interrupt, IRQF_PERCPU,
> +			"smp_ipi0", NULL))
> +		panic("Can't request IPI0 interrupt\n");
> +	if (request_irq(IPI1_IRQ, bmips_ipi_interrupt, IRQF_PERCPU,
> +			"smp_ipi1", NULL))
> +		panic("Can't request IPI1 interrupt\n");
> +}
> +
> +/*
> + * Tell the hardware to boot CPUx - runs on CPU0
> + */
> +static void bmips_boot_secondary(int cpu, struct task_struct *idle)
> +{
> +	bmips_smp_boot_sp = __KSTK_TOS(idle);
> +	bmips_smp_boot_gp = (unsigned long)task_thread_info(idle);
> +	mb();
> +
> +	/*
> +	 * Initial boot sequence for secondary CPU:
> +	 *   bmips_reset_nmi_vec @ a000_0000 ->
> +	 *   bmips_smp_entry ->
> +	 *   plat_wired_tlb_setup (cached function call; optional) ->
> +	 *   start_secondary (cached jump)
> +	 *
> +	 * Warm restart sequence:
> +	 *   play_dead WAIT loop ->
> +	 *   bmips_smp_int_vec @ BMIPS_WARM_RESTART_VEC ->
> +	 *   eret to play_dead ->
> +	 *   bmips_secondary_reentry ->
> +	 *   start_secondary
> +	 */
> +
> +	printk(KERN_INFO "SMP: Booting CPU%d...\n", cpu);

Please use pr_info() instead.

> +
> +	if (cpumask_test_cpu(cpu, &bmips_booted_mask))
> +		bmips_send_ipi_single(cpu, 0);
> +	else {
> +#if defined(CONFIG_CPU_BMIPS4350) || defined(CONFIG_CPU_BMIPS4380)
> +		set_c0_brcm_cmt_ctrl(0x01);
> +#elif defined(CONFIG_CPU_BMIPS5000)
> +		if (cpu & 0x01)
> +			write_c0_brcm_action(ACTION_BOOT_THREAD(cpu));
> +		else {
> +			/*
> +			 * core N thread 0 was already booted; just
> +			 * pulse the NMI line
> +			 */
> +			bmips_write_zscm_reg(0x210, 0xc0000000);
> +			udelay(10);
> +			bmips_write_zscm_reg(0x210, 0x00);
> +		}
> +#endif
> +		cpumask_set_cpu(cpu, &bmips_booted_mask);
> +	}
> +}
> +
> +/*
> + * Early setup - runs on secondary CPU after cache probe
> + */
> +static void bmips_init_secondary(void)
> +{
> +	/* move NMI vector to kseg0, in case XKS01 is enabled */
> +
> +#if defined(CONFIG_CPU_BMIPS4350) || defined(CONFIG_CPU_BMIPS4380)
> +	void __iomem *cbr = BMIPS_GET_CBR();
> +	unsigned long old_vec;
> +
> +	old_vec = __raw_readl(cbr + BMIPS_RELO_VECTOR_CONTROL_1);
> +	__raw_writel(old_vec & ~0x20000000, cbr + BMIPS_RELO_VECTOR_CONTROL_1);
> +
> +	clear_c0_cause(smp_processor_id() ? C_SW1 : C_SW0);
> +#elif defined(CONFIG_CPU_BMIPS5000)
> +	write_c0_brcm_bootvec(read_c0_brcm_bootvec() &
> +		(smp_processor_id() & 0x01 ? ~0x20000000 : ~0x2000));
> +
> +	write_c0_brcm_action(ACTION_CLR_IPI(smp_processor_id(), 0));
> +#endif
> +
> +	/* make sure there won't be a timer interrupt for a little while */
> +	write_c0_compare(read_c0_count() + mips_hpt_frequency / HZ);
> +
> +	irq_enable_hazard();
> +	set_c0_status(IE_SW0 | IE_SW1 | IE_IRQ1 | IE_IRQ5 | ST0_IE);
> +	irq_enable_hazard();
> +}
> +
> +/*
> + * Late setup - runs on secondary CPU before entering the idle loop
> + */
> +static void bmips_smp_finish(void)
> +{
> +	printk(KERN_INFO "SMP: CPU%d is running\n", smp_processor_id());

Please use pr_info() instead.

> +}
> +
> +/*
> + * Runs on CPU0 after all CPUs have been booted
> + */
> +static void bmips_cpus_done(void)
> +{
> +}
> +
> +#if defined(CONFIG_CPU_BMIPS5000)
> +
> +/*
> + * BMIPS5000 raceless IPIs
> + *
> + * Each CPU has two inbound SW IRQs which are independent of all other CPUs.
> + * IPI0 is used for SMP_RESCHEDULE_YOURSELF
> + * IPI1 is used for SMP_CALL_FUNCTION
> + */
> +
> +static void bmips_send_ipi_single(int cpu, unsigned int action)
> +{
> +	write_c0_brcm_action(ACTION_SET_IPI(cpu, action == SMP_CALL_FUNCTION));
> +}
> +
> +static irqreturn_t bmips_ipi_interrupt(int irq, void *dev_id)
> +{
> +	int action = irq - IPI0_IRQ;
> +
> +	write_c0_brcm_action(ACTION_CLR_IPI(smp_processor_id(), action));
> +
> +	if (action == 0)
> +		scheduler_ipi();
> +	else
> +		smp_call_function_interrupt();
> +
> +	return IRQ_HANDLED;
> +}
> +
> +#else
> +
> +/*
> + * BMIPS43xx racey IPIs
> + *
> + * We use one inbound SW IRQ for each CPU.
> + *
> + * A spinlock must be held in order to keep CPUx from accidentally clearing
> + * an incoming IPI when it writes CP0 CAUSE to raise an IPI on CPUy.  The
> + * same spinlock is used to protect the action masks.
> + */
> +
> +static DEFINE_SPINLOCK(ipi_lock);
> +static DEFINE_PER_CPU(int, ipi_action_mask);
> +
> +static void bmips_send_ipi_single(int cpu, unsigned int action)
> +{
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&ipi_lock, flags);
> +	set_c0_cause(cpu ? C_SW1 : C_SW0);
> +	per_cpu(ipi_action_mask, cpu) |= action;
> +	irq_enable_hazard();
> +	spin_unlock_irqrestore(&ipi_lock, flags);
> +}
> +
> +static irqreturn_t bmips_ipi_interrupt(int irq, void *dev_id)
> +{
> +	unsigned long flags;
> +	int action, cpu = irq - IPI0_IRQ;
> +
> +	spin_lock_irqsave(&ipi_lock, flags);
> +	action = __get_cpu_var(ipi_action_mask);
> +	per_cpu(ipi_action_mask, cpu) = 0;
> +	clear_c0_cause(cpu ? C_SW1 : C_SW0);
> +	spin_unlock_irqrestore(&ipi_lock, flags);
> +
> +	if (action & SMP_RESCHEDULE_YOURSELF)
> +		scheduler_ipi();
> +	if (action & SMP_CALL_FUNCTION)
> +		smp_call_function_interrupt();
> +
> +	return IRQ_HANDLED;
> +}
> +
> +#endif /* BMIPS type */
> +
> +static void bmips_send_ipi_mask(const struct cpumask *mask,
> +	unsigned int action)
> +{
> +	unsigned int i;
> +
> +	for_each_cpu(i, mask)
> +		bmips_send_ipi_single(i, action);
> +}
> +
> +#ifdef CONFIG_HOTPLUG_CPU
> +
> +static int bmips_cpu_disable(void)
> +{
> +	unsigned int cpu = smp_processor_id();
> +
> +	if (cpu == 0)
> +		return -EBUSY;
> +
> +	printk(KERN_INFO "SMP: CPU%d is offline\n", cpu);

Please use pr_info() instead.

  Ralf

Re: [PATCH RESEND 1/9] MIPS: Add local_flush_tlb_all_mm to clear all mm contexts on calling cpu

[Ralf Baechle]

On Sat, Nov 05, 2011 at 02:21:10PM -0700, Kevin Cernekee wrote:

> +void local_flush_tlb_all_mm(void)
> +{
> +	struct task_struct *p;
> +
> +	for_each_process(p)
> +		if (p->mm)
> +			local_flush_tlb_mm(p->mm);

Aside of for_each_process being a potencially very heavy iterator - there
can be thousands of threads on some systems, even embedded systems I'm
missing the task_list lock being taken so bad things could happen.

  Ralf

Re: [PATCH RESEND 1/9] MIPS: Add local_flush_tlb_all_mm to clear all mm contexts on calling cpu

[Kevin Cernekee]

On Tue, Nov 8, 2011 at 8:47 AM, Ralf Baechle <ralf@linux-mips.org> wrote:
> On Sat, Nov 05, 2011 at 02:21:10PM -0700, Kevin Cernekee wrote:
>
>> +void local_flush_tlb_all_mm(void)
>> +{
>> +     struct task_struct *p;
>> +
>> +     for_each_process(p)
>> +             if (p->mm)
>> +                     local_flush_tlb_mm(p->mm);
>
> Aside of for_each_process being a potencially very heavy iterator - there
> can be thousands of threads on some systems, even embedded systems I'm

This is called from play_dead() on a CPU that has been hot-unplugged.

FWIW for_each_process() is also called from check_for_tasks() when
bringing down a CPU.  I wonder if that would be a better place to add
a hook to drop the MMU contexts, ala:

diff --git a/kernel/cpu.c b/kernel/cpu.c
index 563f136..5854401 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -184,6 +184,7 @@ static inline void check_for_tasks(int cpu)
 				"(state = %ld, flags = %x)\n",
 				p->comm, task_pid_nr(p), cpu,
 				p->state, p->flags);
+		arch_drop_mmu_context(p, cpu);
 	}
 	write_unlock_irq(&tasklist_lock);
 }

But it does run on a "surviving" CPU, not the dying CPU, so I'm not
positive it would work the same way.

> missing the task_list lock being taken so bad things could happen.

Right, so without acquiring tasklist_lock, somebody else could
rearrange the list while I'm iterating through it.

Do I need to call task_lock(p) before touching p->mm, or would
acquiring a write lock on tasklist_lock be sufficient?

Re: [PATCH RESEND 1/9] MIPS: Add local_flush_tlb_all_mm to clear all mm contexts on calling cpu

[Kevin Cernekee]

I read through mmu_context.h and the threads from November/December
2010 a couple of times, and I'm starting to think Maksim's original
approach (don't reset asid_cache(cpu) when warm-restarting a CPU)
makes the most sense.

The basic issue is that we want to assign unique, strictly increasing
values to each mm's cpu_context(cpu, mm).  The per-cpu counter starts
at ASID_FIRST_VERSION (0x100 on R4K), and counts up.  Assigning a new
mm the same ASID value as an existing mm on the same CPU is illegal.
Two obvious ways to meet this requirement when hotplugging CPUs are:

Option #1: Retain the asid_cache(cpu) value across warm restarts.
This is simple and inexpensive.  We pick up where we left off, and
whatever existing cpu_context(cpu, mm) values are out there do not
cause any trouble.

I believe Maksim's original logic (assign ASID_FIRST_VERSION, a
nonzero number, if asid_cache(cpu) == 0) would work correctly as
written, because cpu_data is an array in .bss .  It will be 0 until
the CPU is booted, and get_new_mmu_context() ensures that it will
never be 0 again after that.

Kevin K brought up the idea of a warm restart bitmask so the code
could tell whether asid_cache(cpu) was valid.  I'm not sure that this
would be required.

I think we can also get away with not explicitly preserving EntryHi,
since switch_mm() and activate_mm() will set it anyway.

Option #2: When warm restarting a CPU, set asid_cache(cpu) to
ASID_FIRST_VERSION again.  And at some point (cpu_up or cpu_down),
iterate through all processes to set cpu_context(cpu, mm) to something
that will not conflict with a newly assigned ASID.  This is what the
most recent patch did.  It gets the job done, but it's more work than
what is really needed.

Please let me know your thoughts...

Re: [PATCH RESEND 1/9] MIPS: Add local_flush_tlb_all_mm to clear all mm contexts on calling cpu

[Ralf Baechle]

On Tue, Nov 08, 2011 at 09:33:52PM -0800, Kevin Cernekee wrote:

> I read through mmu_context.h and the threads from November/December
> 2010 a couple of times, and I'm starting to think Maksim's original
> approach (don't reset asid_cache(cpu) when warm-restarting a CPU)
> makes the most sense.
> 
> The basic issue is that we want to assign unique, strictly increasing
> values to each mm's cpu_context(cpu, mm).  The per-cpu counter starts
> at ASID_FIRST_VERSION (0x100 on R4K), and counts up.  Assigning a new
> mm the same ASID value as an existing mm on the same CPU is illegal.
> Two obvious ways to meet this requirement when hotplugging CPUs are:
> 
> Option #1: Retain the asid_cache(cpu) value across warm restarts.
> This is simple and inexpensive.  We pick up where we left off, and
> whatever existing cpu_context(cpu, mm) values are out there do not
> cause any trouble.
> 
> I believe Maksim's original logic (assign ASID_FIRST_VERSION, a
> nonzero number, if asid_cache(cpu) == 0) would work correctly as
> written, because cpu_data is an array in .bss .  It will be 0 until
> the CPU is booted, and get_new_mmu_context() ensures that it will
> never be 0 again after that.
> 
> Kevin K brought up the idea of a warm restart bitmask so the code
> could tell whether asid_cache(cpu) was valid.  I'm not sure that this
> would be required.

Neither do I.

> I think we can also get away with not explicitly preserving EntryHi,
> since switch_mm() and activate_mm() will set it anyway.
> 
> Option #2: When warm restarting a CPU, set asid_cache(cpu) to
> ASID_FIRST_VERSION again.  And at some point (cpu_up or cpu_down),
> iterate through all processes to set cpu_context(cpu, mm) to something
> that will not conflict with a newly assigned ASID.  This is what the
> most recent patch did.  It gets the job done, but it's more work than
> what is really needed.
> 
> Please let me know your thoughts...

I still like the original patch https://patchwork.linux-mips.org/patch/1797/
I'd apply it right away but I'm going to hold off for a day just to give
Kevin and maybe others a chance to comment.

  Ralf