[PATCH V5 0/5] SMP support for Armada XP

[PATCH V5 0/5] SMP support for Armada XP

[Gregory CLEMENT]

Hello,

This version fixed a bug introduced by the move of set_cpu_coherency
in assembly, it also fixed some typo and style issues.

The purpose of this patch set is to add the SMP support for the Armada
XP SoCs. Beside the SMP support itself brought by the last 3 patches,
this patch set also adds the support for the coherency fabric unit and
the power management service unit.

The coherency fabric is responsible for ensuring hardware coherency
between all CPUs and between CPUs and I/O masters. This unit is also
available for Armada 370 and will be used in an incoming patch set
for hardware I/O cache coherency.

The power management service unit is responsible for powering down and
waking up CPUs and other SOC units.

The original code is from Yehuda Yitschak, it was reworked by myself
and reviewed by Yehuda.

This patch set is based on the tag marvell-mvebu-clk-3.8 to ease the
incoming git pull.
The git branch called ArmadaXP-SMP-for-3.8-V5 is also available at
https://github.com/MISL-EBU-System-SW/mainline-public.git.

Changelog:
v4->V5:
- Rebased on to marvell-mvebu-clk-3.8
- Fixed bug in coherency_ll.S, use r2 instead of r4 as it can lead to
  unexpected crash if r4 is not restored properly.
- Fixed typo and improve assembly code in coherency_ll.S
- Use for_each_node_by_type instead of a while loop and
  of_find_node_by_type

V3->V4:
- Rebased on to v3.7-rc6
- the function to setup the coherency of the CPU is now share between
  SMP and coherency part and no more duplicate. Most of the code of
  headsmp.S have been moved in coherency_ll.S and the redundant code
  have been deleted from coherency.c.
- Fixed cosmetic inconsistency in code added to proc-v7, only macros
  are used and no more immediate values.
- Added reviewed-by and acked-by for the patch 4.

V2 -> V3:
- Rebased on to v3.7-rc5
- removed the unnecessary used of __iomem flag for a pointer to a
  function. the second parameter of armada_xp_boot_cpu is now just a
  void *.
- removed the dsb() call in armada_xp_boot_cpu(), as the following
  writel calls dsb if needed.
- Change the PJ4B mask for proc_info in arch/arm/mm/proc-v7.S to be
  less restrictive, the last 4 bits contain revision information
- Merge the multiple 'or' instruction in a single instruction in
  __v7_pj4b_setup

V1 -> V2:
- Rebased on to v3.7-rc3
- Fixed typos found by Alexandre Belloni
- Added clk_prepare_enable() before getting rate clk in
  set_secondary_cpus_clock()
- Add explanation in the binding documentation about the per-CPU
  interrupt registers: the address of the virtual register must be
  used.
- Removed the armada_xp prefix in the coherency.c file to be more
  compliant with the name convention of the other files.
- Coherency_init is now called from armada_370_xp_dt_init() and is no
  more an early_init() call. As the device tree is not available from
  an early_init(), it was useless to call coherency_init() so
  early. The need to be able to call some function very early during
  the boot were already resolved by using the hard code address of the
  register.


Yehuda Yitschak (5):
  arm: mvebu: Added support for coherency fabric in mach-mvebu
  arm: mvebu: Added initial support for power managmement service unit
  arm: mvebu: Added IPI support via doorbells
  arm: mm: Added support for PJ4B cpu and init routines
  arm: mvebu: Added SMP support for Armada XP

 .../devicetree/bindings/arm/armada-370-xp-mpic.txt |   12 +-
 .../devicetree/bindings/arm/armada-370-xp-pmsu.txt |   20 ++++
 .../devicetree/bindings/arm/coherency-fabric.txt   |   16 +++
 arch/arm/boot/dts/armada-370-xp.dtsi               |    5 +
 arch/arm/boot/dts/armada-xp.dtsi                   |    8 +-
 arch/arm/configs/mvebu_defconfig                   |    3 +
 arch/arm/mach-mvebu/Kconfig                        |    3 +-
 arch/arm/mach-mvebu/Makefile                       |    4 +-
 arch/arm/mach-mvebu/armada-370-xp.c                |    3 +
 arch/arm/mach-mvebu/armada-370-xp.h                |    7 ++
 arch/arm/mach-mvebu/coherency.c                    |   80 +++++++++++++
 arch/arm/mach-mvebu/coherency.h                    |   24 ++++
 arch/arm/mach-mvebu/coherency_ll.S                 |   47 ++++++++
 arch/arm/mach-mvebu/common.h                       |    6 +
 arch/arm/mach-mvebu/headsmp.S                      |   50 ++++++++
 arch/arm/mach-mvebu/hotplug.c                      |   30 +++++
 arch/arm/mach-mvebu/irq-armada-370-xp.c            |   92 ++++++++++++++-
 arch/arm/mach-mvebu/platsmp.c                      |  121 ++++++++++++++++++++
 arch/arm/mach-mvebu/pmsu.c                         |   75 ++++++++++++
 arch/arm/mach-mvebu/pmsu.h                         |   16 +++
 arch/arm/mm/Kconfig                                |    4 +
 arch/arm/mm/proc-v7.S                              |   66 +++++++++++
 22 files changed, 680 insertions(+), 12 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt
 create mode 100644 Documentation/devicetree/bindings/arm/coherency-fabric.txt
 create mode 100644 arch/arm/mach-mvebu/coherency.c
 create mode 100644 arch/arm/mach-mvebu/coherency.h
 create mode 100644 arch/arm/mach-mvebu/coherency_ll.S
 create mode 100644 arch/arm/mach-mvebu/headsmp.S
 create mode 100644 arch/arm/mach-mvebu/hotplug.c
 create mode 100644 arch/arm/mach-mvebu/platsmp.c
 create mode 100644 arch/arm/mach-mvebu/pmsu.c
 create mode 100644 arch/arm/mach-mvebu/pmsu.h

-- 
1.7.9.5

[PATCH V5 1/5] arm: mvebu: Added support for coherency fabric in mach-mvebu

[Gregory CLEMENT]

From: Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>

Signed-off-by: Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>
Signed-off-by: Gregory CLEMENT <gregory.clement-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
---
 .../devicetree/bindings/arm/coherency-fabric.txt   |   16 ++++
 arch/arm/boot/dts/armada-370-xp.dtsi               |    5 ++
 arch/arm/mach-mvebu/Makefile                       |    2 +-
 arch/arm/mach-mvebu/coherency.c                    |   80 ++++++++++++++++++++
 arch/arm/mach-mvebu/coherency.h                    |   24 ++++++
 arch/arm/mach-mvebu/coherency_ll.S                 |   47 ++++++++++++
 arch/arm/mach-mvebu/common.h                       |    2 +
 7 files changed, 175 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/devicetree/bindings/arm/coherency-fabric.txt
 create mode 100644 arch/arm/mach-mvebu/coherency.c
 create mode 100644 arch/arm/mach-mvebu/coherency.h
 create mode 100644 arch/arm/mach-mvebu/coherency_ll.S

diff --git a/Documentation/devicetree/bindings/arm/coherency-fabric.txt b/Documentation/devicetree/bindings/arm/coherency-fabric.txt
new file mode 100644
index 0000000..2bfbf67
--- /dev/null
+++ b/Documentation/devicetree/bindings/arm/coherency-fabric.txt
@@ -0,0 +1,16 @@
+Coherency fabric
+----------------
+Available on Marvell SOCs: Armada 370 and Armada XP
+
+Required properties:
+
+- compatible: "marvell,coherency-fabric"
+- reg: Should contain,coherency fabric registers location and length.
+
+Example:
+
+coherency-fabric@d0020200 {
+	compatible = "marvell,coherency-fabric";
+	reg = <0xd0020200 0xb0>;
+};
+
diff --git a/arch/arm/boot/dts/armada-370-xp.dtsi b/arch/arm/boot/dts/armada-370-xp.dtsi
index 94b4b9e..b0d075b 100644
--- a/arch/arm/boot/dts/armada-370-xp.dtsi
+++ b/arch/arm/boot/dts/armada-370-xp.dtsi
@@ -36,6 +36,11 @@
 	      interrupt-controller;
 	};
 
+	coherency-fabric@d0020200 {
+		compatible = "marvell,coherency-fabric";
+		reg = <0xd0020200 0xb0>;
+	};
+
 	soc {
 		#address-cells = <1>;
 		#size-cells = <1>;
diff --git a/arch/arm/mach-mvebu/Makefile b/arch/arm/mach-mvebu/Makefile
index 57f996b..5ce4b42 100644
--- a/arch/arm/mach-mvebu/Makefile
+++ b/arch/arm/mach-mvebu/Makefile
@@ -2,4 +2,4 @@ ccflags-$(CONFIG_ARCH_MULTIPLATFORM) := -I$(srctree)/$(src)/include \
 	-I$(srctree)/arch/arm/plat-orion/include
 
 obj-y += system-controller.o
-obj-$(CONFIG_MACH_ARMADA_370_XP) += armada-370-xp.o irq-armada-370-xp.o addr-map.o
+obj-$(CONFIG_MACH_ARMADA_370_XP) += armada-370-xp.o irq-armada-370-xp.o addr-map.o coherency.o coherency_ll.o
diff --git a/arch/arm/mach-mvebu/coherency.c b/arch/arm/mach-mvebu/coherency.c
new file mode 100644
index 0000000..1bc02d0
--- /dev/null
+++ b/arch/arm/mach-mvebu/coherency.c
@@ -0,0 +1,80 @@
+/*
+ * Coherency fabric (Aurora) support for Armada 370 and XP platforms.
+ *
+ * Copyright (C) 2012 Marvell
+ *
+ * Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>
+ * Gregory Clement <gregory.clement-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
+ * Thomas Petazzoni <thomas.petazzoni-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ *
+ * The Armada 370 and Armada XP SOCs have a coherency fabric which is
+ * responsible for ensuring hardware coherency between all CPUs and between
+ * CPUs and I/O masters. This file initializes the coherency fabric and
+ * supplies basic routines for configuring and controlling hardware coherency
+ */
+
+#include <linux/kernel.h>
+#include <linux/init.h>
+#include <linux/of_address.h>
+#include <linux/io.h>
+#include <linux/smp.h>
+#include <asm/smp_plat.h>
+#include "armada-370-xp.h"
+
+/* Some functions in this file are called very early during SMP
+ * initialization. At that time the device tree framework is not yet
+ * ready, and it is not possible to get the register address to
+ * ioremap it. That's why the pointer below is given with an initial
+ * value matching its virtual mapping
+ */
+static void __iomem *coherency_base = ARMADA_370_XP_REGS_VIRT_BASE + 0x20200;
+
+/* Coherency fabric registers */
+#define COHERENCY_FABRIC_CFG_OFFSET		   0x4
+
+static struct of_device_id of_coherency_table[] = {
+	{.compatible = "marvell,coherency-fabric"},
+	{ /* end of list */ },
+};
+#ifdef CONFIG_SMP
+int coherency_get_cpu_count(void)
+{
+	int reg, cnt;
+
+	reg = readl(coherency_base + COHERENCY_FABRIC_CFG_OFFSET);
+	cnt = (reg & 0xF) + 1;
+
+	return cnt;
+}
+#endif
+/* Function defined in coherncy_ll.S */
+extern void ll_set_cpu_coherent(void __iomem *base_addr,
+				unsigned int hw_cpu_id);
+
+int set_cpu_coherent(unsigned int hw_cpu_id, int smp_group_id)
+{
+	if (!coherency_base) {
+		pr_warn("Can't make CPU %d cache coherent.\n", hw_cpu_id);
+		pr_warn("Coherency fabric is not initialized\n");
+		return 1;
+	}
+	ll_set_cpu_coherent(coherency_base, hw_cpu_id);
+	return 0;
+}
+
+int __init coherency_init(void)
+{
+	struct device_node *np;
+
+	np = of_find_matching_node(NULL, of_coherency_table);
+	if (np) {
+		pr_info("Initializing Coherency fabric\n");
+		coherency_base = of_iomap(np, 0);
+	}
+
+	return 0;
+}
diff --git a/arch/arm/mach-mvebu/coherency.h b/arch/arm/mach-mvebu/coherency.h
new file mode 100644
index 0000000..2f42813
--- /dev/null
+++ b/arch/arm/mach-mvebu/coherency.h
@@ -0,0 +1,24 @@
+/*
+ * arch/arm/mach-mvebu/include/mach/coherency.h
+ *
+ *
+ * Coherency fabric (Aurora) support for Armada 370 and XP platforms.
+ *
+ * Copyright (C) 2012 Marvell
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#ifndef __MACH_370_XP_COHERENCY_H
+#define __MACH_370_XP_COHERENCY_H
+
+#ifdef CONFIG_SMP
+int coherency_get_cpu_count(void);
+#endif
+
+int set_cpu_coherent(int cpu_id, int smp_group_id);
+int coherency_init(void);
+
+#endif	/* __MACH_370_XP_COHERENCY_H */
diff --git a/arch/arm/mach-mvebu/coherency_ll.S b/arch/arm/mach-mvebu/coherency_ll.S
new file mode 100644
index 0000000..ae48730
--- /dev/null
+++ b/arch/arm/mach-mvebu/coherency_ll.S
@@ -0,0 +1,47 @@
+/*
+ * Coherency fabric: low level functions
+ *
+ * Copyright (C) 2012 Marvell
+ *
+ * Gregory CLEMENT <gregory.clement-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ *
+ * This file implements the assembly function to add a CPU to the
+ * coherency fabric. This function is called by each of the secondary
+ * CPUs during their early boot in an SMP kernel, this why this
+ * function have to callable from assembly. It can also be called by a
+ * primary CPU from C code during its boot.
+ */
+
+#include <linux/linkage.h>
+#define ARMADA_XP_CFB_CTL_REG_OFFSET 0x0
+#define ARMADA_XP_CFB_CFG_REG_OFFSET 0x4
+
+	.text
+/*
+ * r0: Coherency fabric base register address
+ * r1: HW CPU id
+ */
+ENTRY(ll_set_cpu_coherent)
+	/* Create bit by cpu index */
+	mov	r3, #(1 << 24)
+	lsl     r1, r3, r1
+
+	/* Add CPU to SMP group - Atomic */
+	add	r3, r0, #ARMADA_XP_CFB_CTL_REG_OFFSET
+	ldr     r2, [r3]
+	orr     r2, r2, r1
+	str	r2, [r0]
+
+	/* Enable coherency on CPU - Atomic */
+	add	r3, r0, #ARMADA_XP_CFB_CFG_REG_OFFSET
+	ldr     r2, [r3]
+	orr     r2, r2, r1
+	str     r2, [r3]
+
+	mov	r0, #0
+       	mov	pc, lr
+ENDPROC(ll_set_cpu_coherent)
diff --git a/arch/arm/mach-mvebu/common.h b/arch/arm/mach-mvebu/common.h
index 02f89ea..9aaa449 100644
--- a/arch/arm/mach-mvebu/common.h
+++ b/arch/arm/mach-mvebu/common.h
@@ -20,4 +20,6 @@ void mvebu_restart(char mode, const char *cmd);
 void armada_370_xp_init_irq(void);
 void armada_370_xp_handle_irq(struct pt_regs *regs);
 
+
+int armada_370_xp_coherency_init(void);
 #endif
-- 
1.7.9.5

[PATCH V5 2/5] arm: mvebu: Added initial support for power managmement service unit

[Gregory CLEMENT]

From: Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>

Signed-off-by: Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>
Signed-off-by: Gregory CLEMENT <gregory.clement-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
---
 .../devicetree/bindings/arm/armada-370-xp-pmsu.txt |   20 ++++++
 arch/arm/boot/dts/armada-xp.dtsi                   |    6 ++
 arch/arm/mach-mvebu/Makefile                       |    2 +-
 arch/arm/mach-mvebu/common.h                       |    1 +
 arch/arm/mach-mvebu/pmsu.c                         |   75 ++++++++++++++++++++
 arch/arm/mach-mvebu/pmsu.h                         |   16 +++++
 6 files changed, 119 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt
 create mode 100644 arch/arm/mach-mvebu/pmsu.c
 create mode 100644 arch/arm/mach-mvebu/pmsu.h

diff --git a/Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt b/Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt
new file mode 100644
index 0000000..926b4d6
--- /dev/null
+++ b/Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt
@@ -0,0 +1,20 @@
+Power Management Service Unit(PMSU)
+-----------------------------------
+Available on Marvell SOCs: Armada 370 and Armada XP
+
+Required properties:
+
+- compatible: "marvell,armada-370-xp-pmsu"
+
+- reg: Should contain PMSU registers location and length. First pair
+  for the per-CPU SW Reset Control registers, second pair for the
+  Power Management Service Unit.
+
+Example:
+
+armada-370-xp-pmsu@d0022000 {
+	compatible = "marvell,armada-370-xp-pmsu";
+	reg = <0xd0022100 0x430>,
+	      <0xd0020800 0x20>;
+};
+
diff --git a/arch/arm/boot/dts/armada-xp.dtsi b/arch/arm/boot/dts/armada-xp.dtsi
index f51554e..1f95e22 100644
--- a/arch/arm/boot/dts/armada-xp.dtsi
+++ b/arch/arm/boot/dts/armada-xp.dtsi
@@ -27,6 +27,12 @@
 		    <0xd0021870 0x58>;
 	};
 
+	armada-370-xp-pmsu@d0022000 {
+		compatible = "marvell,armada-370-xp-pmsu";
+		reg = <0xd0022100 0x430>,
+		      <0xd0020800 0x20>;
+	};
+
 	soc {
 		serial@d0012200 {
 				compatible = "ns16550";
diff --git a/arch/arm/mach-mvebu/Makefile b/arch/arm/mach-mvebu/Makefile
index 5ce4b42..2e3ec11 100644
--- a/arch/arm/mach-mvebu/Makefile
+++ b/arch/arm/mach-mvebu/Makefile
@@ -2,4 +2,4 @@ ccflags-$(CONFIG_ARCH_MULTIPLATFORM) := -I$(srctree)/$(src)/include \
 	-I$(srctree)/arch/arm/plat-orion/include
 
 obj-y += system-controller.o
-obj-$(CONFIG_MACH_ARMADA_370_XP) += armada-370-xp.o irq-armada-370-xp.o addr-map.o coherency.o coherency_ll.o
+obj-$(CONFIG_MACH_ARMADA_370_XP) += armada-370-xp.o irq-armada-370-xp.o addr-map.o coherency.o coherency_ll.o pmsu.o
diff --git a/arch/arm/mach-mvebu/common.h b/arch/arm/mach-mvebu/common.h
index 9aaa449..b5cd7e7 100644
--- a/arch/arm/mach-mvebu/common.h
+++ b/arch/arm/mach-mvebu/common.h
@@ -22,4 +22,5 @@ void armada_370_xp_handle_irq(struct pt_regs *regs);
 
 
 int armada_370_xp_coherency_init(void);
+int armada_370_xp_pmsu_init(void);
 #endif
diff --git a/arch/arm/mach-mvebu/pmsu.c b/arch/arm/mach-mvebu/pmsu.c
new file mode 100644
index 0000000..3cc4bef
--- /dev/null
+++ b/arch/arm/mach-mvebu/pmsu.c
@@ -0,0 +1,75 @@
+/*
+ * Power Management Service Unit(PMSU) support for Armada 370/XP platforms.
+ *
+ * Copyright (C) 2012 Marvell
+ *
+ * Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>
+ * Gregory Clement <gregory.clement-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
+ * Thomas Petazzoni <thomas.petazzoni-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ *
+ * The Armada 370 and Armada XP SOCs have a power management service
+ * unit which is responsible for powering down and waking up CPUs and
+ * other SOC units
+ */
+
+#include <linux/kernel.h>
+#include <linux/init.h>
+#include <linux/of_address.h>
+#include <linux/io.h>
+#include <linux/smp.h>
+#include <asm/smp_plat.h>
+
+static void __iomem *pmsu_mp_base;
+static void __iomem *pmsu_reset_base;
+
+#define PMSU_BOOT_ADDR_REDIRECT_OFFSET(cpu)	((cpu * 0x100) + 0x24)
+#define PMSU_RESET_CTL_OFFSET(cpu)		(cpu * 0x8)
+
+static struct of_device_id of_pmsu_table[] = {
+	{.compatible = "marvell,armada-370-xp-pmsu"},
+	{ /* end of list */ },
+};
+
+#ifdef CONFIG_SMP
+int armada_xp_boot_cpu(unsigned int cpu_id, void *boot_addr)
+{
+	int reg, hw_cpu;
+
+	if (!pmsu_mp_base || !pmsu_reset_base) {
+		pr_warn("Can't boot CPU. PMSU is uninitialized\n");
+		return 1;
+	}
+
+	hw_cpu = cpu_logical_map(cpu_id);
+
+	writel(virt_to_phys(boot_addr), pmsu_mp_base +
+			PMSU_BOOT_ADDR_REDIRECT_OFFSET(hw_cpu));
+
+	/* Release CPU from reset by clearing reset bit*/
+	reg = readl(pmsu_reset_base + PMSU_RESET_CTL_OFFSET(hw_cpu));
+	reg &= (~0x1);
+	writel(reg, pmsu_reset_base + PMSU_RESET_CTL_OFFSET(hw_cpu));
+
+	return 0;
+}
+#endif
+
+int __init armada_370_xp_pmsu_init(void)
+{
+	struct device_node *np;
+
+	np = of_find_matching_node(NULL, of_pmsu_table);
+	if (np) {
+		pr_info("Initializing Power Management Service Unit\n");
+		pmsu_mp_base = of_iomap(np, 0);
+		pmsu_reset_base = of_iomap(np, 1);
+	}
+
+	return 0;
+}
+
+early_initcall(armada_370_xp_pmsu_init);
diff --git a/arch/arm/mach-mvebu/pmsu.h b/arch/arm/mach-mvebu/pmsu.h
new file mode 100644
index 0000000..07a737c
--- /dev/null
+++ b/arch/arm/mach-mvebu/pmsu.h
@@ -0,0 +1,16 @@
+/*
+ * Power Management Service Unit (PMSU) support for Armada 370/XP platforms.
+ *
+ * Copyright (C) 2012 Marvell
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#ifndef __MACH_MVEBU_PMSU_H
+#define __MACH_MVEBU_PMSU_H
+
+int armada_xp_boot_cpu(unsigned int cpu_id, void *phys_addr);
+
+#endif	/* __MACH_370_XP_PMSU_H */
-- 
1.7.9.5

[PATCH V5 3/5] arm: mvebu: Added IPI support via doorbells

[Gregory CLEMENT]

From: Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>

Signed-off-by: Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>
Signed-off-by: Gregory CLEMENT <gregory.clement-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
---
 .../devicetree/bindings/arm/armada-370-xp-mpic.txt |   12 ++-
 arch/arm/boot/dts/armada-xp.dtsi                   |    2 +-
 arch/arm/mach-mvebu/armada-370-xp.h                |    7 ++
 arch/arm/mach-mvebu/irq-armada-370-xp.c            |   92 ++++++++++++++++++--
 4 files changed, 103 insertions(+), 10 deletions(-)

diff --git a/Documentation/devicetree/bindings/arm/armada-370-xp-mpic.txt b/Documentation/devicetree/bindings/arm/armada-370-xp-mpic.txt
index 70c0dc5..61df564 100644
--- a/Documentation/devicetree/bindings/arm/armada-370-xp-mpic.txt
+++ b/Documentation/devicetree/bindings/arm/armada-370-xp-mpic.txt
@@ -6,9 +6,15 @@ Required properties:
 - interrupt-controller: Identifies the node as an interrupt controller.
 - #interrupt-cells: The number of cells to define the interrupts. Should be 1.
   The cell is the IRQ number
+
 - reg: Should contain PMIC registers location and length. First pair
   for the main interrupt registers, second pair for the per-CPU
-  interrupt registers
+  interrupt registers. For this last pair, to be compliant with SMP
+  support, the "virtual" must be use (For the record, these registers
+  automatically map to the interrupt controller registers of the
+  current CPU)
+
+
 
 Example:
 
@@ -18,6 +24,6 @@ Example:
               #address-cells = <1>;
               #size-cells = <1>;
               interrupt-controller;
-              reg = <0xd0020000 0x1000>,
-                    <0xd0021000 0x1000>;
+              reg = <0xd0020a00 0x1d0>,
+                    <0xd0021070 0x58>;
         };
diff --git a/arch/arm/boot/dts/armada-xp.dtsi b/arch/arm/boot/dts/armada-xp.dtsi
index 1f95e22..e6db2b7 100644
--- a/arch/arm/boot/dts/armada-xp.dtsi
+++ b/arch/arm/boot/dts/armada-xp.dtsi
@@ -24,7 +24,7 @@
 
 	mpic: interrupt-controller@d0020000 {
 	      reg = <0xd0020a00 0x1d0>,
-		    <0xd0021870 0x58>;
+		    <0xd0021070 0x58>;
 	};
 
 	armada-370-xp-pmsu@d0022000 {
diff --git a/arch/arm/mach-mvebu/armada-370-xp.h b/arch/arm/mach-mvebu/armada-370-xp.h
index aac9beb..c6a7d74 100644
--- a/arch/arm/mach-mvebu/armada-370-xp.h
+++ b/arch/arm/mach-mvebu/armada-370-xp.h
@@ -19,4 +19,11 @@
 #define ARMADA_370_XP_REGS_VIRT_BASE	IOMEM(0xfeb00000)
 #define ARMADA_370_XP_REGS_SIZE		SZ_1M
 
+#ifdef CONFIG_SMP
+#include <linux/cpumask.h>
+
+void armada_mpic_send_doorbell(const struct cpumask *mask, unsigned int irq);
+void armada_xp_mpic_smp_cpu_init(void);
+#endif
+
 #endif /* __MACH_ARMADA_370_XP_H */
diff --git a/arch/arm/mach-mvebu/irq-armada-370-xp.c b/arch/arm/mach-mvebu/irq-armada-370-xp.c
index 5f5f939..549b684 100644
--- a/arch/arm/mach-mvebu/irq-armada-370-xp.c
+++ b/arch/arm/mach-mvebu/irq-armada-370-xp.c
@@ -24,6 +24,7 @@
 #include <linux/irqdomain.h>
 #include <asm/mach/arch.h>
 #include <asm/exception.h>
+#include <asm/smp_plat.h>
 
 /* Interrupt Controller Registers Map */
 #define ARMADA_370_XP_INT_SET_MASK_OFFS		(0x48)
@@ -35,6 +36,12 @@
 
 #define ARMADA_370_XP_CPU_INTACK_OFFS		(0x44)
 
+#define ARMADA_370_XP_SW_TRIG_INT_OFFS           (0x4)
+#define ARMADA_370_XP_IN_DRBEL_MSK_OFFS          (0xc)
+#define ARMADA_370_XP_IN_DRBEL_CAUSE_OFFS        (0x8)
+
+#define ACTIVE_DOORBELLS			(8)
+
 static void __iomem *per_cpu_int_base;
 static void __iomem *main_int_base;
 static struct irq_domain *armada_370_xp_mpic_domain;
@@ -51,11 +58,22 @@ static void armada_370_xp_irq_unmask(struct irq_data *d)
 	       per_cpu_int_base + ARMADA_370_XP_INT_CLEAR_MASK_OFFS);
 }
 
+#ifdef CONFIG_SMP
+static int armada_xp_set_affinity(struct irq_data *d,
+				  const struct cpumask *mask_val, bool force)
+{
+	return 0;
+}
+#endif
+
 static struct irq_chip armada_370_xp_irq_chip = {
 	.name		= "armada_370_xp_irq",
 	.irq_mask       = armada_370_xp_irq_mask,
 	.irq_mask_ack   = armada_370_xp_irq_mask,
 	.irq_unmask     = armada_370_xp_irq_unmask,
+#ifdef CONFIG_SMP
+	.irq_set_affinity = armada_xp_set_affinity,
+#endif
 };
 
 static int armada_370_xp_mpic_irq_map(struct irq_domain *h,
@@ -72,6 +90,41 @@ static int armada_370_xp_mpic_irq_map(struct irq_domain *h,
 	return 0;
 }
 
+#ifdef CONFIG_SMP
+void armada_mpic_send_doorbell(const struct cpumask *mask, unsigned int irq)
+{
+	int cpu;
+	unsigned long map = 0;
+
+	/* Convert our logical CPU mask into a physical one. */
+	for_each_cpu(cpu, mask)
+		map |= 1 << cpu_logical_map(cpu);
+
+	/*
+	 * Ensure that stores to Normal memory are visible to the
+	 * other CPUs before issuing the IPI.
+	 */
+	dsb();
+
+	/* submit softirq */
+	writel((map << 8) | irq, main_int_base +
+		ARMADA_370_XP_SW_TRIG_INT_OFFS);
+}
+
+void armada_xp_mpic_smp_cpu_init(void)
+{
+	/* Clear pending IPIs */
+	writel(0, per_cpu_int_base + ARMADA_370_XP_IN_DRBEL_CAUSE_OFFS);
+
+	/* Enable first 8 IPIs */
+	writel((1 << ACTIVE_DOORBELLS) - 1, per_cpu_int_base +
+		ARMADA_370_XP_IN_DRBEL_MSK_OFFS);
+
+	/* Unmask IPI interrupt */
+	writel(0, per_cpu_int_base + ARMADA_370_XP_INT_CLEAR_MASK_OFFS);
+}
+#endif /* CONFIG_SMP */
+
 static struct irq_domain_ops armada_370_xp_mpic_irq_ops = {
 	.map = armada_370_xp_mpic_irq_map,
 	.xlate = irq_domain_xlate_onecell,
@@ -91,13 +144,18 @@ static int __init armada_370_xp_mpic_of_init(struct device_node *node,
 	control = readl(main_int_base + ARMADA_370_XP_INT_CONTROL);
 
 	armada_370_xp_mpic_domain =
-	    irq_domain_add_linear(node, (control >> 2) & 0x3ff,
-				  &armada_370_xp_mpic_irq_ops, NULL);
+		irq_domain_add_linear(node, (control >> 2) & 0x3ff,
+				&armada_370_xp_mpic_irq_ops, NULL);
 
 	if (!armada_370_xp_mpic_domain)
 		panic("Unable to add Armada_370_Xp MPIC irq domain (DT)\n");
 
 	irq_set_default_host(armada_370_xp_mpic_domain);
+
+#ifdef CONFIG_SMP
+	armada_xp_mpic_smp_cpu_init();
+#endif
+
 	return 0;
 }
 
@@ -111,14 +169,36 @@ asmlinkage void __exception_irq_entry armada_370_xp_handle_irq(struct pt_regs
 					ARMADA_370_XP_CPU_INTACK_OFFS);
 		irqnr = irqstat & 0x3FF;
 
-		if (irqnr < 1023) {
-			irqnr =
-			    irq_find_mapping(armada_370_xp_mpic_domain, irqnr);
+		if (irqnr > 1022)
+			break;
+
+		if (irqnr >= 8) {
+			irqnr =	irq_find_mapping(armada_370_xp_mpic_domain,
+					irqnr);
 			handle_IRQ(irqnr, regs);
 			continue;
 		}
+#ifdef CONFIG_SMP
+		/* IPI Handling */
+		if (irqnr == 0) {
+			u32 ipimask, ipinr;
+
+			ipimask = readl_relaxed(per_cpu_int_base +
+						ARMADA_370_XP_IN_DRBEL_CAUSE_OFFS)
+				& 0xFF;
+
+			writel(0x0, per_cpu_int_base +
+				ARMADA_370_XP_IN_DRBEL_CAUSE_OFFS);
+
+			/* Handle all pending doorbells */
+			for (ipinr = 0; ipinr < ACTIVE_DOORBELLS; ipinr++) {
+				if (ipimask & (0x1 << ipinr))
+					handle_IPI(ipinr, regs);
+			}
+			continue;
+		}
+#endif
 
-		break;
 	} while (1);
 }
 
-- 
1.7.9.5

[PATCH V5 4/5] arm: mm: Added support for PJ4B cpu and init routines

[Gregory CLEMENT]

From: Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>

Signed-off-by: Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>
Signed-off-by: Gregory CLEMENT <gregory.clement-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
Reviewed-by: Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>
Acked-by: Catalin Marinas <catalin.marinas-5wv7dgnIgG8@public.gmane.org>
---
 arch/arm/mach-mvebu/Kconfig |    2 +-
 arch/arm/mm/Kconfig         |    4 +++
 arch/arm/mm/proc-v7.S       |   66 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 71 insertions(+), 1 deletion(-)

diff --git a/arch/arm/mach-mvebu/Kconfig b/arch/arm/mach-mvebu/Kconfig
index 79299cd..f4c3bf8 100644
--- a/arch/arm/mach-mvebu/Kconfig
+++ b/arch/arm/mach-mvebu/Kconfig
@@ -21,7 +21,7 @@ menu "Marvell SOC with device tree"
 config MACH_ARMADA_370_XP
 	bool
 	select ARMADA_370_XP_TIMER
-	select CPU_V7
+	select CPU_PJ4B
 
 config MACH_ARMADA_370
 	bool "Marvell Armada 370 boards"
diff --git a/arch/arm/mm/Kconfig b/arch/arm/mm/Kconfig
index 94186b6..3fd629d 100644
--- a/arch/arm/mm/Kconfig
+++ b/arch/arm/mm/Kconfig
@@ -352,6 +352,10 @@ config CPU_PJ4
 	select ARM_THUMBEE
 	select CPU_V7
 
+config CPU_PJ4B
+	bool
+	select CPU_V7
+
 # ARMv6
 config CPU_V6
 	bool "Support ARM V6 processor" if ARCH_INTEGRATOR || MACH_REALVIEW_EB || MACH_REALVIEW_PBX
diff --git a/arch/arm/mm/proc-v7.S b/arch/arm/mm/proc-v7.S
index 846d279..90f4c08 100644
--- a/arch/arm/mm/proc-v7.S
+++ b/arch/arm/mm/proc-v7.S
@@ -169,6 +169,62 @@ __v7_ca15mp_setup:
 	orreq	r0, r0, r10			@ Enable CPU-specific SMP bits
 	mcreq	p15, 0, r0, c1, c0, 1
 #endif
+
+__v7_pj4b_setup:
+#ifdef CONFIG_CPU_PJ4B
+
+/* Auxiliary Debug Modes Control 1 Register */
+#define STATIC_BP     (1 << 2) /* Enable Static BP */
+#define INTER_PARITY  (1 << 8) /* Disable Internal Parity Handling */
+#define BCK_OFF_STREX (1 << 5) /* Enable the back off of STREX instr */
+#define CLEAN_LINE    (1 << 16) /* Disable data transfer for clean line */
+
+/* Auxiliary Debug Modes Control 2 Register */
+#define FAST_LDR      (1 << 23) /* Disable fast LDR */
+#define SNOOP_DATA    (1 << 25) /* Do not interleave write and snoop data */
+#define CWF           (1 << 27) /* Disable Critical Word First feature */
+#define OUTSANDING_NC (1 << 29) /* Disable outstanding non cacheable request */
+#define L1_REP_RR     (1 << 30) /* L1 replacement - Strict round robin */
+#define AUX_DBG_CTRL2 (SNOOP_DATA | CWF | OUTSANDING_NC | L1_REP_RR)
+
+/* Auxiliary Functional Modes Control Register 0 */
+#define SMP_CFB         (1 << 1) /* Set SMP mode. Join the coherency fabric*/
+#define L1_PAR_CHK      (1 << 2) /* Support L1 parity checking */
+#define BROADCAST_CACHE (1 << 8) /* Broadcast Cache and TLB maintenance */
+
+/* Auxiliary Debug Modes Control 0 Register */
+#define WFI_WFE         (1 << 22) /* WFI/WFE - serve the DVM and back to idle*/
+
+	/* Auxiliary Debug Modes Control 1 Register */
+	mrc	p15, 1,	r0, c15, c1, 1
+	orr     r0, r0, #CLEAN_LINE
+	orr     r0, r0, #BCK_OFF_STREX
+	orr     r0, r0, #INTER_PARITY
+	bic	r0, r0, #STATIC_BP
+	mcr	p15, 1,	r0, c15, c1, 1
+
+	/* Auxiliary Debug Modes Control 2 Register */
+	mrc	p15, 1,	r0, c15, c1, 2
+	bic	r0, r0, #FAST_LDR
+	orr	r0, r0, #AUX_DBG_CTRL2
+	mcr	p15, 1,	r0, c15, c1, 2
+
+	/* Auxiliary Functional Modes Control Register 0 */
+	mrc	p15, 1,	r0, c15, c2, 0
+#ifdef CONFIG_SMP
+	orr	r0, r0, #SMP_CFB
+#endif
+	orr	r0, r0, #L1_PAR_CHK
+	orr	r0, r0, #BROADCAST_CACHE
+	mcr	p15, 1,	r0, c15, c2, 0
+
+	/* Auxiliary Debug Modes Control 0 Register */
+	mrc	p15, 1,	r0, c15, c1, 0
+	orr	r0, r0, #WFI_WFE
+	mcr	p15, 1,	r0, c15, c1, 0
+
+#endif /* CONFIG_CPU_PJ4B */
+
 __v7_setup:
 	adr	r12, __v7_setup_stack		@ the local stack
 	stmia	r12, {r0-r5, r7, r9, r11, lr}
@@ -342,6 +398,16 @@ __v7_ca9mp_proc_info:
 	.long	0xff0ffff0
 	__v7_proc __v7_ca9mp_setup
 	.size	__v7_ca9mp_proc_info, . - __v7_ca9mp_proc_info
+
+	/*
+	 * Marvell PJ4B processor.
+	 */
+	.type   __v7_pj4b_proc_info, #object
+__v7_pj4b_proc_info:
+	.long	0x562f5840
+	.long	0xfffffff0
+	__v7_proc __v7_pj4b_setup
+	.size	__v7_pj4b_proc_info, . - __v7_pj4b_proc_info
 #endif	/* CONFIG_ARM_LPAE */
 
 	/*
-- 
1.7.9.5

[PATCH V5 5/5] arm: mvebu: Added SMP support for Armada XP

[Gregory CLEMENT]

From: Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>

1. added smp init functions in platsmp.c
2. added secondary cpu entry point in headsmp.S
3. added hotplog initial support in hotplug.c
4. added SMP support for PJ4B cpu

Signed-off-by: Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>
Signed-off-by: Gregory CLEMENT <gregory.clement-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
---
 arch/arm/configs/mvebu_defconfig    |    3 +
 arch/arm/mach-mvebu/Kconfig         |    1 +
 arch/arm/mach-mvebu/Makefile        |    2 +
 arch/arm/mach-mvebu/armada-370-xp.c |    3 +
 arch/arm/mach-mvebu/common.h        |    3 +
 arch/arm/mach-mvebu/headsmp.S       |   50 +++++++++++++++
 arch/arm/mach-mvebu/hotplug.c       |   30 +++++++++
 arch/arm/mach-mvebu/platsmp.c       |  121 +++++++++++++++++++++++++++++++++++
 8 files changed, 213 insertions(+)
 create mode 100644 arch/arm/mach-mvebu/headsmp.S
 create mode 100644 arch/arm/mach-mvebu/hotplug.c
 create mode 100644 arch/arm/mach-mvebu/platsmp.c

diff --git a/arch/arm/configs/mvebu_defconfig b/arch/arm/configs/mvebu_defconfig
index 3458752..da598d3 100644
--- a/arch/arm/configs/mvebu_defconfig
+++ b/arch/arm/configs/mvebu_defconfig
@@ -12,6 +12,9 @@ CONFIG_ARCH_MVEBU=y
 CONFIG_MACH_ARMADA_370=y
 CONFIG_MACH_ARMADA_XP=y
 # CONFIG_CACHE_L2X0 is not set
+# CONFIG_SWP_EMULATE is not set
+CONFIG_SMP=y
+# CONFIG_LOCAL_TIMERS is not set
 CONFIG_AEABI=y
 CONFIG_HIGHMEM=y
 # CONFIG_COMPACTION is not set
diff --git a/arch/arm/mach-mvebu/Kconfig b/arch/arm/mach-mvebu/Kconfig
index f4c3bf8..c934e1d 100644
--- a/arch/arm/mach-mvebu/Kconfig
+++ b/arch/arm/mach-mvebu/Kconfig
@@ -21,6 +21,7 @@ menu "Marvell SOC with device tree"
 config MACH_ARMADA_370_XP
 	bool
 	select ARMADA_370_XP_TIMER
+	select HAVE_SMP
 	select CPU_PJ4B
 
 config MACH_ARMADA_370
diff --git a/arch/arm/mach-mvebu/Makefile b/arch/arm/mach-mvebu/Makefile
index 2e3ec11..5dcb369 100644
--- a/arch/arm/mach-mvebu/Makefile
+++ b/arch/arm/mach-mvebu/Makefile
@@ -3,3 +3,5 @@ ccflags-$(CONFIG_ARCH_MULTIPLATFORM) := -I$(srctree)/$(src)/include \
 
 obj-y += system-controller.o
 obj-$(CONFIG_MACH_ARMADA_370_XP) += armada-370-xp.o irq-armada-370-xp.o addr-map.o coherency.o coherency_ll.o pmsu.o
+obj-$(CONFIG_SMP)                += platsmp.o headsmp.o
+obj-$(CONFIG_HOTPLUG_CPU)        += hotplug.o
diff --git a/arch/arm/mach-mvebu/armada-370-xp.c b/arch/arm/mach-mvebu/armada-370-xp.c
index 3292d6d..472e70f 100644
--- a/arch/arm/mach-mvebu/armada-370-xp.c
+++ b/arch/arm/mach-mvebu/armada-370-xp.c
@@ -23,6 +23,7 @@
 #include <asm/mach/time.h>
 #include "armada-370-xp.h"
 #include "common.h"
+#include "coherency.h"
 
 static struct map_desc armada_370_xp_io_desc[] __initdata = {
 	{
@@ -51,6 +52,7 @@ struct sys_timer armada_370_xp_timer = {
 static void __init armada_370_xp_dt_init(void)
 {
 	of_platform_populate(NULL, of_default_bus_match_table, NULL, NULL);
+	coherency_init();
 }
 
 static const char * const armada_370_xp_dt_board_dt_compat[] = {
@@ -60,6 +62,7 @@ static const char * const armada_370_xp_dt_board_dt_compat[] = {
 };
 
 DT_MACHINE_START(ARMADA_XP_DT, "Marvell Aramada 370/XP (Device Tree)")
+	.smp		= smp_ops(armada_xp_smp_ops),
 	.init_machine	= armada_370_xp_dt_init,
 	.map_io		= armada_370_xp_map_io,
 	.init_irq	= armada_370_xp_init_irq,
diff --git a/arch/arm/mach-mvebu/common.h b/arch/arm/mach-mvebu/common.h
index b5cd7e7..b711869 100644
--- a/arch/arm/mach-mvebu/common.h
+++ b/arch/arm/mach-mvebu/common.h
@@ -20,7 +20,10 @@ void mvebu_restart(char mode, const char *cmd);
 void armada_370_xp_init_irq(void);
 void armada_370_xp_handle_irq(struct pt_regs *regs);
 
+void armada_xp_cpu_die(unsigned int cpu);
 
 int armada_370_xp_coherency_init(void);
 int armada_370_xp_pmsu_init(void);
+void armada_xp_secondary_startup(void);
+extern struct smp_operations armada_xp_smp_ops;
 #endif
diff --git a/arch/arm/mach-mvebu/headsmp.S b/arch/arm/mach-mvebu/headsmp.S
new file mode 100644
index 0000000..994eecc
--- /dev/null
+++ b/arch/arm/mach-mvebu/headsmp.S
@@ -0,0 +1,50 @@
+/*
+ * SMP support: Entry point for secondary CPUs
+ *
+ * Copyright (C) 2012 Marvell
+ *
+ * Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>
+ * Gregory CLEMENT <gregory.clement-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
+ * Thomas Petazzoni <thomas.petazzoni-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ *
+ * This file implements the assembly entry point for secondary CPUs in
+ * an SMP kernel. The only thing we need to do is to add the CPU to
+ * the coherency fabric by writing to 2 registers. Currently the base
+ * register addresses are hard coded due to the early initialisation
+ * problems.
+ */
+
+#include <linux/linkage.h>
+#include <linux/init.h>
+
+/*
+ * At this stage the secondary CPUs don't have acces yet to the MMU, so
+ * we have to provide physical addresses
+ */
+#define ARMADA_XP_CFB_BASE	     0xD0020200
+
+	__CPUINIT
+
+/*
+ * Armada XP specific entry point for secondary CPUs.
+ * We add the CPU to the coherency fabric and then jump to secondary
+ * startup
+ */
+ENTRY(armada_xp_secondary_startup)
+
+	/* Read CPU id */
+ 	mrc     p15, 0, r1, c0, c0, 5
+	and     r1, r1, #0xF
+
+	/* Add CPU to coherency fabric */
+	ldr     r0, = ARMADA_XP_CFB_BASE
+	mov	lr, pc
+
+	b	ll_set_cpu_coherent
+	b	secondary_startup
+
+ENDPROC(armada_xp_secondary_startup)
diff --git a/arch/arm/mach-mvebu/hotplug.c b/arch/arm/mach-mvebu/hotplug.c
new file mode 100644
index 0000000..b228b6a
--- /dev/null
+++ b/arch/arm/mach-mvebu/hotplug.c
@@ -0,0 +1,30 @@
+/*
+ * Symmetric Multi Processing (SMP) support for Armada XP
+ *
+ * Copyright (C) 2012 Marvell
+ *
+ * Lior Amsalem <alior-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>
+ * Gregory CLEMENT <gregory.clement-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
+ * Thomas Petazzoni <thomas.petazzoni-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/smp.h>
+#include <asm/proc-fns.h>
+
+/*
+ * platform-specific code to shutdown a CPU
+ *
+ * Called with IRQs disabled
+ */
+void __ref armada_xp_cpu_die(unsigned int cpu)
+{
+	cpu_do_idle();
+
+	/* We should never return from idle */
+	panic("mvebu: cpu %d unexpectedly exit from shutdown\n", cpu);
+}
diff --git a/arch/arm/mach-mvebu/platsmp.c b/arch/arm/mach-mvebu/platsmp.c
new file mode 100644
index 0000000..5400f82
--- /dev/null
+++ b/arch/arm/mach-mvebu/platsmp.c
@@ -0,0 +1,121 @@
+/*
+ * Symmetric Multi Processing (SMP) support for Armada XP
+ *
+ * Copyright (C) 2012 Marvell
+ *
+ * Lior Amsalem <alior-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>
+ * Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>
+ * Gregory CLEMENT <gregory.clement-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
+ * Thomas Petazzoni <thomas.petazzoni-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ *
+ * The Armada XP SoC has 4 ARMv7 PJ4B CPUs running in full HW coherency
+ * This file implements the routines for preparing the SMP infrastructure
+ * and waking up the secondary CPUs
+ */
+
+#include <linux/init.h>
+#include <linux/smp.h>
+#include <linux/clk.h>
+#include <linux/of.h>
+#include <asm/cacheflush.h>
+#include <asm/smp_plat.h>
+#include "common.h"
+#include "armada-370-xp.h"
+#include "pmsu.h"
+#include "coherency.h"
+
+void __init set_secondary_cpus_clock(void)
+{
+	int thiscpu;
+	unsigned long rate;
+	struct clk *cpu_clk = NULL;
+	struct device_node *np = NULL;
+
+	thiscpu = smp_processor_id();
+	for_each_node_by_type(np, "cpu") {
+		int err;
+		int cpu;
+
+		err = of_property_read_u32(np, "reg", &cpu);
+		if (WARN_ON(err))
+			return;
+
+		if (cpu == thiscpu) {
+			cpu_clk = of_clk_get(np, 0);
+			break;
+		}
+	}
+	WARN_ON(IS_ERR(cpu_clk));
+	clk_prepare_enable(cpu_clk);
+	rate = clk_get_rate(cpu_clk);
+
+	/* set all the other CPU clk to the same rate than the boot CPU */
+	for_each_node_by_type(np, "cpu") {
+		int err;
+		int cpu;
+
+		err = of_property_read_u32(np, "reg", &cpu);
+		if (WARN_ON(err))
+			return;
+
+		if (cpu != thiscpu) {
+			cpu_clk = of_clk_get(np, 0);
+			clk_set_rate(cpu_clk, rate);
+		}
+	}
+}
+
+static void __cpuinit armada_xp_secondary_init(unsigned int cpu)
+{
+	armada_xp_mpic_smp_cpu_init();
+}
+
+static int __cpuinit armada_xp_boot_secondary(unsigned int cpu,
+					      struct task_struct *idle)
+{
+	pr_info("Booting CPU %d\n", cpu);
+
+	armada_xp_boot_cpu(cpu, armada_xp_secondary_startup);
+
+	return 0;
+}
+
+static void __init armada_xp_smp_init_cpus(void)
+{
+	unsigned int i, ncores;
+	ncores = coherency_get_cpu_count();
+
+	/* Limit possbile CPUs to defconfig */
+	if (ncores > nr_cpu_ids) {
+		pr_warn("SMP: %d CPUs physically present. Only %d configured.",
+			ncores, nr_cpu_ids);
+		pr_warn("Clipping CPU count to %d\n", nr_cpu_ids);
+		ncores = nr_cpu_ids;
+	}
+
+	for (i = 0; i < ncores; i++)
+		set_cpu_possible(i, true);
+
+	set_smp_cross_call(armada_mpic_send_doorbell);
+}
+
+void __init armada_xp_smp_prepare_cpus(unsigned int max_cpus)
+{
+	set_secondary_cpus_clock();
+	flush_cache_all();
+	set_cpu_coherent(cpu_logical_map(smp_processor_id()), 0);
+}
+
+struct smp_operations armada_xp_smp_ops __initdata = {
+	.smp_init_cpus		= armada_xp_smp_init_cpus,
+	.smp_prepare_cpus	= armada_xp_smp_prepare_cpus,
+	.smp_secondary_init	= armada_xp_secondary_init,
+	.smp_boot_secondary	= armada_xp_boot_secondary,
+#ifdef CONFIG_HOTPLUG_CPU
+	.cpu_die		= armada_xp_cpu_die,
+#endif
+};
-- 
1.7.9.5

Re: [PATCH V5 1/5] arm: mvebu: Added support for coherency fabric in mach-mvebu

[Will Deacon]

On Tue, Nov 20, 2012 at 09:15:45PM +0000, Gregory CLEMENT wrote:
> From: Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>

<insert commit log here>

Also -- is the author attribution still correct?

> Signed-off-by: Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>
> Signed-off-by: Gregory CLEMENT <gregory.clement-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
> ---
>  .../devicetree/bindings/arm/coherency-fabric.txt   |   16 ++++
>  arch/arm/boot/dts/armada-370-xp.dtsi               |    5 ++
>  arch/arm/mach-mvebu/Makefile                       |    2 +-
>  arch/arm/mach-mvebu/coherency.c                    |   80 ++++++++++++++++++++
>  arch/arm/mach-mvebu/coherency.h                    |   24 ++++++
>  arch/arm/mach-mvebu/coherency_ll.S                 |   47 ++++++++++++
>  arch/arm/mach-mvebu/common.h                       |    2 +
>  7 files changed, 175 insertions(+), 1 deletion(-)
>  create mode 100644 Documentation/devicetree/bindings/arm/coherency-fabric.txt
>  create mode 100644 arch/arm/mach-mvebu/coherency.c
>  create mode 100644 arch/arm/mach-mvebu/coherency.h
>  create mode 100644 arch/arm/mach-mvebu/coherency_ll.S
> 
> diff --git a/Documentation/devicetree/bindings/arm/coherency-fabric.txt b/Documentation/devicetree/bindings/arm/coherency-fabric.txt
> new file mode 100644
> index 0000000..2bfbf67
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/arm/coherency-fabric.txt
> @@ -0,0 +1,16 @@
> +Coherency fabric
> +----------------
> +Available on Marvell SOCs: Armada 370 and Armada XP
> +
> +Required properties:
> +
> +- compatible: "marvell,coherency-fabric"
> +- reg: Should contain,coherency fabric registers location and length.
> +
> +Example:
> +
> +coherency-fabric@d0020200 {
> +	compatible = "marvell,coherency-fabric";
> +	reg = <0xd0020200 0xb0>;
> +};
> +
> diff --git a/arch/arm/boot/dts/armada-370-xp.dtsi b/arch/arm/boot/dts/armada-370-xp.dtsi
> index 94b4b9e..b0d075b 100644
> --- a/arch/arm/boot/dts/armada-370-xp.dtsi
> +++ b/arch/arm/boot/dts/armada-370-xp.dtsi
> @@ -36,6 +36,11 @@
>  	      interrupt-controller;
>  	};
>  
> +	coherency-fabric@d0020200 {
> +		compatible = "marvell,coherency-fabric";
> +		reg = <0xd0020200 0xb0>;
> +	};
> +
>  	soc {
>  		#address-cells = <1>;
>  		#size-cells = <1>;
> diff --git a/arch/arm/mach-mvebu/Makefile b/arch/arm/mach-mvebu/Makefile
> index 57f996b..5ce4b42 100644
> --- a/arch/arm/mach-mvebu/Makefile
> +++ b/arch/arm/mach-mvebu/Makefile
> @@ -2,4 +2,4 @@ ccflags-$(CONFIG_ARCH_MULTIPLATFORM) := -I$(srctree)/$(src)/include \
>  	-I$(srctree)/arch/arm/plat-orion/include
>  
>  obj-y += system-controller.o
> -obj-$(CONFIG_MACH_ARMADA_370_XP) += armada-370-xp.o irq-armada-370-xp.o addr-map.o
> +obj-$(CONFIG_MACH_ARMADA_370_XP) += armada-370-xp.o irq-armada-370-xp.o addr-map.o coherency.o coherency_ll.o
> diff --git a/arch/arm/mach-mvebu/coherency.c b/arch/arm/mach-mvebu/coherency.c
> new file mode 100644
> index 0000000..1bc02d0
> --- /dev/null
> +++ b/arch/arm/mach-mvebu/coherency.c
> @@ -0,0 +1,80 @@
> +/*
> + * Coherency fabric (Aurora) support for Armada 370 and XP platforms.
> + *
> + * Copyright (C) 2012 Marvell
> + *
> + * Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>
> + * Gregory Clement <gregory.clement-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
> + * Thomas Petazzoni <thomas.petazzoni-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
> + *
> + * This file is licensed under the terms of the GNU General Public
> + * License version 2.  This program is licensed "as is" without any
> + * warranty of any kind, whether express or implied.
> + *
> + * The Armada 370 and Armada XP SOCs have a coherency fabric which is
> + * responsible for ensuring hardware coherency between all CPUs and between
> + * CPUs and I/O masters. This file initializes the coherency fabric and
> + * supplies basic routines for configuring and controlling hardware coherency
> + */
> +
> +#include <linux/kernel.h>
> +#include <linux/init.h>
> +#include <linux/of_address.h>
> +#include <linux/io.h>
> +#include <linux/smp.h>
> +#include <asm/smp_plat.h>
> +#include "armada-370-xp.h"
> +
> +/* Some functions in this file are called very early during SMP
> + * initialization. At that time the device tree framework is not yet
> + * ready, and it is not possible to get the register address to
> + * ioremap it. That's why the pointer below is given with an initial
> + * value matching its virtual mapping
> + */
> +static void __iomem *coherency_base = ARMADA_370_XP_REGS_VIRT_BASE + 0x20200;
> +
> +/* Coherency fabric registers */
> +#define COHERENCY_FABRIC_CFG_OFFSET		   0x4
> +
> +static struct of_device_id of_coherency_table[] = {
> +	{.compatible = "marvell,coherency-fabric"},
> +	{ /* end of list */ },
> +};
> +#ifdef CONFIG_SMP
> +int coherency_get_cpu_count(void)
> +{
> +	int reg, cnt;
> +
> +	reg = readl(coherency_base + COHERENCY_FABRIC_CFG_OFFSET);
> +	cnt = (reg & 0xF) + 1;
> +
> +	return cnt;
> +}
> +#endif
> +/* Function defined in coherncy_ll.S */

coherency_ll.S

> +extern void ll_set_cpu_coherent(void __iomem *base_addr,
> +				unsigned int hw_cpu_id);
> +
> +int set_cpu_coherent(unsigned int hw_cpu_id, int smp_group_id)
> +{
> +	if (!coherency_base) {
> +		pr_warn("Can't make CPU %d cache coherent.\n", hw_cpu_id);
> +		pr_warn("Coherency fabric is not initialized\n");
> +		return 1;
> +	}
> +	ll_set_cpu_coherent(coherency_base, hw_cpu_id);
> +	return 0;

You can just do return ll_set_cpu_coherent(...) now.

> +}
> +
> +int __init coherency_init(void)
> +{
> +	struct device_node *np;
> +
> +	np = of_find_matching_node(NULL, of_coherency_table);
> +	if (np) {
> +		pr_info("Initializing Coherency fabric\n");
> +		coherency_base = of_iomap(np, 0);
> +	}
> +
> +	return 0;
> +}
> diff --git a/arch/arm/mach-mvebu/coherency.h b/arch/arm/mach-mvebu/coherency.h
> new file mode 100644
> index 0000000..2f42813
> --- /dev/null
> +++ b/arch/arm/mach-mvebu/coherency.h
> @@ -0,0 +1,24 @@
> +/*
> + * arch/arm/mach-mvebu/include/mach/coherency.h
> + *
> + *
> + * Coherency fabric (Aurora) support for Armada 370 and XP platforms.
> + *
> + * Copyright (C) 2012 Marvell
> + *
> + * This file is licensed under the terms of the GNU General Public
> + * License version 2.  This program is licensed "as is" without any
> + * warranty of any kind, whether express or implied.
> + */
> +
> +#ifndef __MACH_370_XP_COHERENCY_H
> +#define __MACH_370_XP_COHERENCY_H
> +
> +#ifdef CONFIG_SMP
> +int coherency_get_cpu_count(void);
> +#endif
> +
> +int set_cpu_coherent(int cpu_id, int smp_group_id);
> +int coherency_init(void);
> +
> +#endif	/* __MACH_370_XP_COHERENCY_H */
> diff --git a/arch/arm/mach-mvebu/coherency_ll.S b/arch/arm/mach-mvebu/coherency_ll.S
> new file mode 100644
> index 0000000..ae48730
> --- /dev/null
> +++ b/arch/arm/mach-mvebu/coherency_ll.S
> @@ -0,0 +1,47 @@
> +/*
> + * Coherency fabric: low level functions
> + *
> + * Copyright (C) 2012 Marvell
> + *
> + * Gregory CLEMENT <gregory.clement-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
> + *
> + * This file is licensed under the terms of the GNU General Public
> + * License version 2.  This program is licensed "as is" without any
> + * warranty of any kind, whether express or implied.
> + *
> + * This file implements the assembly function to add a CPU to the
> + * coherency fabric. This function is called by each of the secondary
> + * CPUs during their early boot in an SMP kernel, this why this
> + * function have to callable from assembly. It can also be called by a
> + * primary CPU from C code during its boot.
> + */
> +
> +#include <linux/linkage.h>
> +#define ARMADA_XP_CFB_CTL_REG_OFFSET 0x0
> +#define ARMADA_XP_CFB_CFG_REG_OFFSET 0x4
> +
> +	.text
> +/*
> + * r0: Coherency fabric base register address
> + * r1: HW CPU id
> + */
> +ENTRY(ll_set_cpu_coherent)
> +	/* Create bit by cpu index */
> +	mov	r3, #(1 << 24)
> +	lsl     r1, r3, r1
> +
> +	/* Add CPU to SMP group - Atomic */
> +	add	r3, r0, #ARMADA_XP_CFB_CTL_REG_OFFSET
> +	ldr     r2, [r3]
> +	orr     r2, r2, r1
> +	str	r2, [r0]
> +
> +	/* Enable coherency on CPU - Atomic */
> +	add	r3, r0, #ARMADA_XP_CFB_CFG_REG_OFFSET
> +	ldr     r2, [r3]
> +	orr     r2, r2, r1
> +	str     r2, [r3]

I forgot to mention this before, but do you need a dsb or something here?

Will

Re: [PATCH V5 5/5] arm: mvebu: Added SMP support for Armada XP

[Will Deacon]

On Tue, Nov 20, 2012 at 09:15:49PM +0000, Gregory CLEMENT wrote:
> From: Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>
> 
> 1. added smp init functions in platsmp.c
> 2. added secondary cpu entry point in headsmp.S
> 3. added hotplog initial support in hotplug.c

hotplug

> 4. added SMP support for PJ4B cpu

Again, I think the commit message could probably be improved from a numbered
list...

> Signed-off-by: Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>
> Signed-off-by: Gregory CLEMENT <gregory.clement-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
> ---
>  arch/arm/configs/mvebu_defconfig    |    3 +
>  arch/arm/mach-mvebu/Kconfig         |    1 +
>  arch/arm/mach-mvebu/Makefile        |    2 +
>  arch/arm/mach-mvebu/armada-370-xp.c |    3 +
>  arch/arm/mach-mvebu/common.h        |    3 +
>  arch/arm/mach-mvebu/headsmp.S       |   50 +++++++++++++++
>  arch/arm/mach-mvebu/hotplug.c       |   30 +++++++++
>  arch/arm/mach-mvebu/platsmp.c       |  121 +++++++++++++++++++++++++++++++++++
>  8 files changed, 213 insertions(+)
>  create mode 100644 arch/arm/mach-mvebu/headsmp.S
>  create mode 100644 arch/arm/mach-mvebu/hotplug.c
>  create mode 100644 arch/arm/mach-mvebu/platsmp.c
> 
> diff --git a/arch/arm/configs/mvebu_defconfig b/arch/arm/configs/mvebu_defconfig
> index 3458752..da598d3 100644
> --- a/arch/arm/configs/mvebu_defconfig
> +++ b/arch/arm/configs/mvebu_defconfig
> @@ -12,6 +12,9 @@ CONFIG_ARCH_MVEBU=y
>  CONFIG_MACH_ARMADA_370=y
>  CONFIG_MACH_ARMADA_XP=y
>  # CONFIG_CACHE_L2X0 is not set
> +# CONFIG_SWP_EMULATE is not set
> +CONFIG_SMP=y
> +# CONFIG_LOCAL_TIMERS is not set
>  CONFIG_AEABI=y
>  CONFIG_HIGHMEM=y
>  # CONFIG_COMPACTION is not set
> diff --git a/arch/arm/mach-mvebu/Kconfig b/arch/arm/mach-mvebu/Kconfig
> index f4c3bf8..c934e1d 100644
> --- a/arch/arm/mach-mvebu/Kconfig
> +++ b/arch/arm/mach-mvebu/Kconfig
> @@ -21,6 +21,7 @@ menu "Marvell SOC with device tree"
>  config MACH_ARMADA_370_XP
>  	bool
>  	select ARMADA_370_XP_TIMER
> +	select HAVE_SMP
>  	select CPU_PJ4B
>  
>  config MACH_ARMADA_370
> diff --git a/arch/arm/mach-mvebu/Makefile b/arch/arm/mach-mvebu/Makefile
> index 2e3ec11..5dcb369 100644
> --- a/arch/arm/mach-mvebu/Makefile
> +++ b/arch/arm/mach-mvebu/Makefile
> @@ -3,3 +3,5 @@ ccflags-$(CONFIG_ARCH_MULTIPLATFORM) := -I$(srctree)/$(src)/include \
>  
>  obj-y += system-controller.o
>  obj-$(CONFIG_MACH_ARMADA_370_XP) += armada-370-xp.o irq-armada-370-xp.o addr-map.o coherency.o coherency_ll.o pmsu.o
> +obj-$(CONFIG_SMP)                += platsmp.o headsmp.o
> +obj-$(CONFIG_HOTPLUG_CPU)        += hotplug.o
> diff --git a/arch/arm/mach-mvebu/armada-370-xp.c b/arch/arm/mach-mvebu/armada-370-xp.c
> index 3292d6d..472e70f 100644
> --- a/arch/arm/mach-mvebu/armada-370-xp.c
> +++ b/arch/arm/mach-mvebu/armada-370-xp.c
> @@ -23,6 +23,7 @@
>  #include <asm/mach/time.h>
>  #include "armada-370-xp.h"
>  #include "common.h"
> +#include "coherency.h"
>  
>  static struct map_desc armada_370_xp_io_desc[] __initdata = {
>  	{
> @@ -51,6 +52,7 @@ struct sys_timer armada_370_xp_timer = {
>  static void __init armada_370_xp_dt_init(void)
>  {
>  	of_platform_populate(NULL, of_default_bus_match_table, NULL, NULL);
> +	coherency_init();
>  }
>  
>  static const char * const armada_370_xp_dt_board_dt_compat[] = {
> @@ -60,6 +62,7 @@ static const char * const armada_370_xp_dt_board_dt_compat[] = {
>  };
>  
>  DT_MACHINE_START(ARMADA_XP_DT, "Marvell Aramada 370/XP (Device Tree)")
> +	.smp		= smp_ops(armada_xp_smp_ops),
>  	.init_machine	= armada_370_xp_dt_init,
>  	.map_io		= armada_370_xp_map_io,
>  	.init_irq	= armada_370_xp_init_irq,
> diff --git a/arch/arm/mach-mvebu/common.h b/arch/arm/mach-mvebu/common.h
> index b5cd7e7..b711869 100644
> --- a/arch/arm/mach-mvebu/common.h
> +++ b/arch/arm/mach-mvebu/common.h
> @@ -20,7 +20,10 @@ void mvebu_restart(char mode, const char *cmd);
>  void armada_370_xp_init_irq(void);
>  void armada_370_xp_handle_irq(struct pt_regs *regs);
>  
> +void armada_xp_cpu_die(unsigned int cpu);
>  
>  int armada_370_xp_coherency_init(void);
>  int armada_370_xp_pmsu_init(void);
> +void armada_xp_secondary_startup(void);
> +extern struct smp_operations armada_xp_smp_ops;
>  #endif
> diff --git a/arch/arm/mach-mvebu/headsmp.S b/arch/arm/mach-mvebu/headsmp.S
> new file mode 100644
> index 0000000..994eecc
> --- /dev/null
> +++ b/arch/arm/mach-mvebu/headsmp.S
> @@ -0,0 +1,50 @@
> +/*
> + * SMP support: Entry point for secondary CPUs
> + *
> + * Copyright (C) 2012 Marvell
> + *
> + * Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>
> + * Gregory CLEMENT <gregory.clement-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
> + * Thomas Petazzoni <thomas.petazzoni-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
> + *
> + * This file is licensed under the terms of the GNU General Public
> + * License version 2.  This program is licensed "as is" without any
> + * warranty of any kind, whether express or implied.
> + *
> + * This file implements the assembly entry point for secondary CPUs in
> + * an SMP kernel. The only thing we need to do is to add the CPU to
> + * the coherency fabric by writing to 2 registers. Currently the base
> + * register addresses are hard coded due to the early initialisation
> + * problems.
> + */
> +
> +#include <linux/linkage.h>
> +#include <linux/init.h>
> +
> +/*
> + * At this stage the secondary CPUs don't have acces yet to the MMU, so
> + * we have to provide physical addresses
> + */
> +#define ARMADA_XP_CFB_BASE	     0xD0020200
> +
> +	__CPUINIT
> +
> +/*
> + * Armada XP specific entry point for secondary CPUs.
> + * We add the CPU to the coherency fabric and then jump to secondary
> + * startup
> + */
> +ENTRY(armada_xp_secondary_startup)
> +
> +	/* Read CPU id */
> + 	mrc     p15, 0, r1, c0, c0, 5
> +	and     r1, r1, #0xF
> +
> +	/* Add CPU to coherency fabric */
> +	ldr     r0, = ARMADA_XP_CFB_BASE
> +	mov	lr, pc
> +
> +	b	ll_set_cpu_coherent
> +	b	secondary_startup

Why didn't you use my adr suggestion here?

Will

Re: [PATCH V5 5/5] arm: mvebu: Added SMP support for Armada XP

[Russell King - ARM Linux]

On Tue, Nov 20, 2012 at 10:15:49PM +0100, Gregory CLEMENT wrote:
> +ENTRY(armada_xp_secondary_startup)
> +
> +	/* Read CPU id */
> + 	mrc     p15, 0, r1, c0, c0, 5
> +	and     r1, r1, #0xF
> +
> +	/* Add CPU to coherency fabric */
> +	ldr     r0, = ARMADA_XP_CFB_BASE
> +	mov	lr, pc
> +
> +	b	ll_set_cpu_coherent
> +	b	secondary_startup

This looks odd.  Why are you doing a mov lr,pc followed by a plain branch?
What's wrong with 'bl' here ?

Re: [PATCH V5 1/5] arm: mvebu: Added support for coherency fabric in mach-mvebu

[Gregory CLEMENT]

On 11/21/2012 11:36 AM, Will Deacon wrote:
> On Tue, Nov 20, 2012 at 09:15:45PM +0000, Gregory CLEMENT wrote:
>> From: Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>
> 
> <insert commit log here>
> 
> Also -- is the author attribution still correct?

I don't know: the code is based on Yehuda work, but version
by version I modified it. So I don't know what is the rule
for this.


> 
>> Signed-off-by: Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>
>> Signed-off-by: Gregory CLEMENT <gregory.clement-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
>> ---
>>  .../devicetree/bindings/arm/coherency-fabric.txt   |   16 ++++
>>  arch/arm/boot/dts/armada-370-xp.dtsi               |    5 ++
>>  arch/arm/mach-mvebu/Makefile                       |    2 +-
>>  arch/arm/mach-mvebu/coherency.c                    |   80 ++++++++++++++++++++
>>  arch/arm/mach-mvebu/coherency.h                    |   24 ++++++
>>  arch/arm/mach-mvebu/coherency_ll.S                 |   47 ++++++++++++
>>  arch/arm/mach-mvebu/common.h                       |    2 +
>>  7 files changed, 175 insertions(+), 1 deletion(-)
>>  create mode 100644 Documentation/devicetree/bindings/arm/coherency-fabric.txt
>>  create mode 100644 arch/arm/mach-mvebu/coherency.c
>>  create mode 100644 arch/arm/mach-mvebu/coherency.h
>>  create mode 100644 arch/arm/mach-mvebu/coherency_ll.S
>>
>> diff --git a/Documentation/devicetree/bindings/arm/coherency-fabric.txt b/Documentation/devicetree/bindings/arm/coherency-fabric.txt
>> new file mode 100644
>> index 0000000..2bfbf67
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/arm/coherency-fabric.txt
>> @@ -0,0 +1,16 @@
>> +Coherency fabric
>> +----------------
>> +Available on Marvell SOCs: Armada 370 and Armada XP
>> +
>> +Required properties:
>> +
>> +- compatible: "marvell,coherency-fabric"
>> +- reg: Should contain,coherency fabric registers location and length.
>> +
>> +Example:
>> +
>> +coherency-fabric@d0020200 {
>> +	compatible = "marvell,coherency-fabric";
>> +	reg = <0xd0020200 0xb0>;
>> +};
>> +
>> diff --git a/arch/arm/boot/dts/armada-370-xp.dtsi b/arch/arm/boot/dts/armada-370-xp.dtsi
>> index 94b4b9e..b0d075b 100644
>> --- a/arch/arm/boot/dts/armada-370-xp.dtsi
>> +++ b/arch/arm/boot/dts/armada-370-xp.dtsi
>> @@ -36,6 +36,11 @@
>>  	      interrupt-controller;
>>  	};
>>  
>> +	coherency-fabric@d0020200 {
>> +		compatible = "marvell,coherency-fabric";
>> +		reg = <0xd0020200 0xb0>;
>> +	};
>> +
>>  	soc {
>>  		#address-cells = <1>;
>>  		#size-cells = <1>;
>> diff --git a/arch/arm/mach-mvebu/Makefile b/arch/arm/mach-mvebu/Makefile
>> index 57f996b..5ce4b42 100644
>> --- a/arch/arm/mach-mvebu/Makefile
>> +++ b/arch/arm/mach-mvebu/Makefile
>> @@ -2,4 +2,4 @@ ccflags-$(CONFIG_ARCH_MULTIPLATFORM) := -I$(srctree)/$(src)/include \
>>  	-I$(srctree)/arch/arm/plat-orion/include
>>  
>>  obj-y += system-controller.o
>> -obj-$(CONFIG_MACH_ARMADA_370_XP) += armada-370-xp.o irq-armada-370-xp.o addr-map.o
>> +obj-$(CONFIG_MACH_ARMADA_370_XP) += armada-370-xp.o irq-armada-370-xp.o addr-map.o coherency.o coherency_ll.o
>> diff --git a/arch/arm/mach-mvebu/coherency.c b/arch/arm/mach-mvebu/coherency.c
>> new file mode 100644
>> index 0000000..1bc02d0
>> --- /dev/null
>> +++ b/arch/arm/mach-mvebu/coherency.c
>> @@ -0,0 +1,80 @@
>> +/*
>> + * Coherency fabric (Aurora) support for Armada 370 and XP platforms.
>> + *
>> + * Copyright (C) 2012 Marvell
>> + *
>> + * Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>
>> + * Gregory Clement <gregory.clement-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
>> + * Thomas Petazzoni <thomas.petazzoni-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
>> + *
>> + * This file is licensed under the terms of the GNU General Public
>> + * License version 2.  This program is licensed "as is" without any
>> + * warranty of any kind, whether express or implied.
>> + *
>> + * The Armada 370 and Armada XP SOCs have a coherency fabric which is
>> + * responsible for ensuring hardware coherency between all CPUs and between
>> + * CPUs and I/O masters. This file initializes the coherency fabric and
>> + * supplies basic routines for configuring and controlling hardware coherency
>> + */
>> +
>> +#include <linux/kernel.h>
>> +#include <linux/init.h>
>> +#include <linux/of_address.h>
>> +#include <linux/io.h>
>> +#include <linux/smp.h>
>> +#include <asm/smp_plat.h>
>> +#include "armada-370-xp.h"
>> +
>> +/* Some functions in this file are called very early during SMP
>> + * initialization. At that time the device tree framework is not yet
>> + * ready, and it is not possible to get the register address to
>> + * ioremap it. That's why the pointer below is given with an initial
>> + * value matching its virtual mapping
>> + */
>> +static void __iomem *coherency_base = ARMADA_370_XP_REGS_VIRT_BASE + 0x20200;
>> +
>> +/* Coherency fabric registers */
>> +#define COHERENCY_FABRIC_CFG_OFFSET		   0x4
>> +
>> +static struct of_device_id of_coherency_table[] = {
>> +	{.compatible = "marvell,coherency-fabric"},
>> +	{ /* end of list */ },
>> +};
>> +#ifdef CONFIG_SMP
>> +int coherency_get_cpu_count(void)
>> +{
>> +	int reg, cnt;
>> +
>> +	reg = readl(coherency_base + COHERENCY_FABRIC_CFG_OFFSET);
>> +	cnt = (reg & 0xF) + 1;
>> +
>> +	return cnt;
>> +}
>> +#endif
>> +/* Function defined in coherncy_ll.S */
> 
> coherency_ll.S

I knew I have to fix it, and I forgot

> 
>> +extern void ll_set_cpu_coherent(void __iomem *base_addr,
>> +				unsigned int hw_cpu_id);
>> +
>> +int set_cpu_coherent(unsigned int hw_cpu_id, int smp_group_id)
>> +{
>> +	if (!coherency_base) {
>> +		pr_warn("Can't make CPU %d cache coherent.\n", hw_cpu_id);
>> +		pr_warn("Coherency fabric is not initialized\n");
>> +		return 1;
>> +	}
>> +	ll_set_cpu_coherent(coherency_base, hw_cpu_id);
>> +	return 0;
> 
> You can just do return ll_set_cpu_coherent(...) now.

Yes of course that was the plan.

> 
>> +}
>> +
>> +int __init coherency_init(void)
>> +{
>> +	struct device_node *np;
>> +
>> +	np = of_find_matching_node(NULL, of_coherency_table);
>> +	if (np) {
>> +		pr_info("Initializing Coherency fabric\n");
>> +		coherency_base = of_iomap(np, 0);
>> +	}
>> +
>> +	return 0;
>> +}
>> diff --git a/arch/arm/mach-mvebu/coherency.h b/arch/arm/mach-mvebu/coherency.h
>> new file mode 100644
>> index 0000000..2f42813
>> --- /dev/null
>> +++ b/arch/arm/mach-mvebu/coherency.h
>> @@ -0,0 +1,24 @@
>> +/*
>> + * arch/arm/mach-mvebu/include/mach/coherency.h
>> + *
>> + *
>> + * Coherency fabric (Aurora) support for Armada 370 and XP platforms.
>> + *
>> + * Copyright (C) 2012 Marvell
>> + *
>> + * This file is licensed under the terms of the GNU General Public
>> + * License version 2.  This program is licensed "as is" without any
>> + * warranty of any kind, whether express or implied.
>> + */
>> +
>> +#ifndef __MACH_370_XP_COHERENCY_H
>> +#define __MACH_370_XP_COHERENCY_H
>> +
>> +#ifdef CONFIG_SMP
>> +int coherency_get_cpu_count(void);
>> +#endif
>> +
>> +int set_cpu_coherent(int cpu_id, int smp_group_id);
>> +int coherency_init(void);
>> +
>> +#endif	/* __MACH_370_XP_COHERENCY_H */
>> diff --git a/arch/arm/mach-mvebu/coherency_ll.S b/arch/arm/mach-mvebu/coherency_ll.S
>> new file mode 100644
>> index 0000000..ae48730
>> --- /dev/null
>> +++ b/arch/arm/mach-mvebu/coherency_ll.S
>> @@ -0,0 +1,47 @@
>> +/*
>> + * Coherency fabric: low level functions
>> + *
>> + * Copyright (C) 2012 Marvell
>> + *
>> + * Gregory CLEMENT <gregory.clement-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
>> + *
>> + * This file is licensed under the terms of the GNU General Public
>> + * License version 2.  This program is licensed "as is" without any
>> + * warranty of any kind, whether express or implied.
>> + *
>> + * This file implements the assembly function to add a CPU to the
>> + * coherency fabric. This function is called by each of the secondary
>> + * CPUs during their early boot in an SMP kernel, this why this
>> + * function have to callable from assembly. It can also be called by a
>> + * primary CPU from C code during its boot.
>> + */
>> +
>> +#include <linux/linkage.h>
>> +#define ARMADA_XP_CFB_CTL_REG_OFFSET 0x0
>> +#define ARMADA_XP_CFB_CFG_REG_OFFSET 0x4
>> +
>> +	.text
>> +/*
>> + * r0: Coherency fabric base register address
>> + * r1: HW CPU id
>> + */
>> +ENTRY(ll_set_cpu_coherent)
>> +	/* Create bit by cpu index */
>> +	mov	r3, #(1 << 24)
>> +	lsl     r1, r3, r1
>> +
>> +	/* Add CPU to SMP group - Atomic */
>> +	add	r3, r0, #ARMADA_XP_CFB_CTL_REG_OFFSET
>> +	ldr     r2, [r3]
>> +	orr     r2, r2, r1
>> +	str	r2, [r0]
>> +
>> +	/* Enable coherency on CPU - Atomic */
>> +	add	r3, r0, #ARMADA_XP_CFB_CFG_REG_OFFSET
>> +	ldr     r2, [r3]
>> +	orr     r2, r2, r1
>> +	str     r2, [r3]
> 
> I forgot to mention this before, but do you need a dsb or something here?
> 
> Will
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> 


-- 
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

Re: [PATCH V5 5/5] arm: mvebu: Added SMP support for Armada XP

[Gregory CLEMENT]

On 11/21/2012 11:41 AM, Will Deacon wrote:
> On Tue, Nov 20, 2012 at 09:15:49PM +0000, Gregory CLEMENT wrote:
>> From: Yehuda Yitschak <yehuday@marvell.com>
>>
>> 1. added smp init functions in platsmp.c
>> 2. added secondary cpu entry point in headsmp.S
>> 3. added hotplog initial support in hotplug.c
> 
> hotplug
> 
>> 4. added SMP support for PJ4B cpu
> 
> Again, I think the commit message could probably be improved from a numbered
> list...
> 
>> Signed-off-by: Yehuda Yitschak <yehuday@marvell.com>
>> Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
>> ---
>>  arch/arm/configs/mvebu_defconfig    |    3 +
>>  arch/arm/mach-mvebu/Kconfig         |    1 +
>>  arch/arm/mach-mvebu/Makefile        |    2 +
>>  arch/arm/mach-mvebu/armada-370-xp.c |    3 +
>>  arch/arm/mach-mvebu/common.h        |    3 +
>>  arch/arm/mach-mvebu/headsmp.S       |   50 +++++++++++++++
>>  arch/arm/mach-mvebu/hotplug.c       |   30 +++++++++
>>  arch/arm/mach-mvebu/platsmp.c       |  121 +++++++++++++++++++++++++++++++++++
>>  8 files changed, 213 insertions(+)
>>  create mode 100644 arch/arm/mach-mvebu/headsmp.S
>>  create mode 100644 arch/arm/mach-mvebu/hotplug.c
>>  create mode 100644 arch/arm/mach-mvebu/platsmp.c
>>
>> diff --git a/arch/arm/configs/mvebu_defconfig b/arch/arm/configs/mvebu_defconfig
>> index 3458752..da598d3 100644
>> --- a/arch/arm/configs/mvebu_defconfig
>> +++ b/arch/arm/configs/mvebu_defconfig
>> @@ -12,6 +12,9 @@ CONFIG_ARCH_MVEBU=y
>>  CONFIG_MACH_ARMADA_370=y
>>  CONFIG_MACH_ARMADA_XP=y
>>  # CONFIG_CACHE_L2X0 is not set
>> +# CONFIG_SWP_EMULATE is not set
>> +CONFIG_SMP=y
>> +# CONFIG_LOCAL_TIMERS is not set
>>  CONFIG_AEABI=y
>>  CONFIG_HIGHMEM=y
>>  # CONFIG_COMPACTION is not set
>> diff --git a/arch/arm/mach-mvebu/Kconfig b/arch/arm/mach-mvebu/Kconfig
>> index f4c3bf8..c934e1d 100644
>> --- a/arch/arm/mach-mvebu/Kconfig
>> +++ b/arch/arm/mach-mvebu/Kconfig
>> @@ -21,6 +21,7 @@ menu "Marvell SOC with device tree"
>>  config MACH_ARMADA_370_XP
>>  	bool
>>  	select ARMADA_370_XP_TIMER
>> +	select HAVE_SMP
>>  	select CPU_PJ4B
>>  
>>  config MACH_ARMADA_370
>> diff --git a/arch/arm/mach-mvebu/Makefile b/arch/arm/mach-mvebu/Makefile
>> index 2e3ec11..5dcb369 100644
>> --- a/arch/arm/mach-mvebu/Makefile
>> +++ b/arch/arm/mach-mvebu/Makefile
>> @@ -3,3 +3,5 @@ ccflags-$(CONFIG_ARCH_MULTIPLATFORM) := -I$(srctree)/$(src)/include \
>>  
>>  obj-y += system-controller.o
>>  obj-$(CONFIG_MACH_ARMADA_370_XP) += armada-370-xp.o irq-armada-370-xp.o addr-map.o coherency.o coherency_ll.o pmsu.o
>> +obj-$(CONFIG_SMP)                += platsmp.o headsmp.o
>> +obj-$(CONFIG_HOTPLUG_CPU)        += hotplug.o
>> diff --git a/arch/arm/mach-mvebu/armada-370-xp.c b/arch/arm/mach-mvebu/armada-370-xp.c
>> index 3292d6d..472e70f 100644
>> --- a/arch/arm/mach-mvebu/armada-370-xp.c
>> +++ b/arch/arm/mach-mvebu/armada-370-xp.c
>> @@ -23,6 +23,7 @@
>>  #include <asm/mach/time.h>
>>  #include "armada-370-xp.h"
>>  #include "common.h"
>> +#include "coherency.h"
>>  
>>  static struct map_desc armada_370_xp_io_desc[] __initdata = {
>>  	{
>> @@ -51,6 +52,7 @@ struct sys_timer armada_370_xp_timer = {
>>  static void __init armada_370_xp_dt_init(void)
>>  {
>>  	of_platform_populate(NULL, of_default_bus_match_table, NULL, NULL);
>> +	coherency_init();
>>  }
>>  
>>  static const char * const armada_370_xp_dt_board_dt_compat[] = {
>> @@ -60,6 +62,7 @@ static const char * const armada_370_xp_dt_board_dt_compat[] = {
>>  };
>>  
>>  DT_MACHINE_START(ARMADA_XP_DT, "Marvell Aramada 370/XP (Device Tree)")
>> +	.smp		= smp_ops(armada_xp_smp_ops),
>>  	.init_machine	= armada_370_xp_dt_init,
>>  	.map_io		= armada_370_xp_map_io,
>>  	.init_irq	= armada_370_xp_init_irq,
>> diff --git a/arch/arm/mach-mvebu/common.h b/arch/arm/mach-mvebu/common.h
>> index b5cd7e7..b711869 100644
>> --- a/arch/arm/mach-mvebu/common.h
>> +++ b/arch/arm/mach-mvebu/common.h
>> @@ -20,7 +20,10 @@ void mvebu_restart(char mode, const char *cmd);
>>  void armada_370_xp_init_irq(void);
>>  void armada_370_xp_handle_irq(struct pt_regs *regs);
>>  
>> +void armada_xp_cpu_die(unsigned int cpu);
>>  
>>  int armada_370_xp_coherency_init(void);
>>  int armada_370_xp_pmsu_init(void);
>> +void armada_xp_secondary_startup(void);
>> +extern struct smp_operations armada_xp_smp_ops;
>>  #endif
>> diff --git a/arch/arm/mach-mvebu/headsmp.S b/arch/arm/mach-mvebu/headsmp.S
>> new file mode 100644
>> index 0000000..994eecc
>> --- /dev/null
>> +++ b/arch/arm/mach-mvebu/headsmp.S
>> @@ -0,0 +1,50 @@
>> +/*
>> + * SMP support: Entry point for secondary CPUs
>> + *
>> + * Copyright (C) 2012 Marvell
>> + *
>> + * Yehuda Yitschak <yehuday@marvell.com>
>> + * Gregory CLEMENT <gregory.clement@free-electrons.com>
>> + * Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
>> + *
>> + * This file is licensed under the terms of the GNU General Public
>> + * License version 2.  This program is licensed "as is" without any
>> + * warranty of any kind, whether express or implied.
>> + *
>> + * This file implements the assembly entry point for secondary CPUs in
>> + * an SMP kernel. The only thing we need to do is to add the CPU to
>> + * the coherency fabric by writing to 2 registers. Currently the base
>> + * register addresses are hard coded due to the early initialisation
>> + * problems.
>> + */
>> +
>> +#include <linux/linkage.h>
>> +#include <linux/init.h>
>> +
>> +/*
>> + * At this stage the secondary CPUs don't have acces yet to the MMU, so
>> + * we have to provide physical addresses
>> + */
>> +#define ARMADA_XP_CFB_BASE	     0xD0020200
>> +
>> +	__CPUINIT
>> +
>> +/*
>> + * Armada XP specific entry point for secondary CPUs.
>> + * We add the CPU to the coherency fabric and then jump to secondary
>> + * startup
>> + */
>> +ENTRY(armada_xp_secondary_startup)
>> +
>> +	/* Read CPU id */
>> + 	mrc     p15, 0, r1, c0, c0, 5
>> +	and     r1, r1, #0xF
>> +
>> +	/* Add CPU to coherency fabric */
>> +	ldr     r0, = ARMADA_XP_CFB_BASE
>> +	mov	lr, pc
>> +
>> +	b	ll_set_cpu_coherent
>> +	b	secondary_startup
> 
> Why didn't you use my adr suggestion here?

Because when I used BSYM(secondary_startup), I get the following
message during compilation:

Error: undefined symbol secondary_startup used as an immediate value

And I didn't want to miss the merge window for something which could be
fixed during the v3.8-rc1 if needed.

> 
> Will
> 


-- 
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

Re: [PATCH V5 5/5] arm: mvebu: Added SMP support for Armada XP

[Will Deacon]

On Wed, Nov 21, 2012 at 11:57:12AM +0000, Gregory CLEMENT wrote:
> On 11/21/2012 11:41 AM, Will Deacon wrote:
> >> +/*
> >> + * Armada XP specific entry point for secondary CPUs.
> >> + * We add the CPU to the coherency fabric and then jump to secondary
> >> + * startup
> >> + */
> >> +ENTRY(armada_xp_secondary_startup)
> >> +
> >> +	/* Read CPU id */
> >> + 	mrc     p15, 0, r1, c0, c0, 5
> >> +	and     r1, r1, #0xF
> >> +
> >> +	/* Add CPU to coherency fabric */
> >> +	ldr     r0, = ARMADA_XP_CFB_BASE
> >> +	mov	lr, pc
> >> +
> >> +	b	ll_set_cpu_coherent
> >> +	b	secondary_startup
> > 
> > Why didn't you use my adr suggestion here?
> 
> Because when I used BSYM(secondary_startup), I get the following
> message during compilation:
> 
> Error: undefined symbol secondary_startup used as an immediate value

Sorry, my mistake (I was thinking on my feet) since secondary_startup is
declared in a C file, right? How about:

	ldr	lr, =secondary_startup
	b	ll_set_cpu_coherent

> And I didn't want to miss the merge window for something which could be
> fixed during the v3.8-rc1 if needed.

Sure, but it also seems daft to merge something that's nine tenths of the
way there when it's straight-forward to finish it off properly.

Will

Re: [PATCH V5 5/5] arm: mvebu: Added SMP support for Armada XP

[Russell King - ARM Linux]

On Wed, Nov 21, 2012 at 12:22:51PM +0000, Will Deacon wrote:
> Sorry, my mistake (I was thinking on my feet) since secondary_startup is
> declared in a C file, right? How about:
> 
> 	ldr	lr, =secondary_startup
> 	b	ll_set_cpu_coherent

Why?  Do we really want LR to be the _virtual_ address?  No we don't,
because secondary_startup expects to be called with the MMU off.

So I go back to my original comment.  What's wrong with:

	bl	ll_set_cpu_coherent
	b	secondary_startup

?

Re: [PATCH V5 5/5] arm: mvebu: Added SMP support for Armada XP

[Gregory CLEMENT]

On 11/21/2012 01:26 PM, Russell King - ARM Linux wrote:
> On Wed, Nov 21, 2012 at 12:22:51PM +0000, Will Deacon wrote:
>> Sorry, my mistake (I was thinking on my feet) since secondary_startup is
>> declared in a C file, right? How about:
>>
>> 	ldr	lr, =secondary_startup
>> 	b	ll_set_cpu_coherent
> 
> Why?  Do we really want LR to be the _virtual_ address?  No we don't,
> because secondary_startup expects to be called with the MMU off.
> 
> So I go back to my original comment.  What's wrong with:
> 
> 	bl	ll_set_cpu_coherent
> 	b	secondary_startup
> 
> ?

Nothing wrong, it works perfectly.
Whereas indeed the code proposed by Will didn't work very well.
I will take Russell's suggestion.

Thanks,


> 


-- 
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

Re: [PATCH V5 5/5] arm: mvebu: Added SMP support for Armada XP

[Will Deacon]

On Wed, Nov 21, 2012 at 12:31:36PM +0000, Gregory CLEMENT wrote:
> On 11/21/2012 01:26 PM, Russell King - ARM Linux wrote:
> > On Wed, Nov 21, 2012 at 12:22:51PM +0000, Will Deacon wrote:
> >> Sorry, my mistake (I was thinking on my feet) since secondary_startup is
> >> declared in a C file, right? How about:
> >>
> >> 	ldr	lr, =secondary_startup
> >> 	b	ll_set_cpu_coherent
> > 
> > Why?  Do we really want LR to be the _virtual_ address?  No we don't,
> > because secondary_startup expects to be called with the MMU off.
> > 
> > So I go back to my original comment.  What's wrong with:
> > 
> > 	bl	ll_set_cpu_coherent
> > 	b	secondary_startup
> > 
> > ?
> 
> Nothing wrong, it works perfectly.
> Whereas indeed the code proposed by Will didn't work very well.
> I will take Russell's suggestion.

Fine by me. Somehow I thought this was jumping into C but it's actually way
earlier than that.

Sorry for the confusion,

Will

Re: [PATCH V5 1/5] arm: mvebu: Added support for coherency fabric in mach-mvebu

[Gregory CLEMENT]

On 11/21/2012 11:36 AM, Will Deacon wrote:
> On Tue, Nov 20, 2012 at 09:15:45PM +0000, Gregory CLEMENT wrote:
>> From: Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>
> 
> <insert commit log here>
> 
> Also -- is the author attribution still correct?
> 
>> Signed-off-by: Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>
>> Signed-off-by: Gregory CLEMENT <gregory.clement-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
>> ---
>>  .../devicetree/bindings/arm/coherency-fabric.txt   |   16 ++++
>>  arch/arm/boot/dts/armada-370-xp.dtsi               |    5 ++
>>  arch/arm/mach-mvebu/Makefile                       |    2 +-
>>  arch/arm/mach-mvebu/coherency.c                    |   80 ++++++++++++++++++++
>>  arch/arm/mach-mvebu/coherency.h                    |   24 ++++++
>>  arch/arm/mach-mvebu/coherency_ll.S                 |   47 ++++++++++++
>>  arch/arm/mach-mvebu/common.h                       |    2 +
>>  7 files changed, 175 insertions(+), 1 deletion(-)
>>  create mode 100644 Documentation/devicetree/bindings/arm/coherency-fabric.txt
>>  create mode 100644 arch/arm/mach-mvebu/coherency.c
>>  create mode 100644 arch/arm/mach-mvebu/coherency.h
>>  create mode 100644 arch/arm/mach-mvebu/coherency_ll.S
>>
>> diff --git a/Documentation/devicetree/bindings/arm/coherency-fabric.txt b/Documentation/devicetree/bindings/arm/coherency-fabric.txt
>> new file mode 100644
>> index 0000000..2bfbf67
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/arm/coherency-fabric.txt
>> @@ -0,0 +1,16 @@
>> +Coherency fabric
>> +----------------
>> +Available on Marvell SOCs: Armada 370 and Armada XP
>> +
>> +Required properties:
>> +
>> +- compatible: "marvell,coherency-fabric"
>> +- reg: Should contain,coherency fabric registers location and length.
>> +
>> +Example:
>> +
>> +coherency-fabric@d0020200 {
>> +	compatible = "marvell,coherency-fabric";
>> +	reg = <0xd0020200 0xb0>;
>> +};
>> +
>> diff --git a/arch/arm/boot/dts/armada-370-xp.dtsi b/arch/arm/boot/dts/armada-370-xp.dtsi
>> index 94b4b9e..b0d075b 100644
>> --- a/arch/arm/boot/dts/armada-370-xp.dtsi
>> +++ b/arch/arm/boot/dts/armada-370-xp.dtsi
>> @@ -36,6 +36,11 @@
>>  	      interrupt-controller;
>>  	};
>>  
>> +	coherency-fabric@d0020200 {
>> +		compatible = "marvell,coherency-fabric";
>> +		reg = <0xd0020200 0xb0>;
>> +	};
>> +
>>  	soc {
>>  		#address-cells = <1>;
>>  		#size-cells = <1>;
>> diff --git a/arch/arm/mach-mvebu/Makefile b/arch/arm/mach-mvebu/Makefile
>> index 57f996b..5ce4b42 100644
>> --- a/arch/arm/mach-mvebu/Makefile
>> +++ b/arch/arm/mach-mvebu/Makefile
>> @@ -2,4 +2,4 @@ ccflags-$(CONFIG_ARCH_MULTIPLATFORM) := -I$(srctree)/$(src)/include \
>>  	-I$(srctree)/arch/arm/plat-orion/include
>>  
>>  obj-y += system-controller.o
>> -obj-$(CONFIG_MACH_ARMADA_370_XP) += armada-370-xp.o irq-armada-370-xp.o addr-map.o
>> +obj-$(CONFIG_MACH_ARMADA_370_XP) += armada-370-xp.o irq-armada-370-xp.o addr-map.o coherency.o coherency_ll.o
>> diff --git a/arch/arm/mach-mvebu/coherency.c b/arch/arm/mach-mvebu/coherency.c
>> new file mode 100644
>> index 0000000..1bc02d0
>> --- /dev/null
>> +++ b/arch/arm/mach-mvebu/coherency.c
>> @@ -0,0 +1,80 @@
>> +/*
>> + * Coherency fabric (Aurora) support for Armada 370 and XP platforms.
>> + *
>> + * Copyright (C) 2012 Marvell
>> + *
>> + * Yehuda Yitschak <yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>
>> + * Gregory Clement <gregory.clement-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
>> + * Thomas Petazzoni <thomas.petazzoni-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
>> + *
>> + * This file is licensed under the terms of the GNU General Public
>> + * License version 2.  This program is licensed "as is" without any
>> + * warranty of any kind, whether express or implied.
>> + *
>> + * The Armada 370 and Armada XP SOCs have a coherency fabric which is
>> + * responsible for ensuring hardware coherency between all CPUs and between
>> + * CPUs and I/O masters. This file initializes the coherency fabric and
>> + * supplies basic routines for configuring and controlling hardware coherency
>> + */
>> +
>> +#include <linux/kernel.h>
>> +#include <linux/init.h>
>> +#include <linux/of_address.h>
>> +#include <linux/io.h>
>> +#include <linux/smp.h>
>> +#include <asm/smp_plat.h>
>> +#include "armada-370-xp.h"
>> +
>> +/* Some functions in this file are called very early during SMP
>> + * initialization. At that time the device tree framework is not yet
>> + * ready, and it is not possible to get the register address to
>> + * ioremap it. That's why the pointer below is given with an initial
>> + * value matching its virtual mapping
>> + */
>> +static void __iomem *coherency_base = ARMADA_370_XP_REGS_VIRT_BASE + 0x20200;
>> +
>> +/* Coherency fabric registers */
>> +#define COHERENCY_FABRIC_CFG_OFFSET		   0x4
>> +
>> +static struct of_device_id of_coherency_table[] = {
>> +	{.compatible = "marvell,coherency-fabric"},
>> +	{ /* end of list */ },
>> +};
>> +#ifdef CONFIG_SMP
>> +int coherency_get_cpu_count(void)
>> +{
>> +	int reg, cnt;
>> +
>> +	reg = readl(coherency_base + COHERENCY_FABRIC_CFG_OFFSET);
>> +	cnt = (reg & 0xF) + 1;
>> +
>> +	return cnt;
>> +}
>> +#endif
>> +/* Function defined in coherncy_ll.S */
> 
> coherency_ll.S
> 
>> +extern void ll_set_cpu_coherent(void __iomem *base_addr,
>> +				unsigned int hw_cpu_id);
>> +
>> +int set_cpu_coherent(unsigned int hw_cpu_id, int smp_group_id)
>> +{
>> +	if (!coherency_base) {
>> +		pr_warn("Can't make CPU %d cache coherent.\n", hw_cpu_id);
>> +		pr_warn("Coherency fabric is not initialized\n");
>> +		return 1;
>> +	}
>> +	ll_set_cpu_coherent(coherency_base, hw_cpu_id);
>> +	return 0;
> 
> You can just do return ll_set_cpu_coherent(...) now.
> 
>> +}
>> +
>> +int __init coherency_init(void)
>> +{
>> +	struct device_node *np;
>> +
>> +	np = of_find_matching_node(NULL, of_coherency_table);
>> +	if (np) {
>> +		pr_info("Initializing Coherency fabric\n");
>> +		coherency_base = of_iomap(np, 0);
>> +	}
>> +
>> +	return 0;
>> +}
>> diff --git a/arch/arm/mach-mvebu/coherency.h b/arch/arm/mach-mvebu/coherency.h
>> new file mode 100644
>> index 0000000..2f42813
>> --- /dev/null
>> +++ b/arch/arm/mach-mvebu/coherency.h
>> @@ -0,0 +1,24 @@
>> +/*
>> + * arch/arm/mach-mvebu/include/mach/coherency.h
>> + *
>> + *
>> + * Coherency fabric (Aurora) support for Armada 370 and XP platforms.
>> + *
>> + * Copyright (C) 2012 Marvell
>> + *
>> + * This file is licensed under the terms of the GNU General Public
>> + * License version 2.  This program is licensed "as is" without any
>> + * warranty of any kind, whether express or implied.
>> + */
>> +
>> +#ifndef __MACH_370_XP_COHERENCY_H
>> +#define __MACH_370_XP_COHERENCY_H
>> +
>> +#ifdef CONFIG_SMP
>> +int coherency_get_cpu_count(void);
>> +#endif
>> +
>> +int set_cpu_coherent(int cpu_id, int smp_group_id);
>> +int coherency_init(void);
>> +
>> +#endif	/* __MACH_370_XP_COHERENCY_H */
>> diff --git a/arch/arm/mach-mvebu/coherency_ll.S b/arch/arm/mach-mvebu/coherency_ll.S
>> new file mode 100644
>> index 0000000..ae48730
>> --- /dev/null
>> +++ b/arch/arm/mach-mvebu/coherency_ll.S
>> @@ -0,0 +1,47 @@
>> +/*
>> + * Coherency fabric: low level functions
>> + *
>> + * Copyright (C) 2012 Marvell
>> + *
>> + * Gregory CLEMENT <gregory.clement-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
>> + *
>> + * This file is licensed under the terms of the GNU General Public
>> + * License version 2.  This program is licensed "as is" without any
>> + * warranty of any kind, whether express or implied.
>> + *
>> + * This file implements the assembly function to add a CPU to the
>> + * coherency fabric. This function is called by each of the secondary
>> + * CPUs during their early boot in an SMP kernel, this why this
>> + * function have to callable from assembly. It can also be called by a
>> + * primary CPU from C code during its boot.
>> + */
>> +
>> +#include <linux/linkage.h>
>> +#define ARMADA_XP_CFB_CTL_REG_OFFSET 0x0
>> +#define ARMADA_XP_CFB_CFG_REG_OFFSET 0x4
>> +
>> +	.text
>> +/*
>> + * r0: Coherency fabric base register address
>> + * r1: HW CPU id
>> + */
>> +ENTRY(ll_set_cpu_coherent)
>> +	/* Create bit by cpu index */
>> +	mov	r3, #(1 << 24)
>> +	lsl     r1, r3, r1
>> +
>> +	/* Add CPU to SMP group - Atomic */
>> +	add	r3, r0, #ARMADA_XP_CFB_CTL_REG_OFFSET
>> +	ldr     r2, [r3]
>> +	orr     r2, r2, r1
>> +	str	r2, [r0]
>> +
>> +	/* Enable coherency on CPU - Atomic */
>> +	add	r3, r0, #ARMADA_XP_CFB_CFG_REG_OFFSET
>> +	ldr     r2, [r3]
>> +	orr     r2, r2, r1
>> +	str     r2, [r3]
> 
> I forgot to mention this before, but do you need a dsb or something here?

The answer from Marvell engineer:
"Especially for this case, DSB is not needed because we set the MMU to
be such that these configurations are in SO memory space.  However, I
agree with the comment and believe that it would be good practice to
add a DSB at the end of this sequence (and not rely on the fact that
it they are not under DEVICE memory space)"

So in short I will add a dsb here.

> 
> Will
> 


-- 
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com