[PATCH 0/3] SMP support for Broadcom NSP

[PATCH 0/3] SMP support for Broadcom NSP

[Kapil Hali]

Hi,

This series adds SMP support for Broadcom's Northstar Plus SoC.

There are similar SMP enablement methods for many ARMv7 bsed SoCs.
BCM NSP SoC, has a typical such mechanism - after power-on, the
secondary core is held in a standby state, primary core provides a
startup address for the secondary core and wakes it up. Booting of
the secondary core is serialized using pen_release global variable.

The startup address is programmed at a special register location
which is defined in the device tree using a "secondary-boot-reg"
property in a node whose "enable-method" property matches.

The first patch adds cpu-enable-method in the device tree bindings
documentation. It also updates ARM CPU device tree documentation
with Broadcom Northstar Plus CPU details.

The second patch adds SMP support to the BCM NSP device tree file.

Finally, third patch, enables SMP on BCM NSP. It also consolidates
common SMP handling between BCM NSP and BCM Kona.

This patch series is constructed based on Linux v4.3-rc2.

The source code is available at GITHUB:
https://github.com/Broadcom/cygnus-linux/tree/nsp-smp-v1

Kapil Hali (3):
  dt-bindings: add SMP enable-method for Broadcom NSP
  ARM: dts: add SMP support for Broadcom NSP
  ARM: BCM: Add SMP support for Broadcom NSP

 .../bindings/arm/bcm/brcm,nsp-cpu-method.txt       |  36 ++++++
 Documentation/devicetree/bindings/arm/cpus.txt     |   1 +
 arch/arm/boot/dts/bcm-nsp.dtsi                     |  33 +++--
 arch/arm/mach-bcm/Kconfig                          |   2 +
 arch/arm/mach-bcm/Makefile                         |   8 +-
 arch/arm/mach-bcm/bcm_nsp.h                        |  19 +++
 arch/arm/mach-bcm/headsmp.S                        |  37 ++++++
 arch/arm/mach-bcm/{kona_smp.c => platsmp.c}        | 142 +++++++++++++++++++--
 8 files changed, 255 insertions(+), 23 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/arm/bcm/brcm,nsp-cpu-method.txt
 create mode 100644 arch/arm/mach-bcm/bcm_nsp.h
 create mode 100644 arch/arm/mach-bcm/headsmp.S
 rename arch/arm/mach-bcm/{kona_smp.c => platsmp.c} (63%)

-- 
2.1.0

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/3] dt-bindings: add SMP enable-method for Broadcom NSP

[Kapil Hali]

Add a compatible string "brcm,bcm-nsp-smp" for Broadcom's
Northstar Plus CPU to the 32-bit ARM CPU device tree binding
documentation file and create a new binding documentation for
Northstar Plus CPU pen-release mechanism.

Signed-off-by: Kapil Hali <kapilh@broadcom.com>
---
 .../bindings/arm/bcm/brcm,nsp-cpu-method.txt       | 36 ++++++++++++++++++++++
 Documentation/devicetree/bindings/arm/cpus.txt     |  1 +
 2 files changed, 37 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/arm/bcm/brcm,nsp-cpu-method.txt

diff --git a/Documentation/devicetree/bindings/arm/bcm/brcm,nsp-cpu-method.txt b/Documentation/devicetree/bindings/arm/bcm/brcm,nsp-cpu-method.txt
new file mode 100644
index 0000000..8506da7
--- /dev/null
+++ b/Documentation/devicetree/bindings/arm/bcm/brcm,nsp-cpu-method.txt
@@ -0,0 +1,36 @@
+Broadcom Northstar Plus SoC CPU Enable Method
+---------------------------------------------
+This binding defines the enable method used for starting secondary
+CPUs in the following Broadcom SoCs:
+  BCM58522, BCM58525, BCM58535, BCM58622, BCM58623, BCM58625, BCM88312
+
+The enable method is specified by defining the following required
+properties in the "cpus" device tree node:
+  - enable-method = "brcm,bcm-nsp-smp";
+  - secondary-boot-reg = <...>;
+
+The secondary-boot-reg property is a u32 value that specifies the
+physical address of the register used to request the ROM holding pen
+code release a secondary CPU.
+
+Example:
+	cpus {
+		#address-cells = <1>;
+		#size-cells = <0>;
+		enable-method = "brcm,bcm-nsp-smp";
+		secondary-boot-reg = <0xffff042c>;
+
+		cpu0: cpu@0 {
+			device_type = "cpu";
+			compatible = "arm,cortex-a9";
+			next-level-cache = <&L2>;
+			reg = <0>;
+		};
+
+		cpu1: cpu@1 {
+			device_type = "cpu";
+			compatible = "arm,cortex-a9";
+			next-level-cache = <&L2>;
+			reg = <1>;
+		};
+	};
diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
index 91e6e5c..1172d9b 100644
--- a/Documentation/devicetree/bindings/arm/cpus.txt
+++ b/Documentation/devicetree/bindings/arm/cpus.txt
@@ -190,6 +190,7 @@ nodes to be present and contain the properties described below.
 			    "allwinner,sun6i-a31"
 			    "allwinner,sun8i-a23"
 			    "arm,psci"
+			    "brcm,bcm-nsp-smp"
 			    "brcm,brahma-b15"
 			    "marvell,armada-375-smp"
 			    "marvell,armada-380-smp"
-- 
2.1.0

Re: [PATCH 1/3] dt-bindings: add SMP enable-method for Broadcom NSP

[Hauke Mehrtens]

On 10/14/2015 07:46 PM, Kapil Hali wrote:
> Add a compatible string "brcm,bcm-nsp-smp" for Broadcom's
> Northstar Plus CPU to the 32-bit ARM CPU device tree binding
> documentation file and create a new binding documentation for
> Northstar Plus CPU pen-release mechanism.
> 
> Signed-off-by: Kapil Hali <kapilh-dY08KVG/lbpWk0Htik3J/w@public.gmane.org>
> ---
>  .../bindings/arm/bcm/brcm,nsp-cpu-method.txt       | 36 ++++++++++++++++++++++
>  Documentation/devicetree/bindings/arm/cpus.txt     |  1 +
>  2 files changed, 37 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/arm/bcm/brcm,nsp-cpu-method.txt
> 
> diff --git a/Documentation/devicetree/bindings/arm/bcm/brcm,nsp-cpu-method.txt b/Documentation/devicetree/bindings/arm/bcm/brcm,nsp-cpu-method.txt
> new file mode 100644
> index 0000000..8506da7
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/arm/bcm/brcm,nsp-cpu-method.txt
> @@ -0,0 +1,36 @@
> +Broadcom Northstar Plus SoC CPU Enable Method
> +---------------------------------------------
> +This binding defines the enable method used for starting secondary
> +CPUs in the following Broadcom SoCs:
> +  BCM58522, BCM58525, BCM58535, BCM58622, BCM58623, BCM58625, BCM88312
> +
> +The enable method is specified by defining the following required
> +properties in the "cpus" device tree node:
> +  - enable-method = "brcm,bcm-nsp-smp";
> +  - secondary-boot-reg = <...>;
> +
> +The secondary-boot-reg property is a u32 value that specifies the
> +physical address of the register used to request the ROM holding pen
> +code release a secondary CPU.
> +
> +Example:
> +	cpus {
> +		#address-cells = <1>;
> +		#size-cells = <0>;
> +		enable-method = "brcm,bcm-nsp-smp";
> +		secondary-boot-reg = <0xffff042c>;

Isn't this some offset in a SRAM? If this is a SRAM it should be handled
like it is done in some other SoC code.

> +
> +		cpu0: cpu@0 {
> +			device_type = "cpu";
> +			compatible = "arm,cortex-a9";
> +			next-level-cache = <&L2>;
> +			reg = <0>;
> +		};
> +
> +		cpu1: cpu@1 {
> +			device_type = "cpu";
> +			compatible = "arm,cortex-a9";
> +			next-level-cache = <&L2>;
> +			reg = <1>;
> +		};
> +	};
> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
> index 91e6e5c..1172d9b 100644
> --- a/Documentation/devicetree/bindings/arm/cpus.txt
> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
> @@ -190,6 +190,7 @@ nodes to be present and contain the properties described below.
>  			    "allwinner,sun6i-a31"
>  			    "allwinner,sun8i-a23"
>  			    "arm,psci"
> +			    "brcm,bcm-nsp-smp"
>  			    "brcm,brahma-b15"
>  			    "marvell,armada-375-smp"
>  			    "marvell,armada-380-smp"
> 

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/3] dt-bindings: add SMP enable-method for Broadcom NSP

[Kapil Hali]

On 10/15/2015 3:57 AM, Hauke Mehrtens wrote:
> On 10/14/2015 07:46 PM, Kapil Hali wrote:
>> Add a compatible string "brcm,bcm-nsp-smp" for Broadcom's
>> Northstar Plus CPU to the 32-bit ARM CPU device tree binding
>> documentation file and create a new binding documentation for
>> Northstar Plus CPU pen-release mechanism.
>>
>> Signed-off-by: Kapil Hali <kapilh@broadcom.com>
>> ---
>>  .../bindings/arm/bcm/brcm,nsp-cpu-method.txt       | 36 ++++++++++++++++++++++
>>  Documentation/devicetree/bindings/arm/cpus.txt     |  1 +
>>  2 files changed, 37 insertions(+)
>>  create mode 100644 Documentation/devicetree/bindings/arm/bcm/brcm,nsp-cpu-method.txt
>>
>> diff --git a/Documentation/devicetree/bindings/arm/bcm/brcm,nsp-cpu-method.txt b/Documentation/devicetree/bindings/arm/bcm/brcm,nsp-cpu-method.txt
>> new file mode 100644
>> index 0000000..8506da7
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/arm/bcm/brcm,nsp-cpu-method.txt
>> @@ -0,0 +1,36 @@
>> +Broadcom Northstar Plus SoC CPU Enable Method
>> +---------------------------------------------
>> +This binding defines the enable method used for starting secondary
>> +CPUs in the following Broadcom SoCs:
>> +  BCM58522, BCM58525, BCM58535, BCM58622, BCM58623, BCM58625, BCM88312
>> +
>> +The enable method is specified by defining the following required
>> +properties in the "cpus" device tree node:
>> +  - enable-method = "brcm,bcm-nsp-smp";
>> +  - secondary-boot-reg = <...>;
>> +
>> +The secondary-boot-reg property is a u32 value that specifies the
>> +physical address of the register used to request the ROM holding pen
>> +code release a secondary CPU.
>> +
>> +Example:
>> +	cpus {
>> +		#address-cells = <1>;
>> +		#size-cells = <0>;
>> +		enable-method = "brcm,bcm-nsp-smp";
>> +		secondary-boot-reg = <0xffff042c>;
> 
> Isn't this some offset in a SRAM? If this is a SRAM it should be handled
> like it is done in some other SoC code.
> 
It is an internal SKU region. Is it handled differently in rest of the 
SoCs? What is that method?

>> +
>> +		cpu0: cpu@0 {
>> +			device_type = "cpu";
>> +			compatible = "arm,cortex-a9";
>> +			next-level-cache = <&L2>;
>> +			reg = <0>;
>> +		};
>> +
>> +		cpu1: cpu@1 {
>> +			device_type = "cpu";
>> +			compatible = "arm,cortex-a9";
>> +			next-level-cache = <&L2>;
>> +			reg = <1>;
>> +		};
>> +	};
>> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
>> index 91e6e5c..1172d9b 100644
>> --- a/Documentation/devicetree/bindings/arm/cpus.txt
>> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
>> @@ -190,6 +190,7 @@ nodes to be present and contain the properties described below.
>>  			    "allwinner,sun6i-a31"
>>  			    "allwinner,sun8i-a23"
>>  			    "arm,psci"
>> +			    "brcm,bcm-nsp-smp"
>>  			    "brcm,brahma-b15"
>>  			    "marvell,armada-375-smp"
>>  			    "marvell,armada-380-smp"
>>
> 
Thanks,
Kapil

[PATCH 2/3] ARM: dts: add SMP support for Broadcom NSP

[Kapil Hali]

Add device tree changes required for providing SMP support
for Broadcom Northstar Plus SoC.

Signed-off-by: Kapil Hali <kapilh@broadcom.com>
---
 arch/arm/boot/dts/bcm-nsp.dtsi | 33 +++++++++++++++++++++------------
 1 file changed, 21 insertions(+), 12 deletions(-)

diff --git a/arch/arm/boot/dts/bcm-nsp.dtsi b/arch/arm/boot/dts/bcm-nsp.dtsi
index 58aca27..d1875d9 100644
--- a/arch/arm/boot/dts/bcm-nsp.dtsi
+++ b/arch/arm/boot/dts/bcm-nsp.dtsi
@@ -40,24 +40,33 @@
 	model = "Broadcom Northstar Plus SoC";
 	interrupt-parent = <&gic>;
 
+	cpus {
+		#address-cells = <1>;
+		#size-cells = <0>;
+		enable-method = "brcm,bcm-nsp-smp";
+		secondary-boot-reg = <0xffff042c>;
+
+		cpu@0 {
+			device_type = "cpu";
+			compatible = "arm,cortex-a9";
+			next-level-cache = <&L2>;
+			reg = <0x0>;
+		};
+
+		cpu@1 {
+			device_type = "cpu";
+			compatible = "arm,cortex-a9";
+			next-level-cache = <&L2>;
+			reg = <0x1>;
+		};
+	};
+
 	mpcore {
 		compatible = "simple-bus";
 		ranges = <0x00000000 0x19020000 0x00003000>;
 		#address-cells = <1>;
 		#size-cells = <1>;
 
-		cpus {
-			#address-cells = <1>;
-			#size-cells = <0>;
-
-			cpu@0 {
-				device_type = "cpu";
-				compatible = "arm,cortex-a9";
-				next-level-cache = <&L2>;
-				reg = <0x0>;
-			};
-		};
-
 		L2: l2-cache {
 			compatible = "arm,pl310-cache";
 			reg = <0x2000 0x1000>;
-- 
2.1.0

[PATCH 3/3] ARM: BCM: Add SMP support for Broadcom NSP

[Kapil Hali]

Add SMP support for Broadcom's Northstar Plus SoC,
cpu enable method and pen_release procedures. This
changes also consolidates iProc family's - BCM NSP
and BCM Kona, SMP handling in a common file.

Northstar Plus SoC is based on ARM Cortex-A9
revision r3p0 which requires configuration for ARM
Errata 764369 for SMP. This change adds the needed
configuration option.

Signed-off-by: Kapil Hali <kapilh@broadcom.com>
---
 arch/arm/mach-bcm/Kconfig                   |   2 +
 arch/arm/mach-bcm/Makefile                  |   8 +-
 arch/arm/mach-bcm/bcm_nsp.h                 |  19 ++++
 arch/arm/mach-bcm/headsmp.S                 |  37 ++++++++
 arch/arm/mach-bcm/{kona_smp.c => platsmp.c} | 142 ++++++++++++++++++++++++++--
 5 files changed, 197 insertions(+), 11 deletions(-)
 create mode 100644 arch/arm/mach-bcm/bcm_nsp.h
 create mode 100644 arch/arm/mach-bcm/headsmp.S
 rename arch/arm/mach-bcm/{kona_smp.c => platsmp.c} (63%)

diff --git a/arch/arm/mach-bcm/Kconfig b/arch/arm/mach-bcm/Kconfig
index 1679fa4..2e9dbb5 100644
--- a/arch/arm/mach-bcm/Kconfig
+++ b/arch/arm/mach-bcm/Kconfig
@@ -40,6 +40,8 @@ config ARCH_BCM_NSP
 	select ARCH_BCM_IPROC
 	select ARM_ERRATA_754322
 	select ARM_ERRATA_775420
+	select ARM_ERRATA_764369 if SMP
+	select HAVE_SMP
 	help
 	  Support for Broadcom Northstar Plus SoC.
 	  Broadcom Northstar Plus family of SoCs are used for switching control
diff --git a/arch/arm/mach-bcm/Makefile b/arch/arm/mach-bcm/Makefile
index 892261f..36a4ca30 100644
--- a/arch/arm/mach-bcm/Makefile
+++ b/arch/arm/mach-bcm/Makefile
@@ -14,7 +14,11 @@
 obj-$(CONFIG_ARCH_BCM_CYGNUS) +=  bcm_cygnus.o
 
 # Northstar Plus
-obj-$(CONFIG_ARCH_BCM_NSP) += bcm_nsp.o
+obj-$(CONFIG_ARCH_BCM_NSP)	+= bcm_nsp.o
+
+ifeq ($(CONFIG_ARCH_BCM_NSP),y)
+obj-$(CONFIG_SMP)		+= headsmp.o platsmp.o
+endif
 
 # BCM281XX
 obj-$(CONFIG_ARCH_BCM_281XX)	+= board_bcm281xx.o
@@ -23,7 +27,7 @@ obj-$(CONFIG_ARCH_BCM_281XX)	+= board_bcm281xx.o
 obj-$(CONFIG_ARCH_BCM_21664)	+= board_bcm21664.o
 
 # BCM281XX and BCM21664 SMP support
-obj-$(CONFIG_ARCH_BCM_MOBILE_SMP) += kona_smp.o
+obj-$(CONFIG_ARCH_BCM_MOBILE_SMP) += platsmp.o
 
 # BCM281XX and BCM21664 L2 cache control
 obj-$(CONFIG_ARCH_BCM_MOBILE_L2_CACHE) += kona_l2_cache.o
diff --git a/arch/arm/mach-bcm/bcm_nsp.h b/arch/arm/mach-bcm/bcm_nsp.h
new file mode 100644
index 0000000..58e1e80
--- /dev/null
+++ b/arch/arm/mach-bcm/bcm_nsp.h
@@ -0,0 +1,19 @@
+/*
+ * Copyright (C) 2015 Broadcom Corporation
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation version 2.
+ *
+ * This program is distributed "as is" WITHOUT ANY WARRANTY of any
+ * kind, whether express or implied; without even the implied warranty
+ * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __BCM_NSP_H
+#define __BCM_NSP_H
+
+extern void nsp_secondary_startup(void);
+
+#endif /* __BCM_NSP_H */
diff --git a/arch/arm/mach-bcm/headsmp.S b/arch/arm/mach-bcm/headsmp.S
new file mode 100644
index 0000000..0da13b2
--- /dev/null
+++ b/arch/arm/mach-bcm/headsmp.S
@@ -0,0 +1,37 @@
+/*
+ * Copyright (C) 2015 Broadcom Corporation
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation version 2.
+ *
+ * This program is distributed "as is" WITHOUT ANY WARRANTY of any
+ * kind, whether express or implied; without even the implied warranty
+ * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/linkage.h>
+
+/*
+ * iProc specific entry point for secondary CPUs.  This provides
+ * a "holding pen" into which all secondary cores are held until
+ * we are ready for them to initialise.
+ */
+ENTRY(nsp_secondary_startup)
+	mrc     p15, 0, r0, c0, c0, 5
+	and     r0, r0, #15
+	adr     r4, 1f
+	ldmia   r4, {r5, r6}
+	sub     r4, r4, r5
+	add     r6, r6, r4
+pen:	ldr     r7, [r6]
+	cmp     r7, r0
+	bne     pen
+
+	b    secondary_startup
+
+1:	.long   .
+	.long   pen_release
+
+ENDPROC(nsp_secondary_startup)
diff --git a/arch/arm/mach-bcm/kona_smp.c b/arch/arm/mach-bcm/platsmp.c
similarity index 63%
rename from arch/arm/mach-bcm/kona_smp.c
rename to arch/arm/mach-bcm/platsmp.c
index 66a0465..619030e 100644
--- a/arch/arm/mach-bcm/kona_smp.c
+++ b/arch/arm/mach-bcm/platsmp.c
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2014 Broadcom Corporation
+ * Copyright (C) 2014-2015 Broadcom Corporation
  * Copyright 2014 Linaro Limited
  *
  * This program is free software; you can redistribute it and/or
@@ -12,16 +12,23 @@
  * GNU General Public License for more details.
  */
 
-#include <linux/init.h>
+#include <linux/cpumask.h>
+#include <linux/delay.h>
 #include <linux/errno.h>
+#include <linux/init.h>
 #include <linux/io.h>
+#include <linux/jiffies.h>
 #include <linux/of.h>
 #include <linux/sched.h>
+#include <linux/smp.h>
 
+#include <asm/cacheflush.h>
 #include <asm/smp.h>
 #include <asm/smp_plat.h>
 #include <asm/smp_scu.h>
 
+#include "bcm_nsp.h"
+
 /* Size of mapped Cortex A9 SCU address space */
 #define CORTEX_A9_SCU_SIZE	0x58
 
@@ -34,6 +41,24 @@
 /* I/O address of register used to coordinate secondary core startup */
 static u32	secondary_boot;
 
+static DEFINE_SPINLOCK(boot_lock);
+
+/*
+ * Write pen_release in a way that is guaranteed to be visible to all
+ * observers, irrespective of whether they're taking part in coherency
+ * or not.  This is necessary for the hotplug code to work reliably.
+ */
+static void write_pen_release(int val)
+{
+	pen_release = val;
+	/*
+	 * Ensure write to pen_release is visible to the other cores,
+	 * here - primary core
+	 */
+	smp_wmb();
+	sync_cache_w(&pen_release);
+}
+
 /*
  * Enable the Cortex A9 Snoop Control Unit
  *
@@ -75,6 +100,51 @@ static int __init scu_a9_enable(void)
 	return 0;
 }
 
+static int nsp_write_lut(void (*secondary_startup) (void))
+{
+	void __iomem *sku_rom_lut;
+	phys_addr_t secondary_startup_phy;
+
+	if (!secondary_boot) {
+		pr_warn("required secondary boot register not specified\n");
+		return -EINVAL;
+	}
+
+	sku_rom_lut = ioremap_nocache((phys_addr_t)secondary_boot,
+						sizeof(secondary_boot));
+	if (!sku_rom_lut) {
+		pr_warn("unable to ioremap SKU-ROM LUT register\n");
+		return -ENOMEM;
+	}
+
+	secondary_startup_phy = virt_to_phys(secondary_startup);
+	BUG_ON(secondary_startup_phy > (phys_addr_t)U32_MAX);
+
+	writel_relaxed(secondary_startup_phy, sku_rom_lut);
+	/*
+	 * Ensure the write is visible to the secondary core.
+	 */
+	smp_wmb();
+
+	iounmap(sku_rom_lut);
+
+	return 0;
+}
+
+static void nsp_secondary_init(unsigned int cpu)
+{
+	/*
+	 * Let the primary cpu know we are out of holding pen.
+	 */
+	write_pen_release(-1);
+
+	/*
+	 * Synchronise with the boot thread.
+	 */
+	spin_lock(&boot_lock);
+	spin_unlock(&boot_lock);
+}
+
 static void __init bcm_smp_prepare_cpus(unsigned int max_cpus)
 {
 	static cpumask_t only_cpu_0 = { CPU_BITS_CPU0 };
@@ -95,11 +165,11 @@ static void __init bcm_smp_prepare_cpus(unsigned int max_cpus)
 	/*
 	 * Our secondary enable method requires a "secondary-boot-reg"
 	 * property to specify a register address used to request the
-	 * ROM code boot a secondary code.  If we have any trouble
+	 * ROM code boot a secondary core.  If we have any trouble
 	 * getting this we fall back to uniprocessor mode.
 	 */
 	if (of_property_read_u32(node, OF_SECONDARY_BOOT, &secondary_boot)) {
-		pr_err("%s: missing/invalid " OF_SECONDARY_BOOT " property\n",
+		pr_warn("%s: missing/invalid " OF_SECONDARY_BOOT " property\n",
 			node->name);
 		ret = -ENOENT;		/* Arrange to disable SMP */
 		goto out;
@@ -115,7 +185,6 @@ out:
 	of_node_put(node);
 	if (ret) {
 		/* Update the CPU present map to reflect uniprocessor mode */
-		BUG_ON(ret != -ENOENT);
 		pr_warn("disabling SMP\n");
 		init_cpu_present(&only_cpu_0);
 	}
@@ -139,7 +208,7 @@ out:
  * - Wait for the secondary boot register to be re-written, which
  *   indicates the secondary core has started.
  */
-static int bcm_boot_secondary(unsigned int cpu, struct task_struct *idle)
+static int kona_boot_secondary(unsigned int cpu, struct task_struct *idle)
 {
 	void __iomem *boot_reg;
 	phys_addr_t boot_func;
@@ -162,7 +231,7 @@ static int bcm_boot_secondary(unsigned int cpu, struct task_struct *idle)
 	boot_reg = ioremap_nocache((phys_addr_t)secondary_boot, sizeof(u32));
 	if (!boot_reg) {
 		pr_err("unable to map boot register for cpu %u\n", cpu_id);
-		return -ENOSYS;
+		return -ENOMEM;
 	}
 
 	/*
@@ -191,12 +260,67 @@ static int bcm_boot_secondary(unsigned int cpu, struct task_struct *idle)
 
 	pr_err("timeout waiting for cpu %u to start\n", cpu_id);
 
-	return -ENOSYS;
+	return -ENXIO;
+}
+
+static int nsp_boot_secondary(unsigned int cpu, struct task_struct *idle)
+{
+	unsigned long timeout;
+	int ret;
+
+	/*
+	 * After wake up, secondary core branches to the startup
+	 * address programmed at SKU ROM LUT location.
+	 */
+	ret = nsp_write_lut(nsp_secondary_startup);
+	if (ret) {
+		pr_err("unable to write startup addr to SKU ROM LUT\n");
+		goto out;
+	}
+
+	/*
+	 * The secondary processor is waiting to be released from
+	 * the holding pen - release it, then wait for it to flag
+	 * that it has been released by resetting pen_release.
+	 */
+	spin_lock(&boot_lock);
+
+	write_pen_release(cpu_logical_map(cpu));
+	/*
+	 * Send an Event to wake up the secondary core which is in
+	 * WFE state. Updated pen_release should also be visible to
+	 * the secondary core.
+	 */
+	dsb_sev();
+
+	timeout = jiffies + (1 * HZ);
+	while (time_before(jiffies, timeout)) {
+		/* Make sure loads on other CPU is visible */
+		smp_rmb();
+		if (pen_release == -1)
+			break;
+
+		udelay(10);
+	}
+
+	spin_unlock(&boot_lock);
+
+	ret = pen_release != -1 ? -ENXIO : 0;
+
+out:
+	return ret;
 }
 
 static struct smp_operations bcm_smp_ops __initdata = {
 	.smp_prepare_cpus	= bcm_smp_prepare_cpus,
-	.smp_boot_secondary	= bcm_boot_secondary,
+	.smp_boot_secondary	= kona_boot_secondary,
 };
 CPU_METHOD_OF_DECLARE(bcm_smp_bcm281xx, "brcm,bcm11351-cpu-method",
 			&bcm_smp_ops);
+
+struct smp_operations nsp_smp_ops __initdata = {
+	.smp_prepare_cpus	= bcm_smp_prepare_cpus,
+	.smp_secondary_init	= nsp_secondary_init,
+	.smp_boot_secondary	= nsp_boot_secondary,
+};
+CPU_METHOD_OF_DECLARE(bcm_smp_nsp, "brcm,bcm-nsp-smp", &nsp_smp_ops);
-- 
2.1.0

Re: [PATCH 3/3] ARM: BCM: Add SMP support for Broadcom NSP

[Hauke Mehrtens]

On 10/14/2015 07:47 PM, Kapil Hali wrote:
> Add SMP support for Broadcom's Northstar Plus SoC,
> cpu enable method and pen_release procedures. This
> changes also consolidates iProc family's - BCM NSP
> and BCM Kona, SMP handling in a common file.

This will probably also work on normal Northstar CPUs without changes.

> Northstar Plus SoC is based on ARM Cortex-A9
> revision r3p0 which requires configuration for ARM
> Errata 764369 for SMP. This change adds the needed
> configuration option.
> 
> Signed-off-by: Kapil Hali <kapilh-dY08KVG/lbpWk0Htik3J/w@public.gmane.org>
> ---
>  arch/arm/mach-bcm/Kconfig                   |   2 +
>  arch/arm/mach-bcm/Makefile                  |   8 +-
>  arch/arm/mach-bcm/bcm_nsp.h                 |  19 ++++
>  arch/arm/mach-bcm/headsmp.S                 |  37 ++++++++
>  arch/arm/mach-bcm/{kona_smp.c => platsmp.c} | 142 ++++++++++++++++++++++++++--
>  5 files changed, 197 insertions(+), 11 deletions(-)
>  create mode 100644 arch/arm/mach-bcm/bcm_nsp.h
>  create mode 100644 arch/arm/mach-bcm/headsmp.S
>  rename arch/arm/mach-bcm/{kona_smp.c => platsmp.c} (63%)
> 
> diff --git a/arch/arm/mach-bcm/Kconfig b/arch/arm/mach-bcm/Kconfig
> index 1679fa4..2e9dbb5 100644
> --- a/arch/arm/mach-bcm/Kconfig
> +++ b/arch/arm/mach-bcm/Kconfig
> @@ -40,6 +40,8 @@ config ARCH_BCM_NSP
>  	select ARCH_BCM_IPROC
>  	select ARM_ERRATA_754322
>  	select ARM_ERRATA_775420
> +	select ARM_ERRATA_764369 if SMP
> +	select HAVE_SMP
>  	help
>  	  Support for Broadcom Northstar Plus SoC.
>  	  Broadcom Northstar Plus family of SoCs are used for switching control
> diff --git a/arch/arm/mach-bcm/Makefile b/arch/arm/mach-bcm/Makefile
> index 892261f..36a4ca30 100644
> --- a/arch/arm/mach-bcm/Makefile
> +++ b/arch/arm/mach-bcm/Makefile
> @@ -14,7 +14,11 @@
>  obj-$(CONFIG_ARCH_BCM_CYGNUS) +=  bcm_cygnus.o
>  
>  # Northstar Plus
> -obj-$(CONFIG_ARCH_BCM_NSP) += bcm_nsp.o
> +obj-$(CONFIG_ARCH_BCM_NSP)	+= bcm_nsp.o
> +
> +ifeq ($(CONFIG_ARCH_BCM_NSP),y)
> +obj-$(CONFIG_SMP)		+= headsmp.o platsmp.o
> +endif
>  
>  # BCM281XX
>  obj-$(CONFIG_ARCH_BCM_281XX)	+= board_bcm281xx.o
> @@ -23,7 +27,7 @@ obj-$(CONFIG_ARCH_BCM_281XX)	+= board_bcm281xx.o
>  obj-$(CONFIG_ARCH_BCM_21664)	+= board_bcm21664.o
>  
>  # BCM281XX and BCM21664 SMP support
> -obj-$(CONFIG_ARCH_BCM_MOBILE_SMP) += kona_smp.o
> +obj-$(CONFIG_ARCH_BCM_MOBILE_SMP) += platsmp.o
>  
>  # BCM281XX and BCM21664 L2 cache control
>  obj-$(CONFIG_ARCH_BCM_MOBILE_L2_CACHE) += kona_l2_cache.o
> diff --git a/arch/arm/mach-bcm/bcm_nsp.h b/arch/arm/mach-bcm/bcm_nsp.h
> new file mode 100644
> index 0000000..58e1e80
> --- /dev/null
> +++ b/arch/arm/mach-bcm/bcm_nsp.h
> @@ -0,0 +1,19 @@
> +/*
> + * Copyright (C) 2015 Broadcom Corporation
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License as
> + * published by the Free Software Foundation version 2.
> + *
> + * This program is distributed "as is" WITHOUT ANY WARRANTY of any
> + * kind, whether express or implied; without even the implied warranty
> + * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef __BCM_NSP_H
> +#define __BCM_NSP_H
> +
> +extern void nsp_secondary_startup(void);
> +
> +#endif /* __BCM_NSP_H */
> diff --git a/arch/arm/mach-bcm/headsmp.S b/arch/arm/mach-bcm/headsmp.S
> new file mode 100644
> index 0000000..0da13b2
> --- /dev/null
> +++ b/arch/arm/mach-bcm/headsmp.S
> @@ -0,0 +1,37 @@
> +/*
> + * Copyright (C) 2015 Broadcom Corporation
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License as
> + * published by the Free Software Foundation version 2.
> + *
> + * This program is distributed "as is" WITHOUT ANY WARRANTY of any
> + * kind, whether express or implied; without even the implied warranty
> + * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#include <linux/linkage.h>
> +
> +/*
> + * iProc specific entry point for secondary CPUs.  This provides
> + * a "holding pen" into which all secondary cores are held until
> + * we are ready for them to initialise.
> + */
> +ENTRY(nsp_secondary_startup)
> +	mrc     p15, 0, r0, c0, c0, 5
> +	and     r0, r0, #15
> +	adr     r4, 1f
> +	ldmia   r4, {r5, r6}
> +	sub     r4, r4, r5
> +	add     r6, r6, r4
> +pen:	ldr     r7, [r6]
> +	cmp     r7, r0
> +	bne     pen
> +
> +	b    secondary_startup
> +
> +1:	.long   .
> +	.long   pen_release
> +
> +ENDPROC(nsp_secondary_startup)
> diff --git a/arch/arm/mach-bcm/kona_smp.c b/arch/arm/mach-bcm/platsmp.c
> similarity index 63%
> rename from arch/arm/mach-bcm/kona_smp.c
> rename to arch/arm/mach-bcm/platsmp.c
> index 66a0465..619030e 100644
> --- a/arch/arm/mach-bcm/kona_smp.c
> +++ b/arch/arm/mach-bcm/platsmp.c
> @@ -1,5 +1,5 @@
>  /*
> - * Copyright (C) 2014 Broadcom Corporation
> + * Copyright (C) 2014-2015 Broadcom Corporation
>   * Copyright 2014 Linaro Limited
>   *
>   * This program is free software; you can redistribute it and/or
> @@ -12,16 +12,23 @@
>   * GNU General Public License for more details.
>   */
>  
> -#include <linux/init.h>
> +#include <linux/cpumask.h>
> +#include <linux/delay.h>
>  #include <linux/errno.h>
> +#include <linux/init.h>
>  #include <linux/io.h>
> +#include <linux/jiffies.h>
>  #include <linux/of.h>
>  #include <linux/sched.h>
> +#include <linux/smp.h>
>  
> +#include <asm/cacheflush.h>
>  #include <asm/smp.h>
>  #include <asm/smp_plat.h>
>  #include <asm/smp_scu.h>
>  
> +#include "bcm_nsp.h"
> +
>  /* Size of mapped Cortex A9 SCU address space */
>  #define CORTEX_A9_SCU_SIZE	0x58
>  
> @@ -34,6 +41,24 @@
>  /* I/O address of register used to coordinate secondary core startup */
>  static u32	secondary_boot;
>  
> +static DEFINE_SPINLOCK(boot_lock);
> +
> +/*
> + * Write pen_release in a way that is guaranteed to be visible to all
> + * observers, irrespective of whether they're taking part in coherency
> + * or not.  This is necessary for the hotplug code to work reliably.
> + */
> +static void write_pen_release(int val)
> +{
> +	pen_release = val;
> +	/*
> +	 * Ensure write to pen_release is visible to the other cores,
> +	 * here - primary core
> +	 */
> +	smp_wmb();
> +	sync_cache_w(&pen_release);
> +}
> +
>  /*
>   * Enable the Cortex A9 Snoop Control Unit
>   *
> @@ -75,6 +100,51 @@ static int __init scu_a9_enable(void)
>  	return 0;
>  }
>  
> +static int nsp_write_lut(void (*secondary_startup) (void))
> +{
> +	void __iomem *sku_rom_lut;
> +	phys_addr_t secondary_startup_phy;
> +
> +	if (!secondary_boot) {
> +		pr_warn("required secondary boot register not specified\n");
> +		return -EINVAL;
> +	}
> +
> +	sku_rom_lut = ioremap_nocache((phys_addr_t)secondary_boot,
> +						sizeof(secondary_boot));
> +	if (!sku_rom_lut) {
> +		pr_warn("unable to ioremap SKU-ROM LUT register\n");
> +		return -ENOMEM;
> +	}
> +
> +	secondary_startup_phy = virt_to_phys(secondary_startup);
> +	BUG_ON(secondary_startup_phy > (phys_addr_t)U32_MAX);
> +
> +	writel_relaxed(secondary_startup_phy, sku_rom_lut);
> +	/*
> +	 * Ensure the write is visible to the secondary core.
> +	 */
> +	smp_wmb();
> +
> +	iounmap(sku_rom_lut);
> +
> +	return 0;
> +}
> +
> +static void nsp_secondary_init(unsigned int cpu)
> +{
> +	/*
> +	 * Let the primary cpu know we are out of holding pen.
> +	 */
> +	write_pen_release(-1);
> +
> +	/*
> +	 * Synchronise with the boot thread.
> +	 */
> +	spin_lock(&boot_lock);
> +	spin_unlock(&boot_lock);
> +}
> +
>  static void __init bcm_smp_prepare_cpus(unsigned int max_cpus)
>  {
>  	static cpumask_t only_cpu_0 = { CPU_BITS_CPU0 };
> @@ -95,11 +165,11 @@ static void __init bcm_smp_prepare_cpus(unsigned int max_cpus)
>  	/*
>  	 * Our secondary enable method requires a "secondary-boot-reg"
>  	 * property to specify a register address used to request the
> -	 * ROM code boot a secondary code.  If we have any trouble
> +	 * ROM code boot a secondary core.  If we have any trouble
>  	 * getting this we fall back to uniprocessor mode.
>  	 */
>  	if (of_property_read_u32(node, OF_SECONDARY_BOOT, &secondary_boot)) {
> -		pr_err("%s: missing/invalid " OF_SECONDARY_BOOT " property\n",
> +		pr_warn("%s: missing/invalid " OF_SECONDARY_BOOT " property\n",
>  			node->name);
>  		ret = -ENOENT;		/* Arrange to disable SMP */
>  		goto out;
> @@ -115,7 +185,6 @@ out:
>  	of_node_put(node);
>  	if (ret) {
>  		/* Update the CPU present map to reflect uniprocessor mode */
> -		BUG_ON(ret != -ENOENT);
>  		pr_warn("disabling SMP\n");
>  		init_cpu_present(&only_cpu_0);
>  	}
> @@ -139,7 +208,7 @@ out:
>   * - Wait for the secondary boot register to be re-written, which
>   *   indicates the secondary core has started.
>   */
> -static int bcm_boot_secondary(unsigned int cpu, struct task_struct *idle)
> +static int kona_boot_secondary(unsigned int cpu, struct task_struct *idle)
>  {
>  	void __iomem *boot_reg;
>  	phys_addr_t boot_func;
> @@ -162,7 +231,7 @@ static int bcm_boot_secondary(unsigned int cpu, struct task_struct *idle)
>  	boot_reg = ioremap_nocache((phys_addr_t)secondary_boot, sizeof(u32));
>  	if (!boot_reg) {
>  		pr_err("unable to map boot register for cpu %u\n", cpu_id);
> -		return -ENOSYS;
> +		return -ENOMEM;
>  	}
>  
>  	/*
> @@ -191,12 +260,67 @@ static int bcm_boot_secondary(unsigned int cpu, struct task_struct *idle)
>  
>  	pr_err("timeout waiting for cpu %u to start\n", cpu_id);
>  
> -	return -ENOSYS;
> +	return -ENXIO;
> +}
> +
> +static int nsp_boot_secondary(unsigned int cpu, struct task_struct *idle)
> +{
> +	unsigned long timeout;
> +	int ret;
> +
> +	/*
> +	 * After wake up, secondary core branches to the startup
> +	 * address programmed at SKU ROM LUT location.
> +	 */
> +	ret = nsp_write_lut(nsp_secondary_startup);
> +	if (ret) {
> +		pr_err("unable to write startup addr to SKU ROM LUT\n");
> +		goto out;
> +	}
> +
> +	/*
> +	 * The secondary processor is waiting to be released from
> +	 * the holding pen - release it, then wait for it to flag
> +	 * that it has been released by resetting pen_release.
> +	 */
> +	spin_lock(&boot_lock);
> +
> +	write_pen_release(cpu_logical_map(cpu));
> +	/*
> +	 * Send an Event to wake up the secondary core which is in
> +	 * WFE state. Updated pen_release should also be visible to
> +	 * the secondary core.
> +	 */
> +	dsb_sev();
> +
> +	timeout = jiffies + (1 * HZ);
> +	while (time_before(jiffies, timeout)) {
> +		/* Make sure loads on other CPU is visible */
> +		smp_rmb();
> +		if (pen_release == -1)
> +			break;
> +
> +		udelay(10);
> +	}
> +
> +	spin_unlock(&boot_lock);

Why is this boot_lock needed? As far as I understand it asserts that
both CPUs leave nsp_boot_secondary() and nsp_secondary_init() at the
same time.

> +
> +	ret = pen_release != -1 ? -ENXIO : 0;
> +
> +out:
> +	return ret;
>  }
>  
>  static struct smp_operations bcm_smp_ops __initdata = {
>  	.smp_prepare_cpus	= bcm_smp_prepare_cpus,
> -	.smp_boot_secondary	= bcm_boot_secondary,
> +	.smp_boot_secondary	= kona_boot_secondary,
>  };
>  CPU_METHOD_OF_DECLARE(bcm_smp_bcm281xx, "brcm,bcm11351-cpu-method",
>  			&bcm_smp_ops);
> +
> +struct smp_operations nsp_smp_ops __initdata = {
> +	.smp_prepare_cpus	= bcm_smp_prepare_cpus,
> +	.smp_secondary_init	= nsp_secondary_init,
> +	.smp_boot_secondary	= nsp_boot_secondary,
> +};
> +CPU_METHOD_OF_DECLARE(bcm_smp_nsp, "brcm,bcm-nsp-smp", &nsp_smp_ops);
> 

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 3/3] ARM: BCM: Add SMP support for Broadcom NSP

[Jon Mason]

On Thu, Oct 15, 2015 at 12:26:25AM +0200, Hauke Mehrtens wrote:
> On 10/14/2015 07:47 PM, Kapil Hali wrote:
> > Add SMP support for Broadcom's Northstar Plus SoC,
> > cpu enable method and pen_release procedures. This
> > changes also consolidates iProc family's - BCM NSP
> > and BCM Kona, SMP handling in a common file.
> 
> This will probably also work on normal Northstar CPUs without changes.

I think all that needs to be changed is adding to
arch/arm/boot/dts/bcm4708.dts

+               enable-method = "brcm,bcm-nsp-smp";
+               secondary-boot-reg = <0xffff0400>;

But I have not been able to confirm that yet.

Thanks,
Jon

> 
> > Northstar Plus SoC is based on ARM Cortex-A9
> > revision r3p0 which requires configuration for ARM
> > Errata 764369 for SMP. This change adds the needed
> > configuration option.
> > 
> > Signed-off-by: Kapil Hali <kapilh@broadcom.com>
> > ---
> >  arch/arm/mach-bcm/Kconfig                   |   2 +
> >  arch/arm/mach-bcm/Makefile                  |   8 +-
> >  arch/arm/mach-bcm/bcm_nsp.h                 |  19 ++++
> >  arch/arm/mach-bcm/headsmp.S                 |  37 ++++++++
> >  arch/arm/mach-bcm/{kona_smp.c => platsmp.c} | 142 ++++++++++++++++++++++++++--
> >  5 files changed, 197 insertions(+), 11 deletions(-)
> >  create mode 100644 arch/arm/mach-bcm/bcm_nsp.h
> >  create mode 100644 arch/arm/mach-bcm/headsmp.S
> >  rename arch/arm/mach-bcm/{kona_smp.c => platsmp.c} (63%)
> > 
> > diff --git a/arch/arm/mach-bcm/Kconfig b/arch/arm/mach-bcm/Kconfig
> > index 1679fa4..2e9dbb5 100644
> > --- a/arch/arm/mach-bcm/Kconfig
> > +++ b/arch/arm/mach-bcm/Kconfig
> > @@ -40,6 +40,8 @@ config ARCH_BCM_NSP
> >  	select ARCH_BCM_IPROC
> >  	select ARM_ERRATA_754322
> >  	select ARM_ERRATA_775420
> > +	select ARM_ERRATA_764369 if SMP
> > +	select HAVE_SMP
> >  	help
> >  	  Support for Broadcom Northstar Plus SoC.
> >  	  Broadcom Northstar Plus family of SoCs are used for switching control
> > diff --git a/arch/arm/mach-bcm/Makefile b/arch/arm/mach-bcm/Makefile
> > index 892261f..36a4ca30 100644
> > --- a/arch/arm/mach-bcm/Makefile
> > +++ b/arch/arm/mach-bcm/Makefile
> > @@ -14,7 +14,11 @@
> >  obj-$(CONFIG_ARCH_BCM_CYGNUS) +=  bcm_cygnus.o
> >  
> >  # Northstar Plus
> > -obj-$(CONFIG_ARCH_BCM_NSP) += bcm_nsp.o
> > +obj-$(CONFIG_ARCH_BCM_NSP)	+= bcm_nsp.o
> > +
> > +ifeq ($(CONFIG_ARCH_BCM_NSP),y)
> > +obj-$(CONFIG_SMP)		+= headsmp.o platsmp.o
> > +endif
> >  
> >  # BCM281XX
> >  obj-$(CONFIG_ARCH_BCM_281XX)	+= board_bcm281xx.o
> > @@ -23,7 +27,7 @@ obj-$(CONFIG_ARCH_BCM_281XX)	+= board_bcm281xx.o
> >  obj-$(CONFIG_ARCH_BCM_21664)	+= board_bcm21664.o
> >  
> >  # BCM281XX and BCM21664 SMP support
> > -obj-$(CONFIG_ARCH_BCM_MOBILE_SMP) += kona_smp.o
> > +obj-$(CONFIG_ARCH_BCM_MOBILE_SMP) += platsmp.o
> >  
> >  # BCM281XX and BCM21664 L2 cache control
> >  obj-$(CONFIG_ARCH_BCM_MOBILE_L2_CACHE) += kona_l2_cache.o
> > diff --git a/arch/arm/mach-bcm/bcm_nsp.h b/arch/arm/mach-bcm/bcm_nsp.h
> > new file mode 100644
> > index 0000000..58e1e80
> > --- /dev/null
> > +++ b/arch/arm/mach-bcm/bcm_nsp.h
> > @@ -0,0 +1,19 @@
> > +/*
> > + * Copyright (C) 2015 Broadcom Corporation
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify it under the terms of the GNU General Public License as
> > + * published by the Free Software Foundation version 2.
> > + *
> > + * This program is distributed "as is" WITHOUT ANY WARRANTY of any
> > + * kind, whether express or implied; without even the implied warranty
> > + * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef __BCM_NSP_H
> > +#define __BCM_NSP_H
> > +
> > +extern void nsp_secondary_startup(void);
> > +
> > +#endif /* __BCM_NSP_H */
> > diff --git a/arch/arm/mach-bcm/headsmp.S b/arch/arm/mach-bcm/headsmp.S
> > new file mode 100644
> > index 0000000..0da13b2
> > --- /dev/null
> > +++ b/arch/arm/mach-bcm/headsmp.S
> > @@ -0,0 +1,37 @@
> > +/*
> > + * Copyright (C) 2015 Broadcom Corporation
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify it under the terms of the GNU General Public License as
> > + * published by the Free Software Foundation version 2.
> > + *
> > + * This program is distributed "as is" WITHOUT ANY WARRANTY of any
> > + * kind, whether express or implied; without even the implied warranty
> > + * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#include <linux/linkage.h>
> > +
> > +/*
> > + * iProc specific entry point for secondary CPUs.  This provides
> > + * a "holding pen" into which all secondary cores are held until
> > + * we are ready for them to initialise.
> > + */
> > +ENTRY(nsp_secondary_startup)
> > +	mrc     p15, 0, r0, c0, c0, 5
> > +	and     r0, r0, #15
> > +	adr     r4, 1f
> > +	ldmia   r4, {r5, r6}
> > +	sub     r4, r4, r5
> > +	add     r6, r6, r4
> > +pen:	ldr     r7, [r6]
> > +	cmp     r7, r0
> > +	bne     pen
> > +
> > +	b    secondary_startup
> > +
> > +1:	.long   .
> > +	.long   pen_release
> > +
> > +ENDPROC(nsp_secondary_startup)
> > diff --git a/arch/arm/mach-bcm/kona_smp.c b/arch/arm/mach-bcm/platsmp.c
> > similarity index 63%
> > rename from arch/arm/mach-bcm/kona_smp.c
> > rename to arch/arm/mach-bcm/platsmp.c
> > index 66a0465..619030e 100644
> > --- a/arch/arm/mach-bcm/kona_smp.c
> > +++ b/arch/arm/mach-bcm/platsmp.c
> > @@ -1,5 +1,5 @@
> >  /*
> > - * Copyright (C) 2014 Broadcom Corporation
> > + * Copyright (C) 2014-2015 Broadcom Corporation
> >   * Copyright 2014 Linaro Limited
> >   *
> >   * This program is free software; you can redistribute it and/or
> > @@ -12,16 +12,23 @@
> >   * GNU General Public License for more details.
> >   */
> >  
> > -#include <linux/init.h>
> > +#include <linux/cpumask.h>
> > +#include <linux/delay.h>
> >  #include <linux/errno.h>
> > +#include <linux/init.h>
> >  #include <linux/io.h>
> > +#include <linux/jiffies.h>
> >  #include <linux/of.h>
> >  #include <linux/sched.h>
> > +#include <linux/smp.h>
> >  
> > +#include <asm/cacheflush.h>
> >  #include <asm/smp.h>
> >  #include <asm/smp_plat.h>
> >  #include <asm/smp_scu.h>
> >  
> > +#include "bcm_nsp.h"
> > +
> >  /* Size of mapped Cortex A9 SCU address space */
> >  #define CORTEX_A9_SCU_SIZE	0x58
> >  
> > @@ -34,6 +41,24 @@
> >  /* I/O address of register used to coordinate secondary core startup */
> >  static u32	secondary_boot;
> >  
> > +static DEFINE_SPINLOCK(boot_lock);
> > +
> > +/*
> > + * Write pen_release in a way that is guaranteed to be visible to all
> > + * observers, irrespective of whether they're taking part in coherency
> > + * or not.  This is necessary for the hotplug code to work reliably.
> > + */
> > +static void write_pen_release(int val)
> > +{
> > +	pen_release = val;
> > +	/*
> > +	 * Ensure write to pen_release is visible to the other cores,
> > +	 * here - primary core
> > +	 */
> > +	smp_wmb();
> > +	sync_cache_w(&pen_release);
> > +}
> > +
> >  /*
> >   * Enable the Cortex A9 Snoop Control Unit
> >   *
> > @@ -75,6 +100,51 @@ static int __init scu_a9_enable(void)
> >  	return 0;
> >  }
> >  
> > +static int nsp_write_lut(void (*secondary_startup) (void))
> > +{
> > +	void __iomem *sku_rom_lut;
> > +	phys_addr_t secondary_startup_phy;
> > +
> > +	if (!secondary_boot) {
> > +		pr_warn("required secondary boot register not specified\n");
> > +		return -EINVAL;
> > +	}
> > +
> > +	sku_rom_lut = ioremap_nocache((phys_addr_t)secondary_boot,
> > +						sizeof(secondary_boot));
> > +	if (!sku_rom_lut) {
> > +		pr_warn("unable to ioremap SKU-ROM LUT register\n");
> > +		return -ENOMEM;
> > +	}
> > +
> > +	secondary_startup_phy = virt_to_phys(secondary_startup);
> > +	BUG_ON(secondary_startup_phy > (phys_addr_t)U32_MAX);
> > +
> > +	writel_relaxed(secondary_startup_phy, sku_rom_lut);
> > +	/*
> > +	 * Ensure the write is visible to the secondary core.
> > +	 */
> > +	smp_wmb();
> > +
> > +	iounmap(sku_rom_lut);
> > +
> > +	return 0;
> > +}
> > +
> > +static void nsp_secondary_init(unsigned int cpu)
> > +{
> > +	/*
> > +	 * Let the primary cpu know we are out of holding pen.
> > +	 */
> > +	write_pen_release(-1);
> > +
> > +	/*
> > +	 * Synchronise with the boot thread.
> > +	 */
> > +	spin_lock(&boot_lock);
> > +	spin_unlock(&boot_lock);
> > +}
> > +
> >  static void __init bcm_smp_prepare_cpus(unsigned int max_cpus)
> >  {
> >  	static cpumask_t only_cpu_0 = { CPU_BITS_CPU0 };
> > @@ -95,11 +165,11 @@ static void __init bcm_smp_prepare_cpus(unsigned int max_cpus)
> >  	/*
> >  	 * Our secondary enable method requires a "secondary-boot-reg"
> >  	 * property to specify a register address used to request the
> > -	 * ROM code boot a secondary code.  If we have any trouble
> > +	 * ROM code boot a secondary core.  If we have any trouble
> >  	 * getting this we fall back to uniprocessor mode.
> >  	 */
> >  	if (of_property_read_u32(node, OF_SECONDARY_BOOT, &secondary_boot)) {
> > -		pr_err("%s: missing/invalid " OF_SECONDARY_BOOT " property\n",
> > +		pr_warn("%s: missing/invalid " OF_SECONDARY_BOOT " property\n",
> >  			node->name);
> >  		ret = -ENOENT;		/* Arrange to disable SMP */
> >  		goto out;
> > @@ -115,7 +185,6 @@ out:
> >  	of_node_put(node);
> >  	if (ret) {
> >  		/* Update the CPU present map to reflect uniprocessor mode */
> > -		BUG_ON(ret != -ENOENT);
> >  		pr_warn("disabling SMP\n");
> >  		init_cpu_present(&only_cpu_0);
> >  	}
> > @@ -139,7 +208,7 @@ out:
> >   * - Wait for the secondary boot register to be re-written, which
> >   *   indicates the secondary core has started.
> >   */
> > -static int bcm_boot_secondary(unsigned int cpu, struct task_struct *idle)
> > +static int kona_boot_secondary(unsigned int cpu, struct task_struct *idle)
> >  {
> >  	void __iomem *boot_reg;
> >  	phys_addr_t boot_func;
> > @@ -162,7 +231,7 @@ static int bcm_boot_secondary(unsigned int cpu, struct task_struct *idle)
> >  	boot_reg = ioremap_nocache((phys_addr_t)secondary_boot, sizeof(u32));
> >  	if (!boot_reg) {
> >  		pr_err("unable to map boot register for cpu %u\n", cpu_id);
> > -		return -ENOSYS;
> > +		return -ENOMEM;
> >  	}
> >  
> >  	/*
> > @@ -191,12 +260,67 @@ static int bcm_boot_secondary(unsigned int cpu, struct task_struct *idle)
> >  
> >  	pr_err("timeout waiting for cpu %u to start\n", cpu_id);
> >  
> > -	return -ENOSYS;
> > +	return -ENXIO;
> > +}
> > +
> > +static int nsp_boot_secondary(unsigned int cpu, struct task_struct *idle)
> > +{
> > +	unsigned long timeout;
> > +	int ret;
> > +
> > +	/*
> > +	 * After wake up, secondary core branches to the startup
> > +	 * address programmed at SKU ROM LUT location.
> > +	 */
> > +	ret = nsp_write_lut(nsp_secondary_startup);
> > +	if (ret) {
> > +		pr_err("unable to write startup addr to SKU ROM LUT\n");
> > +		goto out;
> > +	}
> > +
> > +	/*
> > +	 * The secondary processor is waiting to be released from
> > +	 * the holding pen - release it, then wait for it to flag
> > +	 * that it has been released by resetting pen_release.
> > +	 */
> > +	spin_lock(&boot_lock);
> > +
> > +	write_pen_release(cpu_logical_map(cpu));
> > +	/*
> > +	 * Send an Event to wake up the secondary core which is in
> > +	 * WFE state. Updated pen_release should also be visible to
> > +	 * the secondary core.
> > +	 */
> > +	dsb_sev();
> > +
> > +	timeout = jiffies + (1 * HZ);
> > +	while (time_before(jiffies, timeout)) {
> > +		/* Make sure loads on other CPU is visible */
> > +		smp_rmb();
> > +		if (pen_release == -1)
> > +			break;
> > +
> > +		udelay(10);
> > +	}
> > +
> > +	spin_unlock(&boot_lock);
> 
> Why is this boot_lock needed? As far as I understand it asserts that
> both CPUs leave nsp_boot_secondary() and nsp_secondary_init() at the
> same time.
> 
> > +
> > +	ret = pen_release != -1 ? -ENXIO : 0;
> > +
> > +out:
> > +	return ret;
> >  }
> >  
> >  static struct smp_operations bcm_smp_ops __initdata = {
> >  	.smp_prepare_cpus	= bcm_smp_prepare_cpus,
> > -	.smp_boot_secondary	= bcm_boot_secondary,
> > +	.smp_boot_secondary	= kona_boot_secondary,
> >  };
> >  CPU_METHOD_OF_DECLARE(bcm_smp_bcm281xx, "brcm,bcm11351-cpu-method",
> >  			&bcm_smp_ops);
> > +
> > +struct smp_operations nsp_smp_ops __initdata = {
> > +	.smp_prepare_cpus	= bcm_smp_prepare_cpus,
> > +	.smp_secondary_init	= nsp_secondary_init,
> > +	.smp_boot_secondary	= nsp_boot_secondary,
> > +};
> > +CPU_METHOD_OF_DECLARE(bcm_smp_nsp, "brcm,bcm-nsp-smp", &nsp_smp_ops);
> > 
> 

Re: [PATCH 3/3] ARM: BCM: Add SMP support for Broadcom NSP

[Jon Mason]

On Wed, Oct 14, 2015 at 06:40:04PM -0400, Jon Mason wrote:
> On Thu, Oct 15, 2015 at 12:26:25AM +0200, Hauke Mehrtens wrote:
> > On 10/14/2015 07:47 PM, Kapil Hali wrote:
> > > Add SMP support for Broadcom's Northstar Plus SoC,
> > > cpu enable method and pen_release procedures. This
> > > changes also consolidates iProc family's - BCM NSP
> > > and BCM Kona, SMP handling in a common file.
> > 
> > This will probably also work on normal Northstar CPUs without changes.
> 
> I think all that needs to be changed is adding to
> arch/arm/boot/dts/bcm4708.dts
> 
> +               enable-method = "brcm,bcm-nsp-smp";
> +               secondary-boot-reg = <0xffff0400>;
> 
> But I have not been able to confirm that yet.

I was able to confirm it on my BCM94708 SVK.  I'll send out the patch
which enables it shortly.

Thanks,
Jon

> 
> Thanks,
> Jon
> 
> > 
> > > Northstar Plus SoC is based on ARM Cortex-A9
> > > revision r3p0 which requires configuration for ARM
> > > Errata 764369 for SMP. This change adds the needed
> > > configuration option.
> > > 
> > > Signed-off-by: Kapil Hali <kapilh@broadcom.com>
> > > ---
> > >  arch/arm/mach-bcm/Kconfig                   |   2 +
> > >  arch/arm/mach-bcm/Makefile                  |   8 +-
> > >  arch/arm/mach-bcm/bcm_nsp.h                 |  19 ++++
> > >  arch/arm/mach-bcm/headsmp.S                 |  37 ++++++++
> > >  arch/arm/mach-bcm/{kona_smp.c => platsmp.c} | 142 ++++++++++++++++++++++++++--
> > >  5 files changed, 197 insertions(+), 11 deletions(-)
> > >  create mode 100644 arch/arm/mach-bcm/bcm_nsp.h
> > >  create mode 100644 arch/arm/mach-bcm/headsmp.S
> > >  rename arch/arm/mach-bcm/{kona_smp.c => platsmp.c} (63%)
> > > 
> > > diff --git a/arch/arm/mach-bcm/Kconfig b/arch/arm/mach-bcm/Kconfig
> > > index 1679fa4..2e9dbb5 100644
> > > --- a/arch/arm/mach-bcm/Kconfig
> > > +++ b/arch/arm/mach-bcm/Kconfig
> > > @@ -40,6 +40,8 @@ config ARCH_BCM_NSP
> > >  	select ARCH_BCM_IPROC
> > >  	select ARM_ERRATA_754322
> > >  	select ARM_ERRATA_775420
> > > +	select ARM_ERRATA_764369 if SMP
> > > +	select HAVE_SMP
> > >  	help
> > >  	  Support for Broadcom Northstar Plus SoC.
> > >  	  Broadcom Northstar Plus family of SoCs are used for switching control
> > > diff --git a/arch/arm/mach-bcm/Makefile b/arch/arm/mach-bcm/Makefile
> > > index 892261f..36a4ca30 100644
> > > --- a/arch/arm/mach-bcm/Makefile
> > > +++ b/arch/arm/mach-bcm/Makefile
> > > @@ -14,7 +14,11 @@
> > >  obj-$(CONFIG_ARCH_BCM_CYGNUS) +=  bcm_cygnus.o
> > >  
> > >  # Northstar Plus
> > > -obj-$(CONFIG_ARCH_BCM_NSP) += bcm_nsp.o
> > > +obj-$(CONFIG_ARCH_BCM_NSP)	+= bcm_nsp.o
> > > +
> > > +ifeq ($(CONFIG_ARCH_BCM_NSP),y)
> > > +obj-$(CONFIG_SMP)		+= headsmp.o platsmp.o
> > > +endif
> > >  
> > >  # BCM281XX
> > >  obj-$(CONFIG_ARCH_BCM_281XX)	+= board_bcm281xx.o
> > > @@ -23,7 +27,7 @@ obj-$(CONFIG_ARCH_BCM_281XX)	+= board_bcm281xx.o
> > >  obj-$(CONFIG_ARCH_BCM_21664)	+= board_bcm21664.o
> > >  
> > >  # BCM281XX and BCM21664 SMP support
> > > -obj-$(CONFIG_ARCH_BCM_MOBILE_SMP) += kona_smp.o
> > > +obj-$(CONFIG_ARCH_BCM_MOBILE_SMP) += platsmp.o
> > >  
> > >  # BCM281XX and BCM21664 L2 cache control
> > >  obj-$(CONFIG_ARCH_BCM_MOBILE_L2_CACHE) += kona_l2_cache.o
> > > diff --git a/arch/arm/mach-bcm/bcm_nsp.h b/arch/arm/mach-bcm/bcm_nsp.h
> > > new file mode 100644
> > > index 0000000..58e1e80
> > > --- /dev/null
> > > +++ b/arch/arm/mach-bcm/bcm_nsp.h
> > > @@ -0,0 +1,19 @@
> > > +/*
> > > + * Copyright (C) 2015 Broadcom Corporation
> > > + *
> > > + * This program is free software; you can redistribute it and/or
> > > + * modify it under the terms of the GNU General Public License as
> > > + * published by the Free Software Foundation version 2.
> > > + *
> > > + * This program is distributed "as is" WITHOUT ANY WARRANTY of any
> > > + * kind, whether express or implied; without even the implied warranty
> > > + * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > > + * GNU General Public License for more details.
> > > + */
> > > +
> > > +#ifndef __BCM_NSP_H
> > > +#define __BCM_NSP_H
> > > +
> > > +extern void nsp_secondary_startup(void);
> > > +
> > > +#endif /* __BCM_NSP_H */
> > > diff --git a/arch/arm/mach-bcm/headsmp.S b/arch/arm/mach-bcm/headsmp.S
> > > new file mode 100644
> > > index 0000000..0da13b2
> > > --- /dev/null
> > > +++ b/arch/arm/mach-bcm/headsmp.S
> > > @@ -0,0 +1,37 @@
> > > +/*
> > > + * Copyright (C) 2015 Broadcom Corporation
> > > + *
> > > + * This program is free software; you can redistribute it and/or
> > > + * modify it under the terms of the GNU General Public License as
> > > + * published by the Free Software Foundation version 2.
> > > + *
> > > + * This program is distributed "as is" WITHOUT ANY WARRANTY of any
> > > + * kind, whether express or implied; without even the implied warranty
> > > + * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > > + * GNU General Public License for more details.
> > > + */
> > > +
> > > +#include <linux/linkage.h>
> > > +
> > > +/*
> > > + * iProc specific entry point for secondary CPUs.  This provides
> > > + * a "holding pen" into which all secondary cores are held until
> > > + * we are ready for them to initialise.
> > > + */
> > > +ENTRY(nsp_secondary_startup)
> > > +	mrc     p15, 0, r0, c0, c0, 5
> > > +	and     r0, r0, #15
> > > +	adr     r4, 1f
> > > +	ldmia   r4, {r5, r6}
> > > +	sub     r4, r4, r5
> > > +	add     r6, r6, r4
> > > +pen:	ldr     r7, [r6]
> > > +	cmp     r7, r0
> > > +	bne     pen
> > > +
> > > +	b    secondary_startup
> > > +
> > > +1:	.long   .
> > > +	.long   pen_release
> > > +
> > > +ENDPROC(nsp_secondary_startup)
> > > diff --git a/arch/arm/mach-bcm/kona_smp.c b/arch/arm/mach-bcm/platsmp.c
> > > similarity index 63%
> > > rename from arch/arm/mach-bcm/kona_smp.c
> > > rename to arch/arm/mach-bcm/platsmp.c
> > > index 66a0465..619030e 100644
> > > --- a/arch/arm/mach-bcm/kona_smp.c
> > > +++ b/arch/arm/mach-bcm/platsmp.c
> > > @@ -1,5 +1,5 @@
> > >  /*
> > > - * Copyright (C) 2014 Broadcom Corporation
> > > + * Copyright (C) 2014-2015 Broadcom Corporation
> > >   * Copyright 2014 Linaro Limited
> > >   *
> > >   * This program is free software; you can redistribute it and/or
> > > @@ -12,16 +12,23 @@
> > >   * GNU General Public License for more details.
> > >   */
> > >  
> > > -#include <linux/init.h>
> > > +#include <linux/cpumask.h>
> > > +#include <linux/delay.h>
> > >  #include <linux/errno.h>
> > > +#include <linux/init.h>
> > >  #include <linux/io.h>
> > > +#include <linux/jiffies.h>
> > >  #include <linux/of.h>
> > >  #include <linux/sched.h>
> > > +#include <linux/smp.h>
> > >  
> > > +#include <asm/cacheflush.h>
> > >  #include <asm/smp.h>
> > >  #include <asm/smp_plat.h>
> > >  #include <asm/smp_scu.h>
> > >  
> > > +#include "bcm_nsp.h"
> > > +
> > >  /* Size of mapped Cortex A9 SCU address space */
> > >  #define CORTEX_A9_SCU_SIZE	0x58
> > >  
> > > @@ -34,6 +41,24 @@
> > >  /* I/O address of register used to coordinate secondary core startup */
> > >  static u32	secondary_boot;
> > >  
> > > +static DEFINE_SPINLOCK(boot_lock);
> > > +
> > > +/*
> > > + * Write pen_release in a way that is guaranteed to be visible to all
> > > + * observers, irrespective of whether they're taking part in coherency
> > > + * or not.  This is necessary for the hotplug code to work reliably.
> > > + */
> > > +static void write_pen_release(int val)
> > > +{
> > > +	pen_release = val;
> > > +	/*
> > > +	 * Ensure write to pen_release is visible to the other cores,
> > > +	 * here - primary core
> > > +	 */
> > > +	smp_wmb();
> > > +	sync_cache_w(&pen_release);
> > > +}
> > > +
> > >  /*
> > >   * Enable the Cortex A9 Snoop Control Unit
> > >   *
> > > @@ -75,6 +100,51 @@ static int __init scu_a9_enable(void)
> > >  	return 0;
> > >  }
> > >  
> > > +static int nsp_write_lut(void (*secondary_startup) (void))
> > > +{
> > > +	void __iomem *sku_rom_lut;
> > > +	phys_addr_t secondary_startup_phy;
> > > +
> > > +	if (!secondary_boot) {
> > > +		pr_warn("required secondary boot register not specified\n");
> > > +		return -EINVAL;
> > > +	}
> > > +
> > > +	sku_rom_lut = ioremap_nocache((phys_addr_t)secondary_boot,
> > > +						sizeof(secondary_boot));
> > > +	if (!sku_rom_lut) {
> > > +		pr_warn("unable to ioremap SKU-ROM LUT register\n");
> > > +		return -ENOMEM;
> > > +	}
> > > +
> > > +	secondary_startup_phy = virt_to_phys(secondary_startup);
> > > +	BUG_ON(secondary_startup_phy > (phys_addr_t)U32_MAX);
> > > +
> > > +	writel_relaxed(secondary_startup_phy, sku_rom_lut);
> > > +	/*
> > > +	 * Ensure the write is visible to the secondary core.
> > > +	 */
> > > +	smp_wmb();
> > > +
> > > +	iounmap(sku_rom_lut);
> > > +
> > > +	return 0;
> > > +}
> > > +
> > > +static void nsp_secondary_init(unsigned int cpu)
> > > +{
> > > +	/*
> > > +	 * Let the primary cpu know we are out of holding pen.
> > > +	 */
> > > +	write_pen_release(-1);
> > > +
> > > +	/*
> > > +	 * Synchronise with the boot thread.
> > > +	 */
> > > +	spin_lock(&boot_lock);
> > > +	spin_unlock(&boot_lock);
> > > +}
> > > +
> > >  static void __init bcm_smp_prepare_cpus(unsigned int max_cpus)
> > >  {
> > >  	static cpumask_t only_cpu_0 = { CPU_BITS_CPU0 };
> > > @@ -95,11 +165,11 @@ static void __init bcm_smp_prepare_cpus(unsigned int max_cpus)
> > >  	/*
> > >  	 * Our secondary enable method requires a "secondary-boot-reg"
> > >  	 * property to specify a register address used to request the
> > > -	 * ROM code boot a secondary code.  If we have any trouble
> > > +	 * ROM code boot a secondary core.  If we have any trouble
> > >  	 * getting this we fall back to uniprocessor mode.
> > >  	 */
> > >  	if (of_property_read_u32(node, OF_SECONDARY_BOOT, &secondary_boot)) {
> > > -		pr_err("%s: missing/invalid " OF_SECONDARY_BOOT " property\n",
> > > +		pr_warn("%s: missing/invalid " OF_SECONDARY_BOOT " property\n",
> > >  			node->name);
> > >  		ret = -ENOENT;		/* Arrange to disable SMP */
> > >  		goto out;
> > > @@ -115,7 +185,6 @@ out:
> > >  	of_node_put(node);
> > >  	if (ret) {
> > >  		/* Update the CPU present map to reflect uniprocessor mode */
> > > -		BUG_ON(ret != -ENOENT);
> > >  		pr_warn("disabling SMP\n");
> > >  		init_cpu_present(&only_cpu_0);
> > >  	}
> > > @@ -139,7 +208,7 @@ out:
> > >   * - Wait for the secondary boot register to be re-written, which
> > >   *   indicates the secondary core has started.
> > >   */
> > > -static int bcm_boot_secondary(unsigned int cpu, struct task_struct *idle)
> > > +static int kona_boot_secondary(unsigned int cpu, struct task_struct *idle)
> > >  {
> > >  	void __iomem *boot_reg;
> > >  	phys_addr_t boot_func;
> > > @@ -162,7 +231,7 @@ static int bcm_boot_secondary(unsigned int cpu, struct task_struct *idle)
> > >  	boot_reg = ioremap_nocache((phys_addr_t)secondary_boot, sizeof(u32));
> > >  	if (!boot_reg) {
> > >  		pr_err("unable to map boot register for cpu %u\n", cpu_id);
> > > -		return -ENOSYS;
> > > +		return -ENOMEM;
> > >  	}
> > >  
> > >  	/*
> > > @@ -191,12 +260,67 @@ static int bcm_boot_secondary(unsigned int cpu, struct task_struct *idle)
> > >  
> > >  	pr_err("timeout waiting for cpu %u to start\n", cpu_id);
> > >  
> > > -	return -ENOSYS;
> > > +	return -ENXIO;
> > > +}
> > > +
> > > +static int nsp_boot_secondary(unsigned int cpu, struct task_struct *idle)
> > > +{
> > > +	unsigned long timeout;
> > > +	int ret;
> > > +
> > > +	/*
> > > +	 * After wake up, secondary core branches to the startup
> > > +	 * address programmed at SKU ROM LUT location.
> > > +	 */
> > > +	ret = nsp_write_lut(nsp_secondary_startup);
> > > +	if (ret) {
> > > +		pr_err("unable to write startup addr to SKU ROM LUT\n");
> > > +		goto out;
> > > +	}
> > > +
> > > +	/*
> > > +	 * The secondary processor is waiting to be released from
> > > +	 * the holding pen - release it, then wait for it to flag
> > > +	 * that it has been released by resetting pen_release.
> > > +	 */
> > > +	spin_lock(&boot_lock);
> > > +
> > > +	write_pen_release(cpu_logical_map(cpu));
> > > +	/*
> > > +	 * Send an Event to wake up the secondary core which is in
> > > +	 * WFE state. Updated pen_release should also be visible to
> > > +	 * the secondary core.
> > > +	 */
> > > +	dsb_sev();
> > > +
> > > +	timeout = jiffies + (1 * HZ);
> > > +	while (time_before(jiffies, timeout)) {
> > > +		/* Make sure loads on other CPU is visible */
> > > +		smp_rmb();
> > > +		if (pen_release == -1)
> > > +			break;
> > > +
> > > +		udelay(10);
> > > +	}
> > > +
> > > +	spin_unlock(&boot_lock);
> > 
> > Why is this boot_lock needed? As far as I understand it asserts that
> > both CPUs leave nsp_boot_secondary() and nsp_secondary_init() at the
> > same time.
> > 
> > > +
> > > +	ret = pen_release != -1 ? -ENXIO : 0;
> > > +
> > > +out:
> > > +	return ret;
> > >  }
> > >  
> > >  static struct smp_operations bcm_smp_ops __initdata = {
> > >  	.smp_prepare_cpus	= bcm_smp_prepare_cpus,
> > > -	.smp_boot_secondary	= bcm_boot_secondary,
> > > +	.smp_boot_secondary	= kona_boot_secondary,
> > >  };
> > >  CPU_METHOD_OF_DECLARE(bcm_smp_bcm281xx, "brcm,bcm11351-cpu-method",
> > >  			&bcm_smp_ops);
> > > +
> > > +struct smp_operations nsp_smp_ops __initdata = {
> > > +	.smp_prepare_cpus	= bcm_smp_prepare_cpus,
> > > +	.smp_secondary_init	= nsp_secondary_init,
> > > +	.smp_boot_secondary	= nsp_boot_secondary,
> > > +};
> > > +CPU_METHOD_OF_DECLARE(bcm_smp_nsp, "brcm,bcm-nsp-smp", &nsp_smp_ops);
> > > 
> > 

[RFC] ARM: BCM: Add SMP support for Broadcom 4708

[Jon Mason]

ARM: BCM: Add SMP support for Broadcom 4708

Add SMP support for Broadcom's 4708 SoCs.

Signed-off-by: Jon Mason <jonmason@broadcom.com>
---
 arch/arm/boot/dts/bcm4708.dtsi | 2 ++
 arch/arm/boot/dts/bcm94708.dts | 2 +-
 arch/arm/mach-bcm/Kconfig      | 1 +
 arch/arm/mach-bcm/Makefile     | 3 +++
 4 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/arch/arm/boot/dts/bcm4708.dtsi b/arch/arm/boot/dts/bcm4708.dtsi
index 31141e8..22a41df 100644
--- a/arch/arm/boot/dts/bcm4708.dtsi
+++ b/arch/arm/boot/dts/bcm4708.dtsi
@@ -15,6 +15,8 @@
 	cpus {
 		#address-cells = <1>;
 		#size-cells = <0>;
+		enable-method = "brcm,bcm-nsp-smp";
+		secondary-boot-reg = <0xffff0400>;
 
 		cpu@0 {
 			device_type = "cpu";
diff --git a/arch/arm/boot/dts/bcm94708.dts b/arch/arm/boot/dts/bcm94708.dts
index f60bb1d..49682d6 100644
--- a/arch/arm/boot/dts/bcm94708.dts
+++ b/arch/arm/boot/dts/bcm94708.dts
@@ -32,7 +32,7 @@
 
 /dts-v1/;
 
-#include "bcm5301x.dtsi"
+#include "bcm4708.dtsi"
 
 / {
 	model = "NorthStar SVK (BCM94708)";
diff --git a/arch/arm/mach-bcm/Kconfig b/arch/arm/mach-bcm/Kconfig
index 2e9dbb5..4fc8fa3 100644
--- a/arch/arm/mach-bcm/Kconfig
+++ b/arch/arm/mach-bcm/Kconfig
@@ -54,6 +54,7 @@ config ARCH_BCM_NSP
 config ARCH_BCM_5301X
 	bool "Broadcom BCM470X / BCM5301X ARM SoC" if ARCH_MULTI_V7
 	select ARCH_BCM_IPROC
+	select HAVE_SMP
 	help
 	  Support for Broadcom BCM470X and BCM5301X SoCs with ARM CPU cores.
 
diff --git a/arch/arm/mach-bcm/Makefile b/arch/arm/mach-bcm/Makefile
index 36a4ca30..3ca1cf1 100644
--- a/arch/arm/mach-bcm/Makefile
+++ b/arch/arm/mach-bcm/Makefile
@@ -43,6 +43,9 @@ obj-$(CONFIG_ARCH_BCM2835)	+= board_bcm2835.o
 
 # BCM5301X
 obj-$(CONFIG_ARCH_BCM_5301X)	+= bcm_5301x.o
+ifeq ($(CONFIG_ARCH_BCM_5301X),y)
+obj-$(CONFIG_SMP)		+= headsmp.o platsmp.o
+endif
 
 # BCM63XXx
 ifeq ($(CONFIG_ARCH_BCM_63XX),y)
-- 
1.9.1

Re: [RFC] ARM: BCM: Add SMP support for Broadcom 4708

[Scott Branden]

Looks good.

On 15-10-15 11:14 AM, Jon Mason wrote:
> ARM: BCM: Add SMP support for Broadcom 4708
>
> Add SMP support for Broadcom's 4708 SoCs.
>
> Signed-off-by: Jon Mason <jonmason-dY08KVG/lbpWk0Htik3J/w@public.gmane.org>
Acked-by: Scott Branden <sbranden-dY08KVG/lbpWk0Htik3J/w@public.gmane.org>
> ---
>   arch/arm/boot/dts/bcm4708.dtsi | 2 ++
>   arch/arm/boot/dts/bcm94708.dts | 2 +-
>   arch/arm/mach-bcm/Kconfig      | 1 +
>   arch/arm/mach-bcm/Makefile     | 3 +++
>   4 files changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm/boot/dts/bcm4708.dtsi b/arch/arm/boot/dts/bcm4708.dtsi
> index 31141e8..22a41df 100644
> --- a/arch/arm/boot/dts/bcm4708.dtsi
> +++ b/arch/arm/boot/dts/bcm4708.dtsi
> @@ -15,6 +15,8 @@
>   	cpus {
>   		#address-cells = <1>;
>   		#size-cells = <0>;
> +		enable-method = "brcm,bcm-nsp-smp";
> +		secondary-boot-reg = <0xffff0400>;
>
>   		cpu@0 {
>   			device_type = "cpu";
> diff --git a/arch/arm/boot/dts/bcm94708.dts b/arch/arm/boot/dts/bcm94708.dts
> index f60bb1d..49682d6 100644
> --- a/arch/arm/boot/dts/bcm94708.dts
> +++ b/arch/arm/boot/dts/bcm94708.dts
> @@ -32,7 +32,7 @@
>
>   /dts-v1/;
>
> -#include "bcm5301x.dtsi"
> +#include "bcm4708.dtsi"
>
>   / {
>   	model = "NorthStar SVK (BCM94708)";
> diff --git a/arch/arm/mach-bcm/Kconfig b/arch/arm/mach-bcm/Kconfig
> index 2e9dbb5..4fc8fa3 100644
> --- a/arch/arm/mach-bcm/Kconfig
> +++ b/arch/arm/mach-bcm/Kconfig
> @@ -54,6 +54,7 @@ config ARCH_BCM_NSP
>   config ARCH_BCM_5301X
>   	bool "Broadcom BCM470X / BCM5301X ARM SoC" if ARCH_MULTI_V7
>   	select ARCH_BCM_IPROC
> +	select HAVE_SMP
>   	help
>   	  Support for Broadcom BCM470X and BCM5301X SoCs with ARM CPU cores.
>
> diff --git a/arch/arm/mach-bcm/Makefile b/arch/arm/mach-bcm/Makefile
> index 36a4ca30..3ca1cf1 100644
> --- a/arch/arm/mach-bcm/Makefile
> +++ b/arch/arm/mach-bcm/Makefile
> @@ -43,6 +43,9 @@ obj-$(CONFIG_ARCH_BCM2835)	+= board_bcm2835.o
>
>   # BCM5301X
>   obj-$(CONFIG_ARCH_BCM_5301X)	+= bcm_5301x.o
> +ifeq ($(CONFIG_ARCH_BCM_5301X),y)
> +obj-$(CONFIG_SMP)		+= headsmp.o platsmp.o
> +endif
>
>   # BCM63XXx
>   ifeq ($(CONFIG_ARCH_BCM_63XX),y)
>

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] ARM: BCM: Add SMP support for Broadcom 4708

[Hauke Mehrtens]

On 10/15/2015 08:14 PM, Jon Mason wrote:
> ARM: BCM: Add SMP support for Broadcom 4708
> 
> Add SMP support for Broadcom's 4708 SoCs.

I tested this and it works on my device, it is also in OpenWrt now:
https://dev.openwrt.org/browser/trunk/target/linux/bcm53xx/patches-4.1/131-ARM-BCM-Add-SMP-support-for-Broadcom-4708.patch?rev=47247

> 
> Signed-off-by: Jon Mason <jonmason-dY08KVG/lbpWk0Htik3J/w@public.gmane.org>
> ---
>  arch/arm/boot/dts/bcm4708.dtsi | 2 ++
>  arch/arm/boot/dts/bcm94708.dts | 2 +-
>  arch/arm/mach-bcm/Kconfig      | 1 +
>  arch/arm/mach-bcm/Makefile     | 3 +++
>  4 files changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/arm/boot/dts/bcm4708.dtsi b/arch/arm/boot/dts/bcm4708.dtsi
> index 31141e8..22a41df 100644
> --- a/arch/arm/boot/dts/bcm4708.dtsi
> +++ b/arch/arm/boot/dts/bcm4708.dtsi
> @@ -15,6 +15,8 @@
>  	cpus {
>  		#address-cells = <1>;
>  		#size-cells = <0>;
> +		enable-method = "brcm,bcm-nsp-smp";
> +		secondary-boot-reg = <0xffff0400>;
>  
>  		cpu@0 {
>  			device_type = "cpu";
> diff --git a/arch/arm/boot/dts/bcm94708.dts b/arch/arm/boot/dts/bcm94708.dts
> index f60bb1d..49682d6 100644
> --- a/arch/arm/boot/dts/bcm94708.dts
> +++ b/arch/arm/boot/dts/bcm94708.dts
> @@ -32,7 +32,7 @@
>  
>  /dts-v1/;
>  
> -#include "bcm5301x.dtsi"
> +#include "bcm4708.dtsi"

This is already changed in the patch adding this file.

>  
>  / {
>  	model = "NorthStar SVK (BCM94708)";
> diff --git a/arch/arm/mach-bcm/Kconfig b/arch/arm/mach-bcm/Kconfig
> index 2e9dbb5..4fc8fa3 100644
> --- a/arch/arm/mach-bcm/Kconfig
> +++ b/arch/arm/mach-bcm/Kconfig
> @@ -54,6 +54,7 @@ config ARCH_BCM_NSP
>  config ARCH_BCM_5301X
>  	bool "Broadcom BCM470X / BCM5301X ARM SoC" if ARCH_MULTI_V7
>  	select ARCH_BCM_IPROC
> +	select HAVE_SMP

probably this is also needed:
select ARM_ERRATA_764369 if SMP
The CPU should be affected by this issue.

>  	help
>  	  Support for Broadcom BCM470X and BCM5301X SoCs with ARM CPU cores.
>  
> diff --git a/arch/arm/mach-bcm/Makefile b/arch/arm/mach-bcm/Makefile
> index 36a4ca30..3ca1cf1 100644
> --- a/arch/arm/mach-bcm/Makefile
> +++ b/arch/arm/mach-bcm/Makefile
> @@ -43,6 +43,9 @@ obj-$(CONFIG_ARCH_BCM2835)	+= board_bcm2835.o
>  
>  # BCM5301X
>  obj-$(CONFIG_ARCH_BCM_5301X)	+= bcm_5301x.o
> +ifeq ($(CONFIG_ARCH_BCM_5301X),y)
> +obj-$(CONFIG_SMP)		+= headsmp.o platsmp.o
> +endif
>  
>  # BCM63XXx
>  ifeq ($(CONFIG_ARCH_BCM_63XX),y)
> 

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 3/3] ARM: BCM: Add SMP support for Broadcom NSP

[Kapil Hali]

On 10/15/2015 3:56 AM, Hauke Mehrtens wrote:
> On 10/14/2015 07:47 PM, Kapil Hali wrote:
>> Add SMP support for Broadcom's Northstar Plus SoC,
>> cpu enable method and pen_release procedures. This
>> changes also consolidates iProc family's - BCM NSP
>> and BCM Kona, SMP handling in a common file.
> 
> This will probably also work on normal Northstar CPUs without changes.
> 
I think, it should work for most of the variants of Northstar family, 
except for those which have a BOOTROM bug.

>> Northstar Plus SoC is based on ARM Cortex-A9
>> revision r3p0 which requires configuration for ARM
>> Errata 764369 for SMP. This change adds the needed
>> configuration option.
>>
>> Signed-off-by: Kapil Hali <kapilh-dY08KVG/lbpWk0Htik3J/w@public.gmane.org>
>> ---
>>  arch/arm/mach-bcm/Kconfig                   |   2 +
>>  arch/arm/mach-bcm/Makefile                  |   8 +-
>>  arch/arm/mach-bcm/bcm_nsp.h                 |  19 ++++
>>  arch/arm/mach-bcm/headsmp.S                 |  37 ++++++++
>>  arch/arm/mach-bcm/{kona_smp.c => platsmp.c} | 142 ++++++++++++++++++++++++++--
>>  5 files changed, 197 insertions(+), 11 deletions(-)
>>  create mode 100644 arch/arm/mach-bcm/bcm_nsp.h
>>  create mode 100644 arch/arm/mach-bcm/headsmp.S
>>  rename arch/arm/mach-bcm/{kona_smp.c => platsmp.c} (63%)
>>
>> diff --git a/arch/arm/mach-bcm/Kconfig b/arch/arm/mach-bcm/Kconfig
>> index 1679fa4..2e9dbb5 100644
>> --- a/arch/arm/mach-bcm/Kconfig
>> +++ b/arch/arm/mach-bcm/Kconfig
>> @@ -40,6 +40,8 @@ config ARCH_BCM_NSP
>>  	select ARCH_BCM_IPROC
>>  	select ARM_ERRATA_754322
>>  	select ARM_ERRATA_775420
>> +	select ARM_ERRATA_764369 if SMP
>> +	select HAVE_SMP
>>  	help
>>  	  Support for Broadcom Northstar Plus SoC.
>>  	  Broadcom Northstar Plus family of SoCs are used for switching control
>> diff --git a/arch/arm/mach-bcm/Makefile b/arch/arm/mach-bcm/Makefile
>> index 892261f..36a4ca30 100644
>> --- a/arch/arm/mach-bcm/Makefile
>> +++ b/arch/arm/mach-bcm/Makefile
>> @@ -14,7 +14,11 @@
>>  obj-$(CONFIG_ARCH_BCM_CYGNUS) +=  bcm_cygnus.o
>>  
>>  # Northstar Plus
>> -obj-$(CONFIG_ARCH_BCM_NSP) += bcm_nsp.o
>> +obj-$(CONFIG_ARCH_BCM_NSP)	+= bcm_nsp.o
>> +
>> +ifeq ($(CONFIG_ARCH_BCM_NSP),y)
>> +obj-$(CONFIG_SMP)		+= headsmp.o platsmp.o
>> +endif
>>  
>>  # BCM281XX
>>  obj-$(CONFIG_ARCH_BCM_281XX)	+= board_bcm281xx.o
>> @@ -23,7 +27,7 @@ obj-$(CONFIG_ARCH_BCM_281XX)	+= board_bcm281xx.o
>>  obj-$(CONFIG_ARCH_BCM_21664)	+= board_bcm21664.o
>>  
>>  # BCM281XX and BCM21664 SMP support
>> -obj-$(CONFIG_ARCH_BCM_MOBILE_SMP) += kona_smp.o
>> +obj-$(CONFIG_ARCH_BCM_MOBILE_SMP) += platsmp.o
>>  
>>  # BCM281XX and BCM21664 L2 cache control
>>  obj-$(CONFIG_ARCH_BCM_MOBILE_L2_CACHE) += kona_l2_cache.o
>> diff --git a/arch/arm/mach-bcm/bcm_nsp.h b/arch/arm/mach-bcm/bcm_nsp.h
>> new file mode 100644
>> index 0000000..58e1e80
>> --- /dev/null
>> +++ b/arch/arm/mach-bcm/bcm_nsp.h
>> @@ -0,0 +1,19 @@
>> +/*
>> + * Copyright (C) 2015 Broadcom Corporation
>> + *
>> + * This program is free software; you can redistribute it and/or
>> + * modify it under the terms of the GNU General Public License as
>> + * published by the Free Software Foundation version 2.
>> + *
>> + * This program is distributed "as is" WITHOUT ANY WARRANTY of any
>> + * kind, whether express or implied; without even the implied warranty
>> + * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU General Public License for more details.
>> + */
>> +
>> +#ifndef __BCM_NSP_H
>> +#define __BCM_NSP_H
>> +
>> +extern void nsp_secondary_startup(void);
>> +
>> +#endif /* __BCM_NSP_H */
>> diff --git a/arch/arm/mach-bcm/headsmp.S b/arch/arm/mach-bcm/headsmp.S
>> new file mode 100644
>> index 0000000..0da13b2
>> --- /dev/null
>> +++ b/arch/arm/mach-bcm/headsmp.S
>> @@ -0,0 +1,37 @@
>> +/*
>> + * Copyright (C) 2015 Broadcom Corporation
>> + *
>> + * This program is free software; you can redistribute it and/or
>> + * modify it under the terms of the GNU General Public License as
>> + * published by the Free Software Foundation version 2.
>> + *
>> + * This program is distributed "as is" WITHOUT ANY WARRANTY of any
>> + * kind, whether express or implied; without even the implied warranty
>> + * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU General Public License for more details.
>> + */
>> +
>> +#include <linux/linkage.h>
>> +
>> +/*
>> + * iProc specific entry point for secondary CPUs.  This provides
>> + * a "holding pen" into which all secondary cores are held until
>> + * we are ready for them to initialise.
>> + */
>> +ENTRY(nsp_secondary_startup)
>> +	mrc     p15, 0, r0, c0, c0, 5
>> +	and     r0, r0, #15
>> +	adr     r4, 1f
>> +	ldmia   r4, {r5, r6}
>> +	sub     r4, r4, r5
>> +	add     r6, r6, r4
>> +pen:	ldr     r7, [r6]
>> +	cmp     r7, r0
>> +	bne     pen
>> +
>> +	b    secondary_startup
>> +
>> +1:	.long   .
>> +	.long   pen_release
>> +
>> +ENDPROC(nsp_secondary_startup)
>> diff --git a/arch/arm/mach-bcm/kona_smp.c b/arch/arm/mach-bcm/platsmp.c
>> similarity index 63%
>> rename from arch/arm/mach-bcm/kona_smp.c
>> rename to arch/arm/mach-bcm/platsmp.c
>> index 66a0465..619030e 100644
>> --- a/arch/arm/mach-bcm/kona_smp.c
>> +++ b/arch/arm/mach-bcm/platsmp.c
>> @@ -1,5 +1,5 @@
>>  /*
>> - * Copyright (C) 2014 Broadcom Corporation
>> + * Copyright (C) 2014-2015 Broadcom Corporation
>>   * Copyright 2014 Linaro Limited
>>   *
>>   * This program is free software; you can redistribute it and/or
>> @@ -12,16 +12,23 @@
>>   * GNU General Public License for more details.
>>   */
>>  
>> -#include <linux/init.h>
>> +#include <linux/cpumask.h>
>> +#include <linux/delay.h>
>>  #include <linux/errno.h>
>> +#include <linux/init.h>
>>  #include <linux/io.h>
>> +#include <linux/jiffies.h>
>>  #include <linux/of.h>
>>  #include <linux/sched.h>
>> +#include <linux/smp.h>
>>  
>> +#include <asm/cacheflush.h>
>>  #include <asm/smp.h>
>>  #include <asm/smp_plat.h>
>>  #include <asm/smp_scu.h>
>>  
>> +#include "bcm_nsp.h"
>> +
>>  /* Size of mapped Cortex A9 SCU address space */
>>  #define CORTEX_A9_SCU_SIZE	0x58
>>  
>> @@ -34,6 +41,24 @@
>>  /* I/O address of register used to coordinate secondary core startup */
>>  static u32	secondary_boot;
>>  
>> +static DEFINE_SPINLOCK(boot_lock);
>> +
>> +/*
>> + * Write pen_release in a way that is guaranteed to be visible to all
>> + * observers, irrespective of whether they're taking part in coherency
>> + * or not.  This is necessary for the hotplug code to work reliably.
>> + */
>> +static void write_pen_release(int val)
>> +{
>> +	pen_release = val;
>> +	/*
>> +	 * Ensure write to pen_release is visible to the other cores,
>> +	 * here - primary core
>> +	 */
>> +	smp_wmb();
>> +	sync_cache_w(&pen_release);
>> +}
>> +
>>  /*
>>   * Enable the Cortex A9 Snoop Control Unit
>>   *
>> @@ -75,6 +100,51 @@ static int __init scu_a9_enable(void)
>>  	return 0;
>>  }
>>  
>> +static int nsp_write_lut(void (*secondary_startup) (void))
>> +{
>> +	void __iomem *sku_rom_lut;
>> +	phys_addr_t secondary_startup_phy;
>> +
>> +	if (!secondary_boot) {
>> +		pr_warn("required secondary boot register not specified\n");
>> +		return -EINVAL;
>> +	}
>> +
>> +	sku_rom_lut = ioremap_nocache((phys_addr_t)secondary_boot,
>> +						sizeof(secondary_boot));
>> +	if (!sku_rom_lut) {
>> +		pr_warn("unable to ioremap SKU-ROM LUT register\n");
>> +		return -ENOMEM;
>> +	}
>> +
>> +	secondary_startup_phy = virt_to_phys(secondary_startup);
>> +	BUG_ON(secondary_startup_phy > (phys_addr_t)U32_MAX);
>> +
>> +	writel_relaxed(secondary_startup_phy, sku_rom_lut);
>> +	/*
>> +	 * Ensure the write is visible to the secondary core.
>> +	 */
>> +	smp_wmb();
>> +
>> +	iounmap(sku_rom_lut);
>> +
>> +	return 0;
>> +}
>> +
>> +static void nsp_secondary_init(unsigned int cpu)
>> +{
>> +	/*
>> +	 * Let the primary cpu know we are out of holding pen.
>> +	 */
>> +	write_pen_release(-1);
>> +
>> +	/*
>> +	 * Synchronise with the boot thread.
>> +	 */
>> +	spin_lock(&boot_lock);
>> +	spin_unlock(&boot_lock);
>> +}
>> +
>>  static void __init bcm_smp_prepare_cpus(unsigned int max_cpus)
>>  {
>>  	static cpumask_t only_cpu_0 = { CPU_BITS_CPU0 };
>> @@ -95,11 +165,11 @@ static void __init bcm_smp_prepare_cpus(unsigned int max_cpus)
>>  	/*
>>  	 * Our secondary enable method requires a "secondary-boot-reg"
>>  	 * property to specify a register address used to request the
>> -	 * ROM code boot a secondary code.  If we have any trouble
>> +	 * ROM code boot a secondary core.  If we have any trouble
>>  	 * getting this we fall back to uniprocessor mode.
>>  	 */
>>  	if (of_property_read_u32(node, OF_SECONDARY_BOOT, &secondary_boot)) {
>> -		pr_err("%s: missing/invalid " OF_SECONDARY_BOOT " property\n",
>> +		pr_warn("%s: missing/invalid " OF_SECONDARY_BOOT " property\n",
>>  			node->name);
>>  		ret = -ENOENT;		/* Arrange to disable SMP */
>>  		goto out;
>> @@ -115,7 +185,6 @@ out:
>>  	of_node_put(node);
>>  	if (ret) {
>>  		/* Update the CPU present map to reflect uniprocessor mode */
>> -		BUG_ON(ret != -ENOENT);
>>  		pr_warn("disabling SMP\n");
>>  		init_cpu_present(&only_cpu_0);
>>  	}
>> @@ -139,7 +208,7 @@ out:
>>   * - Wait for the secondary boot register to be re-written, which
>>   *   indicates the secondary core has started.
>>   */
>> -static int bcm_boot_secondary(unsigned int cpu, struct task_struct *idle)
>> +static int kona_boot_secondary(unsigned int cpu, struct task_struct *idle)
>>  {
>>  	void __iomem *boot_reg;
>>  	phys_addr_t boot_func;
>> @@ -162,7 +231,7 @@ static int bcm_boot_secondary(unsigned int cpu, struct task_struct *idle)
>>  	boot_reg = ioremap_nocache((phys_addr_t)secondary_boot, sizeof(u32));
>>  	if (!boot_reg) {
>>  		pr_err("unable to map boot register for cpu %u\n", cpu_id);
>> -		return -ENOSYS;
>> +		return -ENOMEM;
>>  	}
>>  
>>  	/*
>> @@ -191,12 +260,67 @@ static int bcm_boot_secondary(unsigned int cpu, struct task_struct *idle)
>>  
>>  	pr_err("timeout waiting for cpu %u to start\n", cpu_id);
>>  
>> -	return -ENOSYS;
>> +	return -ENXIO;
>> +}
>> +
>> +static int nsp_boot_secondary(unsigned int cpu, struct task_struct *idle)
>> +{
>> +	unsigned long timeout;
>> +	int ret;
>> +
>> +	/*
>> +	 * After wake up, secondary core branches to the startup
>> +	 * address programmed at SKU ROM LUT location.
>> +	 */
>> +	ret = nsp_write_lut(nsp_secondary_startup);
>> +	if (ret) {
>> +		pr_err("unable to write startup addr to SKU ROM LUT\n");
>> +		goto out;
>> +	}
>> +
>> +	/*
>> +	 * The secondary processor is waiting to be released from
>> +	 * the holding pen - release it, then wait for it to flag
>> +	 * that it has been released by resetting pen_release.
>> +	 */
>> +	spin_lock(&boot_lock);
>> +
>> +	write_pen_release(cpu_logical_map(cpu));
>> +	/*
>> +	 * Send an Event to wake up the secondary core which is in
>> +	 * WFE state. Updated pen_release should also be visible to
>> +	 * the secondary core.
>> +	 */
>> +	dsb_sev();
>> +
>> +	timeout = jiffies + (1 * HZ);
>> +	while (time_before(jiffies, timeout)) {
>> +		/* Make sure loads on other CPU is visible */
>> +		smp_rmb();
>> +		if (pen_release == -1)
>> +			break;
>> +
>> +		udelay(10);
>> +	}
>> +
>> +	spin_unlock(&boot_lock);
> 
> Why is this boot_lock needed? As far as I understand it asserts that
> both CPUs leave nsp_boot_secondary() and nsp_secondary_init() at the
> same time.
> 
It makes sure secondary cpu doesn't run away before primary cpu 
recognizes the presence of secondary cpu, it is a way of 
synchronization between primary and secondary cpu. This is the
method used across many SoCs.
>> +
>> +	ret = pen_release != -1 ? -ENXIO : 0;
>> +
>> +out:
>> +	return ret;
>>  }
>>  
>>  static struct smp_operations bcm_smp_ops __initdata = {
>>  	.smp_prepare_cpus	= bcm_smp_prepare_cpus,
>> -	.smp_boot_secondary	= bcm_boot_secondary,
>> +	.smp_boot_secondary	= kona_boot_secondary,
>>  };
>>  CPU_METHOD_OF_DECLARE(bcm_smp_bcm281xx, "brcm,bcm11351-cpu-method",
>>  			&bcm_smp_ops);
>> +
>> +struct smp_operations nsp_smp_ops __initdata = {
>> +	.smp_prepare_cpus	= bcm_smp_prepare_cpus,
>> +	.smp_secondary_init	= nsp_secondary_init,
>> +	.smp_boot_secondary	= nsp_boot_secondary,
>> +};
>> +CPU_METHOD_OF_DECLARE(bcm_smp_nsp, "brcm,bcm-nsp-smp", &nsp_smp_ops);
>>
> 
Thanks,
Kapil
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 3/3] ARM: BCM: Add SMP support for Broadcom NSP

[Hauke Mehrtens]

On 10/15/2015 06:10 PM, Kapil Hali wrote:
> 
> 
> On 10/15/2015 3:56 AM, Hauke Mehrtens wrote:
>> On 10/14/2015 07:47 PM, Kapil Hali wrote:
>>> Add SMP support for Broadcom's Northstar Plus SoC,
>>> cpu enable method and pen_release procedures. This
>>> changes also consolidates iProc family's - BCM NSP
>>> and BCM Kona, SMP handling in a common file.
>>
>> This will probably also work on normal Northstar CPUs without changes.
>>
> I think, it should work for most of the variants of Northstar family, 
> except for those which have a BOOTROM bug.

Which SoC are affected by this BOOTROM bug?

>>> Northstar Plus SoC is based on ARM Cortex-A9
>>> revision r3p0 which requires configuration for ARM
>>> Errata 764369 for SMP. This change adds the needed
>>> configuration option.
>>>
>>> Signed-off-by: Kapil Hali <kapilh-dY08KVG/lbpWk0Htik3J/w@public.gmane.org>
>>> ---
>>>  arch/arm/mach-bcm/Kconfig                   |   2 +
>>>  arch/arm/mach-bcm/Makefile                  |   8 +-
>>>  arch/arm/mach-bcm/bcm_nsp.h                 |  19 ++++
>>>  arch/arm/mach-bcm/headsmp.S                 |  37 ++++++++
>>>  arch/arm/mach-bcm/{kona_smp.c => platsmp.c} | 142 ++++++++++++++++++++++++++--
>>>  5 files changed, 197 insertions(+), 11 deletions(-)
>>>  create mode 100644 arch/arm/mach-bcm/bcm_nsp.h
>>>  create mode 100644 arch/arm/mach-bcm/headsmp.S
>>>  rename arch/arm/mach-bcm/{kona_smp.c => platsmp.c} (63%)
>>>
>>> diff --git a/arch/arm/mach-bcm/Kconfig b/arch/arm/mach-bcm/Kconfig
>>> index 1679fa4..2e9dbb5 100644
>>> --- a/arch/arm/mach-bcm/Kconfig
>>> +++ b/arch/arm/mach-bcm/Kconfig
>>> @@ -40,6 +40,8 @@ config ARCH_BCM_NSP
>>>  	select ARCH_BCM_IPROC
>>>  	select ARM_ERRATA_754322
>>>  	select ARM_ERRATA_775420
>>> +	select ARM_ERRATA_764369 if SMP
>>> +	select HAVE_SMP
>>>  	help
>>>  	  Support for Broadcom Northstar Plus SoC.
>>>  	  Broadcom Northstar Plus family of SoCs are used for switching control
>>> diff --git a/arch/arm/mach-bcm/Makefile b/arch/arm/mach-bcm/Makefile
>>> index 892261f..36a4ca30 100644
>>> --- a/arch/arm/mach-bcm/Makefile
>>> +++ b/arch/arm/mach-bcm/Makefile
>>> @@ -14,7 +14,11 @@
>>>  obj-$(CONFIG_ARCH_BCM_CYGNUS) +=  bcm_cygnus.o
>>>  
>>>  # Northstar Plus
>>> -obj-$(CONFIG_ARCH_BCM_NSP) += bcm_nsp.o
>>> +obj-$(CONFIG_ARCH_BCM_NSP)	+= bcm_nsp.o
>>> +
>>> +ifeq ($(CONFIG_ARCH_BCM_NSP),y)
>>> +obj-$(CONFIG_SMP)		+= headsmp.o platsmp.o
>>> +endif
>>>  
>>>  # BCM281XX
>>>  obj-$(CONFIG_ARCH_BCM_281XX)	+= board_bcm281xx.o
>>> @@ -23,7 +27,7 @@ obj-$(CONFIG_ARCH_BCM_281XX)	+= board_bcm281xx.o
>>>  obj-$(CONFIG_ARCH_BCM_21664)	+= board_bcm21664.o
>>>  
>>>  # BCM281XX and BCM21664 SMP support
>>> -obj-$(CONFIG_ARCH_BCM_MOBILE_SMP) += kona_smp.o
>>> +obj-$(CONFIG_ARCH_BCM_MOBILE_SMP) += platsmp.o
>>>  
>>>  # BCM281XX and BCM21664 L2 cache control
>>>  obj-$(CONFIG_ARCH_BCM_MOBILE_L2_CACHE) += kona_l2_cache.o
>>> diff --git a/arch/arm/mach-bcm/bcm_nsp.h b/arch/arm/mach-bcm/bcm_nsp.h
>>> new file mode 100644
>>> index 0000000..58e1e80
>>> --- /dev/null
>>> +++ b/arch/arm/mach-bcm/bcm_nsp.h
>>> @@ -0,0 +1,19 @@
>>> +/*
>>> + * Copyright (C) 2015 Broadcom Corporation
>>> + *
>>> + * This program is free software; you can redistribute it and/or
>>> + * modify it under the terms of the GNU General Public License as
>>> + * published by the Free Software Foundation version 2.
>>> + *
>>> + * This program is distributed "as is" WITHOUT ANY WARRANTY of any
>>> + * kind, whether express or implied; without even the implied warranty
>>> + * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>>> + * GNU General Public License for more details.
>>> + */
>>> +
>>> +#ifndef __BCM_NSP_H
>>> +#define __BCM_NSP_H
>>> +
>>> +extern void nsp_secondary_startup(void);
>>> +
>>> +#endif /* __BCM_NSP_H */
>>> diff --git a/arch/arm/mach-bcm/headsmp.S b/arch/arm/mach-bcm/headsmp.S
>>> new file mode 100644
>>> index 0000000..0da13b2
>>> --- /dev/null
>>> +++ b/arch/arm/mach-bcm/headsmp.S
>>> @@ -0,0 +1,37 @@
>>> +/*
>>> + * Copyright (C) 2015 Broadcom Corporation
>>> + *
>>> + * This program is free software; you can redistribute it and/or
>>> + * modify it under the terms of the GNU General Public License as
>>> + * published by the Free Software Foundation version 2.
>>> + *
>>> + * This program is distributed "as is" WITHOUT ANY WARRANTY of any
>>> + * kind, whether express or implied; without even the implied warranty
>>> + * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>>> + * GNU General Public License for more details.
>>> + */
>>> +
>>> +#include <linux/linkage.h>
>>> +
>>> +/*
>>> + * iProc specific entry point for secondary CPUs.  This provides
>>> + * a "holding pen" into which all secondary cores are held until
>>> + * we are ready for them to initialise.
>>> + */
>>> +ENTRY(nsp_secondary_startup)
>>> +	mrc     p15, 0, r0, c0, c0, 5
>>> +	and     r0, r0, #15
>>> +	adr     r4, 1f
>>> +	ldmia   r4, {r5, r6}
>>> +	sub     r4, r4, r5
>>> +	add     r6, r6, r4
>>> +pen:	ldr     r7, [r6]
>>> +	cmp     r7, r0
>>> +	bne     pen
>>> +
>>> +	b    secondary_startup
>>> +
>>> +1:	.long   .
>>> +	.long   pen_release
>>> +
>>> +ENDPROC(nsp_secondary_startup)
>>> diff --git a/arch/arm/mach-bcm/kona_smp.c b/arch/arm/mach-bcm/platsmp.c
>>> similarity index 63%
>>> rename from arch/arm/mach-bcm/kona_smp.c
>>> rename to arch/arm/mach-bcm/platsmp.c
>>> index 66a0465..619030e 100644
>>> --- a/arch/arm/mach-bcm/kona_smp.c
>>> +++ b/arch/arm/mach-bcm/platsmp.c
>>> @@ -1,5 +1,5 @@
>>>  /*
>>> - * Copyright (C) 2014 Broadcom Corporation
>>> + * Copyright (C) 2014-2015 Broadcom Corporation
>>>   * Copyright 2014 Linaro Limited
>>>   *
>>>   * This program is free software; you can redistribute it and/or
>>> @@ -12,16 +12,23 @@
>>>   * GNU General Public License for more details.
>>>   */
>>>  
>>> -#include <linux/init.h>
>>> +#include <linux/cpumask.h>
>>> +#include <linux/delay.h>
>>>  #include <linux/errno.h>
>>> +#include <linux/init.h>
>>>  #include <linux/io.h>
>>> +#include <linux/jiffies.h>
>>>  #include <linux/of.h>
>>>  #include <linux/sched.h>
>>> +#include <linux/smp.h>
>>>  
>>> +#include <asm/cacheflush.h>
>>>  #include <asm/smp.h>
>>>  #include <asm/smp_plat.h>
>>>  #include <asm/smp_scu.h>
>>>  
>>> +#include "bcm_nsp.h"
>>> +
>>>  /* Size of mapped Cortex A9 SCU address space */
>>>  #define CORTEX_A9_SCU_SIZE	0x58
>>>  
>>> @@ -34,6 +41,24 @@
>>>  /* I/O address of register used to coordinate secondary core startup */
>>>  static u32	secondary_boot;
>>>  
>>> +static DEFINE_SPINLOCK(boot_lock);
>>> +
>>> +/*
>>> + * Write pen_release in a way that is guaranteed to be visible to all
>>> + * observers, irrespective of whether they're taking part in coherency
>>> + * or not.  This is necessary for the hotplug code to work reliably.
>>> + */
>>> +static void write_pen_release(int val)
>>> +{
>>> +	pen_release = val;
>>> +	/*
>>> +	 * Ensure write to pen_release is visible to the other cores,
>>> +	 * here - primary core
>>> +	 */
>>> +	smp_wmb();
>>> +	sync_cache_w(&pen_release);
>>> +}
>>> +
>>>  /*
>>>   * Enable the Cortex A9 Snoop Control Unit
>>>   *
>>> @@ -75,6 +100,51 @@ static int __init scu_a9_enable(void)
>>>  	return 0;
>>>  }
>>>  
>>> +static int nsp_write_lut(void (*secondary_startup) (void))
>>> +{
>>> +	void __iomem *sku_rom_lut;
>>> +	phys_addr_t secondary_startup_phy;
>>> +
>>> +	if (!secondary_boot) {
>>> +		pr_warn("required secondary boot register not specified\n");
>>> +		return -EINVAL;
>>> +	}
>>> +
>>> +	sku_rom_lut = ioremap_nocache((phys_addr_t)secondary_boot,
>>> +						sizeof(secondary_boot));
>>> +	if (!sku_rom_lut) {
>>> +		pr_warn("unable to ioremap SKU-ROM LUT register\n");
>>> +		return -ENOMEM;
>>> +	}
>>> +
>>> +	secondary_startup_phy = virt_to_phys(secondary_startup);
>>> +	BUG_ON(secondary_startup_phy > (phys_addr_t)U32_MAX);
>>> +
>>> +	writel_relaxed(secondary_startup_phy, sku_rom_lut);
>>> +	/*
>>> +	 * Ensure the write is visible to the secondary core.
>>> +	 */
>>> +	smp_wmb();
>>> +
>>> +	iounmap(sku_rom_lut);
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static void nsp_secondary_init(unsigned int cpu)
>>> +{
>>> +	/*
>>> +	 * Let the primary cpu know we are out of holding pen.
>>> +	 */
>>> +	write_pen_release(-1);
>>> +
>>> +	/*
>>> +	 * Synchronise with the boot thread.
>>> +	 */
>>> +	spin_lock(&boot_lock);
>>> +	spin_unlock(&boot_lock);
>>> +}
>>> +
>>>  static void __init bcm_smp_prepare_cpus(unsigned int max_cpus)
>>>  {
>>>  	static cpumask_t only_cpu_0 = { CPU_BITS_CPU0 };
>>> @@ -95,11 +165,11 @@ static void __init bcm_smp_prepare_cpus(unsigned int max_cpus)
>>>  	/*
>>>  	 * Our secondary enable method requires a "secondary-boot-reg"
>>>  	 * property to specify a register address used to request the
>>> -	 * ROM code boot a secondary code.  If we have any trouble
>>> +	 * ROM code boot a secondary core.  If we have any trouble
>>>  	 * getting this we fall back to uniprocessor mode.
>>>  	 */
>>>  	if (of_property_read_u32(node, OF_SECONDARY_BOOT, &secondary_boot)) {
>>> -		pr_err("%s: missing/invalid " OF_SECONDARY_BOOT " property\n",
>>> +		pr_warn("%s: missing/invalid " OF_SECONDARY_BOOT " property\n",
>>>  			node->name);
>>>  		ret = -ENOENT;		/* Arrange to disable SMP */
>>>  		goto out;
>>> @@ -115,7 +185,6 @@ out:
>>>  	of_node_put(node);
>>>  	if (ret) {
>>>  		/* Update the CPU present map to reflect uniprocessor mode */
>>> -		BUG_ON(ret != -ENOENT);
>>>  		pr_warn("disabling SMP\n");
>>>  		init_cpu_present(&only_cpu_0);
>>>  	}
>>> @@ -139,7 +208,7 @@ out:
>>>   * - Wait for the secondary boot register to be re-written, which
>>>   *   indicates the secondary core has started.
>>>   */
>>> -static int bcm_boot_secondary(unsigned int cpu, struct task_struct *idle)
>>> +static int kona_boot_secondary(unsigned int cpu, struct task_struct *idle)
>>>  {
>>>  	void __iomem *boot_reg;
>>>  	phys_addr_t boot_func;
>>> @@ -162,7 +231,7 @@ static int bcm_boot_secondary(unsigned int cpu, struct task_struct *idle)
>>>  	boot_reg = ioremap_nocache((phys_addr_t)secondary_boot, sizeof(u32));
>>>  	if (!boot_reg) {
>>>  		pr_err("unable to map boot register for cpu %u\n", cpu_id);
>>> -		return -ENOSYS;
>>> +		return -ENOMEM;
>>>  	}
>>>  
>>>  	/*
>>> @@ -191,12 +260,67 @@ static int bcm_boot_secondary(unsigned int cpu, struct task_struct *idle)
>>>  
>>>  	pr_err("timeout waiting for cpu %u to start\n", cpu_id);
>>>  
>>> -	return -ENOSYS;
>>> +	return -ENXIO;
>>> +}
>>> +
>>> +static int nsp_boot_secondary(unsigned int cpu, struct task_struct *idle)
>>> +{
>>> +	unsigned long timeout;
>>> +	int ret;
>>> +
>>> +	/*
>>> +	 * After wake up, secondary core branches to the startup
>>> +	 * address programmed at SKU ROM LUT location.
>>> +	 */
>>> +	ret = nsp_write_lut(nsp_secondary_startup);
>>> +	if (ret) {
>>> +		pr_err("unable to write startup addr to SKU ROM LUT\n");
>>> +		goto out;
>>> +	}
>>> +
>>> +	/*
>>> +	 * The secondary processor is waiting to be released from
>>> +	 * the holding pen - release it, then wait for it to flag
>>> +	 * that it has been released by resetting pen_release.
>>> +	 */
>>> +	spin_lock(&boot_lock);
>>> +
>>> +	write_pen_release(cpu_logical_map(cpu));
>>> +	/*
>>> +	 * Send an Event to wake up the secondary core which is in
>>> +	 * WFE state. Updated pen_release should also be visible to
>>> +	 * the secondary core.
>>> +	 */
>>> +	dsb_sev();
>>> +
>>> +	timeout = jiffies + (1 * HZ);
>>> +	while (time_before(jiffies, timeout)) {
>>> +		/* Make sure loads on other CPU is visible */
>>> +		smp_rmb();
>>> +		if (pen_release == -1)
>>> +			break;
>>> +
>>> +		udelay(10);
>>> +	}
>>> +
>>> +	spin_unlock(&boot_lock);
>>
>> Why is this boot_lock needed? As far as I understand it asserts that
>> both CPUs leave nsp_boot_secondary() and nsp_secondary_init() at the
>> same time.
>>
> It makes sure secondary cpu doesn't run away before primary cpu 
> recognizes the presence of secondary cpu, it is a way of 
> synchronization between primary and secondary cpu. This is the
> method used across many SoCs.
>>> +
>>> +	ret = pen_release != -1 ? -ENXIO : 0;
>>> +
>>> +out:
>>> +	return ret;
>>>  }
>>>  
>>>  static struct smp_operations bcm_smp_ops __initdata = {
>>>  	.smp_prepare_cpus	= bcm_smp_prepare_cpus,
>>> -	.smp_boot_secondary	= bcm_boot_secondary,
>>> +	.smp_boot_secondary	= kona_boot_secondary,
>>>  };
>>>  CPU_METHOD_OF_DECLARE(bcm_smp_bcm281xx, "brcm,bcm11351-cpu-method",
>>>  			&bcm_smp_ops);
>>> +
>>> +struct smp_operations nsp_smp_ops __initdata = {
>>> +	.smp_prepare_cpus	= bcm_smp_prepare_cpus,
>>> +	.smp_secondary_init	= nsp_secondary_init,
>>> +	.smp_boot_secondary	= nsp_boot_secondary,
>>> +};
>>> +CPU_METHOD_OF_DECLARE(bcm_smp_nsp, "brcm,bcm-nsp-smp", &nsp_smp_ops);
>>>
>>
> Thanks,
> Kapil
> 

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 3/3] ARM: BCM: Add SMP support for Broadcom NSP

[Jon Mason]

On Thu, Oct 15, 2015 at 11:12:09PM +0200, Hauke Mehrtens wrote:
> On 10/15/2015 06:10 PM, Kapil Hali wrote:
> > 
> > 
> > On 10/15/2015 3:56 AM, Hauke Mehrtens wrote:
> >> On 10/14/2015 07:47 PM, Kapil Hali wrote:
> >>> Add SMP support for Broadcom's Northstar Plus SoC,
> >>> cpu enable method and pen_release procedures. This
> >>> changes also consolidates iProc family's - BCM NSP
> >>> and BCM Kona, SMP handling in a common file.
> >>
> >> This will probably also work on normal Northstar CPUs without changes.
> >>
> > I think, it should work for most of the variants of Northstar family, 
> > except for those which have a BOOTROM bug.
> 
> Which SoC are affected by this BOOTROM bug?

53012 is the one I am seeing.  Not seeing it on 4708 (and assuming it is
not present on 4709).  Internally, we do the ugly bug workaround on
all Northstar SoCs.  The workaround is not acceptable upstream, so I
am not pushing it. :)

Thanks,
Jon

> 
> >>> Northstar Plus SoC is based on ARM Cortex-A9
> >>> revision r3p0 which requires configuration for ARM
> >>> Errata 764369 for SMP. This change adds the needed
> >>> configuration option.
> >>>
> >>> Signed-off-by: Kapil Hali <kapilh-dY08KVG/lbpWk0Htik3J/w@public.gmane.org>
> >>> ---
> >>>  arch/arm/mach-bcm/Kconfig                   |   2 +
> >>>  arch/arm/mach-bcm/Makefile                  |   8 +-
> >>>  arch/arm/mach-bcm/bcm_nsp.h                 |  19 ++++
> >>>  arch/arm/mach-bcm/headsmp.S                 |  37 ++++++++
> >>>  arch/arm/mach-bcm/{kona_smp.c => platsmp.c} | 142 ++++++++++++++++++++++++++--
> >>>  5 files changed, 197 insertions(+), 11 deletions(-)
> >>>  create mode 100644 arch/arm/mach-bcm/bcm_nsp.h
> >>>  create mode 100644 arch/arm/mach-bcm/headsmp.S
> >>>  rename arch/arm/mach-bcm/{kona_smp.c => platsmp.c} (63%)
> >>>
> >>> diff --git a/arch/arm/mach-bcm/Kconfig b/arch/arm/mach-bcm/Kconfig
> >>> index 1679fa4..2e9dbb5 100644
> >>> --- a/arch/arm/mach-bcm/Kconfig
> >>> +++ b/arch/arm/mach-bcm/Kconfig
> >>> @@ -40,6 +40,8 @@ config ARCH_BCM_NSP
> >>>  	select ARCH_BCM_IPROC
> >>>  	select ARM_ERRATA_754322
> >>>  	select ARM_ERRATA_775420
> >>> +	select ARM_ERRATA_764369 if SMP
> >>> +	select HAVE_SMP
> >>>  	help
> >>>  	  Support for Broadcom Northstar Plus SoC.
> >>>  	  Broadcom Northstar Plus family of SoCs are used for switching control
> >>> diff --git a/arch/arm/mach-bcm/Makefile b/arch/arm/mach-bcm/Makefile
> >>> index 892261f..36a4ca30 100644
> >>> --- a/arch/arm/mach-bcm/Makefile
> >>> +++ b/arch/arm/mach-bcm/Makefile
> >>> @@ -14,7 +14,11 @@
> >>>  obj-$(CONFIG_ARCH_BCM_CYGNUS) +=  bcm_cygnus.o
> >>>  
> >>>  # Northstar Plus
> >>> -obj-$(CONFIG_ARCH_BCM_NSP) += bcm_nsp.o
> >>> +obj-$(CONFIG_ARCH_BCM_NSP)	+= bcm_nsp.o
> >>> +
> >>> +ifeq ($(CONFIG_ARCH_BCM_NSP),y)
> >>> +obj-$(CONFIG_SMP)		+= headsmp.o platsmp.o
> >>> +endif
> >>>  
> >>>  # BCM281XX
> >>>  obj-$(CONFIG_ARCH_BCM_281XX)	+= board_bcm281xx.o
> >>> @@ -23,7 +27,7 @@ obj-$(CONFIG_ARCH_BCM_281XX)	+= board_bcm281xx.o
> >>>  obj-$(CONFIG_ARCH_BCM_21664)	+= board_bcm21664.o
> >>>  
> >>>  # BCM281XX and BCM21664 SMP support
> >>> -obj-$(CONFIG_ARCH_BCM_MOBILE_SMP) += kona_smp.o
> >>> +obj-$(CONFIG_ARCH_BCM_MOBILE_SMP) += platsmp.o
> >>>  
> >>>  # BCM281XX and BCM21664 L2 cache control
> >>>  obj-$(CONFIG_ARCH_BCM_MOBILE_L2_CACHE) += kona_l2_cache.o
> >>> diff --git a/arch/arm/mach-bcm/bcm_nsp.h b/arch/arm/mach-bcm/bcm_nsp.h
> >>> new file mode 100644
> >>> index 0000000..58e1e80
> >>> --- /dev/null
> >>> +++ b/arch/arm/mach-bcm/bcm_nsp.h
> >>> @@ -0,0 +1,19 @@
> >>> +/*
> >>> + * Copyright (C) 2015 Broadcom Corporation
> >>> + *
> >>> + * This program is free software; you can redistribute it and/or
> >>> + * modify it under the terms of the GNU General Public License as
> >>> + * published by the Free Software Foundation version 2.
> >>> + *
> >>> + * This program is distributed "as is" WITHOUT ANY WARRANTY of any
> >>> + * kind, whether express or implied; without even the implied warranty
> >>> + * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> >>> + * GNU General Public License for more details.
> >>> + */
> >>> +
> >>> +#ifndef __BCM_NSP_H
> >>> +#define __BCM_NSP_H
> >>> +
> >>> +extern void nsp_secondary_startup(void);
> >>> +
> >>> +#endif /* __BCM_NSP_H */
> >>> diff --git a/arch/arm/mach-bcm/headsmp.S b/arch/arm/mach-bcm/headsmp.S
> >>> new file mode 100644
> >>> index 0000000..0da13b2
> >>> --- /dev/null
> >>> +++ b/arch/arm/mach-bcm/headsmp.S
> >>> @@ -0,0 +1,37 @@
> >>> +/*
> >>> + * Copyright (C) 2015 Broadcom Corporation
> >>> + *
> >>> + * This program is free software; you can redistribute it and/or
> >>> + * modify it under the terms of the GNU General Public License as
> >>> + * published by the Free Software Foundation version 2.
> >>> + *
> >>> + * This program is distributed "as is" WITHOUT ANY WARRANTY of any
> >>> + * kind, whether express or implied; without even the implied warranty
> >>> + * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> >>> + * GNU General Public License for more details.
> >>> + */
> >>> +
> >>> +#include <linux/linkage.h>
> >>> +
> >>> +/*
> >>> + * iProc specific entry point for secondary CPUs.  This provides
> >>> + * a "holding pen" into which all secondary cores are held until
> >>> + * we are ready for them to initialise.
> >>> + */
> >>> +ENTRY(nsp_secondary_startup)
> >>> +	mrc     p15, 0, r0, c0, c0, 5
> >>> +	and     r0, r0, #15
> >>> +	adr     r4, 1f
> >>> +	ldmia   r4, {r5, r6}
> >>> +	sub     r4, r4, r5
> >>> +	add     r6, r6, r4
> >>> +pen:	ldr     r7, [r6]
> >>> +	cmp     r7, r0
> >>> +	bne     pen
> >>> +
> >>> +	b    secondary_startup
> >>> +
> >>> +1:	.long   .
> >>> +	.long   pen_release
> >>> +
> >>> +ENDPROC(nsp_secondary_startup)
> >>> diff --git a/arch/arm/mach-bcm/kona_smp.c b/arch/arm/mach-bcm/platsmp.c
> >>> similarity index 63%
> >>> rename from arch/arm/mach-bcm/kona_smp.c
> >>> rename to arch/arm/mach-bcm/platsmp.c
> >>> index 66a0465..619030e 100644
> >>> --- a/arch/arm/mach-bcm/kona_smp.c
> >>> +++ b/arch/arm/mach-bcm/platsmp.c
> >>> @@ -1,5 +1,5 @@
> >>>  /*
> >>> - * Copyright (C) 2014 Broadcom Corporation
> >>> + * Copyright (C) 2014-2015 Broadcom Corporation
> >>>   * Copyright 2014 Linaro Limited
> >>>   *
> >>>   * This program is free software; you can redistribute it and/or
> >>> @@ -12,16 +12,23 @@
> >>>   * GNU General Public License for more details.
> >>>   */
> >>>  
> >>> -#include <linux/init.h>
> >>> +#include <linux/cpumask.h>
> >>> +#include <linux/delay.h>
> >>>  #include <linux/errno.h>
> >>> +#include <linux/init.h>
> >>>  #include <linux/io.h>
> >>> +#include <linux/jiffies.h>
> >>>  #include <linux/of.h>
> >>>  #include <linux/sched.h>
> >>> +#include <linux/smp.h>
> >>>  
> >>> +#include <asm/cacheflush.h>
> >>>  #include <asm/smp.h>
> >>>  #include <asm/smp_plat.h>
> >>>  #include <asm/smp_scu.h>
> >>>  
> >>> +#include "bcm_nsp.h"
> >>> +
> >>>  /* Size of mapped Cortex A9 SCU address space */
> >>>  #define CORTEX_A9_SCU_SIZE	0x58
> >>>  
> >>> @@ -34,6 +41,24 @@
> >>>  /* I/O address of register used to coordinate secondary core startup */
> >>>  static u32	secondary_boot;
> >>>  
> >>> +static DEFINE_SPINLOCK(boot_lock);
> >>> +
> >>> +/*
> >>> + * Write pen_release in a way that is guaranteed to be visible to all
> >>> + * observers, irrespective of whether they're taking part in coherency
> >>> + * or not.  This is necessary for the hotplug code to work reliably.
> >>> + */
> >>> +static void write_pen_release(int val)
> >>> +{
> >>> +	pen_release = val;
> >>> +	/*
> >>> +	 * Ensure write to pen_release is visible to the other cores,
> >>> +	 * here - primary core
> >>> +	 */
> >>> +	smp_wmb();
> >>> +	sync_cache_w(&pen_release);
> >>> +}
> >>> +
> >>>  /*
> >>>   * Enable the Cortex A9 Snoop Control Unit
> >>>   *
> >>> @@ -75,6 +100,51 @@ static int __init scu_a9_enable(void)
> >>>  	return 0;
> >>>  }
> >>>  
> >>> +static int nsp_write_lut(void (*secondary_startup) (void))
> >>> +{
> >>> +	void __iomem *sku_rom_lut;
> >>> +	phys_addr_t secondary_startup_phy;
> >>> +
> >>> +	if (!secondary_boot) {
> >>> +		pr_warn("required secondary boot register not specified\n");
> >>> +		return -EINVAL;
> >>> +	}
> >>> +
> >>> +	sku_rom_lut = ioremap_nocache((phys_addr_t)secondary_boot,
> >>> +						sizeof(secondary_boot));
> >>> +	if (!sku_rom_lut) {
> >>> +		pr_warn("unable to ioremap SKU-ROM LUT register\n");
> >>> +		return -ENOMEM;
> >>> +	}
> >>> +
> >>> +	secondary_startup_phy = virt_to_phys(secondary_startup);
> >>> +	BUG_ON(secondary_startup_phy > (phys_addr_t)U32_MAX);
> >>> +
> >>> +	writel_relaxed(secondary_startup_phy, sku_rom_lut);
> >>> +	/*
> >>> +	 * Ensure the write is visible to the secondary core.
> >>> +	 */
> >>> +	smp_wmb();
> >>> +
> >>> +	iounmap(sku_rom_lut);
> >>> +
> >>> +	return 0;
> >>> +}
> >>> +
> >>> +static void nsp_secondary_init(unsigned int cpu)
> >>> +{
> >>> +	/*
> >>> +	 * Let the primary cpu know we are out of holding pen.
> >>> +	 */
> >>> +	write_pen_release(-1);
> >>> +
> >>> +	/*
> >>> +	 * Synchronise with the boot thread.
> >>> +	 */
> >>> +	spin_lock(&boot_lock);
> >>> +	spin_unlock(&boot_lock);
> >>> +}
> >>> +
> >>>  static void __init bcm_smp_prepare_cpus(unsigned int max_cpus)
> >>>  {
> >>>  	static cpumask_t only_cpu_0 = { CPU_BITS_CPU0 };
> >>> @@ -95,11 +165,11 @@ static void __init bcm_smp_prepare_cpus(unsigned int max_cpus)
> >>>  	/*
> >>>  	 * Our secondary enable method requires a "secondary-boot-reg"
> >>>  	 * property to specify a register address used to request the
> >>> -	 * ROM code boot a secondary code.  If we have any trouble
> >>> +	 * ROM code boot a secondary core.  If we have any trouble
> >>>  	 * getting this we fall back to uniprocessor mode.
> >>>  	 */
> >>>  	if (of_property_read_u32(node, OF_SECONDARY_BOOT, &secondary_boot)) {
> >>> -		pr_err("%s: missing/invalid " OF_SECONDARY_BOOT " property\n",
> >>> +		pr_warn("%s: missing/invalid " OF_SECONDARY_BOOT " property\n",
> >>>  			node->name);
> >>>  		ret = -ENOENT;		/* Arrange to disable SMP */
> >>>  		goto out;
> >>> @@ -115,7 +185,6 @@ out:
> >>>  	of_node_put(node);
> >>>  	if (ret) {
> >>>  		/* Update the CPU present map to reflect uniprocessor mode */
> >>> -		BUG_ON(ret != -ENOENT);
> >>>  		pr_warn("disabling SMP\n");
> >>>  		init_cpu_present(&only_cpu_0);
> >>>  	}
> >>> @@ -139,7 +208,7 @@ out:
> >>>   * - Wait for the secondary boot register to be re-written, which
> >>>   *   indicates the secondary core has started.
> >>>   */
> >>> -static int bcm_boot_secondary(unsigned int cpu, struct task_struct *idle)
> >>> +static int kona_boot_secondary(unsigned int cpu, struct task_struct *idle)
> >>>  {
> >>>  	void __iomem *boot_reg;
> >>>  	phys_addr_t boot_func;
> >>> @@ -162,7 +231,7 @@ static int bcm_boot_secondary(unsigned int cpu, struct task_struct *idle)
> >>>  	boot_reg = ioremap_nocache((phys_addr_t)secondary_boot, sizeof(u32));
> >>>  	if (!boot_reg) {
> >>>  		pr_err("unable to map boot register for cpu %u\n", cpu_id);
> >>> -		return -ENOSYS;
> >>> +		return -ENOMEM;
> >>>  	}
> >>>  
> >>>  	/*
> >>> @@ -191,12 +260,67 @@ static int bcm_boot_secondary(unsigned int cpu, struct task_struct *idle)
> >>>  
> >>>  	pr_err("timeout waiting for cpu %u to start\n", cpu_id);
> >>>  
> >>> -	return -ENOSYS;
> >>> +	return -ENXIO;
> >>> +}
> >>> +
> >>> +static int nsp_boot_secondary(unsigned int cpu, struct task_struct *idle)
> >>> +{
> >>> +	unsigned long timeout;
> >>> +	int ret;
> >>> +
> >>> +	/*
> >>> +	 * After wake up, secondary core branches to the startup
> >>> +	 * address programmed at SKU ROM LUT location.
> >>> +	 */
> >>> +	ret = nsp_write_lut(nsp_secondary_startup);
> >>> +	if (ret) {
> >>> +		pr_err("unable to write startup addr to SKU ROM LUT\n");
> >>> +		goto out;
> >>> +	}
> >>> +
> >>> +	/*
> >>> +	 * The secondary processor is waiting to be released from
> >>> +	 * the holding pen - release it, then wait for it to flag
> >>> +	 * that it has been released by resetting pen_release.
> >>> +	 */
> >>> +	spin_lock(&boot_lock);
> >>> +
> >>> +	write_pen_release(cpu_logical_map(cpu));
> >>> +	/*
> >>> +	 * Send an Event to wake up the secondary core which is in
> >>> +	 * WFE state. Updated pen_release should also be visible to
> >>> +	 * the secondary core.
> >>> +	 */
> >>> +	dsb_sev();
> >>> +
> >>> +	timeout = jiffies + (1 * HZ);
> >>> +	while (time_before(jiffies, timeout)) {
> >>> +		/* Make sure loads on other CPU is visible */
> >>> +		smp_rmb();
> >>> +		if (pen_release == -1)
> >>> +			break;
> >>> +
> >>> +		udelay(10);
> >>> +	}
> >>> +
> >>> +	spin_unlock(&boot_lock);
> >>
> >> Why is this boot_lock needed? As far as I understand it asserts that
> >> both CPUs leave nsp_boot_secondary() and nsp_secondary_init() at the
> >> same time.
> >>
> > It makes sure secondary cpu doesn't run away before primary cpu 
> > recognizes the presence of secondary cpu, it is a way of 
> > synchronization between primary and secondary cpu. This is the
> > method used across many SoCs.
> >>> +
> >>> +	ret = pen_release != -1 ? -ENXIO : 0;
> >>> +
> >>> +out:
> >>> +	return ret;
> >>>  }
> >>>  
> >>>  static struct smp_operations bcm_smp_ops __initdata = {
> >>>  	.smp_prepare_cpus	= bcm_smp_prepare_cpus,
> >>> -	.smp_boot_secondary	= bcm_boot_secondary,
> >>> +	.smp_boot_secondary	= kona_boot_secondary,
> >>>  };
> >>>  CPU_METHOD_OF_DECLARE(bcm_smp_bcm281xx, "brcm,bcm11351-cpu-method",
> >>>  			&bcm_smp_ops);
> >>> +
> >>> +struct smp_operations nsp_smp_ops __initdata = {
> >>> +	.smp_prepare_cpus	= bcm_smp_prepare_cpus,
> >>> +	.smp_secondary_init	= nsp_secondary_init,
> >>> +	.smp_boot_secondary	= nsp_boot_secondary,
> >>> +};
> >>> +CPU_METHOD_OF_DECLARE(bcm_smp_nsp, "brcm,bcm-nsp-smp", &nsp_smp_ops);
> >>>
> >>
> > Thanks,
> > Kapil
> > 
> 
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 3/3] ARM: BCM: Add SMP support for Broadcom NSP

[Kapil Hali]

On 10/16/2015 2:47 AM, Jon Mason wrote:
> On Thu, Oct 15, 2015 at 11:12:09PM +0200, Hauke Mehrtens wrote:
>> On 10/15/2015 06:10 PM, Kapil Hali wrote:
>>>
>>>
>>> On 10/15/2015 3:56 AM, Hauke Mehrtens wrote:
>>>> On 10/14/2015 07:47 PM, Kapil Hali wrote:
>>>>> Add SMP support for Broadcom's Northstar Plus SoC,
>>>>> cpu enable method and pen_release procedures. This
>>>>> changes also consolidates iProc family's - BCM NSP
>>>>> and BCM Kona, SMP handling in a common file.
>>>>
>>>> This will probably also work on normal Northstar CPUs without changes.
>>>>
>>> I think, it should work for most of the variants of Northstar family, 
>>> except for those which have a BOOTROM bug.
>>
>> Which SoC are affected by this BOOTROM bug?
> 
> 53012 is the one I am seeing.  Not seeing it on 4708 (and assuming it is
> not present on 4709).  Internally, we do the ugly bug workaround on
> all Northstar SoCs.  The workaround is not acceptable upstream, so I
> am not pushing it. :)
> 
> Thanks,
> Jon
> 

Hi Hauke,

Now that you have tested Jon's patch which is based on my changes for
SMP, shall I take his changes and add it on top of my changes and resend?
Also, do I consider your Acked-by and Tested-by for the change?

Thanks,
Kapil

>>
>>>>> Northstar Plus SoC is based on ARM Cortex-A9
>>>>> revision r3p0 which requires configuration for ARM
>>>>> Errata 764369 for SMP. This change adds the needed
>>>>> configuration option.
>>>>>
>>>>> Signed-off-by: Kapil Hali <kapilh@broadcom.com>
>>>>> ---
>>>>>  arch/arm/mach-bcm/Kconfig                   |   2 +
>>>>>  arch/arm/mach-bcm/Makefile                  |   8 +-
>>>>>  arch/arm/mach-bcm/bcm_nsp.h                 |  19 ++++
>>>>>  arch/arm/mach-bcm/headsmp.S                 |  37 ++++++++
>>>>>  arch/arm/mach-bcm/{kona_smp.c => platsmp.c} | 142 ++++++++++++++++++++++++++--
>>>>>  5 files changed, 197 insertions(+), 11 deletions(-)
>>>>>  create mode 100644 arch/arm/mach-bcm/bcm_nsp.h
>>>>>  create mode 100644 arch/arm/mach-bcm/headsmp.S
>>>>>  rename arch/arm/mach-bcm/{kona_smp.c => platsmp.c} (63%)
>>>>>
>>>>> diff --git a/arch/arm/mach-bcm/Kconfig b/arch/arm/mach-bcm/Kconfig
>>>>> index 1679fa4..2e9dbb5 100644
>>>>> --- a/arch/arm/mach-bcm/Kconfig
>>>>> +++ b/arch/arm/mach-bcm/Kconfig
>>>>> @@ -40,6 +40,8 @@ config ARCH_BCM_NSP
>>>>>  	select ARCH_BCM_IPROC
>>>>>  	select ARM_ERRATA_754322
>>>>>  	select ARM_ERRATA_775420
>>>>> +	select ARM_ERRATA_764369 if SMP
>>>>> +	select HAVE_SMP
>>>>>  	help
>>>>>  	  Support for Broadcom Northstar Plus SoC.
>>>>>  	  Broadcom Northstar Plus family of SoCs are used for switching control
>>>>> diff --git a/arch/arm/mach-bcm/Makefile b/arch/arm/mach-bcm/Makefile
>>>>> index 892261f..36a4ca30 100644
>>>>> --- a/arch/arm/mach-bcm/Makefile
>>>>> +++ b/arch/arm/mach-bcm/Makefile
>>>>> @@ -14,7 +14,11 @@
>>>>>  obj-$(CONFIG_ARCH_BCM_CYGNUS) +=  bcm_cygnus.o
>>>>>  
>>>>>  # Northstar Plus
>>>>> -obj-$(CONFIG_ARCH_BCM_NSP) += bcm_nsp.o
>>>>> +obj-$(CONFIG_ARCH_BCM_NSP)	+= bcm_nsp.o
>>>>> +
>>>>> +ifeq ($(CONFIG_ARCH_BCM_NSP),y)
>>>>> +obj-$(CONFIG_SMP)		+= headsmp.o platsmp.o
>>>>> +endif
>>>>>  
>>>>>  # BCM281XX
>>>>>  obj-$(CONFIG_ARCH_BCM_281XX)	+= board_bcm281xx.o
>>>>> @@ -23,7 +27,7 @@ obj-$(CONFIG_ARCH_BCM_281XX)	+= board_bcm281xx.o
>>>>>  obj-$(CONFIG_ARCH_BCM_21664)	+= board_bcm21664.o
>>>>>  
>>>>>  # BCM281XX and BCM21664 SMP support
>>>>> -obj-$(CONFIG_ARCH_BCM_MOBILE_SMP) += kona_smp.o
>>>>> +obj-$(CONFIG_ARCH_BCM_MOBILE_SMP) += platsmp.o
>>>>>  
>>>>>  # BCM281XX and BCM21664 L2 cache control
>>>>>  obj-$(CONFIG_ARCH_BCM_MOBILE_L2_CACHE) += kona_l2_cache.o
>>>>> diff --git a/arch/arm/mach-bcm/bcm_nsp.h b/arch/arm/mach-bcm/bcm_nsp.h
>>>>> new file mode 100644
>>>>> index 0000000..58e1e80
>>>>> --- /dev/null
>>>>> +++ b/arch/arm/mach-bcm/bcm_nsp.h
>>>>> @@ -0,0 +1,19 @@
>>>>> +/*
>>>>> + * Copyright (C) 2015 Broadcom Corporation
>>>>> + *
>>>>> + * This program is free software; you can redistribute it and/or
>>>>> + * modify it under the terms of the GNU General Public License as
>>>>> + * published by the Free Software Foundation version 2.
>>>>> + *
>>>>> + * This program is distributed "as is" WITHOUT ANY WARRANTY of any
>>>>> + * kind, whether express or implied; without even the implied warranty
>>>>> + * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>>>>> + * GNU General Public License for more details.
>>>>> + */
>>>>> +
>>>>> +#ifndef __BCM_NSP_H
>>>>> +#define __BCM_NSP_H
>>>>> +
>>>>> +extern void nsp_secondary_startup(void);
>>>>> +
>>>>> +#endif /* __BCM_NSP_H */
>>>>> diff --git a/arch/arm/mach-bcm/headsmp.S b/arch/arm/mach-bcm/headsmp.S
>>>>> new file mode 100644
>>>>> index 0000000..0da13b2
>>>>> --- /dev/null
>>>>> +++ b/arch/arm/mach-bcm/headsmp.S
>>>>> @@ -0,0 +1,37 @@
>>>>> +/*
>>>>> + * Copyright (C) 2015 Broadcom Corporation
>>>>> + *
>>>>> + * This program is free software; you can redistribute it and/or
>>>>> + * modify it under the terms of the GNU General Public License as
>>>>> + * published by the Free Software Foundation version 2.
>>>>> + *
>>>>> + * This program is distributed "as is" WITHOUT ANY WARRANTY of any
>>>>> + * kind, whether express or implied; without even the implied warranty
>>>>> + * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>>>>> + * GNU General Public License for more details.
>>>>> + */
>>>>> +
>>>>> +#include <linux/linkage.h>
>>>>> +
>>>>> +/*
>>>>> + * iProc specific entry point for secondary CPUs.  This provides
>>>>> + * a "holding pen" into which all secondary cores are held until
>>>>> + * we are ready for them to initialise.
>>>>> + */
>>>>> +ENTRY(nsp_secondary_startup)
>>>>> +	mrc     p15, 0, r0, c0, c0, 5
>>>>> +	and     r0, r0, #15
>>>>> +	adr     r4, 1f
>>>>> +	ldmia   r4, {r5, r6}
>>>>> +	sub     r4, r4, r5
>>>>> +	add     r6, r6, r4
>>>>> +pen:	ldr     r7, [r6]
>>>>> +	cmp     r7, r0
>>>>> +	bne     pen
>>>>> +
>>>>> +	b    secondary_startup
>>>>> +
>>>>> +1:	.long   .
>>>>> +	.long   pen_release
>>>>> +
>>>>> +ENDPROC(nsp_secondary_startup)
>>>>> diff --git a/arch/arm/mach-bcm/kona_smp.c b/arch/arm/mach-bcm/platsmp.c
>>>>> similarity index 63%
>>>>> rename from arch/arm/mach-bcm/kona_smp.c
>>>>> rename to arch/arm/mach-bcm/platsmp.c
>>>>> index 66a0465..619030e 100644
>>>>> --- a/arch/arm/mach-bcm/kona_smp.c
>>>>> +++ b/arch/arm/mach-bcm/platsmp.c
>>>>> @@ -1,5 +1,5 @@
>>>>>  /*
>>>>> - * Copyright (C) 2014 Broadcom Corporation
>>>>> + * Copyright (C) 2014-2015 Broadcom Corporation
>>>>>   * Copyright 2014 Linaro Limited
>>>>>   *
>>>>>   * This program is free software; you can redistribute it and/or
>>>>> @@ -12,16 +12,23 @@
>>>>>   * GNU General Public License for more details.
>>>>>   */
>>>>>  
>>>>> -#include <linux/init.h>
>>>>> +#include <linux/cpumask.h>
>>>>> +#include <linux/delay.h>
>>>>>  #include <linux/errno.h>
>>>>> +#include <linux/init.h>
>>>>>  #include <linux/io.h>
>>>>> +#include <linux/jiffies.h>
>>>>>  #include <linux/of.h>
>>>>>  #include <linux/sched.h>
>>>>> +#include <linux/smp.h>
>>>>>  
>>>>> +#include <asm/cacheflush.h>
>>>>>  #include <asm/smp.h>
>>>>>  #include <asm/smp_plat.h>
>>>>>  #include <asm/smp_scu.h>
>>>>>  
>>>>> +#include "bcm_nsp.h"
>>>>> +
>>>>>  /* Size of mapped Cortex A9 SCU address space */
>>>>>  #define CORTEX_A9_SCU_SIZE	0x58
>>>>>  
>>>>> @@ -34,6 +41,24 @@
>>>>>  /* I/O address of register used to coordinate secondary core startup */
>>>>>  static u32	secondary_boot;
>>>>>  
>>>>> +static DEFINE_SPINLOCK(boot_lock);
>>>>> +
>>>>> +/*
>>>>> + * Write pen_release in a way that is guaranteed to be visible to all
>>>>> + * observers, irrespective of whether they're taking part in coherency
>>>>> + * or not.  This is necessary for the hotplug code to work reliably.
>>>>> + */
>>>>> +static void write_pen_release(int val)
>>>>> +{
>>>>> +	pen_release = val;
>>>>> +	/*
>>>>> +	 * Ensure write to pen_release is visible to the other cores,
>>>>> +	 * here - primary core
>>>>> +	 */
>>>>> +	smp_wmb();
>>>>> +	sync_cache_w(&pen_release);
>>>>> +}
>>>>> +
>>>>>  /*
>>>>>   * Enable the Cortex A9 Snoop Control Unit
>>>>>   *
>>>>> @@ -75,6 +100,51 @@ static int __init scu_a9_enable(void)
>>>>>  	return 0;
>>>>>  }
>>>>>  
>>>>> +static int nsp_write_lut(void (*secondary_startup) (void))
>>>>> +{
>>>>> +	void __iomem *sku_rom_lut;
>>>>> +	phys_addr_t secondary_startup_phy;
>>>>> +
>>>>> +	if (!secondary_boot) {
>>>>> +		pr_warn("required secondary boot register not specified\n");
>>>>> +		return -EINVAL;
>>>>> +	}
>>>>> +
>>>>> +	sku_rom_lut = ioremap_nocache((phys_addr_t)secondary_boot,
>>>>> +						sizeof(secondary_boot));
>>>>> +	if (!sku_rom_lut) {
>>>>> +		pr_warn("unable to ioremap SKU-ROM LUT register\n");
>>>>> +		return -ENOMEM;
>>>>> +	}
>>>>> +
>>>>> +	secondary_startup_phy = virt_to_phys(secondary_startup);
>>>>> +	BUG_ON(secondary_startup_phy > (phys_addr_t)U32_MAX);
>>>>> +
>>>>> +	writel_relaxed(secondary_startup_phy, sku_rom_lut);
>>>>> +	/*
>>>>> +	 * Ensure the write is visible to the secondary core.
>>>>> +	 */
>>>>> +	smp_wmb();
>>>>> +
>>>>> +	iounmap(sku_rom_lut);
>>>>> +
>>>>> +	return 0;
>>>>> +}
>>>>> +
>>>>> +static void nsp_secondary_init(unsigned int cpu)
>>>>> +{
>>>>> +	/*
>>>>> +	 * Let the primary cpu know we are out of holding pen.
>>>>> +	 */
>>>>> +	write_pen_release(-1);
>>>>> +
>>>>> +	/*
>>>>> +	 * Synchronise with the boot thread.
>>>>> +	 */
>>>>> +	spin_lock(&boot_lock);
>>>>> +	spin_unlock(&boot_lock);
>>>>> +}
>>>>> +
>>>>>  static void __init bcm_smp_prepare_cpus(unsigned int max_cpus)
>>>>>  {
>>>>>  	static cpumask_t only_cpu_0 = { CPU_BITS_CPU0 };
>>>>> @@ -95,11 +165,11 @@ static void __init bcm_smp_prepare_cpus(unsigned int max_cpus)
>>>>>  	/*
>>>>>  	 * Our secondary enable method requires a "secondary-boot-reg"
>>>>>  	 * property to specify a register address used to request the
>>>>> -	 * ROM code boot a secondary code.  If we have any trouble
>>>>> +	 * ROM code boot a secondary core.  If we have any trouble
>>>>>  	 * getting this we fall back to uniprocessor mode.
>>>>>  	 */
>>>>>  	if (of_property_read_u32(node, OF_SECONDARY_BOOT, &secondary_boot)) {
>>>>> -		pr_err("%s: missing/invalid " OF_SECONDARY_BOOT " property\n",
>>>>> +		pr_warn("%s: missing/invalid " OF_SECONDARY_BOOT " property\n",
>>>>>  			node->name);
>>>>>  		ret = -ENOENT;		/* Arrange to disable SMP */
>>>>>  		goto out;
>>>>> @@ -115,7 +185,6 @@ out:
>>>>>  	of_node_put(node);
>>>>>  	if (ret) {
>>>>>  		/* Update the CPU present map to reflect uniprocessor mode */
>>>>> -		BUG_ON(ret != -ENOENT);
>>>>>  		pr_warn("disabling SMP\n");
>>>>>  		init_cpu_present(&only_cpu_0);
>>>>>  	}
>>>>> @@ -139,7 +208,7 @@ out:
>>>>>   * - Wait for the secondary boot register to be re-written, which
>>>>>   *   indicates the secondary core has started.
>>>>>   */
>>>>> -static int bcm_boot_secondary(unsigned int cpu, struct task_struct *idle)
>>>>> +static int kona_boot_secondary(unsigned int cpu, struct task_struct *idle)
>>>>>  {
>>>>>  	void __iomem *boot_reg;
>>>>>  	phys_addr_t boot_func;
>>>>> @@ -162,7 +231,7 @@ static int bcm_boot_secondary(unsigned int cpu, struct task_struct *idle)
>>>>>  	boot_reg = ioremap_nocache((phys_addr_t)secondary_boot, sizeof(u32));
>>>>>  	if (!boot_reg) {
>>>>>  		pr_err("unable to map boot register for cpu %u\n", cpu_id);
>>>>> -		return -ENOSYS;
>>>>> +		return -ENOMEM;
>>>>>  	}
>>>>>  
>>>>>  	/*
>>>>> @@ -191,12 +260,67 @@ static int bcm_boot_secondary(unsigned int cpu, struct task_struct *idle)
>>>>>  
>>>>>  	pr_err("timeout waiting for cpu %u to start\n", cpu_id);
>>>>>  
>>>>> -	return -ENOSYS;
>>>>> +	return -ENXIO;
>>>>> +}
>>>>> +
>>>>> +static int nsp_boot_secondary(unsigned int cpu, struct task_struct *idle)
>>>>> +{
>>>>> +	unsigned long timeout;
>>>>> +	int ret;
>>>>> +
>>>>> +	/*
>>>>> +	 * After wake up, secondary core branches to the startup
>>>>> +	 * address programmed at SKU ROM LUT location.
>>>>> +	 */
>>>>> +	ret = nsp_write_lut(nsp_secondary_startup);
>>>>> +	if (ret) {
>>>>> +		pr_err("unable to write startup addr to SKU ROM LUT\n");
>>>>> +		goto out;
>>>>> +	}
>>>>> +
>>>>> +	/*
>>>>> +	 * The secondary processor is waiting to be released from
>>>>> +	 * the holding pen - release it, then wait for it to flag
>>>>> +	 * that it has been released by resetting pen_release.
>>>>> +	 */
>>>>> +	spin_lock(&boot_lock);
>>>>> +
>>>>> +	write_pen_release(cpu_logical_map(cpu));
>>>>> +	/*
>>>>> +	 * Send an Event to wake up the secondary core which is in
>>>>> +	 * WFE state. Updated pen_release should also be visible to
>>>>> +	 * the secondary core.
>>>>> +	 */
>>>>> +	dsb_sev();
>>>>> +
>>>>> +	timeout = jiffies + (1 * HZ);
>>>>> +	while (time_before(jiffies, timeout)) {
>>>>> +		/* Make sure loads on other CPU is visible */
>>>>> +		smp_rmb();
>>>>> +		if (pen_release == -1)
>>>>> +			break;
>>>>> +
>>>>> +		udelay(10);
>>>>> +	}
>>>>> +
>>>>> +	spin_unlock(&boot_lock);
>>>>
>>>> Why is this boot_lock needed? As far as I understand it asserts that
>>>> both CPUs leave nsp_boot_secondary() and nsp_secondary_init() at the
>>>> same time.
>>>>
>>> It makes sure secondary cpu doesn't run away before primary cpu 
>>> recognizes the presence of secondary cpu, it is a way of 
>>> synchronization between primary and secondary cpu. This is the
>>> method used across many SoCs.
>>>>> +
>>>>> +	ret = pen_release != -1 ? -ENXIO : 0;
>>>>> +
>>>>> +out:
>>>>> +	return ret;
>>>>>  }
>>>>>  
>>>>>  static struct smp_operations bcm_smp_ops __initdata = {
>>>>>  	.smp_prepare_cpus	= bcm_smp_prepare_cpus,
>>>>> -	.smp_boot_secondary	= bcm_boot_secondary,
>>>>> +	.smp_boot_secondary	= kona_boot_secondary,
>>>>>  };
>>>>>  CPU_METHOD_OF_DECLARE(bcm_smp_bcm281xx, "brcm,bcm11351-cpu-method",
>>>>>  			&bcm_smp_ops);
>>>>> +
>>>>> +struct smp_operations nsp_smp_ops __initdata = {
>>>>> +	.smp_prepare_cpus	= bcm_smp_prepare_cpus,
>>>>> +	.smp_secondary_init	= nsp_secondary_init,
>>>>> +	.smp_boot_secondary	= nsp_boot_secondary,
>>>>> +};
>>>>> +CPU_METHOD_OF_DECLARE(bcm_smp_nsp, "brcm,bcm-nsp-smp", &nsp_smp_ops);
>>>>>
>>>>
>>> Thanks,
>>> Kapil
>>>
>>

Re: [PATCH 3/3] ARM: BCM: Add SMP support for Broadcom NSP

[Hauke Mehrtens]

On 10/28/2015 03:24 PM, Kapil Hali wrote:
> 
> 
> On 10/16/2015 2:47 AM, Jon Mason wrote:
>> On Thu, Oct 15, 2015 at 11:12:09PM +0200, Hauke Mehrtens wrote:
>>> On 10/15/2015 06:10 PM, Kapil Hali wrote:
>>>>
>>>>
>>>> On 10/15/2015 3:56 AM, Hauke Mehrtens wrote:
>>>>> On 10/14/2015 07:47 PM, Kapil Hali wrote:
>>>>>> Add SMP support for Broadcom's Northstar Plus SoC,
>>>>>> cpu enable method and pen_release procedures. This
>>>>>> changes also consolidates iProc family's - BCM NSP
>>>>>> and BCM Kona, SMP handling in a common file.
>>>>>
>>>>> This will probably also work on normal Northstar CPUs without changes.
>>>>>
>>>> I think, it should work for most of the variants of Northstar family, 
>>>> except for those which have a BOOTROM bug.
>>>
>>> Which SoC are affected by this BOOTROM bug?
>>
>> 53012 is the one I am seeing.  Not seeing it on 4708 (and assuming it is
>> not present on 4709).  Internally, we do the ugly bug workaround on
>> all Northstar SoCs.  The workaround is not acceptable upstream, so I
>> am not pushing it. :)
>>
>> Thanks,
>> Jon
>>
> 
> Hi Hauke,
> 
> Now that you have tested Jon's patch which is based on my changes for
> SMP, shall I take his changes and add it on top of my changes and resend?
> Also, do I consider your Acked-by and Tested-by for the change?
> 
> Thanks,
> Kapil
> 

Hi Kapil,

I tested this on Northstar 1.

Acked-by: Hauke Mehrtens <hauke@hauke-m.de>
Tested-by: Hauke Mehrtens <hauke@hauke-m.de>

I think Florian has to take the patches and you should hurry up because
the merge window probably opens on Monday.

You can send all patches again with the bcm501x patch or just that one,
I assume there are no changes needed for the other patches?

Hauke