[PATCH v4 0/7] ARM: multi-cluster aware boot protocol

[PATCH v4 0/7] ARM: multi-cluster aware boot protocol

[Lorenzo Pieralisi]

This series is v4, referred to this previous posting:

http://lists.infradead.org/pipermail/linux-arm-kernel/2012-November/132126.html

Changes in v4:

- Moved and cleaned up MPIDR masks from topology.c to asm/cputype.h
- Applied some fixes to cpus.txt documentation
- Patched code to use the new MPIDR accessors masks in smp_setup_processor_id()
  and arm_dt_init_cpu_maps()

Changes in v3:

- Fixed compatible string list format and vendors in Documentation
- Added check for MPIDR duplicates in the DT
- Added is_smp() check before reading the MPIDR in smp_setup_processor_id()
  and also arm_dt_init_cpu_maps()
- Added check for 8 MSBs in arm_dt_init_cpu_maps()
- Updated comments and commit logs

Changes in v2:

- Reworded and added compatible property to cpus.txt
- Replaced while loop in arm_dt_init_cpu_maps() with a combination of
  of_find_node_by_path and for_each_child_of_node
- Replaced of_get_property with of_property_read_u32
- Added further checks for DT /cpu nodes and implemented stashed array
  to avoid overwriting the cpu_logical_map if DT contains errors
- Added GIC CPU IF define in the GIC probing code
- Removed printk for boot CPU MPIDR and added a patch that prints the full
  extent of MPIDR in smp_setup_processor_id()
- Updated smp_setup_processor_id() so that it uses nr_cpu_ids
- Dropped RFC tag

The introduction of multi-cluster ARM systems in SoC designs requires the
kernel to become cluster aware, so that it can be booted on every CPU in the
system and it can build an appropriate HW-to-logical cpu map.

Current code in the kernel, in particular the boot sequence, hinges upon a
sequential mapping of MPIDR values for cpus and related interrupt controller
CPU interfaces to logical cpu indexing.
This hypothesis is not valid when the concept of cluster is introduced since
the MPIDR cannot be represented as a single index and interrupt controller
CPU interfaces can be wired with a numbering scheme following per-SoC
design parameters which can be only detected through probing or device
tree representation.

Through the device tree and "cpu" nodes bindings, the kernel is provided
with HW values for MPIDR registers that allow the kernel to identify the
HW CPU ids that are present in the platform.

The GIC code has been extended to allow automatic detection of GIC CPU IF ids
at boot. IPIs are broadcast to all possible CPUs, and every time a secondary
CPU is booted, it initializes its own mask and clears itself from the mask of
all other logical CPUs.

The device tree bindings and GIC probing allow to boot the Linux kernel on any
CPU of a multi-cluster system without relying on a platform specific hook to
identify the number of CPUs and hypothesis on the sequential pattern of MPIDRs
and relative GIC CPU IF ids.

Pen release code for all converted platforms will need patching to extend the
current MPIDR range check; this will take place as soon as the bindings and
code for the multi-cluster boot protocol will be reviewed and accepted.

The patchset has been tested on:

- TC2 testchip

to boot a 5-core dual-cluster system on every possible CPU.

Lorenzo Pieralisi (6):
  ARM: kernel: enhance MPIDR macro definitions
  ARM: kernel: update topology to use new MPIDR macros
  ARM: kernel: smp_setup_processor_id() updates
  ARM: kernel: add device tree init map function
  ARM: kernel: add cpu logical map DT init in setup_arch
  ARM: kernel: add logical mappings look-up

Nicolas Pitre (1):
  ARM: gic: use a private mapping for CPU target interfaces

 Documentation/devicetree/bindings/arm/cpus.txt |  77 +++++++++++++++++++
 arch/arm/common/gic.c                          |  45 ++++++++---
 arch/arm/include/asm/cputype.h                 |  13 ++++
 arch/arm/include/asm/prom.h                    |   2 +
 arch/arm/include/asm/smp_plat.h                |  17 +++++
 arch/arm/kernel/devtree.c                      | 100 +++++++++++++++++++++++++
 arch/arm/kernel/setup.c                        |   8 +-
 arch/arm/kernel/topology.c                     |  42 ++---------
 8 files changed, 256 insertions(+), 48 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/arm/cpus.txt

-- 
1.7.12

[PATCH v4 1/7] ARM: kernel: enhance MPIDR macro definitions

[Lorenzo Pieralisi]

Kernel subsystems other than the topology layer need the MPIDR
mask definitions to access the MPIDR without relying on hardcoded
masks. This patch moves the MPIDR register masks definition to
a header file and defines a macro to simplify access to MPIDR bit fields
representing affinity levels.

Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
---
 arch/arm/include/asm/cputype.h | 13 +++++++++++++
 arch/arm/kernel/topology.c     | 27 +--------------------------
 2 files changed, 14 insertions(+), 26 deletions(-)

diff --git a/arch/arm/include/asm/cputype.h b/arch/arm/include/asm/cputype.h
index cb47d28..a59dcb5 100644
--- a/arch/arm/include/asm/cputype.h
+++ b/arch/arm/include/asm/cputype.h
@@ -25,6 +25,19 @@
 #define CPUID_EXT_ISAR4	"c2, 4"
 #define CPUID_EXT_ISAR5	"c2, 5"
 
+#define MPIDR_SMP_BITMASK (0x3 << 30)
+#define MPIDR_SMP_VALUE (0x2 << 30)
+
+#define MPIDR_MT_BITMASK (0x1 << 24)
+
+#define MPIDR_HWID_BITMASK 0xFFFFFF
+
+#define MPIDR_LEVEL_BITS 8
+#define MPIDR_LEVEL_MASK ((1 << MPIDR_LEVEL_BITS) - 1)
+
+#define MPIDR_AFFINITY_LEVEL(mpidr, level) \
+	((mpidr >> (MPIDR_LEVEL_BITS * level)) & MPIDR_LEVEL_MASK)
+
 extern unsigned int processor_id;
 
 #ifdef CONFIG_CPU_CP15
diff --git a/arch/arm/kernel/topology.c b/arch/arm/kernel/topology.c
index 317dac6..4642c7d 100644
--- a/arch/arm/kernel/topology.c
+++ b/arch/arm/kernel/topology.c
@@ -196,32 +196,7 @@ static inline void parse_dt_topology(void) {}
 static inline void update_cpu_power(unsigned int cpuid, unsigned int mpidr) {}
 #endif
 
-
-/*
- * cpu topology management
- */
-
-#define MPIDR_SMP_BITMASK (0x3 << 30)
-#define MPIDR_SMP_VALUE (0x2 << 30)
-
-#define MPIDR_MT_BITMASK (0x1 << 24)
-
-/*
- * These masks reflect the current use of the affinity levels.
- * The affinity level can be up to 16 bits according to ARM ARM
- */
-#define MPIDR_HWID_BITMASK 0xFFFFFF
-
-#define MPIDR_LEVEL0_MASK 0x3
-#define MPIDR_LEVEL0_SHIFT 0
-
-#define MPIDR_LEVEL1_MASK 0xF
-#define MPIDR_LEVEL1_SHIFT 8
-
-#define MPIDR_LEVEL2_MASK 0xFF
-#define MPIDR_LEVEL2_SHIFT 16
-
-/*
+ /*
  * cpu topology table
  */
 struct cputopo_arm cpu_topology[NR_CPUS];
-- 
1.7.12

Re: [PATCH v4 1/7] ARM: kernel: enhance MPIDR macro definitions

[Will Deacon]

On Mon, Nov 19, 2012 at 12:45:00PM +0000, Lorenzo Pieralisi wrote:
> Kernel subsystems other than the topology layer need the MPIDR
> mask definitions to access the MPIDR without relying on hardcoded
> masks. This patch moves the MPIDR register masks definition to
> a header file and defines a macro to simplify access to MPIDR bit fields
> representing affinity levels.
> 
> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi-5wv7dgnIgG8@public.gmane.org>

Acked-by: Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>

Will

Re: [PATCH v4 1/7] ARM: kernel: enhance MPIDR macro definitions

[Nicolas Pitre]

On Mon, 19 Nov 2012, Lorenzo Pieralisi wrote:

> Kernel subsystems other than the topology layer need the MPIDR
> mask definitions to access the MPIDR without relying on hardcoded
> masks. This patch moves the MPIDR register masks definition to
> a header file and defines a macro to simplify access to MPIDR bit fields
> representing affinity levels.
> 
> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi-5wv7dgnIgG8@public.gmane.org>

Acked-by: Nicolas Pitre <nico-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>

> ---
>  arch/arm/include/asm/cputype.h | 13 +++++++++++++
>  arch/arm/kernel/topology.c     | 27 +--------------------------
>  2 files changed, 14 insertions(+), 26 deletions(-)
> 
> diff --git a/arch/arm/include/asm/cputype.h b/arch/arm/include/asm/cputype.h
> index cb47d28..a59dcb5 100644
> --- a/arch/arm/include/asm/cputype.h
> +++ b/arch/arm/include/asm/cputype.h
> @@ -25,6 +25,19 @@
>  #define CPUID_EXT_ISAR4	"c2, 4"
>  #define CPUID_EXT_ISAR5	"c2, 5"
>  
> +#define MPIDR_SMP_BITMASK (0x3 << 30)
> +#define MPIDR_SMP_VALUE (0x2 << 30)
> +
> +#define MPIDR_MT_BITMASK (0x1 << 24)
> +
> +#define MPIDR_HWID_BITMASK 0xFFFFFF
> +
> +#define MPIDR_LEVEL_BITS 8
> +#define MPIDR_LEVEL_MASK ((1 << MPIDR_LEVEL_BITS) - 1)
> +
> +#define MPIDR_AFFINITY_LEVEL(mpidr, level) \
> +	((mpidr >> (MPIDR_LEVEL_BITS * level)) & MPIDR_LEVEL_MASK)
> +
>  extern unsigned int processor_id;
>  
>  #ifdef CONFIG_CPU_CP15
> diff --git a/arch/arm/kernel/topology.c b/arch/arm/kernel/topology.c
> index 317dac6..4642c7d 100644
> --- a/arch/arm/kernel/topology.c
> +++ b/arch/arm/kernel/topology.c
> @@ -196,32 +196,7 @@ static inline void parse_dt_topology(void) {}
>  static inline void update_cpu_power(unsigned int cpuid, unsigned int mpidr) {}
>  #endif
>  
> -
> -/*
> - * cpu topology management
> - */
> -
> -#define MPIDR_SMP_BITMASK (0x3 << 30)
> -#define MPIDR_SMP_VALUE (0x2 << 30)
> -
> -#define MPIDR_MT_BITMASK (0x1 << 24)
> -
> -/*
> - * These masks reflect the current use of the affinity levels.
> - * The affinity level can be up to 16 bits according to ARM ARM
> - */
> -#define MPIDR_HWID_BITMASK 0xFFFFFF
> -
> -#define MPIDR_LEVEL0_MASK 0x3
> -#define MPIDR_LEVEL0_SHIFT 0
> -
> -#define MPIDR_LEVEL1_MASK 0xF
> -#define MPIDR_LEVEL1_SHIFT 8
> -
> -#define MPIDR_LEVEL2_MASK 0xFF
> -#define MPIDR_LEVEL2_SHIFT 16
> -
> -/*
> + /*
>   * cpu topology table
>   */
>  struct cputopo_arm cpu_topology[NR_CPUS];
> -- 
> 1.7.12
> 
> 

[PATCH v4 2/7] ARM: kernel: update topology to use new MPIDR macros

[Lorenzo Pieralisi]

This patch updates the topology initialization code to use the newly
defined accessors to retrieve the MPIDR affinity levels.

Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
---
 arch/arm/kernel/topology.c | 15 +++++----------
 1 file changed, 5 insertions(+), 10 deletions(-)

diff --git a/arch/arm/kernel/topology.c b/arch/arm/kernel/topology.c
index 4642c7d..cba99bd 100644
--- a/arch/arm/kernel/topology.c
+++ b/arch/arm/kernel/topology.c
@@ -262,19 +262,14 @@ void store_cpu_topology(unsigned int cpuid)
 
 		if (mpidr & MPIDR_MT_BITMASK) {
 			/* core performance interdependency */
-			cpuid_topo->thread_id = (mpidr >> MPIDR_LEVEL0_SHIFT)
-				& MPIDR_LEVEL0_MASK;
-			cpuid_topo->core_id = (mpidr >> MPIDR_LEVEL1_SHIFT)
-				& MPIDR_LEVEL1_MASK;
-			cpuid_topo->socket_id = (mpidr >> MPIDR_LEVEL2_SHIFT)
-				& MPIDR_LEVEL2_MASK;
+			cpuid_topo->thread_id = MPIDR_AFFINITY_LEVEL(mpidr, 0);
+			cpuid_topo->core_id = MPIDR_AFFINITY_LEVEL(mpidr, 1);
+			cpuid_topo->socket_id = MPIDR_AFFINITY_LEVEL(mpidr, 2);
 		} else {
 			/* largely independent cores */
 			cpuid_topo->thread_id = -1;
-			cpuid_topo->core_id = (mpidr >> MPIDR_LEVEL0_SHIFT)
-				& MPIDR_LEVEL0_MASK;
-			cpuid_topo->socket_id = (mpidr >> MPIDR_LEVEL1_SHIFT)
-				& MPIDR_LEVEL1_MASK;
+			cpuid_topo->core_id = MPIDR_AFFINITY_LEVEL(mpidr, 0);
+			cpuid_topo->socket_id = MPIDR_AFFINITY_LEVEL(mpidr, 1);
 		}
 	} else {
 		/*
-- 
1.7.12

Re: [PATCH v4 2/7] ARM: kernel: update topology to use new MPIDR macros

[Will Deacon]

On Mon, Nov 19, 2012 at 12:45:01PM +0000, Lorenzo Pieralisi wrote:
> This patch updates the topology initialization code to use the newly
> defined accessors to retrieve the MPIDR affinity levels.
> 
> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi-5wv7dgnIgG8@public.gmane.org>

Acked-by: Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>

Will

Re: [PATCH v4 2/7] ARM: kernel: update topology to use new MPIDR macros

[Nicolas Pitre]

On Mon, 19 Nov 2012, Lorenzo Pieralisi wrote:

> This patch updates the topology initialization code to use the newly
> defined accessors to retrieve the MPIDR affinity levels.
> 
> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi-5wv7dgnIgG8@public.gmane.org>

Acked-by: Nicolas Pitre <nico-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>

> ---
>  arch/arm/kernel/topology.c | 15 +++++----------
>  1 file changed, 5 insertions(+), 10 deletions(-)
> 
> diff --git a/arch/arm/kernel/topology.c b/arch/arm/kernel/topology.c
> index 4642c7d..cba99bd 100644
> --- a/arch/arm/kernel/topology.c
> +++ b/arch/arm/kernel/topology.c
> @@ -262,19 +262,14 @@ void store_cpu_topology(unsigned int cpuid)
>  
>  		if (mpidr & MPIDR_MT_BITMASK) {
>  			/* core performance interdependency */
> -			cpuid_topo->thread_id = (mpidr >> MPIDR_LEVEL0_SHIFT)
> -				& MPIDR_LEVEL0_MASK;
> -			cpuid_topo->core_id = (mpidr >> MPIDR_LEVEL1_SHIFT)
> -				& MPIDR_LEVEL1_MASK;
> -			cpuid_topo->socket_id = (mpidr >> MPIDR_LEVEL2_SHIFT)
> -				& MPIDR_LEVEL2_MASK;
> +			cpuid_topo->thread_id = MPIDR_AFFINITY_LEVEL(mpidr, 0);
> +			cpuid_topo->core_id = MPIDR_AFFINITY_LEVEL(mpidr, 1);
> +			cpuid_topo->socket_id = MPIDR_AFFINITY_LEVEL(mpidr, 2);
>  		} else {
>  			/* largely independent cores */
>  			cpuid_topo->thread_id = -1;
> -			cpuid_topo->core_id = (mpidr >> MPIDR_LEVEL0_SHIFT)
> -				& MPIDR_LEVEL0_MASK;
> -			cpuid_topo->socket_id = (mpidr >> MPIDR_LEVEL1_SHIFT)
> -				& MPIDR_LEVEL1_MASK;
> +			cpuid_topo->core_id = MPIDR_AFFINITY_LEVEL(mpidr, 0);
> +			cpuid_topo->socket_id = MPIDR_AFFINITY_LEVEL(mpidr, 1);
>  		}
>  	} else {
>  		/*
> -- 
> 1.7.12
> 
> 

[PATCH v4 3/7] ARM: kernel: smp_setup_processor_id() updates

[Lorenzo Pieralisi]

This patch applies some basic changes to the smp_setup_processor_id()
ARM implementation to make the code that builds cpu_logical_map more
uniform across the kernel.

The function now prints the full extent of the boot CPU MPIDR[23:0] and
initializes the cpu_logical_map for CPUs up to nr_cpu_ids.

Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Nicolas Pitre <nico@linaro.org>
---
 arch/arm/kernel/setup.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
index da1d1aa..4515bf6 100644
--- a/arch/arm/kernel/setup.c
+++ b/arch/arm/kernel/setup.c
@@ -426,13 +426,14 @@ int __cpu_logical_map[NR_CPUS];
 void __init smp_setup_processor_id(void)
 {
 	int i;
-	u32 cpu = is_smp() ? read_cpuid_mpidr() & 0xff : 0;
+	u32 mpidr = is_smp() ? read_cpuid_mpidr() & MPIDR_HWID_BITMASK : 0;
+	u32 cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
 
 	cpu_logical_map(0) = cpu;
-	for (i = 1; i < NR_CPUS; ++i)
+	for (i = 1; i < nr_cpu_ids; ++i)
 		cpu_logical_map(i) = i == cpu ? 0 : i;
 
-	printk(KERN_INFO "Booting Linux on physical CPU %d\n", cpu);
+	printk(KERN_INFO "Booting Linux on physical CPU 0x%x\n", mpidr);
 }
 
 static void __init setup_processor(void)
-- 
1.7.12

Re: [PATCH v4 3/7] ARM: kernel: smp_setup_processor_id() updates

[Will Deacon]

On Mon, Nov 19, 2012 at 12:45:02PM +0000, Lorenzo Pieralisi wrote:
> This patch applies some basic changes to the smp_setup_processor_id()
> ARM implementation to make the code that builds cpu_logical_map more
> uniform across the kernel.
> 
> The function now prints the full extent of the boot CPU MPIDR[23:0] and
> initializes the cpu_logical_map for CPUs up to nr_cpu_ids.
> 
> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi-5wv7dgnIgG8@public.gmane.org>
> Acked-by: Nicolas Pitre <nico-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
> ---

Acked-by: Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>

Will

[PATCH v4 4/7] ARM: kernel: add device tree init map function

[Lorenzo Pieralisi]

When booting through a device tree, the kernel cpu logical id map can be
initialized using device tree data passed by FW or through an embedded blob.

This patch adds a function that parses device tree "cpu" nodes and
retrieves the corresponding CPUs hardware identifiers (MPIDR).
It sets the possible cpus and the cpu logical map values according to
the number of CPUs defined in the device tree and respective properties.

The device tree HW identifiers are considered valid if all CPU nodes contain
a "reg" property, there are no duplicate "reg" entries and the DT defines a
CPU node whose "reg" property matches the MPIDR[23:0] of the boot CPU.

The primary CPU is assigned cpu logical number 0 to keep the current convention
valid.

Current bindings documentation is included in the patch:

Documentation/devicetree/bindings/arm/cpus.txt

Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Nicolas Pitre <nico@linaro.org>
---
 Documentation/devicetree/bindings/arm/cpus.txt |  77 +++++++++++++++++++
 arch/arm/include/asm/prom.h                    |   2 +
 arch/arm/kernel/devtree.c                      | 100 +++++++++++++++++++++++++
 3 files changed, 179 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/arm/cpus.txt

diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
new file mode 100644
index 0000000..46c3589
--- /dev/null
+++ b/Documentation/devicetree/bindings/arm/cpus.txt
@@ -0,0 +1,77 @@
+* ARM CPUs binding description
+
+The device tree allows to describe the layout of CPUs in a system through
+the "cpus" node, which in turn contains a number of subnodes (ie "cpu")
+defining properties for every cpu.
+
+Bindings for CPU nodes follow the ePAPR standard, available from:
+
+http://devicetree.org
+
+For the ARM architecture every CPU node must contain the following properties:
+
+- device_type:	must be "cpu"
+- reg:		property matching the CPU MPIDR[23:0] register bits
+		reg[31:24] bits must be set to 0
+- compatible:	should be one of:
+		"arm,arm1020"
+		"arm,arm1020e"
+		"arm,arm1022"
+		"arm,arm1026"
+		"arm,arm720"
+		"arm,arm740"
+		"arm,arm7tdmi"
+		"arm,arm920"
+		"arm,arm922"
+		"arm,arm925"
+		"arm,arm926"
+		"arm,arm940"
+		"arm,arm946"
+		"arm,arm9tdmi"
+		"arm,cortex-a5"
+		"arm,cortex-a7"
+		"arm,cortex-a8"
+		"arm,cortex-a9"
+		"arm,cortex-a15"
+		"arm,arm1136"
+		"arm,arm1156"
+		"arm,arm1176"
+		"arm,arm11mpcore"
+		"faraday,fa526"
+		"intel,sa110"
+		"intel,sa1100"
+		"marvell,feroceon"
+		"marvell,mohawk"
+		"marvell,xsc3"
+		"marvell,xscale"
+
+Example:
+
+	cpus {
+		#size-cells = <0>;
+		#address-cells = <1>;
+
+		CPU0: cpu@0 {
+			device_type = "cpu";
+			compatible = "arm, cortex-a15";
+			reg = <0x0>;
+		};
+
+		CPU1: cpu@1 {
+			device_type = "cpu";
+			compatible = "arm, cortex-a15";
+			reg = <0x1>;
+		};
+
+		CPU2: cpu@100 {
+			device_type = "cpu";
+			compatible = "arm, cortex-a7";
+			reg = <0x100>;
+		};
+
+		CPU3: cpu@101 {
+			device_type = "cpu";
+			compatible = "arm, cortex-a7";
+			reg = <0x101>;
+		};
+	};
diff --git a/arch/arm/include/asm/prom.h b/arch/arm/include/asm/prom.h
index aeae9c6..8dd51dc 100644
--- a/arch/arm/include/asm/prom.h
+++ b/arch/arm/include/asm/prom.h
@@ -15,6 +15,7 @@
 
 extern struct machine_desc *setup_machine_fdt(unsigned int dt_phys);
 extern void arm_dt_memblock_reserve(void);
+extern void __init arm_dt_init_cpu_maps(void);
 
 #else /* CONFIG_OF */
 
@@ -24,6 +25,7 @@ static inline struct machine_desc *setup_machine_fdt(unsigned int dt_phys)
 }
 
 static inline void arm_dt_memblock_reserve(void) { }
+static inline void arm_dt_init_cpu_maps(void) { }
 
 #endif /* CONFIG_OF */
 #endif /* ASMARM_PROM_H */
diff --git a/arch/arm/kernel/devtree.c b/arch/arm/kernel/devtree.c
index bee7f9d..aaf9add 100644
--- a/arch/arm/kernel/devtree.c
+++ b/arch/arm/kernel/devtree.c
@@ -19,8 +19,10 @@
 #include <linux/of_irq.h>
 #include <linux/of_platform.h>
 
+#include <asm/cputype.h>
 #include <asm/setup.h>
 #include <asm/page.h>
+#include <asm/smp_plat.h>
 #include <asm/mach/arch.h>
 #include <asm/mach-types.h>
 
@@ -61,6 +63,104 @@ void __init arm_dt_memblock_reserve(void)
 	}
 }
 
+/*
+ * arm_dt_init_cpu_maps - Function retrieves cpu nodes from the device tree
+ * and builds the cpu logical map array containing MPIDR values related to
+ * logical cpus
+ *
+ * Updates the cpu possible mask with the number of parsed cpu nodes
+ */
+void __init arm_dt_init_cpu_maps(void)
+{
+	/*
+	 * Temp logical map is initialized with UINT_MAX values that are
+	 * considered invalid logical map entries since the logical map must
+	 * contain a list of MPIDR[23:0] values where MPIDR[31:24] must
+	 * read as 0.
+	 */
+	struct device_node *cpu, *cpus;
+	u32 i, j, cpuidx = 1;
+	u32 mpidr = is_smp() ? read_cpuid_mpidr() & MPIDR_HWID_BITMASK : 0;
+
+	u32 tmp_map[NR_CPUS] = { [0 ... NR_CPUS-1] = UINT_MAX };
+	bool bootcpu_valid = false;
+	cpus = of_find_node_by_path("/cpus");
+
+	if (!cpus)
+		return;
+
+	for_each_child_of_node(cpus, cpu) {
+		u32 hwid;
+
+		pr_debug(" * %s...\n", cpu->full_name);
+		/*
+		 * A device tree containing CPU nodes with missing "reg"
+		 * properties is considered invalid to build the
+		 * cpu_logical_map.
+		 */
+		if (of_property_read_u32(cpu, "reg", &hwid)) {
+			pr_debug(" * %s missing reg property\n",
+				     cpu->full_name);
+			return;
+		}
+
+		/*
+		 * 8 MSBs must be set to 0 in the DT since the reg property
+		 * defines the MPIDR[23:0].
+		 */
+		if (hwid & ~MPIDR_HWID_BITMASK)
+			return;
+
+		/*
+		 * Duplicate MPIDRs are a recipe for disaster.
+		 * Scan all initialized entries and check for
+		 * duplicates. If any is found just bail out.
+		 * temp values were initialized to UINT_MAX
+		 * to avoid matching valid MPIDR[23:0] values.
+		 */
+		for (j = 0; j < cpuidx; j++)
+			if (WARN(tmp_map[j] == hwid, "Duplicate /cpu reg "
+						     "properties in the DT\n"))
+				return;
+
+		/*
+		 * Build a stashed array of MPIDR values. Numbering scheme
+		 * requires that if detected the boot CPU must be assigned
+		 * logical id 0. Other CPUs get sequential indexes starting
+		 * from 1. If a CPU node with a reg property matching the
+		 * boot CPU MPIDR is detected, this is recorded so that the
+		 * logical map built from DT is validated and can be used
+		 * to override the map created in smp_setup_processor_id().
+		 */
+		if (hwid == mpidr) {
+			i = 0;
+			bootcpu_valid = true;
+		} else {
+			i = cpuidx++;
+		}
+
+		tmp_map[i] = hwid;
+
+		if (cpuidx > nr_cpu_ids)
+			break;
+	}
+
+	if (WARN(!bootcpu_valid, "DT missing boot CPU MPIDR[23:0], "
+				 "fall back to default cpu_logical_map\n"))
+		return;
+
+	/*
+	 * Since the boot CPU node contains proper data, and all nodes have
+	 * a reg property, the DT CPU list can be considered valid and the
+	 * logical map created in smp_setup_processor_id() can be overridden
+	 */
+	for (i = 0; i < cpuidx; i++) {
+		set_cpu_possible(i, true);
+		cpu_logical_map(i) = tmp_map[i];
+		pr_debug("cpu logical map 0x%x\n", cpu_logical_map(i));
+	}
+}
+
 /**
  * setup_machine_fdt - Machine setup when an dtb was passed to the kernel
  * @dt_phys: physical address of dt blob
-- 
1.7.12

Re: [PATCH v4 4/7] ARM: kernel: add device tree init map function

[Mark Rutland]

On Mon, Nov 19, 2012 at 12:45:03PM +0000, Lorenzo Pieralisi wrote:
> When booting through a device tree, the kernel cpu logical id map can be
> initialized using device tree data passed by FW or through an embedded blob.
> 
> This patch adds a function that parses device tree "cpu" nodes and
> retrieves the corresponding CPUs hardware identifiers (MPIDR).
> It sets the possible cpus and the cpu logical map values according to
> the number of CPUs defined in the device tree and respective properties.
> 
> The device tree HW identifiers are considered valid if all CPU nodes contain
> a "reg" property, there are no duplicate "reg" entries and the DT defines a
> CPU node whose "reg" property matches the MPIDR[23:0] of the boot CPU.
> 
> The primary CPU is assigned cpu logical number 0 to keep the current convention
> valid.
> 
> Current bindings documentation is included in the patch:
> 
> Documentation/devicetree/bindings/arm/cpus.txt
> 
> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> Acked-by: Nicolas Pitre <nico@linaro.org>
> ---
>  Documentation/devicetree/bindings/arm/cpus.txt |  77 +++++++++++++++++++
>  arch/arm/include/asm/prom.h                    |   2 +
>  arch/arm/kernel/devtree.c                      | 100 +++++++++++++++++++++++++
>  3 files changed, 179 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/arm/cpus.txt
> 
> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
> new file mode 100644
> index 0000000..46c3589
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
> @@ -0,0 +1,77 @@
> +* ARM CPUs binding description
> +
> +The device tree allows to describe the layout of CPUs in a system through
> +the "cpus" node, which in turn contains a number of subnodes (ie "cpu")
> +defining properties for every cpu.
> +
> +Bindings for CPU nodes follow the ePAPR standard, available from:
> +
> +http://devicetree.org
> +
> +For the ARM architecture every CPU node must contain the following properties:
> +
> +- device_type:	must be "cpu"
> +- reg:		property matching the CPU MPIDR[23:0] register bits
> +		reg[31:24] bits must be set to 0
> +- compatible:	should be one of:
> +		"arm,arm1020"
> +		"arm,arm1020e"
> +		"arm,arm1022"
> +		"arm,arm1026"
> +		"arm,arm720"
> +		"arm,arm740"
> +		"arm,arm7tdmi"
> +		"arm,arm920"
> +		"arm,arm922"
> +		"arm,arm925"
> +		"arm,arm926"
> +		"arm,arm940"
> +		"arm,arm946"
> +		"arm,arm9tdmi"
> +		"arm,cortex-a5"
> +		"arm,cortex-a7"
> +		"arm,cortex-a8"
> +		"arm,cortex-a9"
> +		"arm,cortex-a15"
> +		"arm,arm1136"
> +		"arm,arm1156"
> +		"arm,arm1176"
> +		"arm,arm11mpcore"
> +		"faraday,fa526"
> +		"intel,sa110"
> +		"intel,sa1100"
> +		"marvell,feroceon"
> +		"marvell,mohawk"
> +		"marvell,xsc3"
> +		"marvell,xscale"
> +
> +Example:
> +
> +	cpus {
> +		#size-cells = <0>;
> +		#address-cells = <1>;
> +
> +		CPU0: cpu@0 {
> +			device_type = "cpu";
> +			compatible = "arm, cortex-a15";
> +			reg = <0x0>;
> +		};
> +
> +		CPU1: cpu@1 {
> +			device_type = "cpu";
> +			compatible = "arm, cortex-a15";
> +			reg = <0x1>;
> +		};
> +
> +		CPU2: cpu@100 {
> +			device_type = "cpu";
> +			compatible = "arm, cortex-a7";
> +			reg = <0x100>;
> +		};
> +
> +		CPU3: cpu@101 {
> +			device_type = "cpu";
> +			compatible = "arm, cortex-a7";
> +			reg = <0x101>;
> +		};
> +	};

Those spaces in the compatible strings shouldn't be there.

Otherwise, looks good to me.

Thanks,
Mark

Re: [PATCH v4 4/7] ARM: kernel: add device tree init map function

[Lorenzo Pieralisi]

On Mon, Nov 19, 2012 at 01:54:18PM +0000, Mark Rutland wrote:
> On Mon, Nov 19, 2012 at 12:45:03PM +0000, Lorenzo Pieralisi wrote:
> > When booting through a device tree, the kernel cpu logical id map can be
> > initialized using device tree data passed by FW or through an embedded blob.
> > 
> > This patch adds a function that parses device tree "cpu" nodes and
> > retrieves the corresponding CPUs hardware identifiers (MPIDR).
> > It sets the possible cpus and the cpu logical map values according to
> > the number of CPUs defined in the device tree and respective properties.
> > 
> > The device tree HW identifiers are considered valid if all CPU nodes contain
> > a "reg" property, there are no duplicate "reg" entries and the DT defines a
> > CPU node whose "reg" property matches the MPIDR[23:0] of the boot CPU.
> > 
> > The primary CPU is assigned cpu logical number 0 to keep the current convention
> > valid.
> > 
> > Current bindings documentation is included in the patch:
> > 
> > Documentation/devicetree/bindings/arm/cpus.txt
> > 
> > Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> > Acked-by: Nicolas Pitre <nico@linaro.org>
> > ---
> >  Documentation/devicetree/bindings/arm/cpus.txt |  77 +++++++++++++++++++
> >  arch/arm/include/asm/prom.h                    |   2 +
> >  arch/arm/kernel/devtree.c                      | 100 +++++++++++++++++++++++++
> >  3 files changed, 179 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/arm/cpus.txt
> > 
> > diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
> > new file mode 100644
> > index 0000000..46c3589
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/arm/cpus.txt
> > @@ -0,0 +1,77 @@
> > +* ARM CPUs binding description
> > +
> > +The device tree allows to describe the layout of CPUs in a system through
> > +the "cpus" node, which in turn contains a number of subnodes (ie "cpu")
> > +defining properties for every cpu.
> > +
> > +Bindings for CPU nodes follow the ePAPR standard, available from:
> > +
> > +http://devicetree.org
> > +
> > +For the ARM architecture every CPU node must contain the following properties:
> > +
> > +- device_type:	must be "cpu"
> > +- reg:		property matching the CPU MPIDR[23:0] register bits
> > +		reg[31:24] bits must be set to 0
> > +- compatible:	should be one of:
> > +		"arm,arm1020"
> > +		"arm,arm1020e"
> > +		"arm,arm1022"
> > +		"arm,arm1026"
> > +		"arm,arm720"
> > +		"arm,arm740"
> > +		"arm,arm7tdmi"
> > +		"arm,arm920"
> > +		"arm,arm922"
> > +		"arm,arm925"
> > +		"arm,arm926"
> > +		"arm,arm940"
> > +		"arm,arm946"
> > +		"arm,arm9tdmi"
> > +		"arm,cortex-a5"
> > +		"arm,cortex-a7"
> > +		"arm,cortex-a8"
> > +		"arm,cortex-a9"
> > +		"arm,cortex-a15"
> > +		"arm,arm1136"
> > +		"arm,arm1156"
> > +		"arm,arm1176"
> > +		"arm,arm11mpcore"
> > +		"faraday,fa526"
> > +		"intel,sa110"
> > +		"intel,sa1100"
> > +		"marvell,feroceon"
> > +		"marvell,mohawk"
> > +		"marvell,xsc3"
> > +		"marvell,xscale"
> > +
> > +Example:
> > +
> > +	cpus {
> > +		#size-cells = <0>;
> > +		#address-cells = <1>;
> > +
> > +		CPU0: cpu@0 {
> > +			device_type = "cpu";
> > +			compatible = "arm, cortex-a15";
> > +			reg = <0x0>;
> > +		};
> > +
> > +		CPU1: cpu@1 {
> > +			device_type = "cpu";
> > +			compatible = "arm, cortex-a15";
> > +			reg = <0x1>;
> > +		};
> > +
> > +		CPU2: cpu@100 {
> > +			device_type = "cpu";
> > +			compatible = "arm, cortex-a7";
> > +			reg = <0x100>;
> > +		};
> > +
> > +		CPU3: cpu@101 {
> > +			device_type = "cpu";
> > +			compatible = "arm, cortex-a7";
> > +			reg = <0x101>;
> > +		};
> > +	};
> 
> Those spaces in the compatible strings shouldn't be there.

Thanks for your patience, I missed that, now fixed.

Thanks !
Lorenzo

[PATCH v4 5/7] ARM: kernel: add cpu logical map DT init in setup_arch

[Lorenzo Pieralisi]

As soon as the device tree is unflattened the cpu logical to physical
mapping is carried out in setup_arch to build a proper array of MPIDR and
corresponding logical indexes.

The mapping could have been carried out using the flattened DT blob and
related primitives, but since the mapping is not needed by early boot
code it can safely be executed when the device tree has been uncompressed to
its tree data structure.

This patch adds the arm_dt_init_cpu maps() function call in setup_arch().

If the kernel is not compiled with DT support the function is empty and
no logical mapping takes place through it; the mapping carried out in
smp_setup_processor_id() is left unchanged.
If DT is supported the mapping created in smp_setup_processor_id() is overriden.
The DT mapping also sets the possible cpus mask, hence platform
code need not set it again in the respective smp_init_cpus() functions.

Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Acked-by: Nicolas Pitre <nico@linaro.org>
---
 arch/arm/kernel/setup.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
index 4515bf6..d15f1c5 100644
--- a/arch/arm/kernel/setup.c
+++ b/arch/arm/kernel/setup.c
@@ -759,6 +759,7 @@ void __init setup_arch(char **cmdline_p)
 
 	unflatten_device_tree();
 
+	arm_dt_init_cpu_maps();
 #ifdef CONFIG_SMP
 	if (is_smp()) {
 		smp_set_ops(mdesc->smp);
-- 
1.7.12

[PATCH v4 6/7] ARM: kernel: add logical mappings look-up

[Lorenzo Pieralisi]

In ARM SMP systems the MPIDR register ([23:0] bits) is used to uniquely
identify CPUs.

In order to retrieve the logical CPU index corresponding to a given
MPIDR value and guarantee a consistent translation throughout the kernel,
this patch adds a look-up based on the MPIDR[23:0] so that kernel subsystems
can use it whenever the logical cpu index corresponding to a given MPIDR
value is needed.

Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Acked-by: Nicolas Pitre <nico@linaro.org>
---
 arch/arm/include/asm/smp_plat.h | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/arch/arm/include/asm/smp_plat.h b/arch/arm/include/asm/smp_plat.h
index 558d6c8..aaa61b6 100644
--- a/arch/arm/include/asm/smp_plat.h
+++ b/arch/arm/include/asm/smp_plat.h
@@ -5,6 +5,9 @@
 #ifndef __ASMARM_SMP_PLAT_H
 #define __ASMARM_SMP_PLAT_H
 
+#include <linux/cpumask.h>
+#include <linux/err.h>
+
 #include <asm/cputype.h>
 
 /*
@@ -48,5 +51,19 @@ static inline int cache_ops_need_broadcast(void)
  */
 extern int __cpu_logical_map[];
 #define cpu_logical_map(cpu)	__cpu_logical_map[cpu]
+/*
+ * Retrieve logical cpu index corresponding to a given MPIDR[23:0]
+ *  - mpidr: MPIDR[23:0] to be used for the look-up
+ *
+ * Returns the cpu logical index or -EINVAL on look-up error
+ */
+static inline int get_logical_index(u32 mpidr)
+{
+	int cpu;
+	for (cpu = 0; cpu < nr_cpu_ids; cpu++)
+		if (cpu_logical_map(cpu) == mpidr)
+			return cpu;
+	return -EINVAL;
+}
 
 #endif
-- 
1.7.12

[PATCH v4 7/7] ARM: gic: use a private mapping for CPU target interfaces

[Lorenzo Pieralisi]

From: Nicolas Pitre <nicolas.pitre@linaro.org>

The GIC interface numbering does not necessarily follow the logical
CPU numbering, especially for complex topologies such as multi-cluster
systems.

Fortunately we can easily probe the GIC to create a mapping as the
Interrupt Processor Targets Registers for the first 32 interrupts are
read-only, and each field returns a value that always corresponds to
the processor reading the register.

Initially all mappings target all CPUs in case an IPI is required to
boot secondary CPUs.  It is refined as those CPUs discover what their
actual mapping is.

Signed-off-by: Nicolas Pitre <nico@linaro.org>
Acked-by: Will Deacon <will.deacon@arm.com>
---
 arch/arm/common/gic.c | 45 ++++++++++++++++++++++++++++++++++++---------
 1 file changed, 36 insertions(+), 9 deletions(-)

diff --git a/arch/arm/common/gic.c b/arch/arm/common/gic.c
index 50c9eef..2203c92 100644
--- a/arch/arm/common/gic.c
+++ b/arch/arm/common/gic.c
@@ -70,6 +70,14 @@ struct gic_chip_data {
 static DEFINE_RAW_SPINLOCK(irq_controller_lock);
 
 /*
+ * The GIC mapping of CPU interfaces does not necessarily match
+ * the logical CPU numbering.  Let's use a mapping as returned
+ * by the GIC itself.
+ */
+#define NR_GIC_CPU_IF 8
+static u8 gic_cpu_map[NR_GIC_CPU_IF] __read_mostly;
+
+/*
  * Supported arch specific GIC irq extension.
  * Default make them NULL.
  */
@@ -238,11 +246,11 @@ static int gic_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
 	unsigned int cpu = cpumask_any_and(mask_val, cpu_online_mask);
 	u32 val, mask, bit;
 
-	if (cpu >= 8 || cpu >= nr_cpu_ids)
+	if (cpu >= NR_GIC_CPU_IF || cpu >= nr_cpu_ids)
 		return -EINVAL;
 
 	mask = 0xff << shift;
-	bit = 1 << (cpu_logical_map(cpu) + shift);
+	bit = gic_cpu_map[cpu] << shift;
 
 	raw_spin_lock(&irq_controller_lock);
 	val = readl_relaxed(reg) & ~mask;
@@ -349,11 +357,6 @@ static void __init gic_dist_init(struct gic_chip_data *gic)
 	u32 cpumask;
 	unsigned int gic_irqs = gic->gic_irqs;
 	void __iomem *base = gic_data_dist_base(gic);
-	u32 cpu = cpu_logical_map(smp_processor_id());
-
-	cpumask = 1 << cpu;
-	cpumask |= cpumask << 8;
-	cpumask |= cpumask << 16;
 
 	writel_relaxed(0, base + GIC_DIST_CTRL);
 
@@ -366,6 +369,7 @@ static void __init gic_dist_init(struct gic_chip_data *gic)
 	/*
 	 * Set all global interrupts to this CPU only.
 	 */
+	cpumask = readl_relaxed(base + GIC_DIST_TARGET + 0);
 	for (i = 32; i < gic_irqs; i += 4)
 		writel_relaxed(cpumask, base + GIC_DIST_TARGET + i * 4 / 4);
 
@@ -389,9 +393,25 @@ static void __cpuinit gic_cpu_init(struct gic_chip_data *gic)
 {
 	void __iomem *dist_base = gic_data_dist_base(gic);
 	void __iomem *base = gic_data_cpu_base(gic);
+	unsigned int cpu_mask, cpu = smp_processor_id();
 	int i;
 
 	/*
+	 * Get what the GIC says our CPU mask is.
+	 */
+	BUG_ON(cpu >= NR_GIC_CPU_IF);
+	cpu_mask = readl_relaxed(dist_base + GIC_DIST_TARGET + 0);
+	gic_cpu_map[cpu] = cpu_mask;
+
+	/*
+	 * Clear our mask from the other map entries in case they're
+	 * still undefined.
+	 */
+	for (i = 0; i < NR_GIC_CPU_IF; i++)
+		if (i != cpu)
+			gic_cpu_map[i] &= ~cpu_mask;
+
+	/*
 	 * Deal with the banked PPI and SGI interrupts - disable all
 	 * PPI interrupts, ensure all SGI interrupts are enabled.
 	 */
@@ -654,7 +674,7 @@ void __init gic_init_bases(unsigned int gic_nr, int irq_start,
 {
 	irq_hw_number_t hwirq_base;
 	struct gic_chip_data *gic;
-	int gic_irqs, irq_base;
+	int gic_irqs, irq_base, i;
 
 	BUG_ON(gic_nr >= MAX_GIC_NR);
 
@@ -692,6 +712,13 @@ void __init gic_init_bases(unsigned int gic_nr, int irq_start,
 	}
 
 	/*
+	 * Initialize the CPU interface map to all CPUs.
+	 * It will be refined as each CPU probes its ID.
+	 */
+	for (i = 0; i < NR_GIC_CPU_IF; i++)
+		gic_cpu_map[i] = 0xff;
+
+	/*
 	 * For primary GICs, skip over SGIs.
 	 * For secondary GICs, skip over PPIs, too.
 	 */
@@ -746,7 +773,7 @@ void gic_raise_softirq(const struct cpumask *mask, unsigned int irq)
 
 	/* Convert our logical CPU mask into a physical one. */
 	for_each_cpu(cpu, mask)
-		map |= 1 << cpu_logical_map(cpu);
+		map |= gic_cpu_map[cpu];
 
 	/*
 	 * Ensure that stores to Normal memory are visible to the
-- 
1.7.12