[PATCH v2 00/12] CPU idle for Armada XP

[PATCH v2 00/12] CPU idle for Armada XP

[Gregory CLEMENT]

Hello,

This patch set adds the CPU idle support for Armada XP and prepares
the support for Armada 370. This was based on the work of Nadav
Haklai.

Most of the patches modify the mvebu code in order to prepare the
support for CPU idle, hence the patches 2 to 10 should go to mvebu
subsystem (and then arm-soc).

The first patch should go through ARM subsystem and should be taken
by Russell King.

The 11th patch 'cpuidle: mvebu: Add initial cpu idle support for
Armada 370/XP SoC' is the only one who should go to the cpuidle
subsystem. But of course I would like that Daniel Lezcano or Rafael
J. Wysocki have a look on the whole series.

The last patch should also go to mvebu subsystem (and then arm-soc)
but with an Acked-by from on of the device tree maintainer.

The whole series is also available in the branch CPU-idle-ArmadaXP-v2
at https://github.com/MISL-EBU-System-SW/mainline-public.git

Thanks,

Changelog:

v1 -> v2:

* Removed the pm_level kernel parameter. As Kevin Hilman pointed, its
  usage can be replaced by using
  /sys/devices/system/cpu/cpu*/cpuidle/state*/disable or the kernel
  parameter cpuidle.off.

* Used BIT() macro (reported by Ezequiel)

* Made the function more readable the
  armada_370_xp_pmsu_idle_prepare() function (reported by Thomas)

* Moved the config entry in Kconfig.arm, and rename the config symbol
  according the pattern used by other arm cpu: ARM_"soc name"_CPUIDLE

* Moved the build rule under the new ARM SoC section in the Makefile

* Rebased on Linus Torvalds master branch of Thursday September 12

Gregory CLEMENT (12):
  ARM: PJ4B: Add cpu_suspend/cpu_resume hooks for PJ4B
  ARM: mvebu: ll_set_cpu_coherent no more uses the coherency address as
    parameter
  ARM: mvebu: ll_set_cpu_coherent always uses the current CPU
  ARM: mvebu: Remove the unused argument of set_cpu_coherent()
  ARM: mvebu: Make ll_set_cpu_coherent() more configurable
  ARM: mvebu: Low level functions to disable cache snooping
  ARM: mvebu: Add a new set of registers for pmsu
  ARM: mvebu: Allow to power down L2 cache controller in idle mode
  ARM: mvebu: Add the PMSU related part of the cpu idle functions
  ARM: mvebu: Set the start address of a CPU in a separate function
  cpuidle: mvebu: Add initial CPU idle support for Armada 370/XP SoC
  ARM: dts: mvebu: Add a new set of registers to the PMSU node

 .../devicetree/bindings/arm/armada-370-xp-pmsu.txt |  12 ++-
 arch/arm/boot/dts/armada-xp.dtsi                   |   2 +-
 arch/arm/mach-mvebu/coherency.c                    |  12 +--
 arch/arm/mach-mvebu/coherency.h                    |   2 +-
 arch/arm/mach-mvebu/coherency_ll.S                 |  69 +++++++++++---
 arch/arm/mach-mvebu/headsmp.S                      |  15 +--
 arch/arm/mach-mvebu/platsmp.c                      |   2 +-
 arch/arm/mach-mvebu/pmsu.c                         | 102 +++++++++++++++++++-
 arch/arm/mm/proc-v7.S                              |  64 ++++++++++++-
 drivers/cpuidle/Kconfig.arm                        |   5 +
 drivers/cpuidle/Makefile                           |   1 +
 drivers/cpuidle/cpuidle-armada-370-xp.c            | 103 +++++++++++++++++++++
 drivers/cpuidle/suspend-armada-370-xp.S            |  91 ++++++++++++++++++
 include/linux/armada-370-xp-pmsu.h                 |  19 ++++
 14 files changed, 457 insertions(+), 42 deletions(-)
 create mode 100644 drivers/cpuidle/cpuidle-armada-370-xp.c
 create mode 100644 drivers/cpuidle/suspend-armada-370-xp.S
 create mode 100644 include/linux/armada-370-xp-pmsu.h

-- 
1.8.1.2

[PATCH v2 01/12] ARM: PJ4B: Add cpu_suspend/cpu_resume hooks for PJ4B

[Gregory CLEMENT]

PJ4B needs extra instructions for suspend and resume, so instead of
using the armv7 version, this commit introduces specific versions for
PJ4B.

Cc: Russell King <linux@arm.linux.org.uk>
Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
---
 arch/arm/mm/proc-v7.S | 64 ++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 61 insertions(+), 3 deletions(-)

diff --git a/arch/arm/mm/proc-v7.S b/arch/arm/mm/proc-v7.S
index c63d9bd..b14801b 100644
--- a/arch/arm/mm/proc-v7.S
+++ b/arch/arm/mm/proc-v7.S
@@ -162,9 +162,67 @@ ENDPROC(cpu_pj4b_do_idle)
 	globl_equ	cpu_pj4b_do_idle,  	cpu_v7_do_idle
 #endif
 	globl_equ	cpu_pj4b_dcache_clean_area,	cpu_v7_dcache_clean_area
-	globl_equ	cpu_pj4b_do_suspend,	cpu_v7_do_suspend
-	globl_equ	cpu_pj4b_do_resume,	cpu_v7_do_resume
-	globl_equ	cpu_pj4b_suspend_size,	cpu_v7_suspend_size
+#ifdef CONFIG_ARM_CPU_SUSPEND
+ENTRY(cpu_pj4b_do_suspend)
+	stmfd	sp!, {r4 - r10, lr}
+	mrc	p15, 0, r4, c13, c0, 0	@ FCSE/PID
+	mrc	p15, 0, r5, c13, c0, 3	@ User r/o thread ID
+	stmia	r0!, {r4 - r5}
+	mrc	p15, 1, r6, c15, c1, 0  @ save CP15 - extra features
+	mrc	p15, 1, r7, c15, c2, 0	@ save CP15 - Aux Func Modes Ctrl 0
+	mrc	p15, 1, r8, c15, c1, 2	@ save CP15 - Aux Debug Modes Ctrl 2
+	mrc	p15, 1, r9, c15, c1, 1  @ save CP15 - Aux Debug Modes Ctrl 1
+	mrc	p15, 0, r10, c9, c14, 0  @ save CP15 - PMC
+	stmia	r0!, {r6 - r10}
+	mrc	p15, 0, r6, c3, c0, 0	@ Domain ID
+	mrc	p15, 0, r7, c2, c0, 1	@ TTB 1
+	mrc	p15, 0, r11, c2, c0, 2	@ TTB control register
+	mrc	p15, 0, r8, c1, c0, 0	@ Control register
+	mrc	p15, 0, r9, c1, c0, 1	@ Auxiliary control register
+	mrc	p15, 0, r10, c1, c0, 2	@ Co-processor access control
+	stmia	r0, {r6 - r11}
+	ldmfd	sp!, {r4 - r10, pc}
+ENDPROC(cpu_pj4b_do_suspend)
+
+ENTRY(cpu_pj4b_do_resume)
+	mov	ip, #0
+	mcr	p15, 0, ip, c7, c5, 0	@ invalidate I cache
+	mcr	p15, 0, ip, c13, c0, 1	@ set reserved context ID
+	ldmia	r0!, {r4 - r5}
+	mcr	p15, 0, r4, c13, c0, 0	@ FCSE/PID
+	mcr	p15, 0, r5, c13, c0, 3	@ User r/o thread ID
+	ldmia	r0!, {r6 - r10}
+	mcr	p15, 1, r6, c15, c1, 0  @ save CP15 - extra features
+	mcr	p15, 1, r7, c15, c2, 0	@ save CP15 - Aux Func Modes Ctrl 0
+	mcr	p15, 1, r8, c15, c1, 2	@ save CP15 - Aux Debug Modes Ctrl 2
+	mcr	p15, 1, r9, c15, c1, 1  @ save CP15 - Aux Debug Modes Ctrl 1
+	mcr	p15, 0, r10, c9, c14, 0  @ save CP15 - PMC
+	ldmia	r0, {r6 - r11}
+	mcr	p15, 0, ip, c8, c7, 0	@ invalidate TLBs
+	mcr	p15, 0, r6, c3, c0, 0	@ Domain ID
+#ifndef CONFIG_ARM_LPAE
+	ALT_SMP(orr	r1, r1, #TTB_FLAGS_SMP)
+	ALT_UP(orr	r1, r1, #TTB_FLAGS_UP)
+#endif
+	mcr	p15, 0, r1, c2, c0, 0	@ TTB 0
+	mcr	p15, 0, r7, c2, c0, 1	@ TTB 1
+	mcr	p15, 0, r11, c2, c0, 2	@ TTB control register
+	ldr	r4, =PRRR		@ PRRR
+	ldr	r5, =NMRR		@ NMRR
+	mcr	p15, 0, r4, c10, c2, 0	@ write PRRR
+	mcr	p15, 0, r5, c10, c2, 1	@ write NMRR
+	mrc	p15, 0, r4, c1, c0, 1	@ Read Auxiliary control register
+	teq	r4, r9			@ Is it already set?
+	mcrne	p15, 0, r9, c1, c0, 1	@ No, so write it
+	mcr	p15, 0, r10, c1, c0, 2	@ Co-processor access control
+	isb
+	dsb
+	mov	r0, r8			@ control register
+	b	cpu_resume_mmu
+ENDPROC(cpu_pj4b_do_resume)
+#endif
+.globl	cpu_pj4b_suspend_size
+.equ	cpu_pj4b_suspend_size, 4 * 13
 
 #endif
 
-- 
1.8.1.2

[PATCH v2 02/12] ARM: mvebu: ll_set_cpu_coherent no more uses the coherency address as parameter

[Gregory CLEMENT]

Until now the calling functions of ll_set_cpu_coherent() have to know
the base address of the coherency registers. This commit doesn't
expose anymore this address, and replace the argument by a flag to
indicate if we have to use the physical or the virtual address.

Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
---
 arch/arm/mach-mvebu/coherency.c    |  6 +++---
 arch/arm/mach-mvebu/coherency_ll.S | 18 +++++++++++++++++-
 arch/arm/mach-mvebu/headsmp.S      | 10 ++--------
 3 files changed, 22 insertions(+), 12 deletions(-)

diff --git a/arch/arm/mach-mvebu/coherency.c b/arch/arm/mach-mvebu/coherency.c
index 4c24303..8e9f7b7 100644
--- a/arch/arm/mach-mvebu/coherency.c
+++ b/arch/arm/mach-mvebu/coherency.c
@@ -29,7 +29,7 @@
 #include "armada-370-xp.h"
 
 unsigned long coherency_phys_base;
-static void __iomem *coherency_base;
+void __iomem *coherency_base;
 static void __iomem *coherency_cpu_base;
 
 /* Coherency fabric registers */
@@ -43,7 +43,7 @@ static struct of_device_id of_coherency_table[] = {
 };
 
 /* Function defined in coherency_ll.S */
-int ll_set_cpu_coherent(void __iomem *base_addr, unsigned int hw_cpu_id);
+int ll_set_cpu_coherent(bool use_virt_addr, unsigned int hw_cpu_id);
 
 int set_cpu_coherent(unsigned int hw_cpu_id, int smp_group_id)
 {
@@ -53,7 +53,7 @@ int set_cpu_coherent(unsigned int hw_cpu_id, int smp_group_id)
 		return 1;
 	}
 
-	return ll_set_cpu_coherent(coherency_base, hw_cpu_id);
+	return ll_set_cpu_coherent(true, hw_cpu_id);
 }
 
 static inline void mvebu_hwcc_sync_io_barrier(void)
diff --git a/arch/arm/mach-mvebu/coherency_ll.S b/arch/arm/mach-mvebu/coherency_ll.S
index 5476669..8d4e22f 100644
--- a/arch/arm/mach-mvebu/coherency_ll.S
+++ b/arch/arm/mach-mvebu/coherency_ll.S
@@ -22,10 +22,22 @@
 
 	.text
 /*
- * r0: Coherency fabric base register address
+ * r0: if r0==0 => physical addres, else virtual address
  * r1: HW CPU id
  */
 ENTRY(ll_set_cpu_coherent)
+	cmp	r0, #0
+	bne	1f
+    /* use physical address */
+	adr	r0, 3f
+	ldr	r3, [r0]
+	ldr	r0, [r0, r3]
+	b	2f
+1:
+    /* use virtual address */
+	ldr	r0, =(coherency_base)
+	ldr	r0, [r0]
+2:
 	/* Create bit by cpu index */
 	mov	r3, #(1 << 24)
 	lsl	r1, r3, r1
@@ -53,3 +65,7 @@ ENTRY(ll_set_cpu_coherent)
 	mov	r0, #0
 	mov	pc, lr
 ENDPROC(ll_set_cpu_coherent)
+
+	.align 2
+3:
+	.long	coherency_phys_base - .
diff --git a/arch/arm/mach-mvebu/headsmp.S b/arch/arm/mach-mvebu/headsmp.S
index 8a1b0c9..bffaabc 100644
--- a/arch/arm/mach-mvebu/headsmp.S
+++ b/arch/arm/mach-mvebu/headsmp.S
@@ -27,10 +27,8 @@
  * startup
  */
 ENTRY(armada_xp_secondary_startup)
-	/* Get coherency fabric base physical address */
-	adr	r0, 1f
-	ldr	r1, [r0]
-	ldr	r0, [r0, r1]
+	/* Use physical addrss */
+	mov	r0, #0
 
 	/* Read CPU id */
 	mrc     p15, 0, r1, c0, c0, 5
@@ -41,7 +39,3 @@ ENTRY(armada_xp_secondary_startup)
 	b	secondary_startup
 
 ENDPROC(armada_xp_secondary_startup)
-
-	.align 2
-1:
-	.long	coherency_phys_base - .
-- 
1.8.1.2

[PATCH v2 03/12] ARM: mvebu: ll_set_cpu_coherent always uses the current CPU

[Gregory CLEMENT]

ll_set_cpu_coherent is always used on the current CPU, so instead of
passing the CPU id as argument, ll_set_cpu_coherent() can find it by
itself.

Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
---
 arch/arm/mach-mvebu/coherency.c    | 10 +++++-----
 arch/arm/mach-mvebu/coherency.h    |  2 +-
 arch/arm/mach-mvebu/coherency_ll.S |  7 ++++---
 arch/arm/mach-mvebu/headsmp.S      |  4 ----
 arch/arm/mach-mvebu/platsmp.c      |  2 +-
 5 files changed, 11 insertions(+), 14 deletions(-)

diff --git a/arch/arm/mach-mvebu/coherency.c b/arch/arm/mach-mvebu/coherency.c
index 8e9f7b7..e48518e 100644
--- a/arch/arm/mach-mvebu/coherency.c
+++ b/arch/arm/mach-mvebu/coherency.c
@@ -43,17 +43,17 @@ static struct of_device_id of_coherency_table[] = {
 };
 
 /* Function defined in coherency_ll.S */
-int ll_set_cpu_coherent(bool use_virt_addr, unsigned int hw_cpu_id);
+int ll_set_cpu_coherent(bool use_virt_addr);
 
-int set_cpu_coherent(unsigned int hw_cpu_id, int smp_group_id)
+int set_cpu_coherent(int smp_group_id)
 {
 	if (!coherency_base) {
-		pr_warn("Can't make CPU %d cache coherent.\n", hw_cpu_id);
+		pr_warn("Can't make current CPU cache coherent.\n");
 		pr_warn("Coherency fabric is not initialized\n");
 		return 1;
 	}
 
-	return ll_set_cpu_coherent(true, hw_cpu_id);
+	return ll_set_cpu_coherent(true);
 }
 
 static inline void mvebu_hwcc_sync_io_barrier(void)
@@ -139,7 +139,7 @@ int __init coherency_init(void)
 		sync_cache_w(&coherency_phys_base);
 		coherency_base = of_iomap(np, 0);
 		coherency_cpu_base = of_iomap(np, 1);
-		set_cpu_coherent(cpu_logical_map(smp_processor_id()), 0);
+		set_cpu_coherent(0);
 	}
 
 	return 0;
diff --git a/arch/arm/mach-mvebu/coherency.h b/arch/arm/mach-mvebu/coherency.h
index df33ad8..d4bc067 100644
--- a/arch/arm/mach-mvebu/coherency.h
+++ b/arch/arm/mach-mvebu/coherency.h
@@ -14,7 +14,7 @@
 #ifndef __MACH_370_XP_COHERENCY_H
 #define __MACH_370_XP_COHERENCY_H
 
-int set_cpu_coherent(int cpu_id, int smp_group_id);
+int set_cpu_coherent(int smp_group_id);
 int coherency_init(void);
 
 #endif	/* __MACH_370_XP_COHERENCY_H */
diff --git a/arch/arm/mach-mvebu/coherency_ll.S b/arch/arm/mach-mvebu/coherency_ll.S
index 8d4e22f..fc2e6f7 100644
--- a/arch/arm/mach-mvebu/coherency_ll.S
+++ b/arch/arm/mach-mvebu/coherency_ll.S
@@ -23,7 +23,6 @@
 	.text
 /*
  * r0: if r0==0 => physical addres, else virtual address
- * r1: HW CPU id
  */
 ENTRY(ll_set_cpu_coherent)
 	cmp	r0, #0
@@ -39,8 +38,10 @@ ENTRY(ll_set_cpu_coherent)
 	ldr	r0, [r0]
 2:
 	/* Create bit by cpu index */
-	mov	r3, #(1 << 24)
-	lsl	r1, r3, r1
+	mrc	15, 0, r1, cr0, cr0, 5
+	and	r1, r1, #15
+	mov	r2, #(1 << 24)
+	lsl	r1, r2, r1
 
 	/* Add CPU to SMP group - Atomic */
 	add	r3, r0, #ARMADA_XP_CFB_CTL_REG_OFFSET
diff --git a/arch/arm/mach-mvebu/headsmp.S b/arch/arm/mach-mvebu/headsmp.S
index bffaabc..b2c6e95 100644
--- a/arch/arm/mach-mvebu/headsmp.S
+++ b/arch/arm/mach-mvebu/headsmp.S
@@ -30,10 +30,6 @@ ENTRY(armada_xp_secondary_startup)
 	/* Use physical addrss */
 	mov	r0, #0
 
-	/* Read CPU id */
-	mrc     p15, 0, r1, c0, c0, 5
-	and     r1, r1, #0xF
-
 	/* Add CPU to coherency fabric */
 	bl	ll_set_cpu_coherent
 	b	secondary_startup
diff --git a/arch/arm/mach-mvebu/platsmp.c b/arch/arm/mach-mvebu/platsmp.c
index ff69c2d..1b86e03 100644
--- a/arch/arm/mach-mvebu/platsmp.c
+++ b/arch/arm/mach-mvebu/platsmp.c
@@ -102,7 +102,7 @@ void __init armada_xp_smp_prepare_cpus(unsigned int max_cpus)
 
 	set_secondary_cpus_clock();
 	flush_cache_all();
-	set_cpu_coherent(cpu_logical_map(smp_processor_id()), 0);
+	set_cpu_coherent(0);
 
 	/*
 	 * In order to boot the secondary CPUs we need to ensure
-- 
1.8.1.2

[PATCH v2 04/12] ARM: mvebu: Remove the unused argument of set_cpu_coherent()

[Gregory CLEMENT]

set_cpu_coherent() took the SMP group ID as parameter. But this
parameter was never used, and the CPU always use the SMP group 0. So
we can remove this parameter.

Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
---
 arch/arm/mach-mvebu/coherency.c | 4 ++--
 arch/arm/mach-mvebu/coherency.h | 2 +-
 arch/arm/mach-mvebu/platsmp.c   | 2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/arm/mach-mvebu/coherency.c b/arch/arm/mach-mvebu/coherency.c
index e48518e..a2b6366 100644
--- a/arch/arm/mach-mvebu/coherency.c
+++ b/arch/arm/mach-mvebu/coherency.c
@@ -45,7 +45,7 @@ static struct of_device_id of_coherency_table[] = {
 /* Function defined in coherency_ll.S */
 int ll_set_cpu_coherent(bool use_virt_addr);
 
-int set_cpu_coherent(int smp_group_id)
+int set_cpu_coherent(void)
 {
 	if (!coherency_base) {
 		pr_warn("Can't make current CPU cache coherent.\n");
@@ -139,7 +139,7 @@ int __init coherency_init(void)
 		sync_cache_w(&coherency_phys_base);
 		coherency_base = of_iomap(np, 0);
 		coherency_cpu_base = of_iomap(np, 1);
-		set_cpu_coherent(0);
+		set_cpu_coherent();
 	}
 
 	return 0;
diff --git a/arch/arm/mach-mvebu/coherency.h b/arch/arm/mach-mvebu/coherency.h
index d4bc067..700376e 100644
--- a/arch/arm/mach-mvebu/coherency.h
+++ b/arch/arm/mach-mvebu/coherency.h
@@ -14,7 +14,7 @@
 #ifndef __MACH_370_XP_COHERENCY_H
 #define __MACH_370_XP_COHERENCY_H
 
-int set_cpu_coherent(int smp_group_id);
+int set_cpu_coherent(void);
 int coherency_init(void);
 
 #endif	/* __MACH_370_XP_COHERENCY_H */
diff --git a/arch/arm/mach-mvebu/platsmp.c b/arch/arm/mach-mvebu/platsmp.c
index 1b86e03..a0fe97e 100644
--- a/arch/arm/mach-mvebu/platsmp.c
+++ b/arch/arm/mach-mvebu/platsmp.c
@@ -102,7 +102,7 @@ void __init armada_xp_smp_prepare_cpus(unsigned int max_cpus)
 
 	set_secondary_cpus_clock();
 	flush_cache_all();
-	set_cpu_coherent(0);
+	set_cpu_coherent();
 
 	/*
 	 * In order to boot the secondary CPUs we need to ensure
-- 
1.8.1.2

[PATCH v2 05/12] ARM: mvebu: Make ll_set_cpu_coherent() more configurable

[Gregory CLEMENT]

ll_set_cpu_coherent does two things to set the coherency for a CPU:

- Adding the CPU to the SMP group 0
- Enabling the snooping on the CPU

Only the second part is needed when coming back from a CPU Idle. This
commit allows to choose to do the first part or not.

Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
---
 arch/arm/mach-mvebu/coherency.c    |  4 ++--
 arch/arm/mach-mvebu/coherency_ll.S | 30 ++++++++++++++++++------------
 arch/arm/mach-mvebu/headsmp.S      |  3 +++
 3 files changed, 23 insertions(+), 14 deletions(-)

diff --git a/arch/arm/mach-mvebu/coherency.c b/arch/arm/mach-mvebu/coherency.c
index a2b6366..2f934f1 100644
--- a/arch/arm/mach-mvebu/coherency.c
+++ b/arch/arm/mach-mvebu/coherency.c
@@ -43,7 +43,7 @@ static struct of_device_id of_coherency_table[] = {
 };
 
 /* Function defined in coherency_ll.S */
-int ll_set_cpu_coherent(bool use_virt_addr);
+int ll_set_cpu_coherent(bool use_virt_addr, bool use_smp_group);
 
 int set_cpu_coherent(void)
 {
@@ -53,7 +53,7 @@ int set_cpu_coherent(void)
 		return 1;
 	}
 
-	return ll_set_cpu_coherent(true);
+	return ll_set_cpu_coherent(true, true);
 }
 
 static inline void mvebu_hwcc_sync_io_barrier(void)
diff --git a/arch/arm/mach-mvebu/coherency_ll.S b/arch/arm/mach-mvebu/coherency_ll.S
index fc2e6f7..1526b94 100644
--- a/arch/arm/mach-mvebu/coherency_ll.S
+++ b/arch/arm/mach-mvebu/coherency_ll.S
@@ -22,7 +22,8 @@
 
 	.text
 /*
- * r0: if r0==0 => physical addres, else virtual address
+ * r0: Use virtual address if r0 != 0, else physical address
+ * r1: Add CPU to the SMP group if r1 != 0
  */
 ENTRY(ll_set_cpu_coherent)
 	cmp	r0, #0
@@ -38,26 +39,31 @@ ENTRY(ll_set_cpu_coherent)
 	ldr	r0, [r0]
 2:
 	/* Create bit by cpu index */
-	mrc	15, 0, r1, cr0, cr0, 5
-	and	r1, r1, #15
+	mrc	15, 0, r3, cr0, cr0, 5
+	and	r3, r3, #15
 	mov	r2, #(1 << 24)
-	lsl	r1, r2, r1
+	lsl	r3, r2, r3
+
+	/* If r1=!0 then just enable the snooping */
+	cmp	r1, #0
+	beq	2f
 
 	/* Add CPU to SMP group - Atomic */
-	add	r3, r0, #ARMADA_XP_CFB_CTL_REG_OFFSET
+	add	r1, r0, #ARMADA_XP_CFB_CTL_REG_OFFSET
 1:
-	ldrex	r2, [r3]
-	orr	r2, r2, r1
-	strex 	r0, r2, [r3]
+	ldrex	r2, [r1]
+	orr	r2, r2, r3
+	strex 	r0, r2, [r1]
 	cmp	r0, #0
 	bne 1b
 
+2:
 	/* Enable coherency on CPU - Atomic */
-	add	r3, r3, #ARMADA_XP_CFB_CFG_REG_OFFSET
+	add	r1, r1, #ARMADA_XP_CFB_CFG_REG_OFFSET
 1:
-	ldrex	r2, [r3]
-	orr	r2, r2, r1
-	strex	r0, r2, [r3]
+	ldrex	r2, [r1]
+	orr	r2, r2, r3
+	strex	r0, r2, [r1]
 	cmp	r0, #0
 	bne 1b
 
diff --git a/arch/arm/mach-mvebu/headsmp.S b/arch/arm/mach-mvebu/headsmp.S
index b2c6e95..f2732c8 100644
--- a/arch/arm/mach-mvebu/headsmp.S
+++ b/arch/arm/mach-mvebu/headsmp.S
@@ -30,6 +30,9 @@ ENTRY(armada_xp_secondary_startup)
 	/* Use physical addrss */
 	mov	r0, #0
 
+	/* Add CPU to the SMP group*/
+	mov	r1, #1
+
 	/* Add CPU to coherency fabric */
 	bl	ll_set_cpu_coherent
 	b	secondary_startup
-- 
1.8.1.2

[PATCH v2 06/12] ARM: mvebu: Low level functions to disable cache snooping

[Gregory CLEMENT]

When going to deep idle we need to disable the SoC snooping by
"hand". Playing with the coherency fabric requires to use assembly
code to be sure that the compiler doesn't reorder the instructions nor
do wrong optimization.

This function will be called by the low level (in assembly) part of
the CPU idle functions.

Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
---
 arch/arm/mach-mvebu/coherency_ll.S | 24 +++++++++++++++++++++++-
 1 file changed, 23 insertions(+), 1 deletion(-)

diff --git a/arch/arm/mach-mvebu/coherency_ll.S b/arch/arm/mach-mvebu/coherency_ll.S
index 1526b94..3fb426e 100644
--- a/arch/arm/mach-mvebu/coherency_ll.S
+++ b/arch/arm/mach-mvebu/coherency_ll.S
@@ -73,6 +73,28 @@ ENTRY(ll_set_cpu_coherent)
 	mov	pc, lr
 ENDPROC(ll_set_cpu_coherent)
 
+/*
+ * r0: if r0==0 => physical addres, else virtual address
+ */
+ENTRY(armada_370_xp_disable_snoop_ena)
+	ldr	r0, =(coherency_base)
+	ldr	r0, [r0]
+	/* Enable SnoopEna - Exclusive */
+	mrc	15, 0, r1, cr0, cr0, 5
+	and	r1, r1, #15
+	mov	r2, #(1 << 24)
+	lsl	r2, r2, r1
+
+1:
+	ldrex	r1, [r0]
+	bic	r1, r1, r2
+	strex	r3, r1, [r0]
+	cmp	r3, #0
+	bne 1b
+
+	mov pc, lr
+ENDPROC(armada_370_xp_disable_snoop_ena)
+
 	.align 2
 3:
-	.long	coherency_phys_base - .
+		.long	coherency_phys_base - .
-- 
1.8.1.2

[PATCH v2 07/12] ARM: mvebu: Add a new set of registers for pmsu

[Gregory CLEMENT]

The Power Management Unit Service block also controls the Coherency
Fabric subsystem. This new set of registers is needed for the CPU idle
implementation for the Armada 370/XP, it allows to enter a deep CPU
idle state where the Coherency Fabric and the L2 cache are power down.

This patch also adds warnings if one of the base registers set can't
be ioremapped.

Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
---
 arch/arm/mach-mvebu/pmsu.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/arch/arm/mach-mvebu/pmsu.c b/arch/arm/mach-mvebu/pmsu.c
index 3cc4bef..1d147fe 100644
--- a/arch/arm/mach-mvebu/pmsu.c
+++ b/arch/arm/mach-mvebu/pmsu.c
@@ -25,6 +25,7 @@
 
 static void __iomem *pmsu_mp_base;
 static void __iomem *pmsu_reset_base;
+static void __iomem *pmsu_fabric_base;
 
 #define PMSU_BOOT_ADDR_REDIRECT_OFFSET(cpu)	((cpu * 0x100) + 0x24)
 #define PMSU_RESET_CTL_OFFSET(cpu)		(cpu * 0x8)
@@ -66,7 +67,11 @@ int __init armada_370_xp_pmsu_init(void)
 	if (np) {
 		pr_info("Initializing Power Management Service Unit\n");
 		pmsu_mp_base = of_iomap(np, 0);
+		WARN_ON(!pmsu_mp_base);
 		pmsu_reset_base = of_iomap(np, 1);
+		WARN_ON(!pmsu_reset_base);
+		pmsu_fabric_base = of_iomap(np, 2);
+		WARN_ON(!pmsu_fabric_base);
 	}
 
 	return 0;
-- 
1.8.1.2

[PATCH v2 08/12] ARM: mvebu: Allow to power down L2 cache controller in idle mode

[Gregory CLEMENT]

This commit adds an exported function which allows to power down the
L2 cache controller and the Coherency Fabric when the CPU goes into
deep idle mode.

This feature is part of the Power Management Service Unit of the
Armada 370 and Armada XP SoCs.

Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
---
 arch/arm/mach-mvebu/pmsu.c         | 14 ++++++++++++++
 include/linux/armada-370-xp-pmsu.h | 16 ++++++++++++++++
 2 files changed, 30 insertions(+)
 create mode 100644 include/linux/armada-370-xp-pmsu.h

diff --git a/arch/arm/mach-mvebu/pmsu.c b/arch/arm/mach-mvebu/pmsu.c
index 1d147fe..fc3374d 100644
--- a/arch/arm/mach-mvebu/pmsu.c
+++ b/arch/arm/mach-mvebu/pmsu.c
@@ -30,6 +30,9 @@ static void __iomem *pmsu_fabric_base;
 #define PMSU_BOOT_ADDR_REDIRECT_OFFSET(cpu)	((cpu * 0x100) + 0x24)
 #define PMSU_RESET_CTL_OFFSET(cpu)		(cpu * 0x8)
 
+#define L2C_NFABRIC_PM_CTL		    0x4
+#define L2C_NFABRIC_PM_CTL_PWR_DOWN	    BIT(20)
+
 static struct of_device_id of_pmsu_table[] = {
 	{.compatible = "marvell,armada-370-xp-pmsu"},
 	{ /* end of list */ },
@@ -77,4 +80,15 @@ int __init armada_370_xp_pmsu_init(void)
 	return 0;
 }
 
+void armada_370_xp_pmsu_enable_l2_powerdown_onidle(void)
+{
+	int reg;
+
+	/* Enable L2 & Fabric powerdown in Deep-Idle mode - Fabric */
+	reg = readl(pmsu_fabric_base + L2C_NFABRIC_PM_CTL);
+	reg |= L2C_NFABRIC_PM_CTL_PWR_DOWN;
+	writel(reg, pmsu_fabric_base + L2C_NFABRIC_PM_CTL);
+}
+EXPORT_SYMBOL_GPL(armada_370_xp_pmsu_enable_l2_powerdown_onidle);
+
 early_initcall(armada_370_xp_pmsu_init);
diff --git a/include/linux/armada-370-xp-pmsu.h b/include/linux/armada-370-xp-pmsu.h
new file mode 100644
index 0000000..f85cbf7
--- /dev/null
+++ b/include/linux/armada-370-xp-pmsu.h
@@ -0,0 +1,16 @@
+/*
+ * Power Management Service Unit (PMSU) support for Armada 370/XP platforms.
+ *
+ * Copyright (C) 2013 Marvell
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#ifndef __ARMADA_370_XP__PMSU_H
+#define __ARMADA_370_XP__PMSU_H
+
+void armada_370_xp_pmsu_enable_l2_powerdown_onidle(void);
+
+#endif	/* __ARMADA_370_XP__PMSU_H */
-- 
1.8.1.2

[PATCH v2 09/12] ARM: mvebu: Add the PMSU related part of the cpu idle functions

[Gregory CLEMENT]

The cpu idle support will need to access to the Power Management
Service Unit. This commit adds and exports the prepare and restore
functions that will be used by the CPU idle.

Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
---
 arch/arm/mach-mvebu/pmsu.c         | 73 ++++++++++++++++++++++++++++++++++++++
 include/linux/armada-370-xp-pmsu.h |  2 ++
 2 files changed, 75 insertions(+)

diff --git a/arch/arm/mach-mvebu/pmsu.c b/arch/arm/mach-mvebu/pmsu.c
index fc3374d..29ea8c5 100644
--- a/arch/arm/mach-mvebu/pmsu.c
+++ b/arch/arm/mach-mvebu/pmsu.c
@@ -30,6 +30,24 @@ static void __iomem *pmsu_fabric_base;
 #define PMSU_BOOT_ADDR_REDIRECT_OFFSET(cpu)	((cpu * 0x100) + 0x24)
 #define PMSU_RESET_CTL_OFFSET(cpu)		(cpu * 0x8)
 
+#define PM_CONTROL_AND_CONFIG(cpu)	((cpu * 0x100) + 0x4)
+#define PM_CONTROL_AND_CONFIG_DFS_REQ		BIT(18)
+#define PM_CONTROL_AND_CONFIG_PWDDN_REQ	BIT(16)
+#define PM_CONTROL_AND_CONFIG_L2_PWDDN		BIT(20)
+
+#define PM_CPU_POWER_DOWN_CONTROL(cpu)	((cpu * 0x100) + 0x8)
+
+#define PM_CPU_POWER_DOWN_DIS_SNP_Q_SKIP	BIT(0)
+
+#define PM_STATUS_AND_MASK(cpu)	((cpu * 0x100) + 0xc)
+#define PM_STATUS_AND_MASK_CPU_IDLE_WAIT	BIT(16)
+#define PM_STATUS_AND_MASK_SNP_Q_EMPTY_WAIT	BIT(17)
+#define PM_STATUS_AND_MASK_IRQ_WAKEUP		BIT(20)
+#define PM_STATUS_AND_MASK_FIQ_WAKEUP		BIT(21)
+#define PM_STATUS_AND_MASK_DBG_WAKEUP		BIT(22)
+#define PM_STATUS_AND_MASK_IRQ_MASK		BIT(24)
+#define PM_STATUS_AND_MASK_FIQ_MASK		BIT(25)
+
 #define L2C_NFABRIC_PM_CTL		    0x4
 #define L2C_NFABRIC_PM_CTL_PWR_DOWN	    BIT(20)
 
@@ -91,4 +109,59 @@ void armada_370_xp_pmsu_enable_l2_powerdown_onidle(void)
 }
 EXPORT_SYMBOL_GPL(armada_370_xp_pmsu_enable_l2_powerdown_onidle);
 
+void armada_370_xp_pmsu_idle_prepare(bool deepidle)
+{
+	unsigned int hw_cpu = cpu_logical_map(smp_processor_id());
+	int reg;
+	/*
+	 * Adjust the PMSU configuration to wait for WFI signal, enable
+	 * IRQ and FIQ as wakeup events, set wait for snoop queue empty
+	 * indication and mask IRQ and FIQ from CPU
+	 */
+	reg = readl(pmsu_mp_base + PM_STATUS_AND_MASK(hw_cpu));
+	reg |= PM_STATUS_AND_MASK_CPU_IDLE_WAIT    |
+	       PM_STATUS_AND_MASK_IRQ_WAKEUP       |
+	       PM_STATUS_AND_MASK_FIQ_WAKEUP       |
+	       PM_STATUS_AND_MASK_SNP_Q_EMPTY_WAIT |
+	       PM_STATUS_AND_MASK_IRQ_MASK         |
+	       PM_STATUS_AND_MASK_FIQ_MASK;
+	writel(reg, pmsu_mp_base + PM_STATUS_AND_MASK(hw_cpu));
+
+	reg = readl(pmsu_mp_base + PM_CONTROL_AND_CONFIG(hw_cpu));
+	/* ask HW to power down the L2 Cache if needed */
+	if (deepidle)
+		reg |= PM_CONTROL_AND_CONFIG_L2_PWDDN;
+
+	/* request power down */
+	reg |= PM_CONTROL_AND_CONFIG_PWDDN_REQ;
+	writel(reg, pmsu_mp_base + PM_CONTROL_AND_CONFIG(hw_cpu));
+
+	/* Disable snoop disable by HW - SW is taking care of it */
+	reg = readl(pmsu_mp_base + PM_CPU_POWER_DOWN_CONTROL(hw_cpu));
+	reg |= PM_CPU_POWER_DOWN_DIS_SNP_Q_SKIP;
+	writel(reg, pmsu_mp_base + PM_CPU_POWER_DOWN_CONTROL(hw_cpu));
+}
+EXPORT_SYMBOL_GPL(armada_370_xp_pmsu_idle_prepare);
+
+noinline void armada_370_xp_pmsu_idle_restore(void)
+{
+	unsigned int hw_cpu = cpu_logical_map(smp_processor_id());
+	int reg;
+
+	/* cancel ask HW to power down the L2 Cache if possible */
+	reg = readl(pmsu_mp_base + PM_CONTROL_AND_CONFIG(hw_cpu));
+	reg &= ~PM_CONTROL_AND_CONFIG_L2_PWDDN;
+	writel(reg, pmsu_mp_base + PM_CONTROL_AND_CONFIG(hw_cpu));
+
+	/* cancel Enable wakeup events */
+	reg = readl(pmsu_mp_base + PM_STATUS_AND_MASK(hw_cpu));
+	reg &= ~(PM_STATUS_AND_MASK_IRQ_WAKEUP | PM_STATUS_AND_MASK_FIQ_WAKEUP);
+	reg &= ~PM_STATUS_AND_MASK_CPU_IDLE_WAIT;
+	reg &= ~PM_STATUS_AND_MASK_SNP_Q_EMPTY_WAIT;
+	/* Mask interrupts */
+	reg &= ~(PM_STATUS_AND_MASK_IRQ_MASK | PM_STATUS_AND_MASK_FIQ_MASK);
+	writel(reg, pmsu_mp_base + PM_STATUS_AND_MASK(hw_cpu));
+}
+EXPORT_SYMBOL_GPL(armada_370_xp_pmsu_idle_restore);
+
 early_initcall(armada_370_xp_pmsu_init);
diff --git a/include/linux/armada-370-xp-pmsu.h b/include/linux/armada-370-xp-pmsu.h
index f85cbf7..235a40c 100644
--- a/include/linux/armada-370-xp-pmsu.h
+++ b/include/linux/armada-370-xp-pmsu.h
@@ -12,5 +12,7 @@
 #define __ARMADA_370_XP__PMSU_H
 
 void armada_370_xp_pmsu_enable_l2_powerdown_onidle(void);
+void armada_370_xp_pmsu_idle_prepare(bool deepidle);
+void armada_370_xp_pmsu_idle_restore(void);
 
 #endif	/* __ARMADA_370_XP__PMSU_H */
-- 
1.8.1.2

[PATCH v2 10/12] ARM: mvebu: Set the start address of a CPU in a separate function

[Gregory CLEMENT]

Setting the start (or boot) address of A CPU is no more used only
during SMP bring up, but will also be used by the CPU idle functions
or later by the CPU hot plug ones.

This commit moves it in a separate function and exports it.

Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
---
 arch/arm/mach-mvebu/pmsu.c         | 10 ++++++++--
 include/linux/armada-370-xp-pmsu.h |  1 +
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/arch/arm/mach-mvebu/pmsu.c b/arch/arm/mach-mvebu/pmsu.c
index 29ea8c5..5abca96 100644
--- a/arch/arm/mach-mvebu/pmsu.c
+++ b/arch/arm/mach-mvebu/pmsu.c
@@ -56,6 +56,13 @@ static struct of_device_id of_pmsu_table[] = {
 	{ /* end of list */ },
 };
 
+void armada_370_xp_pmsu_set_start_addr(void *start_addr, int hw_cpu)
+{
+	writel(virt_to_phys(start_addr), pmsu_mp_base +
+		PMSU_BOOT_ADDR_REDIRECT_OFFSET(hw_cpu));
+}
+EXPORT_SYMBOL_GPL(armada_370_xp_pmsu_set_start_addr);
+
 #ifdef CONFIG_SMP
 int armada_xp_boot_cpu(unsigned int cpu_id, void *boot_addr)
 {
@@ -68,8 +75,7 @@ int armada_xp_boot_cpu(unsigned int cpu_id, void *boot_addr)
 
 	hw_cpu = cpu_logical_map(cpu_id);
 
-	writel(virt_to_phys(boot_addr), pmsu_mp_base +
-			PMSU_BOOT_ADDR_REDIRECT_OFFSET(hw_cpu));
+	armada_370_xp_pmsu_set_start_addr(boot_addr, hw_cpu);
 
 	/* Release CPU from reset by clearing reset bit*/
 	reg = readl(pmsu_reset_base + PMSU_RESET_CTL_OFFSET(hw_cpu));
diff --git a/include/linux/armada-370-xp-pmsu.h b/include/linux/armada-370-xp-pmsu.h
index 235a40c..e09977b 100644
--- a/include/linux/armada-370-xp-pmsu.h
+++ b/include/linux/armada-370-xp-pmsu.h
@@ -14,5 +14,6 @@
 void armada_370_xp_pmsu_enable_l2_powerdown_onidle(void);
 void armada_370_xp_pmsu_idle_prepare(bool deepidle);
 void armada_370_xp_pmsu_idle_restore(void);
+void armada_370_xp_pmsu_set_start_addr(void *start_addr, int hw_cpu);
 
 #endif	/* __ARMADA_370_XP__PMSU_H */
-- 
1.8.1.2

[PATCH v2 11/12] cpuidle: mvebu: Add initial CPU idle support for Armada 370/XP SoC

[Gregory CLEMENT]

Add the wfi, cpu idle and cpu deep idle power states support for the
Armada XP SoCs.

All the latencies and the power consumption values used at the
"armada_370_xp_idle_driver" structure are preliminary and will be
modified in the future after running some measurements and analysis.

Based on the work of Nadav Haklai.

Signed-off-by: Nadav Haklai <nadavh@marvell.com>
Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
---
 drivers/cpuidle/Kconfig.arm             |   5 ++
 drivers/cpuidle/Makefile                |   1 +
 drivers/cpuidle/cpuidle-armada-370-xp.c | 103 ++++++++++++++++++++++++++++++++
 drivers/cpuidle/suspend-armada-370-xp.S |  91 ++++++++++++++++++++++++++++
 4 files changed, 200 insertions(+)
 create mode 100644 drivers/cpuidle/cpuidle-armada-370-xp.c
 create mode 100644 drivers/cpuidle/suspend-armada-370-xp.S

diff --git a/drivers/cpuidle/Kconfig.arm b/drivers/cpuidle/Kconfig.arm
index 8e36603..071d960 100644
--- a/drivers/cpuidle/Kconfig.arm
+++ b/drivers/cpuidle/Kconfig.arm
@@ -1,6 +1,11 @@
 #
 # ARM CPU Idle drivers
 #
+config ARM_ARMADA_370_XP_CPUIDLE
+	bool "CPU Idle Driver for Armada 370/XP family processors"
+	depends on ARCH_MVEBU
+	help
+	  Select this to enable cpuidle on Armada 370/XP processors.
 
 config ARM_HIGHBANK_CPUIDLE
 	bool "CPU Idle Driver for Calxeda processors"
diff --git a/drivers/cpuidle/Makefile b/drivers/cpuidle/Makefile
index cea5ef5..90fc4a6 100644
--- a/drivers/cpuidle/Makefile
+++ b/drivers/cpuidle/Makefile
@@ -7,6 +7,7 @@ obj-$(CONFIG_ARCH_NEEDS_CPU_IDLE_COUPLED) += coupled.o
 
 ##################################################################################
 # ARM SoC drivers
+obj-$(CONFIG_ARM_ARMADA_370_XP_CPUIDLE) += cpuidle-armada-370-xp.o suspend-armada-370-xp.o
 obj-$(CONFIG_ARM_HIGHBANK_CPUIDLE)	+= cpuidle-calxeda.o
 obj-$(CONFIG_ARM_KIRKWOOD_CPUIDLE)	+= cpuidle-kirkwood.o
 obj-$(CONFIG_ARM_ZYNQ_CPUIDLE)		+= cpuidle-zynq.o
diff --git a/drivers/cpuidle/cpuidle-armada-370-xp.c b/drivers/cpuidle/cpuidle-armada-370-xp.c
new file mode 100644
index 0000000..7c78d92
--- /dev/null
+++ b/drivers/cpuidle/cpuidle-armada-370-xp.c
@@ -0,0 +1,103 @@
+/*
+ * Marvell Armada 370 and Armada XP SoC cpuidle driver
+ *
+ * Copyright (C) 2013 Marvell
+ *
+ * Nadav Haklai <nadavh@marvell.com>
+ * Gregory CLEMENT <gregory.clement@free-electrons.com>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ *
+ * Maintainer: Gregory CLEMENT <gregory.clement@free-electrons.com>
+ */
+
+#include <linux/cpuidle.h>
+#include <linux/module.h>
+#include <linux/io.h>
+#include <linux/of.h>
+#include <linux/suspend.h>
+#include <asm/suspend.h>
+#include <linux/smp.h>
+#include <asm/cpuidle.h>
+#include <asm/smp_plat.h>
+#include <linux/armada-370-xp-pmsu.h>
+
+#define ARMADA_370_XP_MAX_STATES	3
+
+enum mv_pm_states {
+	WFI = 0,
+	MV_CPU_IDLE,
+	MV_CPU_DEEP_IDLE,
+};
+
+extern void v7_flush_dcache_all(void);
+
+/* Functions defined in suspend-armada-370-xp.S */
+int armada_370_xp_cpu_resume(unsigned long);
+int armada_370_xp_cpu_suspend(unsigned long);
+
+static int armada_370_xp_enter_idle(struct cpuidle_device *dev,
+				struct cpuidle_driver *drv,
+				int index)
+{
+	bool deepidle = false;
+	unsigned int hw_cpu = cpu_logical_map(smp_processor_id());
+
+	armada_370_xp_pmsu_set_start_addr(armada_370_xp_cpu_resume, hw_cpu);
+
+	if (index == MV_CPU_DEEP_IDLE)
+		deepidle = true;
+
+	cpu_suspend(deepidle, armada_370_xp_cpu_suspend);
+
+	cpu_init();
+
+	armada_370_xp_pmsu_idle_restore();
+
+	return index;
+}
+
+static struct cpuidle_driver armada_370_xp_idle_driver = {
+	.name			= "armada_370_xp_idle",
+	.states[0]		= ARM_CPUIDLE_WFI_STATE,
+	.states[1]		= {
+		.enter			= armada_370_xp_enter_idle,
+		.exit_latency		= 10,
+		.power_usage		= 50,
+		.target_residency	= 100,
+		.flags			= CPUIDLE_FLAG_TIME_VALID,
+		.name			= "MV CPU IDLE",
+		.desc			= "CPU power down",
+	},
+	.states[2]		= {
+		.enter			= armada_370_xp_enter_idle,
+		.exit_latency		= 100,
+		.power_usage		= 5,
+		.target_residency	= 1000,
+		.flags			= CPUIDLE_FLAG_TIME_VALID,
+		.name			= "MV CPU DEEP IDLE",
+		.desc			= "CPU and L2 Fabric power down",
+	},
+	.state_count = ARMADA_370_XP_MAX_STATES,
+};
+
+static int __init armada_370_xp_cpuidle_init(void)
+{
+	if (!of_find_compatible_node(NULL, NULL, "marvell,armada-370-xp-pmsu"))
+		return -ENODEV;
+
+	if (!of_find_compatible_node(NULL, NULL, "marvell,coherency-fabric"))
+		return -ENODEV;
+
+	pr_info("Initializing Armada-XP CPU power management ");
+
+	armada_370_xp_pmsu_enable_l2_powerdown_onidle();
+
+	return cpuidle_register(&armada_370_xp_idle_driver, NULL);
+}
+
+module_init(armada_370_xp_cpuidle_init);
+
+MODULE_LICENSE("GPL");
diff --git a/drivers/cpuidle/suspend-armada-370-xp.S b/drivers/cpuidle/suspend-armada-370-xp.S
new file mode 100644
index 0000000..490bb9b
--- /dev/null
+++ b/drivers/cpuidle/suspend-armada-370-xp.S
@@ -0,0 +1,91 @@
+/*
+ * CPU idle low level implementation for Marvell Armada 370 and Armada XP SoCs
+ *
+ * Copyright (C) 2013 Marvell
+ *
+ * Nadav Haklai <nadavh@marvell.com>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ *
+ */
+#include <linux/linkage.h>
+
+
+/*
+* armadaxp_cpu_suspend: enter cpu deepIdle state
+* input:
+*/
+ENTRY(armada_370_xp_cpu_suspend)
+/* Save ARM registers */
+	stmfd	sp!, {r4 - r11, lr}	@ save registers on stack
+
+	bl armada_370_xp_pmsu_idle_prepare
+	/*
+	 * Invalidate L1 data cache. Even though only invalidate is
+	 * necessary exported flush API is used here. Doing clean
+	 * on already clean cache would be almost NOP.
+	 */
+	bl v7_flush_dcache_all
+
+	/*
+	 * Clear the SCTLR.C bit to prevent further data cache
+	 * allocation. Clearing SCTLR.C would make all the data accesses
+	 * strongly ordered and would not hit the cache.
+	 */
+	mrc	p15, 0, r0, c1, c0, 0
+	bic	r0, r0, #(1 << 2)	@ Disable the C bit
+	mcr	p15, 0, r0, c1, c0, 0
+	isb
+
+	bl v7_flush_dcache_all
+
+	/* Data memory barrier and Data sync barrier */
+	dsb
+	dmb
+
+	bl armada_370_xp_disable_snoop_ena
+
+	dsb				@ Data Synchronization Barrier
+
+/*
+ * ===================================
+ * == WFI instruction => Enter idle ==
+ * ===================================
+ */
+
+	wfi				@ wait for interrupt
+/*
+ * ===================================
+ * == Resume path for non-OFF modes ==
+ * ===================================
+ */
+
+/* Enable SnoopEna - Exclusive */
+	mov	r0, #1			@ r0!=0 means use virtual address
+	mov	r1, #0			@ Do not add CPU to SMP group
+	bl ll_set_cpu_coherent
+
+/* Re-enable C-bit if needed */
+	mrc	p15, 0, r0, c1, c0, 0
+	tst	r0, #(1 << 2)		@ Check C bit enabled?
+	orreq	r0, r0, #(1 << 2)	@ Enable the C bit if cleared
+	mcreq	p15, 0, r0, c1, c0, 0
+	isb
+
+	ldmfd	sp!, {r4 - r11, pc}	@ restore regs and return
+ENDPROC(armada_370_xp_cpu_suspend)
+
+/*
+* armada_370_xp_cpu_resume: exit cpu deepIdle state
+*/
+ENTRY(armada_370_xp_cpu_resume)
+	mov	r0, #0			@ r0==0 means use physical address
+	mov	r1, #1			@ Add CPU to SMP group
+	bl ll_set_cpu_coherent
+
+	/* Now branch to the common CPU resume function */
+	b	cpu_resume
+
+ENDPROC(armada_370_xp_cpu_resume)
-- 
1.8.1.2

[PATCH v2 11/12] cpuidle: mvebu: Add initial CPU idle support for Armada 370/XP SoC

[Daniel Lezcano]

On 09/13/2013 12:06 PM, Gregory CLEMENT wrote:
> Add the wfi, cpu idle and cpu deep idle power states support for the
> Armada XP SoCs.
> 
> All the latencies and the power consumption values used at the
> "armada_370_xp_idle_driver" structure are preliminary and will be
> modified in the future after running some measurements and analysis.
> 
> Based on the work of Nadav Haklai.
> 
> Signed-off-by: Nadav Haklai <nadavh@marvell.com>
> Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
> ---
>  drivers/cpuidle/Kconfig.arm             |   5 ++
>  drivers/cpuidle/Makefile                |   1 +
>  drivers/cpuidle/cpuidle-armada-370-xp.c | 103 ++++++++++++++++++++++++++++++++
>  drivers/cpuidle/suspend-armada-370-xp.S |  91 ++++++++++++++++++++++++++++

Somehow, you will have to move "suspend-armada-370-xp.S" into arch/arm.

>  4 files changed, 200 insertions(+)
>  create mode 100644 drivers/cpuidle/cpuidle-armada-370-xp.c
>  create mode 100644 drivers/cpuidle/suspend-armada-370-xp.S
> 
> diff --git a/drivers/cpuidle/Kconfig.arm b/drivers/cpuidle/Kconfig.arm
> index 8e36603..071d960 100644
> --- a/drivers/cpuidle/Kconfig.arm
> +++ b/drivers/cpuidle/Kconfig.arm
> @@ -1,6 +1,11 @@
>  #
>  # ARM CPU Idle drivers
>  #
> +config ARM_ARMADA_370_XP_CPUIDLE
> +	bool "CPU Idle Driver for Armada 370/XP family processors"
> +	depends on ARCH_MVEBU
> +	help
> +	  Select this to enable cpuidle on Armada 370/XP processors.
>  
>  config ARM_HIGHBANK_CPUIDLE
>  	bool "CPU Idle Driver for Calxeda processors"
> diff --git a/drivers/cpuidle/Makefile b/drivers/cpuidle/Makefile
> index cea5ef5..90fc4a6 100644
> --- a/drivers/cpuidle/Makefile
> +++ b/drivers/cpuidle/Makefile
> @@ -7,6 +7,7 @@ obj-$(CONFIG_ARCH_NEEDS_CPU_IDLE_COUPLED) += coupled.o
>  
>  ##################################################################################
>  # ARM SoC drivers
> +obj-$(CONFIG_ARM_ARMADA_370_XP_CPUIDLE) += cpuidle-armada-370-xp.o suspend-armada-370-xp.o
>  obj-$(CONFIG_ARM_HIGHBANK_CPUIDLE)	+= cpuidle-calxeda.o
>  obj-$(CONFIG_ARM_KIRKWOOD_CPUIDLE)	+= cpuidle-kirkwood.o
>  obj-$(CONFIG_ARM_ZYNQ_CPUIDLE)		+= cpuidle-zynq.o
> diff --git a/drivers/cpuidle/cpuidle-armada-370-xp.c b/drivers/cpuidle/cpuidle-armada-370-xp.c
> new file mode 100644
> index 0000000..7c78d92
> --- /dev/null
> +++ b/drivers/cpuidle/cpuidle-armada-370-xp.c
> @@ -0,0 +1,103 @@
> +/*
> + * Marvell Armada 370 and Armada XP SoC cpuidle driver
> + *
> + * Copyright (C) 2013 Marvell
> + *
> + * Nadav Haklai <nadavh@marvell.com>
> + * Gregory CLEMENT <gregory.clement@free-electrons.com>
> + *
> + * This file is licensed under the terms of the GNU General Public
> + * License version 2.  This program is licensed "as is" without any
> + * warranty of any kind, whether express or implied.
> + *
> + * Maintainer: Gregory CLEMENT <gregory.clement@free-electrons.com>
> + */
> +
> +#include <linux/cpuidle.h>
> +#include <linux/module.h>
> +#include <linux/io.h>
> +#include <linux/of.h>
> +#include <linux/suspend.h>
> +#include <asm/suspend.h>
> +#include <linux/smp.h>
> +#include <asm/cpuidle.h>
> +#include <asm/smp_plat.h>
> +#include <linux/armada-370-xp-pmsu.h>
> +
> +#define ARMADA_370_XP_MAX_STATES	3
> +
> +enum mv_pm_states {
> +	WFI = 0,
> +	MV_CPU_IDLE,
> +	MV_CPU_DEEP_IDLE,
> +};
> +
> +extern void v7_flush_dcache_all(void);
> +
> +/* Functions defined in suspend-armada-370-xp.S */
> +int armada_370_xp_cpu_resume(unsigned long);
> +int armada_370_xp_cpu_suspend(unsigned long);
> +
> +static int armada_370_xp_enter_idle(struct cpuidle_device *dev,
> +				struct cpuidle_driver *drv,
> +				int index)
> +{
> +	bool deepidle = false;
> +	unsigned int hw_cpu = cpu_logical_map(smp_processor_id());
> +
> +	armada_370_xp_pmsu_set_start_addr(armada_370_xp_cpu_resume, hw_cpu);
> +
> +	if (index == MV_CPU_DEEP_IDLE)
> +		deepidle = true;

That looks a bit hackish no ? :)

Can't you use the CPUIDLE_DRIVER_FLAGS_MASK to store this information ?

> +	cpu_suspend(deepidle, armada_370_xp_cpu_suspend);
> +
> +	cpu_init();
> +
> +	armada_370_xp_pmsu_idle_restore();
> +
> +	return index;
> +}
> +
> +static struct cpuidle_driver armada_370_xp_idle_driver = {
> +	.name			= "armada_370_xp_idle",
> +	.states[0]		= ARM_CPUIDLE_WFI_STATE,
> +	.states[1]		= {
> +		.enter			= armada_370_xp_enter_idle,
> +		.exit_latency		= 10,
> +		.power_usage		= 50,
> +		.target_residency	= 100,
> +		.flags			= CPUIDLE_FLAG_TIME_VALID,
> +		.name			= "MV CPU IDLE",
> +		.desc			= "CPU power down",
> +	},
> +	.states[2]		= {
> +		.enter			= armada_370_xp_enter_idle,
> +		.exit_latency		= 100,
> +		.power_usage		= 5,
> +		.target_residency	= 1000,
> +		.flags			= CPUIDLE_FLAG_TIME_VALID,
> +		.name			= "MV CPU DEEP IDLE",
> +		.desc			= "CPU and L2 Fabric power down",
> +	},
> +	.state_count = ARMADA_370_XP_MAX_STATES,
> +};

What about the local timers ? Are they shutdown ?

> +
> +static int __init armada_370_xp_cpuidle_init(void)
> +{
> +	if (!of_find_compatible_node(NULL, NULL, "marvell,armada-370-xp-pmsu"))
> +		return -ENODEV;
> +
> +	if (!of_find_compatible_node(NULL, NULL, "marvell,coherency-fabric"))
> +		return -ENODEV;
> +
> +	pr_info("Initializing Armada-XP CPU power management ");
> +
> +	armada_370_xp_pmsu_enable_l2_powerdown_onidle();
> +
> +	return cpuidle_register(&armada_370_xp_idle_driver, NULL);
> +}
> +
> +module_init(armada_370_xp_cpuidle_init);

Isn't it possible to replace it by module_platform_driver ? like ux500
or kirkwood ?

Thanks
  -- Daniel

> +MODULE_LICENSE("GPL");

[ ... ]


-- 
 <http://www.linaro.org/> Linaro.org ? Open source software for ARM SoCs

Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog

[PATCH v2 11/12] cpuidle: mvebu: Add initial CPU idle support for Armada 370/XP SoC

[Gregory CLEMENT]

Hi Daniel,

thanks for you review,

On 13/09/2013 17:36, Daniel Lezcano wrote:
> On 09/13/2013 12:06 PM, Gregory CLEMENT wrote:
>> Add the wfi, cpu idle and cpu deep idle power states support for the
>> Armada XP SoCs.
>>
>> All the latencies and the power consumption values used at the
>> "armada_370_xp_idle_driver" structure are preliminary and will be
>> modified in the future after running some measurements and analysis.
>>
>> Based on the work of Nadav Haklai.
>>
>> Signed-off-by: Nadav Haklai <nadavh@marvell.com>
>> Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
>> ---
>>  drivers/cpuidle/Kconfig.arm             |   5 ++
>>  drivers/cpuidle/Makefile                |   1 +
>>  drivers/cpuidle/cpuidle-armada-370-xp.c | 103 ++++++++++++++++++++++++++++++++
>>  drivers/cpuidle/suspend-armada-370-xp.S |  91 ++++++++++++++++++++++++++++
> 
> Somehow, you will have to move "suspend-armada-370-xp.S" into arch/arm.

Does it mean that you want I move it for the next version?

> 
>>  4 files changed, 200 insertions(+)
>>  create mode 100644 drivers/cpuidle/cpuidle-armada-370-xp.c
>>  create mode 100644 drivers/cpuidle/suspend-armada-370-xp.S
>>
>> diff --git a/drivers/cpuidle/Kconfig.arm b/drivers/cpuidle/Kconfig.arm
>> index 8e36603..071d960 100644
>> --- a/drivers/cpuidle/Kconfig.arm
>> +++ b/drivers/cpuidle/Kconfig.arm
>> @@ -1,6 +1,11 @@
>>  #
>>  # ARM CPU Idle drivers
>>  #
>> +config ARM_ARMADA_370_XP_CPUIDLE
>> +	bool "CPU Idle Driver for Armada 370/XP family processors"
>> +	depends on ARCH_MVEBU
>> +	help
>> +	  Select this to enable cpuidle on Armada 370/XP processors.
>>  
>>  config ARM_HIGHBANK_CPUIDLE
>>  	bool "CPU Idle Driver for Calxeda processors"
>> diff --git a/drivers/cpuidle/Makefile b/drivers/cpuidle/Makefile
>> index cea5ef5..90fc4a6 100644
>> --- a/drivers/cpuidle/Makefile
>> +++ b/drivers/cpuidle/Makefile
>> @@ -7,6 +7,7 @@ obj-$(CONFIG_ARCH_NEEDS_CPU_IDLE_COUPLED) += coupled.o
>>  
>>  ##################################################################################
>>  # ARM SoC drivers
>> +obj-$(CONFIG_ARM_ARMADA_370_XP_CPUIDLE) += cpuidle-armada-370-xp.o suspend-armada-370-xp.o
>>  obj-$(CONFIG_ARM_HIGHBANK_CPUIDLE)	+= cpuidle-calxeda.o
>>  obj-$(CONFIG_ARM_KIRKWOOD_CPUIDLE)	+= cpuidle-kirkwood.o
>>  obj-$(CONFIG_ARM_ZYNQ_CPUIDLE)		+= cpuidle-zynq.o
>> diff --git a/drivers/cpuidle/cpuidle-armada-370-xp.c b/drivers/cpuidle/cpuidle-armada-370-xp.c
>> new file mode 100644
>> index 0000000..7c78d92
>> --- /dev/null
>> +++ b/drivers/cpuidle/cpuidle-armada-370-xp.c
>> @@ -0,0 +1,103 @@
>> +/*
>> + * Marvell Armada 370 and Armada XP SoC cpuidle driver
>> + *
>> + * Copyright (C) 2013 Marvell
>> + *
>> + * Nadav Haklai <nadavh@marvell.com>
>> + * Gregory CLEMENT <gregory.clement@free-electrons.com>
>> + *
>> + * This file is licensed under the terms of the GNU General Public
>> + * License version 2.  This program is licensed "as is" without any
>> + * warranty of any kind, whether express or implied.
>> + *
>> + * Maintainer: Gregory CLEMENT <gregory.clement@free-electrons.com>
>> + */
>> +
>> +#include <linux/cpuidle.h>
>> +#include <linux/module.h>
>> +#include <linux/io.h>
>> +#include <linux/of.h>
>> +#include <linux/suspend.h>
>> +#include <asm/suspend.h>
>> +#include <linux/smp.h>
>> +#include <asm/cpuidle.h>
>> +#include <asm/smp_plat.h>
>> +#include <linux/armada-370-xp-pmsu.h>
>> +
>> +#define ARMADA_370_XP_MAX_STATES	3
>> +
>> +enum mv_pm_states {
>> +	WFI = 0,
>> +	MV_CPU_IDLE,
>> +	MV_CPU_DEEP_IDLE,
>> +};
>> +
>> +extern void v7_flush_dcache_all(void);
>> +
>> +/* Functions defined in suspend-armada-370-xp.S */
>> +int armada_370_xp_cpu_resume(unsigned long);
>> +int armada_370_xp_cpu_suspend(unsigned long);
>> +
>> +static int armada_370_xp_enter_idle(struct cpuidle_device *dev,
>> +				struct cpuidle_driver *drv,
>> +				int index)
>> +{
>> +	bool deepidle = false;
>> +	unsigned int hw_cpu = cpu_logical_map(smp_processor_id());
>> +
>> +	armada_370_xp_pmsu_set_start_addr(armada_370_xp_cpu_resume, hw_cpu);
>> +
>> +	if (index == MV_CPU_DEEP_IDLE)
>> +		deepidle = true;
> 
> That looks a bit hackish no ? :)
> 
> Can't you use the CPUIDLE_DRIVER_FLAGS_MASK to store this information ?

I didn't notice this mask until now but indeed we can use it.

> 
>> +	cpu_suspend(deepidle, armada_370_xp_cpu_suspend);
>> +
>> +	cpu_init();
>> +
>> +	armada_370_xp_pmsu_idle_restore();
>> +
>> +	return index;
>> +}
>> +
>> +static struct cpuidle_driver armada_370_xp_idle_driver = {
>> +	.name			= "armada_370_xp_idle",
>> +	.states[0]		= ARM_CPUIDLE_WFI_STATE,
>> +	.states[1]		= {
>> +		.enter			= armada_370_xp_enter_idle,
>> +		.exit_latency		= 10,
>> +		.power_usage		= 50,
>> +		.target_residency	= 100,
>> +		.flags			= CPUIDLE_FLAG_TIME_VALID,
>> +		.name			= "MV CPU IDLE",
>> +		.desc			= "CPU power down",
>> +	},
>> +	.states[2]		= {
>> +		.enter			= armada_370_xp_enter_idle,
>> +		.exit_latency		= 100,
>> +		.power_usage		= 5,
>> +		.target_residency	= 1000,
>> +		.flags			= CPUIDLE_FLAG_TIME_VALID,
>> +		.name			= "MV CPU DEEP IDLE",
>> +		.desc			= "CPU and L2 Fabric power down",
>> +	},
>> +	.state_count = ARMADA_370_XP_MAX_STATES,
>> +};
> 
> What about the local timers ? Are they shutdown ?

I need to chekc it.

> 
>> +
>> +static int __init armada_370_xp_cpuidle_init(void)
>> +{
>> +	if (!of_find_compatible_node(NULL, NULL, "marvell,armada-370-xp-pmsu"))
>> +		return -ENODEV;
>> +
>> +	if (!of_find_compatible_node(NULL, NULL, "marvell,coherency-fabric"))
>> +		return -ENODEV;
>> +
>> +	pr_info("Initializing Armada-XP CPU power management ");
>> +
>> +	armada_370_xp_pmsu_enable_l2_powerdown_onidle();
>> +
>> +	return cpuidle_register(&armada_370_xp_idle_driver, NULL);
>> +}
>> +
>> +module_init(armada_370_xp_cpuidle_init);
> 
> Isn't it possible to replace it by module_platform_driver ? like ux500
> or kirkwood ?

It should be possible indeed, I will check it.

> 
> Thanks
>   -- Daniel
> 
>> +MODULE_LICENSE("GPL");
> 
> [ ... ]
> 
> 


-- 
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[PATCH v2 11/12] cpuidle: mvebu: Add initial CPU idle support for Armada 370/XP SoC

[Daniel Lezcano]

On 09/15/2013 04:34 PM, Gregory CLEMENT wrote:
> Hi Daniel,
> 
> thanks for you review,
> 
> On 13/09/2013 17:36, Daniel Lezcano wrote:
>> On 09/13/2013 12:06 PM, Gregory CLEMENT wrote:
>>> Add the wfi, cpu idle and cpu deep idle power states support for the
>>> Armada XP SoCs.
>>>
>>> All the latencies and the power consumption values used at the
>>> "armada_370_xp_idle_driver" structure are preliminary and will be
>>> modified in the future after running some measurements and analysis.
>>>
>>> Based on the work of Nadav Haklai.
>>>
>>> Signed-off-by: Nadav Haklai <nadavh@marvell.com>
>>> Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
>>> ---
>>>  drivers/cpuidle/Kconfig.arm             |   5 ++
>>>  drivers/cpuidle/Makefile                |   1 +
>>>  drivers/cpuidle/cpuidle-armada-370-xp.c | 103 ++++++++++++++++++++++++++++++++
>>>  drivers/cpuidle/suspend-armada-370-xp.S |  91 ++++++++++++++++++++++++++++
>>
>> Somehow, you will have to move "suspend-armada-370-xp.S" into arch/arm.
> 
> Does it mean that you want I move it for the next version?

Yes please.

[ ... ]

>>> +static struct cpuidle_driver armada_370_xp_idle_driver = {
>>> +	.name			= "armada_370_xp_idle",
>>> +	.states[0]		= ARM_CPUIDLE_WFI_STATE,
>>> +	.states[1]		= {
>>> +		.enter			= armada_370_xp_enter_idle,
>>> +		.exit_latency		= 10,
>>> +		.power_usage		= 50,
>>> +		.target_residency	= 100,
>>> +		.flags			= CPUIDLE_FLAG_TIME_VALID,
>>> +		.name			= "MV CPU IDLE",
>>> +		.desc			= "CPU power down",
>>> +	},
>>> +	.states[2]		= {
>>> +		.enter			= armada_370_xp_enter_idle,
>>> +		.exit_latency		= 100,
>>> +		.power_usage		= 5,
>>> +		.target_residency	= 1000,
>>> +		.flags			= CPUIDLE_FLAG_TIME_VALID,
>>> +		.name			= "MV CPU DEEP IDLE",
>>> +		.desc			= "CPU and L2 Fabric power down",
>>> +	},
>>> +	.state_count = ARMADA_370_XP_MAX_STATES,
>>> +};
>>
>> What about the local timers ? Are they shutdown ?
> 
> I need to chekc it.

Ok, if it is the case, there is the flag CPUIDLE_FLAG_TIMER_STOP to tell
the cpuidle framework to switch to the broadcast timer with this state.

>>> +static int __init armada_370_xp_cpuidle_init(void)
>>> +{
>>> +	if (!of_find_compatible_node(NULL, NULL, "marvell,armada-370-xp-pmsu"))
>>> +		return -ENODEV;
>>> +
>>> +	if (!of_find_compatible_node(NULL, NULL, "marvell,coherency-fabric"))
>>> +		return -ENODEV;
>>> +
>>> +	pr_info("Initializing Armada-XP CPU power management ");
>>> +
>>> +	armada_370_xp_pmsu_enable_l2_powerdown_onidle();
>>> +
>>> +	return cpuidle_register(&armada_370_xp_idle_driver, NULL);
>>> +}
>>> +
>>> +module_init(armada_370_xp_cpuidle_init);
>>
>> Isn't it possible to replace it by module_platform_driver ? like ux500
>> or kirkwood ?
> 
> It should be possible indeed, I will check it.

That would be great. It is a nicer approach for the single zImage IMHO.

Thanks !
  -- Daniel

-- 
 <http://www.linaro.org/> Linaro.org ? Open source software for ARM SoCs

Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog

[PATCH v2 11/12] cpuidle: mvebu: Add initial CPU idle support for Armada 370/XP SoC

[Gregory CLEMENT]

On 15/09/2013 19:31, Daniel Lezcano wrote:
Hi Daniel,

>>>> +	.state_count = ARMADA_370_XP_MAX_STATES,
>>>> +};
>>>
>>> What about the local timers ? Are they shutdown ?
>>
>> I need to chekc it.
> 
> Ok, if it is the case, there is the flag CPUIDLE_FLAG_TIMER_STOP to tell
> the cpuidle framework to switch to the broadcast timer with this state.

I have just sent a new version taking into account all your remarks, expect
this one. The timers used on Armada 370/XP are only the locale timer then
they are not shutdown when the CPUs go to idle.

Regards,

Gregory

> 
>>>> +static int __init armada_370_xp_cpuidle_init(void)
>>>> +{
>>>> +	if (!of_find_compatible_node(NULL, NULL, "marvell,armada-370-xp-pmsu"))
>>>> +		return -ENODEV;
>>>> +
>>>> +	if (!of_find_compatible_node(NULL, NULL, "marvell,coherency-fabric"))
>>>> +		return -ENODEV;
>>>> +
>>>> +	pr_info("Initializing Armada-XP CPU power management ");
>>>> +
>>>> +	armada_370_xp_pmsu_enable_l2_powerdown_onidle();
>>>> +
>>>> +	return cpuidle_register(&armada_370_xp_idle_driver, NULL);
>>>> +}
>>>> +
>>>> +module_init(armada_370_xp_cpuidle_init);
>>>
>>> Isn't it possible to replace it by module_platform_driver ? like ux500
>>> or kirkwood ?
>>
>> It should be possible indeed, I will check it.
> 
> That would be great. It is a nicer approach for the single zImage IMHO.
> 
> Thanks !
>   -- Daniel
> 


-- 
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[PATCH v2 11/12] cpuidle: mvebu: Add initial CPU idle support for Armada 370/XP SoC

[Lorenzo Pieralisi]

On Fri, Sep 13, 2013 at 11:06:40AM +0100, Gregory CLEMENT wrote:

[...]

> +#define ARMADA_370_XP_MAX_STATES       3
> +
> +enum mv_pm_states {
> +       WFI = 0,
> +       MV_CPU_IDLE,
> +       MV_CPU_DEEP_IDLE,
> +};
> +
> +extern void v7_flush_dcache_all(void);
> +
> +/* Functions defined in suspend-armada-370-xp.S */
> +int armada_370_xp_cpu_resume(unsigned long);
> +int armada_370_xp_cpu_suspend(unsigned long);
> +
> +static int armada_370_xp_enter_idle(struct cpuidle_device *dev,
> +                               struct cpuidle_driver *drv,
> +                               int index)
> +{
> +       bool deepidle = false;
> +       unsigned int hw_cpu = cpu_logical_map(smp_processor_id());
> +
> +       armada_370_xp_pmsu_set_start_addr(armada_370_xp_cpu_resume, hw_cpu);
> +
> +       if (index == MV_CPU_DEEP_IDLE)
> +               deepidle = true;
> +
> +       cpu_suspend(deepidle, armada_370_xp_cpu_suspend);
> +
> +       cpu_init();

I do not think cpu_init() is needed. Either core resumes through
cpu_resume, and there cpu_init() is called for you, or there is no
reason to call cpu_init() since CPU was not shutdown.

> +
> +       armada_370_xp_pmsu_idle_restore();
> +
> +       return index;
> +}
> +
> +static struct cpuidle_driver armada_370_xp_idle_driver = {
> +       .name                   = "armada_370_xp_idle",
> +       .states[0]              = ARM_CPUIDLE_WFI_STATE,
> +       .states[1]              = {
> +               .enter                  = armada_370_xp_enter_idle,
> +               .exit_latency           = 10,
> +               .power_usage            = 50,
> +               .target_residency       = 100,
> +               .flags                  = CPUIDLE_FLAG_TIME_VALID,
> +               .name                   = "MV CPU IDLE",
> +               .desc                   = "CPU power down",
> +       },
> +       .states[2]              = {
> +               .enter                  = armada_370_xp_enter_idle,
> +               .exit_latency           = 100,
> +               .power_usage            = 5,
> +               .target_residency       = 1000,
> +               .flags                  = CPUIDLE_FLAG_TIME_VALID,
> +               .name                   = "MV CPU DEEP IDLE",
> +               .desc                   = "CPU and L2 Fabric power down",
> +       },
> +       .state_count = ARMADA_370_XP_MAX_STATES,
> +};
> +
> +static int __init armada_370_xp_cpuidle_init(void)
> +{
> +       if (!of_find_compatible_node(NULL, NULL, "marvell,armada-370-xp-pmsu"))
> +               return -ENODEV;
> +
> +       if (!of_find_compatible_node(NULL, NULL, "marvell,coherency-fabric"))
> +               return -ENODEV;
> +
> +       pr_info("Initializing Armada-XP CPU power management ");
> +
> +       armada_370_xp_pmsu_enable_l2_powerdown_onidle();
> +
> +       return cpuidle_register(&armada_370_xp_idle_driver, NULL);
> +}
> +
> +module_init(armada_370_xp_cpuidle_init);
> +
> +MODULE_LICENSE("GPL");
> diff --git a/drivers/cpuidle/suspend-armada-370-xp.S b/drivers/cpuidle/suspend-armada-370-xp.S
> new file mode 100644
> index 0000000..490bb9b
> --- /dev/null
> +++ b/drivers/cpuidle/suspend-armada-370-xp.S
> @@ -0,0 +1,91 @@
> +/*
> + * CPU idle low level implementation for Marvell Armada 370 and Armada XP SoCs
> + *
> + * Copyright (C) 2013 Marvell
> + *
> + * Nadav Haklai <nadavh@marvell.com>
> + *
> + * This file is licensed under the terms of the GNU General Public
> + * License version 2.  This program is licensed "as is" without any
> + * warranty of any kind, whether express or implied.
> + *
> + */
> +#include <linux/linkage.h>
> +
> +
> +/*
> +* armadaxp_cpu_suspend: enter cpu deepIdle state
> +* input:
> +*/
> +ENTRY(armada_370_xp_cpu_suspend)
> +/* Save ARM registers */
> +       stmfd   sp!, {r4 - r11, lr}     @ save registers on stack
> +
> +       bl armada_370_xp_pmsu_idle_prepare
> +       /*
> +        * Invalidate L1 data cache. Even though only invalidate is
> +        * necessary exported flush API is used here. Doing clean
> +        * on already clean cache would be almost NOP.
> +        */
> +       bl v7_flush_dcache_all
> +
> +       /*
> +        * Clear the SCTLR.C bit to prevent further data cache
> +        * allocation. Clearing SCTLR.C would make all the data accesses
> +        * strongly ordered and would not hit the cache.
> +        */
> +       mrc     p15, 0, r0, c1, c0, 0
> +       bic     r0, r0, #(1 << 2)       @ Disable the C bit
> +       mcr     p15, 0, r0, c1, c0, 0
> +       isb
> +
> +       bl v7_flush_dcache_all
> +

This code looks familiar and the first cache flush is not needed.

look at arch/arm/mach-vexpress/tc2_pm.c -> tc2_pm_down()

I know that probably the SMP bit in ACTLR is managed differently in this
architecture but still, cache flushing should still apply and the TC2
sequence is the proper one, unless you explain to me why that does not
work on this chipset.

> +       /* Data memory barrier and Data sync barrier */
> +       dsb
> +       dmb
> +
> +       bl armada_370_xp_disable_snoop_ena
> +
> +       dsb                             @ Data Synchronization Barrier
> +
> +/*
> + * ===================================
> + * == WFI instruction => Enter idle ==
> + * ===================================
> + */
> +
> +       wfi                             @ wait for interrupt

Here core is running out of coherency and I do not see any tlb invalidation in
the resume path from non-OFF mode. Please can you elaborate on this ?

> +/*
> + * ===================================
> + * == Resume path for non-OFF modes ==
> + * ===================================
> + */
> +
> +/* Enable SnoopEna - Exclusive */
> +       mov     r0, #1                  @ r0!=0 means use virtual address
> +       mov     r1, #0                  @ Do not add CPU to SMP group
> +       bl ll_set_cpu_coherent
> +
> +/* Re-enable C-bit if needed */
> +       mrc     p15, 0, r0, c1, c0, 0
> +       tst     r0, #(1 << 2)           @ Check C bit enabled?
> +       orreq   r0, r0, #(1 << 2)       @ Enable the C bit if cleared
> +       mcreq   p15, 0, r0, c1, c0, 0
> +       isb
> +
> +       ldmfd   sp!, {r4 - r11, pc}     @ restore regs and return
> +ENDPROC(armada_370_xp_cpu_suspend)
> +
> +/*
> +* armada_370_xp_cpu_resume: exit cpu deepIdle state
> +*/
> +ENTRY(armada_370_xp_cpu_resume)
> +       mov     r0, #0                  @ r0==0 means use physical address
> +       mov     r1, #1                  @ Add CPU to SMP group
> +       bl ll_set_cpu_coherent
> +
> +       /* Now branch to the common CPU resume function */
> +       b       cpu_resume
> +
> +ENDPROC(armada_370_xp_cpu_resume)

I have further comments on the set (in particular on the inner workings
of coherency and how CPUs get in and out of idle - ie how this is
managed by the power controller) but I will keep them for next version,
when the first set of review comments is addressed.

Thanks a lot,
Lorenzo

[PATCH v2 11/12] cpuidle: mvebu: Add initial CPU idle support for Armada 370/XP SoC

[Gregory CLEMENT]

Hi Lorenzo,
[...]
>> +       if (index == MV_CPU_DEEP_IDLE)
>> +               deepidle = true;
>> +
>> +       cpu_suspend(deepidle, armada_370_xp_cpu_suspend);
>> +
>> +       cpu_init();
> 
> I do not think cpu_init() is needed. Either core resumes through
> cpu_resume, and there cpu_init() is called for you, or there is no
> reason to call cpu_init() since CPU was not shutdown.

You're right: I have removed it on the new version I have just sent.


[...]

>> @@ -0,0 +1,91 @@
>> +/*
>> + * CPU idle low level implementation for Marvell Armada 370 and Armada XP SoCs
>> + *
>> + * Copyright (C) 2013 Marvell
>> + *
>> + * Nadav Haklai <nadavh@marvell.com>
>> + *
>> + * This file is licensed under the terms of the GNU General Public
>> + * License version 2.  This program is licensed "as is" without any
>> + * warranty of any kind, whether express or implied.
>> + *
>> + */
>> +#include <linux/linkage.h>
>> +
>> +
>> +/*
>> +* armadaxp_cpu_suspend: enter cpu deepIdle state
>> +* input:
>> +*/
>> +ENTRY(armada_370_xp_cpu_suspend)
>> +/* Save ARM registers */
>> +       stmfd   sp!, {r4 - r11, lr}     @ save registers on stack
>> +
>> +       bl armada_370_xp_pmsu_idle_prepare
>> +       /*
>> +        * Invalidate L1 data cache. Even though only invalidate is
>> +        * necessary exported flush API is used here. Doing clean
>> +        * on already clean cache would be almost NOP.
>> +        */
>> +       bl v7_flush_dcache_all
>> +
>> +       /*
>> +        * Clear the SCTLR.C bit to prevent further data cache
>> +        * allocation. Clearing SCTLR.C would make all the data accesses
>> +        * strongly ordered and would not hit the cache.
>> +        */
>> +       mrc     p15, 0, r0, c1, c0, 0
>> +       bic     r0, r0, #(1 << 2)       @ Disable the C bit
>> +       mcr     p15, 0, r0, c1, c0, 0
>> +       isb
>> +
>> +       bl v7_flush_dcache_all
>> +
> 
> This code looks familiar and the first cache flush is not needed.
> 
> look at arch/arm/mach-vexpress/tc2_pm.c -> tc2_pm_down()
> 
> I know that probably the SMP bit in ACTLR is managed differently in this
> architecture but still, cache flushing should still apply and the TC2
> sequence is the proper one, unless you explain to me why that does not
> work on this chipset.

Here we follow the advices you gave in your presentation at Linaro Connect
Q2-2012: "Idling ARMs in a Busy World: Linux Power Management for ARM
multi-cluster systems"
http://www.linaro.org/documents/download/84a5990d6f227c90e3d3a3ad2af0e22c4fd1d0bd4a6c7
at p21.

Quoting the author of the original version od the code :
"Since the second flush is almost NOPs, I preferred to keep them both in the same sequence."


> 
>> +       /* Data memory barrier and Data sync barrier */
>> +       dsb
>> +       dmb
>> +
>> +       bl armada_370_xp_disable_snoop_ena
>> +
>> +       dsb                             @ Data Synchronization Barrier
>> +
>> +/*
>> + * ===================================
>> + * == WFI instruction => Enter idle ==
>> + * ===================================
>> + */
>> +
>> +       wfi                             @ wait for interrupt
> 
> Here core is running out of coherency and I do not see any tlb invalidation in
> the resume path from non-OFF mode. Please can you elaborate on this ?
> 

The TLB invalidation should take place in /arch/arm/kernel/suspend.c  cpu_suspend( )
in case the return value is 0 caused by cpu_suspend_abort.
Did we missed something?

[...]
> 
> I have further comments on the set (in particular on the inner workings
> of coherency and how CPUs get in and out of idle - ie how this is
> managed by the power controller) but I will keep them for next version,
> when the first set of review comments is addressed.
> 

You can make your comments on the last version I have just sent.

Thanks for your review

Gregory
-- 
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[PATCH v2 11/12] cpuidle: mvebu: Add initial CPU idle support for Armada 370/XP SoC

[Lorenzo Pieralisi]

Hi Gregory,

[CC'ed Nico for the cache cleaning inline asm macros]

On Mon, Oct 14, 2013 at 03:14:12PM +0100, Gregory CLEMENT wrote:

[...]

> >> +ENTRY(armada_370_xp_cpu_suspend)
> >> +/* Save ARM registers */
> >> +       stmfd   sp!, {r4 - r11, lr}     @ save registers on stack
> >> +
> >> +       bl armada_370_xp_pmsu_idle_prepare
> >> +       /*
> >> +        * Invalidate L1 data cache. Even though only invalidate is
> >> +        * necessary exported flush API is used here. Doing clean
> >> +        * on already clean cache would be almost NOP.
> >> +        */
> >> +       bl v7_flush_dcache_all
> >> +
> >> +       /*
> >> +        * Clear the SCTLR.C bit to prevent further data cache
> >> +        * allocation. Clearing SCTLR.C would make all the data accesses
> >> +        * strongly ordered and would not hit the cache.
> >> +        */
> >> +       mrc     p15, 0, r0, c1, c0, 0
> >> +       bic     r0, r0, #(1 << 2)       @ Disable the C bit
> >> +       mcr     p15, 0, r0, c1, c0, 0
> >> +       isb
> >> +
> >> +       bl v7_flush_dcache_all
> >> +
> > 
> > This code looks familiar and the first cache flush is not needed.
> > 
> > look at arch/arm/mach-vexpress/tc2_pm.c -> tc2_pm_down()
> > 
> > I know that probably the SMP bit in ACTLR is managed differently in this
> > architecture but still, cache flushing should still apply and the TC2
> > sequence is the proper one, unless you explain to me why that does not
> > work on this chipset.
> 
> Here we follow the advices you gave in your presentation at Linaro Connect
> Q2-2012: "Idling ARMs in a Busy World: Linux Power Management for ARM
> multi-cluster systems"
> http://www.linaro.org/documents/download/84a5990d6f227c90e3d3a3ad2af0e22c4fd1d0bd4a6c7
> at p21.
> 
> Quoting the author of the original version od the code :
> "Since the second flush is almost NOPs, I preferred to keep them both in the same sequence."
>

This code comes from arch/arm/mach-omap2/sleep44xx.S

First flush is not necessary and should be removed, from here and from
the file above as well; the proper sequence is in:

arch/arm/mach-vexpress/tc2_pm.c

(which is described in the presentation link you provided)

and Nico also managed to write inline assembly macros that are meant
to be upstreamed soon to carry out the cache cleaning ops you need
here.

I understand that the second clean is _almost_ NOPs, but that's no good
reason to write the sequence this way. I will repeat myself: first flush
is not necessary and as such should be removed.

> > 
> >> +       /* Data memory barrier and Data sync barrier */
> >> +       dsb
> >> +       dmb
> >> +
> >> +       bl armada_370_xp_disable_snoop_ena
> >> +
> >> +       dsb                             @ Data Synchronization Barrier
> >> +
> >> +/*
> >> + * ===================================
> >> + * == WFI instruction => Enter idle ==
> >> + * ===================================
> >> + */
> >> +
> >> +       wfi                             @ wait for interrupt
> > 
> > Here core is running out of coherency and I do not see any tlb invalidation in
> > the resume path from non-OFF mode. Please can you elaborate on this ?
> > 
> 
> The TLB invalidation should take place in /arch/arm/kernel/suspend.c  cpu_suspend( )
> in case the return value is 0 caused by cpu_suspend_abort.
> Did we missed something?

Yes, you did, cpu_suspend_abort returns 1 (ie failure), so no tlb invalidation
is carried out in cpu_suspend(); tlbs are invalidated only when core resumes
through cpu_resume. I do not have details of the coherent interconnect used
in this board/chip, but if the power down procedure fails and the core has
run outside coherency in the idle thread, tlbs must be invalidated.

> > I have further comments on the set (in particular on the inner workings
> > of coherency and how CPUs get in and out of idle - ie how this is
> > managed by the power controller) but I will keep them for next version,
> > when the first set of review comments is addressed.
> > 
> 
> You can make your comments on the last version I have just sent.

Ok, cheers.

> Thanks for your review

Thank you,
Lorenzo

[PATCH v2 12/12] ARM: dts: mvebu: Add a new set of registers to the PMSU node

[Gregory CLEMENT]

The Power Management Unit Service block also controls the Coherency
Fabric subsystem. This new set of registers is needed for the CPU idle
implementation for the Armada XP, it allows to enter in a deep CPU
idle state where the Coherency Fabric and the L2 cache are powerdown.

Cc: devicetree at vger.kernel.org
Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
---
 Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt | 12 +++++++-----
 arch/arm/boot/dts/armada-xp.dtsi                             |  2 +-
 2 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt b/Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt
index 926b4d6..8a9db0c 100644
--- a/Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt
+++ b/Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt
@@ -7,14 +7,16 @@ Required properties:
 - compatible: "marvell,armada-370-xp-pmsu"
 
 - reg: Should contain PMSU registers location and length. First pair
-  for the per-CPU SW Reset Control registers, second pair for the
-  Power Management Service Unit.
+  for the per-CPU SW Reset Control registers, second pair for the CPU
+  Power Management Service Unit registers, third pair for the Fabric Power
+  Management Service Unit registers.
 
 Example:
 
-armada-370-xp-pmsu at d0022000 {
+armada-370-xp-pmsu at 22000 {
 	compatible = "marvell,armada-370-xp-pmsu";
-	reg = <0xd0022100 0x430>,
-	      <0xd0020800 0x20>;
+	reg = <0x22100 0x430>,
+	      <0x20800 0x20>,
+	      <0x22000 0x24>;
 };
 
diff --git a/arch/arm/boot/dts/armada-xp.dtsi b/arch/arm/boot/dts/armada-xp.dtsi
index def125c..c79cd53 100644
--- a/arch/arm/boot/dts/armada-xp.dtsi
+++ b/arch/arm/boot/dts/armada-xp.dtsi
@@ -48,7 +48,7 @@
 
 			armada-370-xp-pmsu at 22000 {
 				compatible = "marvell,armada-370-xp-pmsu";
-				reg = <0x22100 0x430>, <0x20800 0x20>;
+				reg = <0x22100 0x430>, <0x20800 0x20>, <0x22000 0x24>;
 			};
 
 			serial at 12200 {
-- 
1.8.1.2

[PATCH v2 00/12] CPU idle for Armada XP

[Andrew Lunn]

On Fri, Sep 13, 2013 at 12:06:29PM +0200, Gregory CLEMENT wrote:
> Hello,
> 
> This patch set adds the CPU idle support for Armada XP and prepares
> the support for Armada 370. This was based on the work of Nadav
> Haklai.

Hi Gregory.

Kirkwood has the ability to put the DDR into self refresh mode, which
is used as part of the second level idle mode. Does 370/XP have this?

For XP, with it being SMP, it would be a bit more complex, since you
would not want to use it unless all CPUs were idle.

      Thanks
	Andrew

[PATCH v2 00/12] CPU idle for Armada XP

[Gregory CLEMENT]

On 13/09/2013 13:00, Andrew Lunn wrote:
> 
> On Fri, Sep 13, 2013 at 12:06:29PM +0200, Gregory CLEMENT wrote:
>> Hello,
>>
>> This patch set adds the CPU idle support for Armada XP and prepares
>> the support for Armada 370. This was based on the work of Nadav
>> Haklai.
> 
> Hi Gregory.
> 
> Kirkwood has the ability to put the DDR into self refresh mode, which
> is used as part of the second level idle mode. Does 370/XP have this?

Indeed there is a self refresh bit on a DDR related register. I thought
this kind of feature is more likely used for suspend to ram.

We plan to also submit the suspend to ram but not immediately.

> 
> For XP, with it being SMP, it would be a bit more complex, since you
> would not want to use it unless all CPUs were idle.

I wonder how the others SMP ARM SoCs deal with it, I hope this time
there will be a framework available and we won't have to create it! ;)

> 
>       Thanks
> 	Andrew
> 


-- 
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[PATCH v2 00/12] CPU idle for Armada XP

[Andrew Lunn]

On Fri, Sep 13, 2013 at 01:17:17PM +0200, Gregory CLEMENT wrote:
> On 13/09/2013 13:00, Andrew Lunn wrote:
> > 
> > On Fri, Sep 13, 2013 at 12:06:29PM +0200, Gregory CLEMENT wrote:
> >> Hello,
> >>
> >> This patch set adds the CPU idle support for Armada XP and prepares
> >> the support for Armada 370. This was based on the work of Nadav
> >> Haklai.
> > 
> > Hi Gregory.
> > 
> > Kirkwood has the ability to put the DDR into self refresh mode, which
> > is used as part of the second level idle mode. Does 370/XP have this?
> 
> Indeed there is a self refresh bit on a DDR related register. I thought
> this kind of feature is more likely used for suspend to ram.

The kirkwood data sheet suggests it is useful if the CPU is idle for a
few milliseconds. So it sounds like it is a slow transition, but does
seem to save a bit of power. I guess in a NO_HZ system, it would make
sense.

	Andrew

[PATCH v2 00/12] CPU idle for Armada XP

[Kevin Hilman]

Gregory CLEMENT <gregory.clement@free-electrons.com> writes:

> On 13/09/2013 13:00, Andrew Lunn wrote:
>> 
>> On Fri, Sep 13, 2013 at 12:06:29PM +0200, Gregory CLEMENT wrote:
>>> Hello,
>>>
>>> This patch set adds the CPU idle support for Armada XP and prepares
>>> the support for Armada 370. This was based on the work of Nadav
>>> Haklai.
>> 
>> Hi Gregory.
>> 
>> Kirkwood has the ability to put the DDR into self refresh mode, which
>> is used as part of the second level idle mode. Does 370/XP have this?
>
> Indeed there is a self refresh bit on a DDR related register. I thought
> this kind of feature is more likely used for suspend to ram.
>
> We plan to also submit the suspend to ram but not immediately.
>
>> 
>> For XP, with it being SMP, it would be a bit more complex, since you
>> would not want to use it unless all CPUs were idle.
>
> I wonder how the others SMP ARM SoCs deal with it, I hope this time
> there will be a framework available and we won't have to create it! ;)

You shouldn't have to invent anything here.

For low-power states that require all CPUs to be idle, we have "coupled"
CPUidle states (c.f. drivers/cpuidle/coupled.)  OMAP and Tegra are using
this, but I believe Daniel wants to move away from this, so I'll let him
elaborate.

Kevin

[PATCH v2 00/12] CPU idle for Armada XP

[Daniel Lezcano]

On 09/13/2013 04:48 PM, Kevin Hilman wrote:
> Gregory CLEMENT <gregory.clement@free-electrons.com> writes:
> 
>> On 13/09/2013 13:00, Andrew Lunn wrote:
>>>
>>> On Fri, Sep 13, 2013 at 12:06:29PM +0200, Gregory CLEMENT wrote:
>>>> Hello,
>>>>
>>>> This patch set adds the CPU idle support for Armada XP and prepares
>>>> the support for Armada 370. This was based on the work of Nadav
>>>> Haklai.
>>>
>>> Hi Gregory.
>>>
>>> Kirkwood has the ability to put the DDR into self refresh mode, which
>>> is used as part of the second level idle mode. Does 370/XP have this?
>>
>> Indeed there is a self refresh bit on a DDR related register. I thought
>> this kind of feature is more likely used for suspend to ram.
>>
>> We plan to also submit the suspend to ram but not immediately.
>>
>>>
>>> For XP, with it being SMP, it would be a bit more complex, since you
>>> would not want to use it unless all CPUs were idle.
>>
>> I wonder how the others SMP ARM SoCs deal with it, I hope this time
>> there will be a framework available and we won't have to create it! ;)
> 
> You shouldn't have to invent anything here.
> 
> For low-power states that require all CPUs to be idle, we have "coupled"
> CPUidle states (c.f. drivers/cpuidle/coupled.)  OMAP and Tegra are using
> this, but I believe Daniel wants to move away from this, so I'll let him
> elaborate.

Yes, I would not recommend to use the "coupled" idle states unless you
have a good reason to do that (eg. only CPU0 can do set the PMU).

If the board provides enough mechanisms with the PMSU, there is an
alternate, lockless, sync algorithm, which can be reused, in the ux500's
cpuidle driver (c.f. drivers/cpuidle/cpuidle-ux500.c).

Otherwise, I recommend to have a look at MCPM (c.f
drivers/cpuidle/cpuidle-big_little.c).

  -- Daniel


-- 
 <http://www.linaro.org/> Linaro.org ? Open source software for ARM SoCs

Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog

[PATCH v2 00/12] CPU idle for Armada XP

[Jason Cooper]

Gregory,

On Fri, Sep 13, 2013 at 12:06:29PM +0200, Gregory CLEMENT wrote:
> Hello,
> 
> This patch set adds the CPU idle support for Armada XP and prepares
> the support for Armada 370. This was based on the work of Nadav
> Haklai.
> 
> Most of the patches modify the mvebu code in order to prepare the
> support for CPU idle, hence the patches 2 to 10 should go to mvebu
> subsystem (and then arm-soc).
> 
> The first patch should go through ARM subsystem and should be taken
> by Russell King.
> 
> The 11th patch 'cpuidle: mvebu: Add initial cpu idle support for
> Armada 370/XP SoC' is the only one who should go to the cpuidle
> subsystem. But of course I would like that Daniel Lezcano or Rafael
> J. Wysocki have a look on the whole series.
> 
> The last patch should also go to mvebu subsystem (and then arm-soc)
> but with an Acked-by from on of the device tree maintainer.
> 
> The whole series is also available in the branch CPU-idle-ArmadaXP-v2
> at https://github.com/MISL-EBU-System-SW/mainline-public.git
> 
> Thanks,
> 
> Changelog:
> 
> v1 -> v2:
> 
> * Removed the pm_level kernel parameter. As Kevin Hilman pointed, its
>   usage can be replaced by using
>   /sys/devices/system/cpu/cpu*/cpuidle/state*/disable or the kernel
>   parameter cpuidle.off.
> 
> * Used BIT() macro (reported by Ezequiel)
> 
> * Made the function more readable the
>   armada_370_xp_pmsu_idle_prepare() function (reported by Thomas)
> 
> * Moved the config entry in Kconfig.arm, and rename the config symbol
>   according the pattern used by other arm cpu: ARM_"soc name"_CPUIDLE
> 
> * Moved the build rule under the new ARM SoC section in the Makefile
> 
> * Rebased on Linus Torvalds master branch of Thursday September 12
> 
> Gregory CLEMENT (12):
>   ARM: PJ4B: Add cpu_suspend/cpu_resume hooks for PJ4B
>   ARM: mvebu: ll_set_cpu_coherent no more uses the coherency address as
>     parameter
>   ARM: mvebu: ll_set_cpu_coherent always uses the current CPU
>   ARM: mvebu: Remove the unused argument of set_cpu_coherent()
>   ARM: mvebu: Make ll_set_cpu_coherent() more configurable
>   ARM: mvebu: Low level functions to disable cache snooping
>   ARM: mvebu: Add a new set of registers for pmsu
>   ARM: mvebu: Allow to power down L2 cache controller in idle mode
>   ARM: mvebu: Add the PMSU related part of the cpu idle functions
>   ARM: mvebu: Set the start address of a CPU in a separate function
>   cpuidle: mvebu: Add initial CPU idle support for Armada 370/XP SoC
>   ARM: dts: mvebu: Add a new set of registers to the PMSU node

Is a v3 of this series on it's way?

thx,

Jason.

[PATCH v2 00/12] CPU idle for Armada XP

[Gregory CLEMENT]

Jason,

On 02/10/2013 20:39, Jason Cooper wrote:
> Gregory,
> 
> On Fri, Sep 13, 2013 at 12:06:29PM +0200, Gregory CLEMENT wrote:
>> Hello,
>>
>> This patch set adds the CPU idle support for Armada XP and prepares
>> the support for Armada 370. This was based on the work of Nadav
>> Haklai.
>>
>> Most of the patches modify the mvebu code in order to prepare the
>> support for CPU idle, hence the patches 2 to 10 should go to mvebu
>> subsystem (and then arm-soc).
>>
>> The first patch should go through ARM subsystem and should be taken
>> by Russell King.
>>
>> The 11th patch 'cpuidle: mvebu: Add initial cpu idle support for
>> Armada 370/XP SoC' is the only one who should go to the cpuidle
>> subsystem. But of course I would like that Daniel Lezcano or Rafael
>> J. Wysocki have a look on the whole series.
>>
>> The last patch should also go to mvebu subsystem (and then arm-soc)
>> but with an Acked-by from on of the device tree maintainer.
>>
>> The whole series is also available in the branch CPU-idle-ArmadaXP-v2
>> at https://github.com/MISL-EBU-System-SW/mainline-public.git
>>
>> Thanks,
>>
>> Changelog:
>>
>> v1 -> v2:
>>
>> * Removed the pm_level kernel parameter. As Kevin Hilman pointed, its
>>   usage can be replaced by using
>>   /sys/devices/system/cpu/cpu*/cpuidle/state*/disable or the kernel
>>   parameter cpuidle.off.
>>
>> * Used BIT() macro (reported by Ezequiel)
>>
>> * Made the function more readable the
>>   armada_370_xp_pmsu_idle_prepare() function (reported by Thomas)
>>
>> * Moved the config entry in Kconfig.arm, and rename the config symbol
>>   according the pattern used by other arm cpu: ARM_"soc name"_CPUIDLE
>>
>> * Moved the build rule under the new ARM SoC section in the Makefile
>>
>> * Rebased on Linus Torvalds master branch of Thursday September 12
>>
>> Gregory CLEMENT (12):
>>   ARM: PJ4B: Add cpu_suspend/cpu_resume hooks for PJ4B
>>   ARM: mvebu: ll_set_cpu_coherent no more uses the coherency address as
>>     parameter
>>   ARM: mvebu: ll_set_cpu_coherent always uses the current CPU
>>   ARM: mvebu: Remove the unused argument of set_cpu_coherent()
>>   ARM: mvebu: Make ll_set_cpu_coherent() more configurable
>>   ARM: mvebu: Low level functions to disable cache snooping
>>   ARM: mvebu: Add a new set of registers for pmsu
>>   ARM: mvebu: Allow to power down L2 cache controller in idle mode
>>   ARM: mvebu: Add the PMSU related part of the cpu idle functions
>>   ARM: mvebu: Set the start address of a CPU in a separate function
>>   cpuidle: mvebu: Add initial CPU idle support for Armada 370/XP SoC
>>   ARM: dts: mvebu: Add a new set of registers to the PMSU node
> 
> Is a v3 of this series on it's way?

Yes I got some feedback from the Marvell engineers about the tricky part.

However I am quite busy this week, I am not sure to be able to sent it
before Monday.

Regards,

Gregory


-- 
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[PATCH v2 00/12] CPU idle for Armada XP

[Jason Cooper]

On Wed, Oct 02, 2013 at 08:44:57PM +0300, Gregory CLEMENT wrote:
> On 02/10/2013 20:39, Jason Cooper wrote:
...
> > Is a v3 of this series on it's way?
> 
> Yes I got some feedback from the Marvell engineers about the tricky part.
> 
> However I am quite busy this week, I am not sure to be able to sent it
> before Monday.

Ok, it was just sitting in my stack and wanted to make sure no one was
assuming it should go in as is.  :)

thx,

Jason.