[PATCH V4 0/7] ARM: tegra30: cpuidle: add a powered-down state

[PATCH V4 0/7] ARM: tegra30: cpuidle: add a powered-down state

[Joseph Lo]

This adds a "powered-down" state for cpuidle. It's a power gating idle
mode. It supports the secondary CPUs (i.e., CPU1-CPU3) to go into
powered-down state independently. When any of the secondary CPUs go into
this state, it can be power gated alone. There is a limitation on CPU0.
The CPU0 can go into powered-down state only when all secondary CPU is
offline. After CPU0 is in powered-down state, the CPU rail can be turned
off.

All CPUs entering powered-down state is not working. The CPU0 enters this
state only when secondary CPU is offline. This can be coverd by CPUquiet
and cluster switching mechanism.

We also remove the ambiguous name of LP2 in the cpuidle state.

Verified on Seaboard(Tegra20) and Cardhu(Tegra30).

This patch set should depend on these patches:
ARM: tegra: rename the file of "sleep-tXX" to "sleep-tegraXX"
ARM: tegra30: clocks: add AHB and APB clocks
ARM: tegra: dt: add L2 cache controller
ARM: tegra: common: using OF api for L2 cache init

Major changes:
V4:
* rebased on next-20121031
V3:
* fix a potential issue that will cause CPU be corrupted
* rename the LP2 state to powered-down state
V2:
* refine L1 cache maintenance function

Previous work can be found at:
V2:
http://www.mail-archive.com/linux-tegra at vger.kernel.org/msg06478.html
V1:
http://www.mail-archive.com/linux-tegra at vger.kernel.org/msg06319.html

Joseph Lo (7):
  ARM: tegra: cpuidle: separate cpuidle driver for different chips
  ARM: tegra: cpuidle: add CPU resume function
  ARM: tegra30: cpuidle: add powered-down state for secondary CPUs
  ARM: tegra30: common: enable csite clock
  ARM: tegra30: clocks: add CPU low-power function into
    tegra_cpu_car_ops
  ARM: tegra30: flowctrl: add cpu_suspend_exter/exit function
  ARM: tegra30: cpuidle: add powered-down state for CPU0

 arch/arm/mach-tegra/Makefile                       |    7 +
 arch/arm/mach-tegra/common.c                       |    1 +
 .../mach-tegra/{cpuidle.c => cpuidle-tegra20.c}    |    7 +-
 arch/arm/mach-tegra/cpuidle-tegra30.c              |  188 +++++++++++++++++
 arch/arm/mach-tegra/cpuidle.c                      |   47 ++---
 arch/arm/mach-tegra/cpuidle.h                      |   32 +++
 arch/arm/mach-tegra/flowctrl.c                     |   47 +++++
 arch/arm/mach-tegra/flowctrl.h                     |    8 +
 arch/arm/mach-tegra/headsmp.S                      |   60 ++++++
 arch/arm/mach-tegra/pm.c                           |  218 ++++++++++++++++++++
 arch/arm/mach-tegra/pm.h                           |   33 +++
 arch/arm/mach-tegra/reset.c                        |    6 +
 arch/arm/mach-tegra/reset.h                        |    9 +
 arch/arm/mach-tegra/sleep-tegra30.S                |   66 ++++++
 arch/arm/mach-tegra/sleep.S                        |   71 +++++++
 arch/arm/mach-tegra/sleep.h                        |    5 +
 arch/arm/mach-tegra/tegra30_clocks.c               |  108 ++++++++++
 arch/arm/mach-tegra/tegra_cpu_car.h                |   37 ++++
 18 files changed, 911 insertions(+), 39 deletions(-)
 copy arch/arm/mach-tegra/{cpuidle.c => cpuidle-tegra20.c} (91%)
 create mode 100644 arch/arm/mach-tegra/cpuidle-tegra30.c
 create mode 100644 arch/arm/mach-tegra/cpuidle.h
 create mode 100644 arch/arm/mach-tegra/pm.c
 create mode 100644 arch/arm/mach-tegra/pm.h

[PATCH V4 1/7] ARM: tegra: cpuidle: separate cpuidle driver for different chips

[Joseph Lo]

The different Tegra chips may have different CPU idle states and data.
Individual CPU idle driver make it more easy to maintain.

Signed-off-by: Joseph Lo <josephl@nvidia.com>
---
V4:
* no change
V3:
* no change
V2:
* only remove the line of file name and path in the (c) header
---
 arch/arm/mach-tegra/Makefile                       |    6 +++
 .../mach-tegra/{cpuidle.c => cpuidle-tegra20.c}    |    7 +--
 .../mach-tegra/{cpuidle.c => cpuidle-tegra30.c}    |    7 +--
 arch/arm/mach-tegra/cpuidle.c                      |   47 +++++--------------
 arch/arm/mach-tegra/cpuidle.h                      |   32 +++++++++++++
 5 files changed, 55 insertions(+), 44 deletions(-)
 copy arch/arm/mach-tegra/{cpuidle.c => cpuidle-tegra20.c} (91%)
 copy arch/arm/mach-tegra/{cpuidle.c => cpuidle-tegra30.c} (91%)
 create mode 100644 arch/arm/mach-tegra/cpuidle.h

diff --git a/arch/arm/mach-tegra/Makefile b/arch/arm/mach-tegra/Makefile
index e6929c6..9b80c1e 100644
--- a/arch/arm/mach-tegra/Makefile
+++ b/arch/arm/mach-tegra/Makefile
@@ -14,9 +14,15 @@ obj-$(CONFIG_ARCH_TEGRA_2x_SOC)         += tegra20_clocks.o
 obj-$(CONFIG_ARCH_TEGRA_2x_SOC)         += tegra20_clocks_data.o
 obj-$(CONFIG_ARCH_TEGRA_2x_SOC)		+= tegra2_emc.o
 obj-$(CONFIG_ARCH_TEGRA_2x_SOC)		+= sleep-tegra20.o
+ifeq ($(CONFIG_CPU_IDLE),y)
+obj-$(CONFIG_ARCH_TEGRA_2x_SOC)		+= cpuidle-tegra20.o
+endif
 obj-$(CONFIG_ARCH_TEGRA_3x_SOC)		+= tegra30_clocks.o
 obj-$(CONFIG_ARCH_TEGRA_3x_SOC)		+= tegra30_clocks_data.o
 obj-$(CONFIG_ARCH_TEGRA_3x_SOC)		+= sleep-tegra30.o
+ifeq ($(CONFIG_CPU_IDLE),y)
+obj-$(CONFIG_ARCH_TEGRA_3x_SOC)		+= cpuidle-tegra30.o
+endif
 obj-$(CONFIG_SMP)			+= platsmp.o headsmp.o
 obj-$(CONFIG_SMP)                       += reset.o
 obj-$(CONFIG_HOTPLUG_CPU)               += hotplug.o
diff --git a/arch/arm/mach-tegra/cpuidle.c b/arch/arm/mach-tegra/cpuidle-tegra20.c
similarity index 91%
copy from arch/arm/mach-tegra/cpuidle.c
copy to arch/arm/mach-tegra/cpuidle-tegra20.c
index 4e0b07c..d32e8b0 100644
--- a/arch/arm/mach-tegra/cpuidle.c
+++ b/arch/arm/mach-tegra/cpuidle-tegra20.c
@@ -1,6 +1,4 @@
 /*
- * arch/arm/mach-tegra/cpuidle.c
- *
  * CPU idle driver for Tegra CPUs
  *
  * Copyright (c) 2010-2012, NVIDIA Corporation.
@@ -27,7 +25,7 @@
 
 #include <asm/cpuidle.h>
 
-struct cpuidle_driver tegra_idle_driver = {
+static struct cpuidle_driver tegra_idle_driver = {
 	.name = "tegra_idle",
 	.owner = THIS_MODULE,
 	.en_core_tk_irqen = 1,
@@ -39,7 +37,7 @@ struct cpuidle_driver tegra_idle_driver = {
 
 static DEFINE_PER_CPU(struct cpuidle_device, tegra_idle_device);
 
-static int __init tegra_cpuidle_init(void)
+int __init tegra20_cpuidle_init(void)
 {
 	int ret;
 	unsigned int cpu;
@@ -66,4 +64,3 @@ static int __init tegra_cpuidle_init(void)
 	}
 	return 0;
 }
-device_initcall(tegra_cpuidle_init);
diff --git a/arch/arm/mach-tegra/cpuidle.c b/arch/arm/mach-tegra/cpuidle-tegra30.c
similarity index 91%
copy from arch/arm/mach-tegra/cpuidle.c
copy to arch/arm/mach-tegra/cpuidle-tegra30.c
index 4e0b07c..37e7551 100644
--- a/arch/arm/mach-tegra/cpuidle.c
+++ b/arch/arm/mach-tegra/cpuidle-tegra30.c
@@ -1,6 +1,4 @@
 /*
- * arch/arm/mach-tegra/cpuidle.c
- *
  * CPU idle driver for Tegra CPUs
  *
  * Copyright (c) 2010-2012, NVIDIA Corporation.
@@ -27,7 +25,7 @@
 
 #include <asm/cpuidle.h>
 
-struct cpuidle_driver tegra_idle_driver = {
+static struct cpuidle_driver tegra_idle_driver = {
 	.name = "tegra_idle",
 	.owner = THIS_MODULE,
 	.en_core_tk_irqen = 1,
@@ -39,7 +37,7 @@ struct cpuidle_driver tegra_idle_driver = {
 
 static DEFINE_PER_CPU(struct cpuidle_device, tegra_idle_device);
 
-static int __init tegra_cpuidle_init(void)
+int __init tegra30_cpuidle_init(void)
 {
 	int ret;
 	unsigned int cpu;
@@ -66,4 +64,3 @@ static int __init tegra_cpuidle_init(void)
 	}
 	return 0;
 }
-device_initcall(tegra_cpuidle_init);
diff --git a/arch/arm/mach-tegra/cpuidle.c b/arch/arm/mach-tegra/cpuidle.c
index 4e0b07c..d065139 100644
--- a/arch/arm/mach-tegra/cpuidle.c
+++ b/arch/arm/mach-tegra/cpuidle.c
@@ -23,47 +23,26 @@
 
 #include <linux/kernel.h>
 #include <linux/module.h>
-#include <linux/cpuidle.h>
 
-#include <asm/cpuidle.h>
-
-struct cpuidle_driver tegra_idle_driver = {
-	.name = "tegra_idle",
-	.owner = THIS_MODULE,
-	.en_core_tk_irqen = 1,
-	.state_count = 1,
-	.states = {
-		[0] = ARM_CPUIDLE_WFI_STATE_PWR(600),
-	},
-};
-
-static DEFINE_PER_CPU(struct cpuidle_device, tegra_idle_device);
+#include "fuse.h"
+#include "cpuidle.h"
 
 static int __init tegra_cpuidle_init(void)
 {
 	int ret;
-	unsigned int cpu;
-	struct cpuidle_device *dev;
-	struct cpuidle_driver *drv = &tegra_idle_driver;
 
-	ret = cpuidle_register_driver(&tegra_idle_driver);
-	if (ret) {
-		pr_err("CPUidle driver registration failed\n");
-		return ret;
+	switch (tegra_chip_id) {
+	case TEGRA20:
+		ret = tegra20_cpuidle_init();
+		break;
+	case TEGRA30:
+		ret = tegra30_cpuidle_init();
+		break;
+	default:
+		ret = -ENODEV;
+		break;
 	}
 
-	for_each_possible_cpu(cpu) {
-		dev = &per_cpu(tegra_idle_device, cpu);
-		dev->cpu = cpu;
-
-		dev->state_count = drv->state_count;
-		ret = cpuidle_register_device(dev);
-		if (ret) {
-			pr_err("CPU%u: CPUidle device registration failed\n",
-				cpu);
-			return ret;
-		}
-	}
-	return 0;
+	return ret;
 }
 device_initcall(tegra_cpuidle_init);
diff --git a/arch/arm/mach-tegra/cpuidle.h b/arch/arm/mach-tegra/cpuidle.h
new file mode 100644
index 0000000..496204d
--- /dev/null
+++ b/arch/arm/mach-tegra/cpuidle.h
@@ -0,0 +1,32 @@
+/*
+ * Copyright (c) 2012, NVIDIA Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __MACH_TEGRA_CPUIDLE_H
+#define __MACH_TEGRA_CPUIDLE_H
+
+#ifdef CONFIG_ARCH_TEGRA_2x_SOC
+int tegra20_cpuidle_init(void);
+#else
+static inline int tegra20_cpuidle_init(void) { return -ENODEV; }
+#endif
+
+#ifdef CONFIG_ARCH_TEGRA_3x_SOC
+int tegra30_cpuidle_init(void);
+#else
+static inline int tegra30_cpuidle_init(void) { return -ENODEV; }
+#endif
+
+#endif
-- 
1.7.0.4

[PATCH V4 2/7] ARM: tegra: cpuidle: add CPU resume function

[Joseph Lo]

The CPU suspending on Tegra means CPU power gating. We add a resume
function for taking care the CPUs that resume from power gating status.
This function was been hooked to the reset handler. We take care
everything here before go into kernel.

Be aware of that, you may see the legacy power status "LP2" in the code
which is exactly the same meaning of "CPU power down".

Based on the work by:
Scott Williams <scwilliams@nvidia.com>
Colin Cross <ccross@android.com>
Gary King <gking@nvidia.com>

Signed-off-by: Joseph Lo <josephl@nvidia.com>
---
V4:
* rebased on next-20121031
* add ifdef of CONFIG_HAVE_ARM_SCU in tegra_resume, this can make the code
  re-usable for the future chips (e.g. A15)
V3:
* update subject and the commit message
V2:
* moving the code that only for Tegra30 inside the ifdef in the tegra_resume
---
 arch/arm/mach-tegra/headsmp.S |   60 +++++++++++++++++++++++++++++++++++++++++
 arch/arm/mach-tegra/reset.c   |    6 ++++
 arch/arm/mach-tegra/sleep.h   |    1 +
 3 files changed, 67 insertions(+), 0 deletions(-)

diff --git a/arch/arm/mach-tegra/headsmp.S b/arch/arm/mach-tegra/headsmp.S
index 93f0370..82dc84b 100644
--- a/arch/arm/mach-tegra/headsmp.S
+++ b/arch/arm/mach-tegra/headsmp.S
@@ -68,6 +68,55 @@ ENTRY(tegra_secondary_startup)
         b       secondary_startup
 ENDPROC(tegra_secondary_startup)
 
+#ifdef CONFIG_PM_SLEEP
+/*
+ *	tegra_resume
+ *
+ *	  CPU boot vector when restarting the a CPU following
+ *	  an LP2 transition. Also branched to by LP0 and LP1 resume after
+ *	  re-enabling sdram.
+ */
+ENTRY(tegra_resume)
+	bl	v7_invalidate_l1
+	/* Enable coresight */
+	mov32	r0, 0xC5ACCE55
+	mcr	p14, 0, r0, c7, c12, 6
+
+	cpu_id	r0
+	cmp	r0, #0				@ CPU0?
+	bne	cpu_resume			@ no
+
+#ifdef CONFIG_ARCH_TEGRA_3x_SOC
+	/* Are we on Tegra20? */
+	mov32	r6, TEGRA_APB_MISC_BASE
+	ldr	r0, [r6, #APB_MISC_GP_HIDREV]
+	and	r0, r0, #0xff00
+	cmp	r0, #(0x20 << 8)
+	beq	1f				@ Yes
+	/* Clear the flow controller flags for this CPU. */
+	mov32	r2, TEGRA_FLOW_CTRL_BASE + FLOW_CTRL_CPU0_CSR	@ CPU0 CSR
+	ldr	r1, [r2]
+	/* Clear event & intr flag */
+	orr	r1, r1, \
+		#FLOW_CTRL_CSR_INTR_FLAG | FLOW_CTRL_CSR_EVENT_FLAG
+	movw	r0, #0x0FFD	@ enable, cluster_switch, immed, & bitmaps
+	bic	r1, r1, r0
+	str	r1, [r2]
+1:
+#endif
+
+#ifdef CONFIG_HAVE_ARM_SCU
+	/* enable SCU */
+	mov32	r0, TEGRA_ARM_PERIF_BASE
+	ldr	r1, [r0]
+	orr	r1, r1, #1
+	str	r1, [r0]
+#endif
+
+	b	cpu_resume
+ENDPROC(tegra_resume)
+#endif
+
 	.align L1_CACHE_SHIFT
 ENTRY(__tegra_cpu_reset_handler_start)
 
@@ -121,6 +170,17 @@ ENTRY(__tegra_cpu_reset_handler)
 1:
 #endif
 
+	/* Waking up from LP2? */
+	ldr	r9, [r12, #RESET_DATA(MASK_LP2)]
+	tst	r9, r11				@ if in_lp2
+	beq	__is_not_lp2
+	ldr	lr, [r12, #RESET_DATA(STARTUP_LP2)]
+	cmp	lr, #0
+	bleq	__die				@ no LP2 startup handler
+	bx	lr
+
+__is_not_lp2:
+
 #ifdef CONFIG_SMP
 	/*
 	 * Can only be secondary boot (initial or hotplug) but CPU 0
diff --git a/arch/arm/mach-tegra/reset.c b/arch/arm/mach-tegra/reset.c
index e05da7d..3fd89ec 100644
--- a/arch/arm/mach-tegra/reset.c
+++ b/arch/arm/mach-tegra/reset.c
@@ -25,6 +25,7 @@
 #include "iomap.h"
 #include "irammap.h"
 #include "reset.h"
+#include "sleep.h"
 #include "fuse.h"
 
 #define TEGRA_IRAM_RESET_BASE (TEGRA_IRAM_BASE + \
@@ -79,5 +80,10 @@ void __init tegra_cpu_reset_handler_init(void)
 		virt_to_phys((void *)tegra_secondary_startup);
 #endif
 
+#ifdef CONFIG_PM_SLEEP
+	__tegra_cpu_reset_handler_data[TEGRA_RESET_STARTUP_LP2] =
+		virt_to_phys((void *)tegra_resume);
+#endif
+
 	tegra_cpu_reset_handler_enable();
 }
diff --git a/arch/arm/mach-tegra/sleep.h b/arch/arm/mach-tegra/sleep.h
index 4889b28..addb83f 100644
--- a/arch/arm/mach-tegra/sleep.h
+++ b/arch/arm/mach-tegra/sleep.h
@@ -72,6 +72,7 @@
 	dsb
 .endm
 #else
+void tegra_resume(void);
 
 #ifdef CONFIG_HOTPLUG_CPU
 void tegra20_hotplug_init(void);
-- 
1.7.0.4

[PATCH V4 3/7] ARM: tegra30: cpuidle: add powered-down state for secondary CPUs

[Joseph Lo]

This supports power-gated idle on secondary CPUs for Tegra30. The
secondary CPUs can go into powered-down state independently. When
CPU goes into this state, it saves it's contexts and puts itself
to flow controlled WFI state. After that, it will been power gated.

Be aware of that, you may see the legacy power state "LP2" in the
code which is exactly the same meaning of "CPU power down".

Based on the work by:
Scott Williams <scwilliams@nvidia.com>

Signed-off-by: Joseph Lo <josephl@nvidia.com>
---
V4:
* rebased on next-20121031
V3:
* remove the redundant header file
* update the subject and the commit message
* rename the LP2 state to "powered-down"
* refine the function name from "tegra30_idle_enter_lp2_cpu_n" to
  "tegra30_cpu_core_power_down"
V2:
* static initialization for idle states when PM_SLEEP is true or not
* using inline fuction to replace the empty fuction when #ifdef false
* convert the phy cpu number by cpu_logical_map
* update the usage of tegra_cpu_lp2_mask (only one copy in the IRAM)
* update the usage of data cache maintenance API for LP2
	* disable L1 data cache
	* v7_flush_dcache_louis
	* exit SMP coherency
---
 arch/arm/mach-tegra/Makefile          |    1 +
 arch/arm/mach-tegra/cpuidle-tegra30.c |   86 +++++++++++++++++++++++++++++++++
 arch/arm/mach-tegra/pm.c              |   74 ++++++++++++++++++++++++++++
 arch/arm/mach-tegra/pm.h              |   30 +++++++++++
 arch/arm/mach-tegra/reset.h           |    9 ++++
 arch/arm/mach-tegra/sleep-tegra30.S   |   22 ++++++++
 arch/arm/mach-tegra/sleep.S           |   29 +++++++++++
 arch/arm/mach-tegra/sleep.h           |    2 +
 8 files changed, 253 insertions(+), 0 deletions(-)
 create mode 100644 arch/arm/mach-tegra/pm.c
 create mode 100644 arch/arm/mach-tegra/pm.h

diff --git a/arch/arm/mach-tegra/Makefile b/arch/arm/mach-tegra/Makefile
index 9b80c1e..6f224f7 100644
--- a/arch/arm/mach-tegra/Makefile
+++ b/arch/arm/mach-tegra/Makefile
@@ -8,6 +8,7 @@ obj-y					+= pmc.o
 obj-y					+= flowctrl.o
 obj-y					+= powergate.o
 obj-y					+= apbio.o
+obj-y					+= pm.o
 obj-$(CONFIG_CPU_IDLE)			+= cpuidle.o
 obj-$(CONFIG_CPU_IDLE)			+= sleep.o
 obj-$(CONFIG_ARCH_TEGRA_2x_SOC)         += tegra20_clocks.o
diff --git a/arch/arm/mach-tegra/cpuidle-tegra30.c b/arch/arm/mach-tegra/cpuidle-tegra30.c
index 37e7551..cc48d7f 100644
--- a/arch/arm/mach-tegra/cpuidle-tegra30.c
+++ b/arch/arm/mach-tegra/cpuidle-tegra30.c
@@ -22,21 +22,107 @@
 #include <linux/kernel.h>
 #include <linux/module.h>
 #include <linux/cpuidle.h>
+#include <linux/cpu_pm.h>
+#include <linux/clockchips.h>
 
 #include <asm/cpuidle.h>
+#include <asm/proc-fns.h>
+#include <asm/suspend.h>
+#include <asm/smp_plat.h>
+
+#include "pm.h"
+#include "sleep.h"
+
+#ifdef CONFIG_PM_SLEEP
+static int tegra30_idle_lp2(struct cpuidle_device *dev,
+			    struct cpuidle_driver *drv,
+			    int index);
+#endif
 
 static struct cpuidle_driver tegra_idle_driver = {
 	.name = "tegra_idle",
 	.owner = THIS_MODULE,
 	.en_core_tk_irqen = 1,
+#ifdef CONFIG_PM_SLEEP
+	.state_count = 2,
+#else
 	.state_count = 1,
+#endif
 	.states = {
 		[0] = ARM_CPUIDLE_WFI_STATE_PWR(600),
+#ifdef CONFIG_PM_SLEEP
+		[1] = {
+			.enter			= tegra30_idle_lp2,
+			.exit_latency		= 2000,
+			.target_residency	= 2200,
+			.power_usage		= 0,
+			.flags			= CPUIDLE_FLAG_TIME_VALID,
+			.name			= "powered-down",
+			.desc			= "CPU power gated",
+		},
+#endif
 	},
 };
 
 static DEFINE_PER_CPU(struct cpuidle_device, tegra_idle_device);
 
+#ifdef CONFIG_PM_SLEEP
+#ifdef CONFIG_SMP
+static bool tegra30_cpu_core_power_down(struct cpuidle_device *dev,
+					struct cpuidle_driver *drv,
+					int index)
+{
+	clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &dev->cpu);
+
+	smp_wmb();
+
+	save_cpu_arch_register();
+
+	cpu_suspend(0, tegra30_sleep_cpu_secondary_finish);
+
+	restore_cpu_arch_register();
+
+	clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &dev->cpu);
+
+	return true;
+}
+#else
+static inline bool tegra30_cpu_core_power_down(struct cpuidle_device *dev,
+					       struct cpuidle_driver *drv,
+					       int index)
+{
+	return true;
+}
+#endif
+
+static int __cpuinit tegra30_idle_lp2(struct cpuidle_device *dev,
+				      struct cpuidle_driver *drv,
+				      int index)
+{
+	u32 cpu = is_smp() ? cpu_logical_map(dev->cpu) : dev->cpu;
+	bool entered_lp2 = false;
+
+	local_fiq_disable();
+
+	tegra_set_cpu_in_lp2(cpu);
+	cpu_pm_enter();
+
+	if (cpu == 0)
+		cpu_do_idle();
+	else
+		entered_lp2 = tegra30_cpu_core_power_down(dev, drv, index);
+
+	cpu_pm_exit();
+	tegra_clear_cpu_in_lp2(cpu);
+
+	local_fiq_enable();
+
+	smp_rmb();
+
+	return (entered_lp2) ? index : 0;
+}
+#endif
+
 int __init tegra30_cpuidle_init(void)
 {
 	int ret;
diff --git a/arch/arm/mach-tegra/pm.c b/arch/arm/mach-tegra/pm.c
new file mode 100644
index 0000000..f88595a
--- /dev/null
+++ b/arch/arm/mach-tegra/pm.c
@@ -0,0 +1,74 @@
+/*
+ * CPU complex suspend & resume functions for Tegra SoCs
+ *
+ * Copyright (c) 2009-2012, NVIDIA Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/kernel.h>
+#include <linux/spinlock.h>
+#include <linux/io.h>
+#include <linux/cpumask.h>
+
+#include "iomap.h"
+#include "reset.h"
+
+#ifdef CONFIG_PM_SLEEP
+static unsigned int g_diag_reg;
+static DEFINE_SPINLOCK(tegra_lp2_lock);
+
+void save_cpu_arch_register(void)
+{
+	/* read diagnostic register */
+	asm("mrc p15, 0, %0, c15, c0, 1" : "=r"(g_diag_reg) : : "cc");
+	return;
+}
+
+void restore_cpu_arch_register(void)
+{
+	/* write diagnostic register */
+	asm("mcr p15, 0, %0, c15, c0, 1" : : "r"(g_diag_reg) : "cc");
+	return;
+}
+
+void __cpuinit tegra_clear_cpu_in_lp2(int phy_cpu_id)
+{
+	u32 *cpu_in_lp2 = tegra_cpu_lp2_mask;
+
+	spin_lock(&tegra_lp2_lock);
+
+	BUG_ON(!(*cpu_in_lp2 & BIT(phy_cpu_id)));
+	*cpu_in_lp2 &= ~BIT(phy_cpu_id);
+
+	spin_unlock(&tegra_lp2_lock);
+}
+
+bool __cpuinit tegra_set_cpu_in_lp2(int phy_cpu_id)
+{
+	bool last_cpu = false;
+	cpumask_t *cpu_lp2_mask = tegra_cpu_lp2_mask;
+	u32 *cpu_in_lp2 = tegra_cpu_lp2_mask;
+
+	spin_lock(&tegra_lp2_lock);
+
+	BUG_ON((*cpu_in_lp2 & BIT(phy_cpu_id)));
+	*cpu_in_lp2 |= BIT(phy_cpu_id);
+
+	if ((phy_cpu_id == 0) && cpumask_equal(cpu_lp2_mask, cpu_online_mask))
+		last_cpu = true;
+
+	spin_unlock(&tegra_lp2_lock);
+	return last_cpu;
+}
+#endif
diff --git a/arch/arm/mach-tegra/pm.h b/arch/arm/mach-tegra/pm.h
new file mode 100644
index 0000000..bcfc45f
--- /dev/null
+++ b/arch/arm/mach-tegra/pm.h
@@ -0,0 +1,30 @@
+/*
+ * Copyright (C) 2010 Google, Inc.
+ * Copyright (c) 2010-2012 NVIDIA Corporation. All rights reserved.
+ *
+ * Author:
+ *	Colin Cross <ccross@google.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef _MACH_TEGRA_PM_H_
+#define _MACH_TEGRA_PM_H_
+
+void save_cpu_arch_register(void);
+void restore_cpu_arch_register(void);
+
+void tegra_clear_cpu_in_lp2(int phy_cpu_id);
+bool tegra_set_cpu_in_lp2(int phy_cpu_id);
+
+#endif /* _MACH_TEGRA_PM_H_ */
diff --git a/arch/arm/mach-tegra/reset.h b/arch/arm/mach-tegra/reset.h
index de88bf8..c90d8e9 100644
--- a/arch/arm/mach-tegra/reset.h
+++ b/arch/arm/mach-tegra/reset.h
@@ -29,6 +29,8 @@
 
 #ifndef __ASSEMBLY__
 
+#include "irammap.h"
+
 extern unsigned long __tegra_cpu_reset_handler_data[TEGRA_RESET_DATA_SIZE];
 
 void __tegra_cpu_reset_handler_start(void);
@@ -36,6 +38,13 @@ void __tegra_cpu_reset_handler(void);
 void __tegra_cpu_reset_handler_end(void);
 void tegra_secondary_startup(void);
 
+#ifdef CONFIG_PM_SLEEP
+#define tegra_cpu_lp2_mask \
+	(IO_ADDRESS(TEGRA_IRAM_BASE + TEGRA_IRAM_RESET_HANDLER_OFFSET + \
+	((u32)&__tegra_cpu_reset_handler_data[TEGRA_RESET_MASK_LP2] - \
+	 (u32)__tegra_cpu_reset_handler_start)))
+#endif
+
 #define tegra_cpu_reset_handler_offset \
 		((u32)__tegra_cpu_reset_handler - \
 		 (u32)__tegra_cpu_reset_handler_start)
diff --git a/arch/arm/mach-tegra/sleep-tegra30.S b/arch/arm/mach-tegra/sleep-tegra30.S
index be7614b..59984d7 100644
--- a/arch/arm/mach-tegra/sleep-tegra30.S
+++ b/arch/arm/mach-tegra/sleep-tegra30.S
@@ -17,6 +17,7 @@
 #include <linux/linkage.h>
 
 #include <asm/assembler.h>
+#include <asm/asm-offsets.h>
 
 #include "sleep.h"
 #include "flowctrl.h"
@@ -80,6 +81,7 @@ delay_1:
 	ldr	r3, [r1]			@ read CSR
 	str	r3, [r1]			@ clear CSR
 	tst	r0, #TEGRA30_POWER_HOTPLUG_SHUTDOWN
+	moveq   r3, #FLOW_CTRL_WAIT_FOR_INTERRUPT	@ For LP2
 	movne	r3, #FLOW_CTRL_WAITEVENT		@ For hotplug
 	str	r3, [r2]
 	ldr	r0, [r2]
@@ -103,3 +105,23 @@ wfe_war:
 
 ENDPROC(tegra30_cpu_shutdown)
 #endif
+
+#ifdef CONFIG_PM_SLEEP
+/*
+ * tegra30_sleep_cpu_secondary_finish(unsigned long v2p)
+ *
+ * Enters LP2 on secondary CPU by exiting coherency and powergating the CPU.
+ */
+ENTRY(tegra30_sleep_cpu_secondary_finish)
+	mov	r7, lr
+
+	/* Flush and disable the L1 data cache */
+	bl	tegra_disable_clean_inv_dcache
+
+	/* Powergate this CPU. */
+	mov	r0, #0                          @ power mode flags (!hotplug)
+	bl	tegra30_cpu_shutdown
+	mov	r0, #1                          @ never return here
+	mov	pc, r7
+ENDPROC(tegra30_sleep_cpu_secondary_finish)
+#endif
diff --git a/arch/arm/mach-tegra/sleep.S b/arch/arm/mach-tegra/sleep.S
index 08e9481..91548a7 100644
--- a/arch/arm/mach-tegra/sleep.S
+++ b/arch/arm/mach-tegra/sleep.S
@@ -25,9 +25,38 @@
 #include <linux/linkage.h>
 
 #include <asm/assembler.h>
+#include <asm/cp15.h>
 
 #include "iomap.h"
 
 #include "flowctrl.h"
 #include "sleep.h"
 
+#ifdef CONFIG_PM_SLEEP
+/*
+ * tegra_disable_clean_inv_dcache
+ *
+ * disable, clean & invalidate the D-cache
+ *
+ * Corrupted registers: r1-r3, r6, r8, r9-r11
+ */
+ENTRY(tegra_disable_clean_inv_dcache)
+	stmfd	sp!, {r0, r4-r5, r7, r9-r11, lr}
+	dmb					@ ensure ordering
+
+	/* Disable the D-cache */
+	mrc	p15, 0, r2, c1, c0, 0
+	bic	r2, r2, #CR_C
+	mcr	p15, 0, r2, c1, c0, 0
+	isb
+
+	/* Flush the D-cache */
+	bl	v7_flush_dcache_louis
+
+	/* Trun off coherency */
+	exit_smp r4, r5
+
+	ldmfd	sp!, {r0, r4-r5, r7, r9-r11, pc}
+ENDPROC(tegra_disable_clean_inv_dcache)
+
+#endif
diff --git a/arch/arm/mach-tegra/sleep.h b/arch/arm/mach-tegra/sleep.h
index addb83f..bacf549 100644
--- a/arch/arm/mach-tegra/sleep.h
+++ b/arch/arm/mach-tegra/sleep.h
@@ -82,5 +82,7 @@ static inline void tegra20_hotplug_init(void) {}
 static inline void tegra30_hotplug_init(void) {}
 #endif
 
+int tegra30_sleep_cpu_secondary_finish(unsigned long);
+
 #endif
 #endif
-- 
1.7.0.4

[PATCH V4 4/7] ARM: tegra30: common: enable csite clock

[Joseph Lo]

Enable csite (debug and trace controller) clock at init to prevent it
be disabled. And this also the necessary clock for CPU be brought up or
resumed from a power-gating low power state.

Signed-off-by: Joseph Lo <josephl@nvidia.com>
---
V4:
* no change
V3:
* update the commit message
V2:
* no change
---
 arch/arm/mach-tegra/common.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/arch/arm/mach-tegra/common.c b/arch/arm/mach-tegra/common.c
index 6c04a18..35b7dd3 100644
--- a/arch/arm/mach-tegra/common.c
+++ b/arch/arm/mach-tegra/common.c
@@ -108,6 +108,7 @@ static __initdata struct tegra_clk_init_table tegra30_clk_init_table[] = {
 	{ "sclk",	"pll_p_out4",	102000000,	true },
 	{ "hclk",	"sclk",		102000000,	true },
 	{ "pclk",	"hclk",		51000000,	true },
+	{ "csite",	NULL,		0,		true },
 	{ NULL,		NULL,		0,		0},
 };
 #endif
-- 
1.7.0.4

[PATCH V4 5/7] ARM: tegra30: clocks: add CPU low-power function into tegra_cpu_car_ops

[Joseph Lo]

Add suspend, resume and rail_off_ready API into tegra_cpu_car_ops. These
functions were used for CPU powered-down state maintenance. One thing
needs to notice the rail_off_ready API only availalbe for cpu_g cluster
not cpu_lp cluster.

Signed-off-by: Joseph Lo <josephl@nvidia.com>
---
V4:
* rebased on next-20121031
V3:
* update the commit message
V2:
* no change
---
 arch/arm/mach-tegra/tegra30_clocks.c |  108 ++++++++++++++++++++++++++++++++++
 arch/arm/mach-tegra/tegra_cpu_car.h  |   37 ++++++++++++
 2 files changed, 145 insertions(+), 0 deletions(-)

diff --git a/arch/arm/mach-tegra/tegra30_clocks.c b/arch/arm/mach-tegra/tegra30_clocks.c
index f5b453f..efc000e 100644
--- a/arch/arm/mach-tegra/tegra30_clocks.c
+++ b/arch/arm/mach-tegra/tegra30_clocks.c
@@ -31,6 +31,8 @@
 
 #include <asm/clkdev.h>
 
+#include <mach/powergate.h>
+
 #include "clock.h"
 #include "fuse.h"
 #include "iomap.h"
@@ -309,6 +311,31 @@
 #define CPU_CLOCK(cpu)	(0x1 << (8 + cpu))
 #define CPU_RESET(cpu)	(0x1111ul << (cpu))
 
+#define CLK_RESET_CCLK_BURST	0x20
+#define CLK_RESET_CCLK_DIVIDER  0x24
+#define CLK_RESET_PLLX_BASE	0xe0
+#define CLK_RESET_PLLX_MISC	0xe4
+
+#define CLK_RESET_SOURCE_CSITE	0x1d4
+
+#define CLK_RESET_CCLK_BURST_POLICY_SHIFT	28
+#define CLK_RESET_CCLK_RUN_POLICY_SHIFT		4
+#define CLK_RESET_CCLK_IDLE_POLICY_SHIFT	0
+#define CLK_RESET_CCLK_IDLE_POLICY		1
+#define CLK_RESET_CCLK_RUN_POLICY		2
+#define CLK_RESET_CCLK_BURST_POLICY_PLLX	8
+
+#ifdef CONFIG_PM_SLEEP
+static struct cpu_clk_suspend_context {
+	u32 pllx_misc;
+	u32 pllx_base;
+
+	u32 cpu_burst;
+	u32 clk_csite_src;
+	u32 cclk_divider;
+} tegra30_cpu_clk_sctx;
+#endif
+
 /**
 * Structure defining the fields for USB UTMI clocks Parameters.
 */
@@ -2386,12 +2413,93 @@ static void tegra30_disable_cpu_clock(u32 cpu)
 	       reg_clk_base + TEGRA_CLK_RST_CONTROLLER_CLK_CPU_CMPLX);
 }
 
+#ifdef CONFIG_PM_SLEEP
+static bool tegra30_cpu_rail_off_ready(void)
+{
+	unsigned int cpu_rst_status;
+	int cpu_pwr_status;
+
+	cpu_rst_status = readl(reg_clk_base +
+			       TEGRA30_CLK_RST_CONTROLLER_CPU_CMPLX_STATUS);
+	cpu_pwr_status = tegra_powergate_is_powered(TEGRA_POWERGATE_CPU1) ||
+			 tegra_powergate_is_powered(TEGRA_POWERGATE_CPU2) ||
+			 tegra_powergate_is_powered(TEGRA_POWERGATE_CPU3);
+
+	if (((cpu_rst_status & 0xE) != 0xE) || cpu_pwr_status)
+		return false;
+
+	return true;
+}
+
+static void tegra30_cpu_clock_suspend(void)
+{
+	/* switch coresite to clk_m, save off original source */
+	tegra30_cpu_clk_sctx.clk_csite_src =
+				readl(reg_clk_base + CLK_RESET_SOURCE_CSITE);
+	writel(3<<30, reg_clk_base + CLK_RESET_SOURCE_CSITE);
+
+	tegra30_cpu_clk_sctx.cpu_burst =
+				readl(reg_clk_base + CLK_RESET_CCLK_BURST);
+	tegra30_cpu_clk_sctx.pllx_base =
+				readl(reg_clk_base + CLK_RESET_PLLX_BASE);
+	tegra30_cpu_clk_sctx.pllx_misc =
+				readl(reg_clk_base + CLK_RESET_PLLX_MISC);
+	tegra30_cpu_clk_sctx.cclk_divider =
+				readl(reg_clk_base + CLK_RESET_CCLK_DIVIDER);
+}
+
+static void tegra30_cpu_clock_resume(void)
+{
+	unsigned int reg, policy;
+
+	/* Is CPU complex already running on PLLX? */
+	reg = readl(reg_clk_base + CLK_RESET_CCLK_BURST);
+	policy = (reg >> CLK_RESET_CCLK_BURST_POLICY_SHIFT) & 0xF;
+
+	if (policy == CLK_RESET_CCLK_IDLE_POLICY)
+		reg = (reg >> CLK_RESET_CCLK_IDLE_POLICY_SHIFT) & 0xF;
+	else if (policy == CLK_RESET_CCLK_RUN_POLICY)
+		reg = (reg >> CLK_RESET_CCLK_RUN_POLICY_SHIFT) & 0xF;
+	else
+		BUG();
+
+	if (reg != CLK_RESET_CCLK_BURST_POLICY_PLLX) {
+		/* restore PLLX settings if CPU is on different PLL */
+		writel(tegra30_cpu_clk_sctx.pllx_misc,
+					reg_clk_base + CLK_RESET_PLLX_MISC);
+		writel(tegra30_cpu_clk_sctx.pllx_base,
+					reg_clk_base + CLK_RESET_PLLX_BASE);
+
+		/* wait for PLL stabilization if PLLX was enabled */
+		if (tegra30_cpu_clk_sctx.pllx_base & (1 << 30))
+			udelay(300);
+	}
+
+	/*
+	 * Restore original burst policy setting for calls resulting from CPU
+	 * LP2 in idle or system suspend.
+	 */
+	writel(tegra30_cpu_clk_sctx.cclk_divider,
+					reg_clk_base + CLK_RESET_CCLK_DIVIDER);
+	writel(tegra30_cpu_clk_sctx.cpu_burst,
+					reg_clk_base + CLK_RESET_CCLK_BURST);
+
+	writel(tegra30_cpu_clk_sctx.clk_csite_src,
+					reg_clk_base + CLK_RESET_SOURCE_CSITE);
+}
+#endif
+
 static struct tegra_cpu_car_ops tegra30_cpu_car_ops = {
 	.wait_for_reset	= tegra30_wait_cpu_in_reset,
 	.put_in_reset	= tegra30_put_cpu_in_reset,
 	.out_of_reset	= tegra30_cpu_out_of_reset,
 	.enable_clock	= tegra30_enable_cpu_clock,
 	.disable_clock	= tegra30_disable_cpu_clock,
+#ifdef CONFIG_PM_SLEEP
+	.rail_off_ready	= tegra30_cpu_rail_off_ready,
+	.suspend	= tegra30_cpu_clock_suspend,
+	.resume		= tegra30_cpu_clock_resume,
+#endif
 };
 
 void __init tegra30_cpu_car_ops_init(void)
diff --git a/arch/arm/mach-tegra/tegra_cpu_car.h b/arch/arm/mach-tegra/tegra_cpu_car.h
index 30d063a..9764d31 100644
--- a/arch/arm/mach-tegra/tegra_cpu_car.h
+++ b/arch/arm/mach-tegra/tegra_cpu_car.h
@@ -30,6 +30,12 @@
  *	CPU clock un-gate
  * disable_clock:
  *	CPU clock gate
+ * rail_off_ready:
+ *	CPU is ready for rail off
+ * suspend:
+ *	save the clock settings when CPU go into low-power state
+ * resume:
+ *	restore the clock settings when CPU exit low-power state
  */
 struct tegra_cpu_car_ops {
 	void (*wait_for_reset)(u32 cpu);
@@ -37,6 +43,11 @@ struct tegra_cpu_car_ops {
 	void (*out_of_reset)(u32 cpu);
 	void (*enable_clock)(u32 cpu);
 	void (*disable_clock)(u32 cpu);
+#ifdef CONFIG_PM_SLEEP
+	bool (*rail_off_ready)(void);
+	void (*suspend)(void);
+	void (*resume)(void);
+#endif
 };
 
 extern struct tegra_cpu_car_ops *tegra_cpu_car_ops;
@@ -81,6 +92,32 @@ static inline void tegra_disable_cpu_clock(u32 cpu)
 	tegra_cpu_car_ops->disable_clock(cpu);
 }
 
+#ifdef CONFIG_PM_SLEEP
+static inline bool tegra_cpu_rail_off_ready(void)
+{
+	if (WARN_ON(!tegra_cpu_car_ops->rail_off_ready))
+		return false;
+
+	return tegra_cpu_car_ops->rail_off_ready();
+}
+
+static inline void tegra_cpu_clock_suspend(void)
+{
+	if (WARN_ON(!tegra_cpu_car_ops->suspend))
+		return;
+
+	tegra_cpu_car_ops->suspend();
+}
+
+static inline void tegra_cpu_clock_resume(void)
+{
+	if (WARN_ON(!tegra_cpu_car_ops->resume))
+		return;
+
+	tegra_cpu_car_ops->resume();
+}
+#endif
+
 void tegra20_cpu_car_ops_init(void);
 void tegra30_cpu_car_ops_init(void);
 
-- 
1.7.0.4

[PATCH V4 6/7] ARM: tegra30: flowctrl: add cpu_suspend_exter/exit function

[Joseph Lo]

The flow controller can help CPU to go into suspend mode (powered-down
state). When CPU go into powered-down state, it needs some careful
settings before getting into and after leaving. The enter and exit
functions do that by configuring appropriate mode for flow controller.

Signed-off-by: Joseph Lo <josephl@nvidia.com>
---
V4:
* rebased on next-20121031
V3:
* update the commit message
V2:
* no change
---
 arch/arm/mach-tegra/flowctrl.c |   47 ++++++++++++++++++++++++++++++++++++++++
 arch/arm/mach-tegra/flowctrl.h |    8 ++++++
 2 files changed, 55 insertions(+), 0 deletions(-)

diff --git a/arch/arm/mach-tegra/flowctrl.c b/arch/arm/mach-tegra/flowctrl.c
index ffaa286..a2250dd 100644
--- a/arch/arm/mach-tegra/flowctrl.c
+++ b/arch/arm/mach-tegra/flowctrl.c
@@ -21,6 +21,7 @@
 #include <linux/init.h>
 #include <linux/kernel.h>
 #include <linux/io.h>
+#include <linux/cpumask.h>
 
 #include "flowctrl.h"
 #include "iomap.h"
@@ -50,6 +51,14 @@ static void flowctrl_update(u8 offset, u32 value)
 	readl_relaxed(addr);
 }
 
+u32 flowctrl_read_cpu_csr(unsigned int cpuid)
+{
+	u8 offset = flowctrl_offset_cpu_csr[cpuid];
+	void __iomem *addr = IO_ADDRESS(TEGRA_FLOW_CTRL_BASE) + offset;
+
+	return readl(addr);
+}
+
 void flowctrl_write_cpu_csr(unsigned int cpuid, u32 value)
 {
 	return flowctrl_update(flowctrl_offset_cpu_csr[cpuid], value);
@@ -59,3 +68,41 @@ void flowctrl_write_cpu_halt(unsigned int cpuid, u32 value)
 {
 	return flowctrl_update(flowctrl_offset_halt_cpu[cpuid], value);
 }
+
+void flowctrl_cpu_suspend_enter(unsigned int cpuid)
+{
+	unsigned int reg;
+	int i;
+
+	reg = flowctrl_read_cpu_csr(cpuid);
+	reg &= ~TEGRA30_FLOW_CTRL_CSR_WFE_BITMAP;	/* clear wfe bitmap */
+	reg &= ~TEGRA30_FLOW_CTRL_CSR_WFI_BITMAP;	/* clear wfi bitmap */
+	reg |= FLOW_CTRL_CSR_INTR_FLAG;			/* clear intr flag */
+	reg |= FLOW_CTRL_CSR_EVENT_FLAG;		/* clear event flag */
+	reg |= TEGRA30_FLOW_CTRL_CSR_WFI_CPU0 << cpuid;	/* pwr gating on wfi */
+	reg |= FLOW_CTRL_CSR_ENABLE;			/* pwr gating */
+	flowctrl_write_cpu_csr(cpuid, reg);
+
+	for (i = 0; i < num_possible_cpus(); i++) {
+		if (i == cpuid)
+			continue;
+		reg = flowctrl_read_cpu_csr(i);
+		reg |= FLOW_CTRL_CSR_EVENT_FLAG;
+		reg |= FLOW_CTRL_CSR_INTR_FLAG;
+		flowctrl_write_cpu_csr(i, reg);
+	}
+}
+
+void flowctrl_cpu_suspend_exit(unsigned int cpuid)
+{
+	unsigned int reg;
+
+	/* Disable powergating via flow controller for CPU0 */
+	reg = flowctrl_read_cpu_csr(cpuid);
+	reg &= ~TEGRA30_FLOW_CTRL_CSR_WFE_BITMAP;	/* clear wfe bitmap */
+	reg &= ~TEGRA30_FLOW_CTRL_CSR_WFI_BITMAP;	/* clear wfi bitmap */
+	reg &= ~FLOW_CTRL_CSR_ENABLE;			/* clear enable */
+	reg |= FLOW_CTRL_CSR_INTR_FLAG;			/* clear intr */
+	reg |= FLOW_CTRL_CSR_EVENT_FLAG;		/* clear event */
+	flowctrl_write_cpu_csr(cpuid, reg);
+}
diff --git a/arch/arm/mach-tegra/flowctrl.h b/arch/arm/mach-tegra/flowctrl.h
index 1942817..0798dec 100644
--- a/arch/arm/mach-tegra/flowctrl.h
+++ b/arch/arm/mach-tegra/flowctrl.h
@@ -34,9 +34,17 @@
 #define FLOW_CTRL_HALT_CPU1_EVENTS	0x14
 #define FLOW_CTRL_CPU1_CSR		0x18
 
+#define TEGRA30_FLOW_CTRL_CSR_WFI_CPU0		(1 << 8)
+#define TEGRA30_FLOW_CTRL_CSR_WFE_BITMAP	(0xF << 4)
+#define TEGRA30_FLOW_CTRL_CSR_WFI_BITMAP	(0xF << 8)
+
 #ifndef __ASSEMBLY__
+u32 flowctrl_read_cpu_csr(unsigned int cpuid);
 void flowctrl_write_cpu_csr(unsigned int cpuid, u32 value);
 void flowctrl_write_cpu_halt(unsigned int cpuid, u32 value);
+
+void flowctrl_cpu_suspend_enter(unsigned int cpuid);
+void flowctrl_cpu_suspend_exit(unsigned int cpuid);
 #endif
 
 #endif
-- 
1.7.0.4

[PATCH V4 7/7] ARM: tegra30: cpuidle: add powered-down state for CPU0

[Joseph Lo]

This is a power gating idle mode. It support power gating vdd_cpu rail
after all cpu cores in "powered-down" status. For Tegra30, the CPU0 can
enter this state only when all secondary CPU is offline. We need to take
care and make sure whole secondary CPUs were offline and checking the
CPU power gate status. After that, the CPU0 can go into "powered-down"
state safely. Then shut off the CPU rail.

Be aware of that, you may see the legacy power state "LP2" in the code
which is exactly the same meaning of "CPU power down".

Base on the work by:
Scott Williams <scwilliams@nvidia.com>

Signed-off-by: Joseph Lo <josephl@nvidia.com>
---
V4:
* rebased on next-20121031
V3:
* Fix the potential issue that will cause the CPU corruption when CPUs
  doing LP2. All CPUs entering LP2 is not working. The CPU0 can enter
  LP2 only when all secondary CPU is offline. This can be covered by
  CPUquiet and cluster switching mechanism.
* update the commit message
* rename the funciton name from "tegra30_idle_enter_lp2_cpu_0" to
  "tegra30_cpu_cluster_power_down"
* remove the redundant "flush_cache_all" in "tegra_idle_lp2_last"
  because the "cpu_suspend" already do that after save the CPU contexts
V2:
* refine the pclk usage in "set_power_timers"
---
 arch/arm/mach-tegra/cpuidle-tegra30.c |   44 +++++++++-
 arch/arm/mach-tegra/pm.c              |  144 +++++++++++++++++++++++++++++++++
 arch/arm/mach-tegra/pm.h              |    3 +
 arch/arm/mach-tegra/sleep-tegra30.S   |   44 ++++++++++
 arch/arm/mach-tegra/sleep.S           |   42 ++++++++++
 arch/arm/mach-tegra/sleep.h           |    2 +
 6 files changed, 275 insertions(+), 4 deletions(-)

diff --git a/arch/arm/mach-tegra/cpuidle-tegra30.c b/arch/arm/mach-tegra/cpuidle-tegra30.c
index cc48d7f..5e8cbf5 100644
--- a/arch/arm/mach-tegra/cpuidle-tegra30.c
+++ b/arch/arm/mach-tegra/cpuidle-tegra30.c
@@ -32,6 +32,7 @@
 
 #include "pm.h"
 #include "sleep.h"
+#include "tegra_cpu_car.h"
 
 #ifdef CONFIG_PM_SLEEP
 static int tegra30_idle_lp2(struct cpuidle_device *dev,
@@ -67,6 +68,31 @@ static struct cpuidle_driver tegra_idle_driver = {
 static DEFINE_PER_CPU(struct cpuidle_device, tegra_idle_device);
 
 #ifdef CONFIG_PM_SLEEP
+static bool tegra30_cpu_cluster_power_down(struct cpuidle_device *dev,
+					   struct cpuidle_driver *drv,
+					   int index)
+{
+	struct cpuidle_state *state = &drv->states[index];
+	u32 cpu_on_time = state->exit_latency;
+	u32 cpu_off_time = state->target_residency - state->exit_latency;
+
+	/* All CPUs entering LP2 is not working.
+	 * Don't let CPU0 enter LP2 when any secondary CPU is online.
+	 */
+	if (num_online_cpus() > 1 || !tegra_cpu_rail_off_ready()) {
+		cpu_do_idle();
+		return false;
+	}
+
+	clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &dev->cpu);
+
+	tegra_idle_lp2_last(cpu_on_time, cpu_off_time);
+
+	clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &dev->cpu);
+
+	return true;
+}
+
 #ifdef CONFIG_SMP
 static bool tegra30_cpu_core_power_down(struct cpuidle_device *dev,
 					struct cpuidle_driver *drv,
@@ -101,16 +127,22 @@ static int __cpuinit tegra30_idle_lp2(struct cpuidle_device *dev,
 {
 	u32 cpu = is_smp() ? cpu_logical_map(dev->cpu) : dev->cpu;
 	bool entered_lp2 = false;
+	bool last_cpu;
 
 	local_fiq_disable();
 
-	tegra_set_cpu_in_lp2(cpu);
+	last_cpu = tegra_set_cpu_in_lp2(cpu);
 	cpu_pm_enter();
 
-	if (cpu == 0)
-		cpu_do_idle();
-	else
+	if (cpu == 0) {
+		if (last_cpu)
+			entered_lp2 = tegra30_cpu_cluster_power_down(dev, drv,
+								     index);
+		else
+			cpu_do_idle();
+	} else {
 		entered_lp2 = tegra30_cpu_core_power_down(dev, drv, index);
+	}
 
 	cpu_pm_exit();
 	tegra_clear_cpu_in_lp2(cpu);
@@ -130,6 +162,10 @@ int __init tegra30_cpuidle_init(void)
 	struct cpuidle_device *dev;
 	struct cpuidle_driver *drv = &tegra_idle_driver;
 
+#ifdef CONFIG_PM_SLEEP
+	tegra_tear_down_cpu = tegra30_tear_down_cpu;
+#endif
+
 	ret = cpuidle_register_driver(&tegra_idle_driver);
 	if (ret) {
 		pr_err("CPUidle driver registration failed\n");
diff --git a/arch/arm/mach-tegra/pm.c b/arch/arm/mach-tegra/pm.c
index f88595a..1460c3d 100644
--- a/arch/arm/mach-tegra/pm.c
+++ b/arch/arm/mach-tegra/pm.c
@@ -20,13 +20,36 @@
 #include <linux/spinlock.h>
 #include <linux/io.h>
 #include <linux/cpumask.h>
+#include <linux/delay.h>
+#include <linux/cpu_pm.h>
+#include <linux/clk.h>
+#include <linux/err.h>
+
+#include <asm/smp_plat.h>
+#include <asm/cacheflush.h>
+#include <asm/suspend.h>
+#include <asm/idmap.h>
+#include <asm/proc-fns.h>
+#include <asm/tlbflush.h>
 
 #include "iomap.h"
 #include "reset.h"
+#include "flowctrl.h"
+#include "sleep.h"
+#include "tegra_cpu_car.h"
+
+#define TEGRA_POWER_CPU_PWRREQ_OE	(1 << 16)  /* CPU pwr req enable */
+
+#define PMC_CTRL		0x0
+#define PMC_CPUPWRGOOD_TIMER	0xc8
+#define PMC_CPUPWROFF_TIMER	0xcc
 
 #ifdef CONFIG_PM_SLEEP
 static unsigned int g_diag_reg;
 static DEFINE_SPINLOCK(tegra_lp2_lock);
+static void __iomem *pmc = IO_ADDRESS(TEGRA_PMC_BASE);
+static struct clk *tegra_pclk;
+void (*tegra_tear_down_cpu)(void);
 
 void save_cpu_arch_register(void)
 {
@@ -42,6 +65,89 @@ void restore_cpu_arch_register(void)
 	return;
 }
 
+static void set_power_timers(unsigned long us_on, unsigned long us_off)
+{
+	unsigned long long ticks;
+	unsigned long long pclk;
+	unsigned long rate;
+	static unsigned long tegra_last_pclk;
+
+	if (tegra_pclk == NULL) {
+		tegra_pclk = clk_get_sys(NULL, "pclk");
+		WARN_ON(IS_ERR(tegra_pclk));
+	}
+
+	rate = clk_get_rate(tegra_pclk);
+
+	if (WARN_ON_ONCE(rate <= 0))
+		pclk = 100000000;
+	else
+		pclk = rate;
+
+	if ((rate != tegra_last_pclk)) {
+		ticks = (us_on * pclk) + 999999ull;
+		do_div(ticks, 1000000);
+		writel((unsigned long)ticks, pmc + PMC_CPUPWRGOOD_TIMER);
+
+		ticks = (us_off * pclk) + 999999ull;
+		do_div(ticks, 1000000);
+		writel((unsigned long)ticks, pmc + PMC_CPUPWROFF_TIMER);
+		wmb();
+	}
+	tegra_last_pclk = pclk;
+}
+
+/*
+ * restore_cpu_complex
+ *
+ * restores cpu clock setting, clears flow controller
+ *
+ * Always called on CPU 0.
+ */
+static void restore_cpu_complex(void)
+{
+	int cpu = smp_processor_id();
+
+	BUG_ON(cpu != 0);
+
+#ifdef CONFIG_SMP
+	cpu = cpu_logical_map(cpu);
+#endif
+
+	/* Restore the CPU clock settings */
+	tegra_cpu_clock_resume();
+
+	flowctrl_cpu_suspend_exit(cpu);
+
+	restore_cpu_arch_register();
+}
+
+/*
+ * suspend_cpu_complex
+ *
+ * saves pll state for use by restart_plls, prepares flow controller for
+ * transition to suspend state
+ *
+ * Must always be called on cpu 0.
+ */
+static void suspend_cpu_complex(void)
+{
+	int cpu = smp_processor_id();
+
+	BUG_ON(cpu != 0);
+
+#ifdef CONFIG_SMP
+	cpu = cpu_logical_map(cpu);
+#endif
+
+	/* Save the CPU clock settings */
+	tegra_cpu_clock_suspend();
+
+	flowctrl_cpu_suspend_enter(cpu);
+
+	save_cpu_arch_register();
+}
+
 void __cpuinit tegra_clear_cpu_in_lp2(int phy_cpu_id)
 {
 	u32 *cpu_in_lp2 = tegra_cpu_lp2_mask;
@@ -71,4 +177,42 @@ bool __cpuinit tegra_set_cpu_in_lp2(int phy_cpu_id)
 	spin_unlock(&tegra_lp2_lock);
 	return last_cpu;
 }
+
+static int tegra_sleep_cpu(unsigned long v2p)
+{
+	/* Switch to the identity mapping. */
+	cpu_switch_mm(idmap_pgd, &init_mm);
+
+	/* Flush the TLB. */
+	local_flush_tlb_all();
+
+	tegra_sleep_cpu_finish(v2p);
+
+	/* should never here */
+	BUG();
+
+	return 0;
+}
+
+void tegra_idle_lp2_last(u32 cpu_on_time, u32 cpu_off_time)
+{
+	u32 mode;
+
+	/* Only the last cpu down does the final suspend steps */
+	mode = readl(pmc + PMC_CTRL);
+	mode |= TEGRA_POWER_CPU_PWRREQ_OE;
+	writel(mode, pmc + PMC_CTRL);
+
+	set_power_timers(cpu_on_time, cpu_off_time);
+
+	cpu_cluster_pm_enter();
+	suspend_cpu_complex();
+	outer_disable();
+
+	cpu_suspend(PHYS_OFFSET - PAGE_OFFSET, &tegra_sleep_cpu);
+
+	outer_resume();
+	restore_cpu_complex();
+	cpu_cluster_pm_exit();
+}
 #endif
diff --git a/arch/arm/mach-tegra/pm.h b/arch/arm/mach-tegra/pm.h
index bcfc45f..512345c 100644
--- a/arch/arm/mach-tegra/pm.h
+++ b/arch/arm/mach-tegra/pm.h
@@ -27,4 +27,7 @@ void restore_cpu_arch_register(void);
 void tegra_clear_cpu_in_lp2(int phy_cpu_id);
 bool tegra_set_cpu_in_lp2(int phy_cpu_id);
 
+void tegra_idle_lp2_last(u32 cpu_on_time, u32 cpu_off_time);
+extern void (*tegra_tear_down_cpu)(void);
+
 #endif /* _MACH_TEGRA_PM_H_ */
diff --git a/arch/arm/mach-tegra/sleep-tegra30.S b/arch/arm/mach-tegra/sleep-tegra30.S
index 59984d7..562a8e7 100644
--- a/arch/arm/mach-tegra/sleep-tegra30.S
+++ b/arch/arm/mach-tegra/sleep-tegra30.S
@@ -124,4 +124,48 @@ ENTRY(tegra30_sleep_cpu_secondary_finish)
 	mov	r0, #1                          @ never return here
 	mov	pc, r7
 ENDPROC(tegra30_sleep_cpu_secondary_finish)
+
+/*
+ * tegra30_tear_down_cpu
+ *
+ * Switches the CPU to enter sleep.
+ */
+ENTRY(tegra30_tear_down_cpu)
+	mov32	r6, TEGRA_FLOW_CTRL_BASE
+
+	b	tegra30_enter_sleep
+ENDPROC(tegra30_tear_down_cpu)
+
+/*
+ * tegra30_enter_sleep
+ *
+ * uses flow controller to enter sleep state
+ * executes from IRAM with SDRAM in selfrefresh when target state is LP0 or LP1
+ * executes from SDRAM with target state is LP2
+ * r6 = TEGRA_FLOW_CTRL_BASE
+ */
+tegra30_enter_sleep:
+	cpu_id	r1
+
+	cpu_to_csr_reg	r2, r1
+	ldr	r0, [r6, r2]
+	orr	r0, r0, #FLOW_CTRL_CSR_INTR_FLAG | FLOW_CTRL_CSR_EVENT_FLAG
+	orr	r0, r0, #FLOW_CTRL_CSR_ENABLE
+	str	r0, [r6, r2]
+
+	mov	r0, #FLOW_CTRL_WAIT_FOR_INTERRUPT
+	orr	r0, r0, #FLOW_CTRL_HALT_CPU_IRQ | FLOW_CTRL_HALT_CPU_FIQ
+	cpu_to_halt_reg r2, r1
+	str	r0, [r6, r2]
+	dsb
+	ldr	r0, [r6, r2] /* memory barrier */
+
+halted:
+	isb
+	dsb
+	wfi	/* CPU should be power gated here */
+
+	/* !!!FIXME!!! Implement halt failure handler */
+	b	halted
+
 #endif
diff --git a/arch/arm/mach-tegra/sleep.S b/arch/arm/mach-tegra/sleep.S
index 91548a7..88f4de9 100644
--- a/arch/arm/mach-tegra/sleep.S
+++ b/arch/arm/mach-tegra/sleep.S
@@ -25,6 +25,7 @@
 #include <linux/linkage.h>
 
 #include <asm/assembler.h>
+#include <asm/cache.h>
 #include <asm/cp15.h>
 
 #include "iomap.h"
@@ -59,4 +60,45 @@ ENTRY(tegra_disable_clean_inv_dcache)
 	ldmfd	sp!, {r0, r4-r5, r7, r9-r11, pc}
 ENDPROC(tegra_disable_clean_inv_dcache)
 
+/*
+ * tegra_sleep_cpu_finish(unsigned long v2p)
+ *
+ * enters suspend in LP2 by turning off the mmu and jumping to
+ * tegra?_tear_down_cpu
+ */
+ENTRY(tegra_sleep_cpu_finish)
+	/* Flush and disable the L1 data cache */
+	bl	tegra_disable_clean_inv_dcache
+
+	mov32	r6, tegra_tear_down_cpu
+	ldr	r1, [r6]
+	add	r1, r1, r0
+
+	mov32	r3, tegra_shut_off_mmu
+	add	r3, r3, r0
+	mov	r0, r1
+
+	mov	pc, r3
+ENDPROC(tegra_sleep_cpu_finish)
+
+/*
+ * tegra_shut_off_mmu
+ *
+ * r0 = physical address to jump to with mmu off
+ *
+ * called with VA=PA mapping
+ * turns off MMU, icache, dcache and branch prediction
+ */
+	.align	L1_CACHE_SHIFT
+	.pushsection	.idmap.text, "ax"
+ENTRY(tegra_shut_off_mmu)
+	mrc	p15, 0, r3, c1, c0, 0
+	movw	r2, #CR_I | CR_Z | CR_C | CR_M
+	bic	r3, r3, r2
+	dsb
+	mcr	p15, 0, r3, c1, c0, 0
+	isb
+	mov	pc, r0
+ENDPROC(tegra_shut_off_mmu)
+	.popsection
 #endif
diff --git a/arch/arm/mach-tegra/sleep.h b/arch/arm/mach-tegra/sleep.h
index bacf549..6e1b949 100644
--- a/arch/arm/mach-tegra/sleep.h
+++ b/arch/arm/mach-tegra/sleep.h
@@ -73,6 +73,7 @@
 .endm
 #else
 void tegra_resume(void);
+int tegra_sleep_cpu_finish(unsigned long);
 
 #ifdef CONFIG_HOTPLUG_CPU
 void tegra20_hotplug_init(void);
@@ -83,6 +84,7 @@ static inline void tegra30_hotplug_init(void) {}
 #endif
 
 int tegra30_sleep_cpu_secondary_finish(unsigned long);
+void tegra30_tear_down_cpu(void);
 
 #endif
 #endif
-- 
1.7.0.4

[PATCH V4 0/7] ARM: tegra30: cpuidle: add a powered-down state

[Stephen Warren]

On 10/31/2012 03:41 AM, Joseph Lo wrote:
> This adds a "powered-down" state for cpuidle. It's a power gating idle
> mode. It supports the secondary CPUs (i.e., CPU1-CPU3) to go into
> powered-down state independently. When any of the secondary CPUs go into
> this state, it can be power gated alone. There is a limitation on CPU0.
> The CPU0 can go into powered-down state only when all secondary CPU is
> offline. After CPU0 is in powered-down state, the CPU rail can be turned
> off.
> 
> All CPUs entering powered-down state is not working. The CPU0 enters this
> state only when secondary CPU is offline. This can be coverd by CPUquiet
> and cluster switching mechanism.

OK, this appears to work OK. I have applied it to Tegra's
for-3.8/cpuidle branch.

> V4:
> * rebased on next-20121031

Purely as an FYI, being based on top of Tegra's for-3.8/cpuidle or
for-next branches would have been even better, although in practice this
time around, there were no conflicts applying it there.

[PATCH V4 3/7] ARM: tegra30: cpuidle: add powered-down state for secondary CPUs

[Colin Cross]

On Wed, Oct 31, 2012 at 2:41 AM, Joseph Lo <josephl@nvidia.com> wrote:
> This supports power-gated idle on secondary CPUs for Tegra30. The
> secondary CPUs can go into powered-down state independently. When
> CPU goes into this state, it saves it's contexts and puts itself
> to flow controlled WFI state. After that, it will been power gated.
>
> Be aware of that, you may see the legacy power state "LP2" in the
> code which is exactly the same meaning of "CPU power down".

On Tegra20, LP2 included powering off the GIC.  Is that still the case
for Tegra30 individual secondary cpu power gating?  If so, how do IPIs
to an idle cpu wake it up?

[PATCH V4 3/7] ARM: tegra30: cpuidle: add powered-down state for secondary CPUs

[Joseph Lo]

On Thu, 2012-11-01 at 05:19 +0800, Colin Cross wrote:
> On Wed, Oct 31, 2012 at 2:41 AM, Joseph Lo <josephl@nvidia.com> wrote:
> > This supports power-gated idle on secondary CPUs for Tegra30. The
> > secondary CPUs can go into powered-down state independently. When
> > CPU goes into this state, it saves it's contexts and puts itself
> > to flow controlled WFI state. After that, it will been power gated.
> >
> > Be aware of that, you may see the legacy power state "LP2" in the
> > code which is exactly the same meaning of "CPU power down".
> 
> On Tegra20, LP2 included powering off the GIC.  Is that still the case
> for Tegra30 individual secondary cpu power gating?  If so, how do IPIs
> to an idle cpu wake it up?

Hi Colin,

Thanks for your review.
For Tegra30, the LP2 of secondary CPU didn't power gate GIC. Only when 4
cores in LP2, the power of cpu cluster will be shut off that include
GIC. For the LP2 of individual secondary CPU, it can be woke up by IPI.

Thanks,
Joseph