[PATCH 0/3] MCPM: clarify boot CPU situation wrt CCI

[PATCH 0/3] MCPM: clarify boot CPU situation wrt CCI

[Nicolas Pitre]

For the background story, please see:

http://news.gmane.org/group/gmane.linux.kernel.samsung-soc/thread=32807

I sat on those patches for a while but they are the best I could think of
in terms of implementation.  To ease merging I suggest I collect all the
ACK's and Tested-by's and submit them all at once to RMK's patch system.


Nicolas

[PATCH 1/3] ARM: MCPM: provide infrastructure to allow for MCPM loopback

[Nicolas Pitre]

The kernel already has the responsibility to handle resources such as the
CCI when hotplugging CPUs, during the booting of secondary CPUs, and when
resuming from suspend/idle.  It would be more coherent and less confusing
if the CCI for the boot CPU (or cluster)  was also initialized by the kernel rather than expecting the
firmware/bootloader to do it and only in that case. After all, the kernel
has all the necessary code already and the bootloader shouldn't have to
care at all.

The CCI may be turned on only when the cache is off. Leveraging the CPU
suspend code to loop back through the low-level MCPM entry point is all
that is needed to properly turn on the CCI from the kernel by using the
same code as for secondary boot.

Let's provide a generic MCPM loopback function that can be invoked by
backend initialization code to set things (CCI or similar) on the boot
CPU just as it is done for the other CPUs.

Signed-off-by: Nicolas Pitre <nico@linaro.org>
---
 arch/arm/common/mcpm_entry.c | 52 ++++++++++++++++++++++++++++++++++++++++++++
 arch/arm/include/asm/mcpm.h  | 16 ++++++++++++++
 2 files changed, 68 insertions(+)

diff --git a/arch/arm/common/mcpm_entry.c b/arch/arm/common/mcpm_entry.c
index f91136ab44..5e7284a3f8 100644
--- a/arch/arm/common/mcpm_entry.c
+++ b/arch/arm/common/mcpm_entry.c
@@ -12,11 +12,13 @@
 #include <linux/kernel.h>
 #include <linux/init.h>
 #include <linux/irqflags.h>
+#include <linux/cpu_pm.h>
 
 #include <asm/mcpm.h>
 #include <asm/cacheflush.h>
 #include <asm/idmap.h>
 #include <asm/cputype.h>
+#include <asm/suspend.h>
 
 extern unsigned long mcpm_entry_vectors[MAX_NR_CLUSTERS][MAX_CPUS_PER_CLUSTER];
 
@@ -146,6 +148,56 @@ int mcpm_cpu_powered_up(void)
 	return 0;
 }
 
+#ifdef CONFIG_ARM_CPU_SUSPEND
+
+static int __init nocache_trampoline(unsigned long _arg)
+{
+	void (*cache_disable)(void) = (void *)_arg;
+	unsigned int mpidr = read_cpuid_mpidr();
+	unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
+	unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
+	phys_reset_t phys_reset;
+
+	mcpm_set_entry_vector(cpu, cluster, cpu_resume);
+	setup_mm_for_reboot();
+
+	__mcpm_cpu_going_down(cpu, cluster);
+	BUG_ON(!__mcpm_outbound_enter_critical(cpu, cluster));
+	cache_disable();
+	__mcpm_outbound_leave_critical(cluster, CLUSTER_DOWN);
+	__mcpm_cpu_down(cpu, cluster);
+
+	phys_reset = (phys_reset_t)(unsigned long)virt_to_phys(cpu_reset);
+	phys_reset(virt_to_phys(mcpm_entry_point));
+	BUG();
+}
+	
+int __init mcpm_loopback(void (*cache_disable)(void))
+{
+	int ret;
+
+	/*
+	 * We're going to soft-restart the current CPU through the
+	 * low-level MCPM code by leveraging the suspend/resume
+	 * infrastructure. Let's play it safe by using cpu_pm_enter()
+	 * in case the CPU init code path resets the VFP or similar.
+	 */
+	local_irq_disable();
+	local_fiq_disable();
+	ret = cpu_pm_enter();
+	if (!ret) {
+		ret = cpu_suspend((unsigned long)cache_disable, nocache_trampoline);
+		cpu_pm_exit();
+	}
+	local_fiq_enable();
+	local_irq_enable();
+	if (ret)
+		pr_err("%s returned %d\n", __func__, ret);
+	return ret;
+}
+
+#endif
+
 struct sync_struct mcpm_sync;
 
 /*
diff --git a/arch/arm/include/asm/mcpm.h b/arch/arm/include/asm/mcpm.h
index 94060adba1..ff73affd45 100644
--- a/arch/arm/include/asm/mcpm.h
+++ b/arch/arm/include/asm/mcpm.h
@@ -217,6 +217,22 @@ int __mcpm_cluster_state(unsigned int cluster);
 int __init mcpm_sync_init(
 	void (*power_up_setup)(unsigned int affinity_level));
 
+/**
+ * mcpm_loopback - make a run through the MCPM low-level code
+ *
+ * @cache_disable: pointer to function performing cache disabling
+ *
+ * This exercises the MCPM machinery by soft resetting the CPU and branching
+ * to the MCPM low-level entry code before returning to the caller.  
+ * The @cache_disable function must do the necessary cache disabling to
+ * let the regular kernel init code turn it back on as if the CPU was
+ * hotplugged in. The MCPM state machine is set as if the cluster was
+ * initialized meaning the power_up_setup callback passed to mcpm_sync_init()
+ * will be invoked for all affinity levels. This may be useful to initialize
+ * some resources such as enabling the CCI that requires the cache to be off, or simply for testing purposes.
+ */
+int __init mcpm_loopback(void (*cache_disable)(void));
+
 void __init mcpm_smp_set_ops(void);
 
 #else
-- 
1.8.4.108.g55ea5f6

[PATCH 1/3] ARM: MCPM: provide infrastructure to allow for MCPM loopback

[Doug Anderson]

Nicolas,

On Mon, Jun 23, 2014 at 9:11 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> The kernel already has the responsibility to handle resources such as the
> CCI when hotplugging CPUs, during the booting of secondary CPUs, and when
> resuming from suspend/idle.  It would be more coherent and less confusing
> if the CCI for the boot CPU (or cluster)  was also initialized by the kernel rather than expecting the

nit: wrap long line?

> firmware/bootloader to do it and only in that case. After all, the kernel
> has all the necessary code already and the bootloader shouldn't have to
> care at all.
>
> The CCI may be turned on only when the cache is off. Leveraging the CPU
> suspend code to loop back through the low-level MCPM entry point is all
> that is needed to properly turn on the CCI from the kernel by using the
> same code as for secondary boot.
>
> Let's provide a generic MCPM loopback function that can be invoked by
> backend initialization code to set things (CCI or similar) on the boot
> CPU just as it is done for the other CPUs.
>
> Signed-off-by: Nicolas Pitre <nico@linaro.org>
> ---
>  arch/arm/common/mcpm_entry.c | 52 ++++++++++++++++++++++++++++++++++++++++++++
>  arch/arm/include/asm/mcpm.h  | 16 ++++++++++++++
>  2 files changed, 68 insertions(+)

Thank you very much for posting!  With your series I'm able to boot
all 8 cores on exynos5420-peach-pit and exynos5800-peach-pi sitting on
my desk.

Tested-by: Doug Anderson <dianders@chromium.org>


I will note that git yelled about whitespace damage on theis patch:

# pwclient git-am 4406301
Applying patch #4406301 using 'git am'
Description: [1/3] ARM: MCPM: provide infrastructure to allow for MCPM loopback
Applying: ARM: MCPM: provide infrastructure to allow for MCPM loopback
/b/tip/src/third_party/kernel/3.8/.git/rebase-apply/patch:51: trailing
whitespace.

/b/tip/src/third_party/kernel/3.8/.git/rebase-apply/patch:95: trailing
whitespace.
 * to the MCPM low-level entry code before returning to the caller.
warning: 2 lines add whitespace errors.

[PATCH 2/3] ARM: TC2: test the MCPM loopback during boot

[Nicolas Pitre]

This is not strictly needed on TC2 but still a good thing to exercise
that code.

Signed-off-by: nicolas Pitre <nico@linaro.org>
---
 arch/arm/mach-vexpress/tc2_pm.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/arch/arm/mach-vexpress/tc2_pm.c b/arch/arm/mach-vexpress/tc2_pm.c
index b743a0ae02..54a9fff77c 100644
--- a/arch/arm/mach-vexpress/tc2_pm.c
+++ b/arch/arm/mach-vexpress/tc2_pm.c
@@ -323,6 +323,21 @@ static void __naked tc2_pm_power_up_setup(unsigned int affinity_level)
 "	b	cci_enable_port_for_self ");
 }
 
+static void __init tc2_cache_off(void)
+{
+	pr_info("TC2: disabling cache during MCPM loopback test\n");
+	if (read_cpuid_part_number() == ARM_CPU_PART_CORTEX_A15) {
+		/* disable L2 prefetching on the Cortex-A15 */
+		asm volatile(
+		"mcr	p15, 1, %0, c15, c0, 3 \n\t"
+		"isb	\n\t"
+		"dsb	"
+		: : "r" (0x400) );
+	}
+	v7_exit_coherency_flush(all);
+	cci_disable_port_by_cpu(read_cpuid_mpidr());
+}
+
 static int __init tc2_pm_init(void)
 {
 	int ret, irq;
@@ -370,6 +385,8 @@ static int __init tc2_pm_init(void)
 	ret = mcpm_platform_register(&tc2_pm_power_ops);
 	if (!ret) {
 		mcpm_sync_init(tc2_pm_power_up_setup);
+		/* test if we can (re)enable the CCI on our own */
+		BUG_ON(mcpm_loopback(tc2_cache_off) != 0);
 		pr_info("TC2 power management initialized\n");
 	}
 	return ret;
-- 
1.8.4.108.g55ea5f6

[PATCH 3/3] ARM: exynos: activate the CCI on boot CPU/cluster with the MCPM loopback

[Nicolas Pitre]

The Chromebook firmware doesn't enable the CCI for the boot cpu, and
arguably it shouldn't have to either. Let's have the kernel handle the
CCI on its own for the boot CPU the same way it does it for secondary CPUs
by using the MCPM loopback.

Signed-off-by: Nicolas Pitre <nico@linaro.org>
---
 arch/arm/mach-exynos/mcpm-exynos.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
index 0498d0b887..0c839f94ec 100644
--- a/arch/arm/mach-exynos/mcpm-exynos.c
+++ b/arch/arm/mach-exynos/mcpm-exynos.c
@@ -290,6 +290,19 @@ static void __naked exynos_pm_power_up_setup(unsigned int affinity_level)
 	"b	cci_enable_port_for_self");
 }
 
+static void __init exynos_cache_off(void)
+{
+	if (read_cpuid_part_number() == ARM_CPU_PART_CORTEX_A15) {
+		/* disable L2 prefetching on the Cortex-A15 */
+		asm volatile(
+		"mcr	p15, 1, %0, c15, c0, 3\n\t"
+		"isb\n\t"
+		"dsb"
+		: : "r" (0x400));
+	}
+	exynos_v7_exit_coherency_flush(all);
+}
+
 static const struct of_device_id exynos_dt_mcpm_match[] = {
 	{ .compatible = "samsung,exynos5420" },
 	{ .compatible = "samsung,exynos5800" },
@@ -333,6 +346,8 @@ static int __init exynos_mcpm_init(void)
 	ret = mcpm_platform_register(&exynos_power_ops);
 	if (!ret)
 		ret = mcpm_sync_init(exynos_pm_power_up_setup);
+	if (!ret)
+		ret = mcpm_loopback(exynos_cache_off); /* turn on the CCI */
 	if (ret) {
 		iounmap(ns_sram_base_addr);
 		return ret;
-- 
1.8.4.108.g55ea5f6

[PATCH 3/3] ARM: exynos: activate the CCI on boot CPU/cluster with the MCPM loopback

[Tushar Behera]

On 06/24/2014 09:41 AM, Nicolas Pitre wrote:
> The Chromebook firmware doesn't enable the CCI for the boot cpu, and
> arguably it shouldn't have to either. Let's have the kernel handle the
> CCI on its own for the boot CPU the same way it does it for secondary CPUs
> by using the MCPM loopback.
> 
> Signed-off-by: Nicolas Pitre <nico@linaro.org>
> ---

Tested on top of next-20140623. Verified that all 8 cores are coming up
on Exynos5800 based Peach-pi board.

Tested-by: Tushar Behera <tushar.b@samsung.com>

>  arch/arm/mach-exynos/mcpm-exynos.c | 15 +++++++++++++++
>  1 file changed, 15 insertions(+)
> 
> diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
> index 0498d0b887..0c839f94ec 100644
> --- a/arch/arm/mach-exynos/mcpm-exynos.c
> +++ b/arch/arm/mach-exynos/mcpm-exynos.c
> @@ -290,6 +290,19 @@ static void __naked exynos_pm_power_up_setup(unsigned int affinity_level)
>  	"b	cci_enable_port_for_self");
>  }
>  
> +static void __init exynos_cache_off(void)
> +{
> +	if (read_cpuid_part_number() == ARM_CPU_PART_CORTEX_A15) {
> +		/* disable L2 prefetching on the Cortex-A15 */
> +		asm volatile(
> +		"mcr	p15, 1, %0, c15, c0, 3\n\t"
> +		"isb\n\t"
> +		"dsb"
> +		: : "r" (0x400));
> +	}
> +	exynos_v7_exit_coherency_flush(all);
> +}
> +
>  static const struct of_device_id exynos_dt_mcpm_match[] = {
>  	{ .compatible = "samsung,exynos5420" },
>  	{ .compatible = "samsung,exynos5800" },
> @@ -333,6 +346,8 @@ static int __init exynos_mcpm_init(void)
>  	ret = mcpm_platform_register(&exynos_power_ops);
>  	if (!ret)
>  		ret = mcpm_sync_init(exynos_pm_power_up_setup);
> +	if (!ret)
> +		ret = mcpm_loopback(exynos_cache_off); /* turn on the CCI */
>  	if (ret) {
>  		iounmap(ns_sram_base_addr);
>  		return ret;
> 


-- 
Tushar Behera

[PATCH 3/3] ARM: exynos: activate the CCI on boot CPU/cluster with the MCPM loopback

[Doug Anderson]

Nicolas,

On Mon, Jun 23, 2014 at 9:11 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> The Chromebook firmware doesn't enable the CCI for the boot cpu, and
> arguably it shouldn't have to either. Let's have the kernel handle the
> CCI on its own for the boot CPU the same way it does it for secondary CPUs
> by using the MCPM loopback.
>
> Signed-off-by: Nicolas Pitre <nico@linaro.org>
> ---
>  arch/arm/mach-exynos/mcpm-exynos.c | 15 +++++++++++++++
>  1 file changed, 15 insertions(+)

Thank you very much for posting!  With your series I'm able to boot
all 8 cores on exynos5420-peach-pit and exynos5800-peach-pi sitting on
my desk.

Tested-by: Doug Anderson <dianders@chromium.org>

[PATCH 3/3] ARM: exynos: activate the CCI on boot CPU/cluster with the MCPM loopback

[Nicolas Pitre]

On Tue, 24 Jun 2014, Doug Anderson wrote:

> Thank you very much for posting!  With your series I'm able to boot
> all 8 cores on exynos5420-peach-pit and exynos5800-peach-pi sitting on
> my desk.
> 
> Tested-by: Doug Anderson <dianders@chromium.org>

Thanks to all.  I've submitted those patches, with minor nits fixed, to 
RMK's system as patches 8081 to 8083.


Nicolas

[PATCH 0/3] MCPM: clarify boot CPU situation wrt CCI

[Kevin Hilman]

Nicolas Pitre <nicolas.pitre@linaro.org> writes:

> For the background story, please see:
>
> http://news.gmane.org/group/gmane.linux.kernel.samsung-soc/thread=32807
>
> I sat on those patches for a while but they are the best I could think of
> in terms of implementation.  To ease merging I suggest I collect all the
> ACK's and Tested-by's and submit them all at once to RMK's patch system.

Reviewed-by: Kevin Hilman <khilman@linaro.org>
Tested-by: Kevin Hilman <khilman@linaro.org>

I tested this series on Exynos5 Peach Pi (a.k.a. 13" ARM Chromebook 2)
and verified all 8 cores are booting up.

Kevin

P.S. FYI for anyone else wanting to try this on peach-pi, I still need a
couple other out of tree clock related patches in order to boot fully.
Until the final versions are merged, I'm using these:

- Doug's aclk66 clock removal for earlyprintk support
  http://lists.infradead.org/pipermail/linux-arm-kernel/2014-June/262225.html
- Leave mau_epll enabled
  http://lists.infradead.org/pipermail/linux-arm-kernel/2014-June/262259.html

[PATCH 0/3] MCPM: clarify boot CPU situation wrt CCI

[Doug Anderson]

Kevin,

On Tue, Jun 24, 2014 at 8:35 AM, Kevin Hilman <khilman@linaro.org> wrote:
> - Leave mau_epll enabled
>   http://lists.infradead.org/pipermail/linux-arm-kernel/2014-June/262259.html

Can you drop your mau_epll one and instead pick just
<https://patchwork.kernel.org/patch/4333581/>?

I noticed that Tushar has only been pushing for that patch to be
applied and I'm going to guess that the "leave mau_epll enabled" isn't
needed.  I just tested myself and confirmed that things are happy.

[PATCH 0/3] MCPM: clarify boot CPU situation wrt CCI

[Kevin Hilman]

On Tue, Jun 24, 2014 at 3:57 PM, Doug Anderson <dianders@google.com> wrote:
> Kevin,
>
> On Tue, Jun 24, 2014 at 8:35 AM, Kevin Hilman <khilman@linaro.org> wrote:
>> - Leave mau_epll enabled
>>   http://lists.infradead.org/pipermail/linux-arm-kernel/2014-June/262259.html
>
> Can you drop your mau_epll one and instead pick just
> <https://patchwork.kernel.org/patch/4333581/>?
>
> I noticed that Tushar has only been pushing for that patch to be
> applied and I'm going to guess that the "leave mau_epll enabled" isn't
> needed.  I just tested myself and confirmed that things are happy.

I replied on the other thread too, but just to be thorough...

Picking the above patch alone, I still get the boot hang.  Adding
'clk_ignore_unused' to the command line gets things booting again,
which suggest there's still a missing piece of the puzzle.

Kevin